Conceptual Framework for Course Evaluations
This website provides a brief overview of the most important aspects of the conceptual framework for course evaluations.
Purpose of evaluations
The standardized course evaluation serves a dual purpose: on the one hand, it provides specific suggestions to improve teaching and learning situations, and on the other hand provides information about the current quality of teaching. The ultimate goal of all activities is to not only maintain but continuously advance high-level teaching quality.
Proof of quality (accountability)
The results of the evaluations, specifically the final evaluation, at the University of Bern serve, among other things, as proof of quality. Since universities serve a publicly financed and legally regulated educational mandate, they are required to provide transparent information to the public regarding the implementation and goal achievement of the measures they offer (i.e. the courses).
In addition, course evaluations are anchored in the University's quality assurance concept and thus play a key role in the University's successful accreditation.
Advancement (improvement-focused)
However, the evaluation results also serve to continuously advance the quality of teaching, as described in the University's quality assurance concept. The goal is to identify strengths and weaknesses of teaching at the course level in order to help generate ideas for modifications. The interim evaluation and the evaluation of performance assessments are specifically designed for this purpose. However, the final evaluation serves this purposed to a certain extent as well.
The results of the three evaluations are used to improve courses on two levels:
- Lecturers use the results to improve their own teaching concepts (learning outcomes, teaching/learning activities, assessments).
- The institutes/faculties use insights from the aggregated results to optimize (pre-)conditions for good teaching.
Conditional factors, target criteria & impact model for Good Teaching
In order for course evaluations to fulfill their purposes of ensuring and improving the quality of teaching at the University of Bern, they must provide answers to the following question:
To what extent is Good Teaching implemented in courses at the University of Bern?
In order to define Good Teaching at the University of Bern, one of the most recent and comprehensive studies on this issue (Hattie, 2014) was consulted (among others). In addition, it was taken into account that teaching is generally considered «good» when three crucial factors are coherently interrelated («constructive alignment»)
- Learning outcomes
- Teaching and learning activities
- Assessment formats
Based on these and other findings of current teaching and evaluation research, conditional factors and criteria that define Good Teaching at the University of Bern were selected for course evaluations. These conditional factors and target criteria are divided into three areas:
| Conditional factors |
Target criteria |
|---|---|
|
|
The conditional factors and target criteria as well as their interrelations were summarized in an impact model (see figure), and the questionnaires for the respective evaluations were developed on the basis of this impact model. The data obtained with these questionnaires reveal the specific areas (conditional factors) of a course that have potential for improvement.
Method, Sample questionnaires, content & goals
Method
At the University of Bern, course evaluations are primarily based on student feedback, a valid form of assessing course quality (e.g., Fondel et al., 2015). In most cases, this feedback is collected by means of a written questionnaire.
Sample questionnaires
- Interim evaluation: online
- Final evaluation: online
- Supplementary modules to final evaluation: e-learning / Student contributions / Team teaching / Application orientation / Interdisciplinarity
- Evaluation of performance assessment: online
Contents & goals
The survey instruments (questionnaires) of the three evaluations are coordinated in terms of content:
Interim Evaluation
The questionnaire for the interim evaluation is designed to identify important conditions for Good Teaching. The content of the questionnaire for the interim evaluation therefore focuses on the defined conditional factors of Good Teaching: course, lecturers, students.
Final evaluation
The questionnaire for the final evaluation is intended to measure the achievement of the target criteria of Good Teaching as briefly and succinctly as possible. Therefore, the questionnaire for the final evaluation focuses on two of the three defined target criteria: Satisfaction and learning progress.
In order to cover additional, course-specific aspects in the surveys, it is possible to add one or more supplementary module(s) to the core questionnaire for the final evaluation. The results for supplementary modules are intended solely for informational purposes for the instructors and do not count towards the overall result (success level rating) of the final evaluation.
The following supplementary modules are available:
- e-learning
- student contributions (e.g. seminars)
- team teaching
- application orientation (e.g. labs)
- interdisciplinarity
Evaluation of the performance assessment
The questionnaire for the evaluation of performance assessments includes both the conditional factors performance assessment and students as well as the achievement of the target criterion learning success. The questionnaire focuses on constructive alignment: the match between learning outcomes, teaching/learning activities and the content and form of the performance assessment; the learning behavior (student factor), and the self-assessed learning success (achievement of objectives).
Not included in the survey are aspects of fairness, satisfaction and grading. The opinions on these aspects are often very subjective and can have undesirable effects on the design of performance assessments.
Frequency of evaluations
While the interim evaluation does not follow a prescribed rotation, the final evaluation and the evaluation of the performance assessments have to be carried out on a regular basis:
Final evaluation
The conceptual framework requires that, at a minimum, each course and all instructors are evaluated at least once every three years. Within this framework, individual faculties plan final evaluations according to their own regulations and monitor compliance with them. The final evaluation is mandatory for new types of courses and newly hired instructors, as well as for instructors in qualification positions (e.g. assistant professorships with tenure track).
Evaluation of the performance assessment
The conceptual framework requires that 50% of all performance assessments are evaluated on a regular basis (at least every 3 years). The faculties plan the evaluations accordingly and select the performance assessments to be evaluated, taking into account the faculty's QM regulations.
Timing of evaluations
Course evaluations are conducted at three different points in time during the semester.
For specific dates for the current semester, please see Current semester.
Interim evaluation
- 1st half of the semester
- Default: no later than week 9 of the semester
Final evaluation:
- Toward the end of the semester
- Default: starting week 11 of the semester
Evaluation of the performance assessment
- Immediately after the assessment, before grades are released
- Default: starting week 11 of the semester
Rating criteria & consequences of rating
The results from the final evaluation are categorized into one of 4 performance levels:
- excellent,
- good,
- sufficient and
- insufficient teaching.
The categorization is based on the overall score of a final evaluation, which results from the average scores of the individual questions. Included in this calculation are questions 1.1 - 1.3 (Learning progress) and 2.1 - 2.5 (Satisfaction).
Owing to the Corona situation and because of the need to introduce the thresholds in a constructive manner, the following preliminary thresholds have been defined since the spring semester 2020:
- <4.0 = «insufficient» (Red level)
- ≥4.0 <4.5 = «sufficient» (Yellow level)
- ≥4.5 <5.7 = «good» (Light green level)
- ≥5.7 = «excellent» (Dark green level)
The highest rating, «excellent teaching», is linked to the ALL project (Recognition of excellent teaching performance).
«Insufficient» results lead to the initiation of a quality development process. Faculties define their own measures for this purpose in accordance with their respective QM regulations.
Interim evaluations are not assessed a priori, as they are designed to provide formative and improvement-oriented feedback only.
Structure of the QM process for course evaluations
The part of our quality management that is based on course evaluations is designed as a control loop which includes 3 interrelated levels:
First level: Course
The first level, the course, primarily concerns the instructors.The instructors plan the course taking into account the upcoming evaluations for a given semester (1) and conduct the evaluations on the given dates (2). From the results of the evaluations (reports) they derive ideas for improving the course and, if applicable, the performance assessment (2/3). The results and any modifications to the course and/or performance assessment (4) are discussed with the students if possible.
Second level: Faculty
At the end of each evaluation period, the faculties receive the mean scores of the final evaluations from the Evaluation Office, which include the performance rating for each course and each set of questions. Once the performance assessments have all taken place, the faculties also receive the pooled results of these evaluations. The faculties analyze these results (7) and take appropriate measures (8).
Which measures are taken in which cases is stipulated in the conceptual framework and in the faculties' QM regulations. In general, courses that are rated as insufficient must be re-evaluated the next time they are taught (interim and final evaluation). After completion of the repeated evaluations, instructors are asked to comment on the three evaluations in a short self-report and to reflect on their own teaching. If the results of the repeated evaluation turn out to be "insufficient" as well, the quality manager organizes a discussion with the instructors and potential other persons, depending on the faculty's QM regulations.
In addition, the faculties determine when and which courses must be evaluated (5) and ensure that this plan is implemented according to their regulations (6).
Third level: University management
The faculties report to the university management on the completed evaluations, the insights gained from them, and the quality measures that have been planned or taken. The university management analyzes these reports and discusses them with the faculties during the strategy meetings (11). If necessary, specific goals are defined and measures derived (12), which in turn entail adjustments to the quality strategy (9) and its implementation (10).