Student Ratings of Teaching: Validity of Several Rating Factors

Compatibility of Students' Learning and Teaching Styles on Their Ratings of College Teaching

Student's ratings of teaching performance were compared for those whose preferences for learning style were strongly matched ( n = 77) and strongly mismatched ( n = 40) with their instructors' preference of teaching style. Canfield's Learning Styles Inventory and Canfield and Canfield's Instructional Styles Inventory measured learning and teaching style preferences. The rating scale was designed from a catalog of items. Analysis of variance indicated no significant differences in instructors' ratings between the two groups.

Using Multiple Measures in the Interpretation of Student Ratings of Instruction

This study investigated a multiple measurement approach to validating the inter pretation of student ratings of a medical school course. Concerns of priority to students were isolated from the ratings and data on student opinion collected by an open-ended question and student faculty documents. An interpretation consonant with all data was derived. Influences on student opinion, including demographic, academic, and personal characteristics of students, teaching practices, and the instructional context were explored through a comparison of the present class with a previous class. Ratings of the course by the two classes differed in the absence of significant differences in the students or teaching practices. Divergences in the ratings were attributed to changes in the instruc tional context reported in the multiple measures of student opinion. It was concluded that a multiple measurement approach allows greater precision in the interpretation of student opinion than is afforded by ratings alone.

Evaluating the performance of academic medical education administrators

Educators have devoted little attention to formal evaluation of educational administrative personnel. The authors surveyed the educational administrators working in McMaster University's Department of Medicine residency program and found they felt they were receiving little useful feedback. The authors also surveyed the colleagues, residents, and administrative staff with whom the administrators worked and found they felt they had inadequate input into the administrators' evaluation. In response to these results, a measurement instrument was developed based on existing job descriptions and feedback was obtained on administrators' performance from relevant individuals. After three yearly evaluations, administrators and evaluators acknowledged much broader input into evaluation but saw little evaluation-related improvement in performance. Of the administrators, 85% felt the process should continue as did 91% of the evaluators. An evaluation process may not alter perceived performance when it is already good but there may be other benefits to rigorous evaluation.

Supervisor evaluation : from theory to implementation

The evaluation of supervisors and assessment of teaching effectiveness are vital for academic psychiatry programs. The literature on teaching evaluation in classroom settings is reviewed briefly, followed by a summary of the available literature on supervisor evaluation in clinical medicine and psychiatry. While student evaluation forms of supervisor's teaching ability may be reliable and valid, this popular form of assessment is insufficient to fulfill all the purposes required of such evaluations. The development and implementation of a new process for supervisor evaluation is described for the University of Toronto's Department of Psychiatry.

A factor analytic study of teaching in off-campus general practice clerkships

The Preceptor-evaluation questionnaire (PEQ) was developed at the University of Mississippi Medical Center to evaluate the clinical teaching behavior of general practice doctors in private practice who have one or more third-year medical students per year in their offices for a required general practice clerkship. The PEQ was designed to provide feedback to these doctors and the sponsoring department in an effort to improve teaching in this setting. Developed from the input of family practice doctors who have supervised students in their offices and from a review of related literature, the instrument was given to students to determine the level of importance they attributed to each teaching characteristic. A principle components factor analysis and subsequent varimax rotation produced six significant teaching factors in this setting. These six factors were: (1) demonstrates a humanistic orientation toward patients and staff; (2) demonstrates a humanistic orientation toward student; (3) provides opportunity for practice; (4) motivates and stimulates student's thinking; (5)communicates effectively with student; and (6) demonstrates comfort and confidence in the roles of doctor and teacher. The factor relating to the humanistic orientation toward student has not appeared as a separate factor in other studies of clinical teaching. This may be due to the unique aspects of the relationship between student and doctor in this setting which may not occur in other settings. The PEQ and similar reliable and valid instruments can provide valuable information for the doctors in these off-campus teaching situations as well as provide useful programme planning information for the sponsoring academic departments. This information may also be valuable for off-campus residency training.

Validity of students' ratings of clinical instructors

The validity of students' rating of clinical instruction was correlated with students' performance in a clinical examination. One-way analysis of variance showed that the students' ratings discriminated among instructors, and that the instructors were a significant source of variance in their students' adjusted examination scores on a written clinical simulation. There was no significant correlation between students' performance in the examinations and their ratings of the clinical instructors.

Prior Subject Interest, Students' Evaluations, And Instructional Effectiveness

Students' Prior Subject Interest in a course showed similar correlations with student ratings of instructional effectiveness in two university settings (N = 1102 classes). Correlations between Prior Subject Interest and different dimensions of instructional effectiveness varied from approximately zero to .44. Though these correlations were not high, Prior Subject Interest predicted student ratings better than any of 15 other student/course/instructor characteristics considered (e.g., Expected Grade, Class Size, Workload/Difficulty, Teacher Rank). Instructor self-evaluations of their own teaching effectiveness (N = 329 classes) were also positively correlated with both their own and their students' perceptions of Prior Subject Interest; the dimensions that were most highly correlated -- particularly Learning/Value -- were the same as observed with student ratings. Since both student and instructor self evaluations were similarly related to Prior Subject Interest, it appears that this variable actually affects instructional effectiveness in a way that is accurately reflected in the student ratings.

The relationship between student ratings of course effectiveness and student achievement

For many years student ratings have been used to assess the strengths and weaknesses of various courses and the teaching abilities of the instructors involved. Investigators have either supported or attacked the validity of this assessment procedure on the basis of the direction and degree of the relationship found between student rating of a course and student achievement in that course. This validation study supplemented correlation analysis with analysis of variance to compare groups at various achievement levels and to control for the possible confounding effects of overall group analysis. Medical students responded to a Likert attitude scale about aspects of an anatomy course. The mean attitude scores for a high and a low course achievement group were significantly different at the 0.005 level; the high achievement group had a more positive course attitude.