Evaluating clinicians' teaching performance

Improving Faculty Teaching Practices: Fostering the "Ideal ICU Teacher" Using a Positive Deviance Strategy

We describe an innovative approach to medical education, using "positive deviants," or faculty who emerge as role models, despite responsibilities that compete with teaching efforts. In small group sessions, "positive deviants" taught effective strategies, resulting in faculty implementing new strategies.

Characteristics of good clinical teachers in anesthesiology from medical students' perspective: a qualitative descriptive study

PURPOSE

Learning needs are influenced by the stage of learning and medical specialty. We sought to investigate the characteristics of a good clinical teacher in anesthesiology from the medical students' perspective.

METHODS

We conducted a qualitative descriptive study to analyze written comments of medical students about their clinical teachers' performances. Our analysis strategy was the inductive content analysis method. The results are reported as a descriptive summary with major themes as the final product.

RESULTS

Our study identified four themes. The first theme, teachers' individual characteristics, includes characteristics that are usually more related to students' subjective experiences and feelings. The second theme, teachers' characteristics that advance student learning, seems to be one of the most important contributions to learning because it increases the practice of procedural skills. The third theme, teachers' characteristics that prepare students for success, shows characteristics that facilitate students' learning by promoting a healthy and safe environment. Lastly, the fourth theme, characteristics related to teaching approaches, includes characteristics that can guide clinical teachers more objectively.

CONCLUSION

Our analysis of the written comments of medical students identified many characteristics of a good clinical teacher that were organized in four different themes. These themes contribute to expand on existing understandings of clinical teaching in the anesthesiology clerkship environment, and add new interpretations that can be reflected upon and explored by other clinical educators.

Psychometric Properties of Visual Indicators of Teaching and Learning Success "VITALS" Instrument for Evaluation of Clinical Teachers

PURPOSE

We have previously developed an instrument for students' evaluation of clinical teachers that we called Visual Indicators of Clinical Teaching and Learning Success (VITALS). This study measures the reliability of VITALS as an instrument for student evaluation of clinical tutors. Additionally, the study explores the minimum number of student raters necessary for an acceptable reliability, and provides evidence of construct validity of the evaluation scores.

MATERIALS AND METHODS

This retrospective study included 1825 evaluation forms completed by medical students evaluating clinical tutors using the VITALS instrument. Reliability was measured by applying generalizability theory (G-theory) analysis using a two-facet design (raters and items). A D-study was used to determine the minimum number of raters required to achieve a reliability ≥0.80. Face validity was tested by measuring tutors' degree of agreement with the items of the study instrument.

RESULTS

The overall G-coefficient was 0.89. The subject of measurement (clinical tutors' scores) represented 15.8% of the variance across all tutors and items. The variance due to the interaction between raters (students) and tutors contributed to 43.5%, while the variance due to items was negligible. The remaining 40% of the variance was due to unexplained sources of error. The D-study demonstrated that a minimum of 12 raters (students) are required to achieve a reliability of 0.80. Finally, most of the clinical tutors agreed that all items in the instrument were appropriate.

CONCLUSION

We demonstrate that VITALS exhibits good psychometric properties. There should be at least 12 students rating each clinical tutor to have an acceptable level of reliability for the study instrument. Face validity of the study instrument was evidenced by its high level of approval among clinical tutors.

Variability of residents' ratings of faculty's teaching performance measured by five- and seven-point response scales

BACKGROUND

Medical faculty's teaching performance is often measured using residents' feedback, collected by questionnaires. Researchers extensively studied the psychometric qualities of resulting ratings. However, these studies rarely consider the number of response categories and its consequences for residents' ratings of faculty's teaching performance. We compared the variability of residents' ratings measured by five- and seven-point response scales.

METHODS

This retrospective study used teaching performance data from Dutch anaesthesiology residency training programs. Questionnaires with five- and seven-point response scales from the extensively studied System for Evaluation of Teaching Qualities (SETQ) collected the ratings. We inspected ratings' variability by comparing standard deviations, interquartile ranges, and frequency (percentage) distributions. Relevant statistical tests were used to test differences in frequency distributions and teaching performance scores.

RESULTS

We examined 3379 residents' ratings and 480 aggregated faculty scores. Residents used the additional response categories provided by the seven-point scale - especially those differentiating between positive performances. Residents' ratings and aggregated faculty scores showed a more even distribution on the seven-point scale compared to the five-point scale. Also, the seven-point scale showed a smaller ceiling effect. After rescaling, the mean scores and (most) standard deviations of ratings from both scales were comparable.

CONCLUSIONS

Ratings from the seven-point scale were more evenly distributed and could potentially yield more nuanced, specific and user-friendly feedback. Still, both scales measured (almost) similar teaching performance outcomes. In teaching performance practice, residents and faculty members should discuss whether response scales fit their preferences and goals.