Faculty Teaching Scores: Validating Evaluations, Evaluating Validation

Teaching quality in neurosurgery: quantitating outcomes over time

OBJECTIVE

High-quality neurosurgery resident training is essential to developing competent neurosurgeons. Validated formative tools to assess faculty teaching performance exist, but are not used widely among Accreditation Council for Graduate Medical Education (ACGME) residency programs in the United States. Furthermore, their longer-term impact on teaching performance improvement and educational outcomes remains unclear. The goal of this study was to assess the impact of implementing an evaluation system to provide faculty with feedback on teaching performance in a neurosurgery residency training program over a 4-year period.

METHODS

The authors performed a prospective cohort study in which a modified version of the System for Evaluation of Teaching Qualities (SETQ) instrument was administered to neurosurgical trainees in their department regularly every 6 months. The authors analyzed subscale score dynamics to identify the strongest correlates of faculty teaching performance improvement. ACGME program survey results and trainee performance on written board examinations were compared for the 3 years before and after SETQ implementation.

RESULTS

The overall response rate among trainees was 91.8%, with 1044 surveys completed for 41 faculty. Performance scores improved progressively from cycle 1 to cycle 6. The strongest correlate of overall performance was providing positive feedback to trainees. Compared to the 3 years prior, the 3 years following SETQ implementation saw significant increases in written board examination and ACGME resident survey scores compared to the national mean.

CONCLUSIONS

Implementation of SETQ was associated with significant improvements in faculty teaching performance as judged by trainees over a 4-year period, and guided curricular changes in the authors' training program that resulted in improved educational outcomes.