Factors influencing residents' evaluations of clinical faculty member teaching qualities and role model status

Higher degree of impostor characteristics relates to lower self-perceived teaching ability and emotional intelligence

BACKGROUND

Emotional intelligence (EI) has been previously associated with teaching ability and impostor phenomenon (IP) in medical education; however, studies have demonstrated mixed findings and have largely focused on trainees only. Therefore, we sought to explore the potential association between the degree of IP characteristics, EI and teaching ability in obstetrics and gynaecology (Ob/Gyn) faculty physicians.

METHODS

A cross-sectional, survey-based pilot study was completed at a single academic institution. Ob/Gyn attending (faculty) physicians were queried using surveys related to IP, EI and teaching ability. Resident (trainee) physicians also completed anonymous evaluations of faculty teaching ability.

FINDINGS

The degree of IP characteristics correlated negatively with self-perceived teaching ability, with no significant differences in resident assessment of faculty teaching. IP also correlated negatively with EI. Although there were no statistically significant differences in resident assessment of teaching ability based on EI, both EI and IP demonstrated inverse relationships to faculty assessment of teaching ability compared with resident assessment.

CONCLUSION

IP appears to relate to lower perceived teaching ability in Ob/Gyn faculty that does not correspond to resident evaluation of teaching performance. The demonstrated negative correlation between the degree of impostor characteristics and EI suggests that EI could potentially play a protective role in the development of IP and burnout, as well as influence teaching. This relationship may have implications for faculty willingness to continue in academic medicine.

Gender Disparity in Teaching Evaluations of Pediatric Faculty by Residents

OBJECTIVES

Despite their overrepresentation, female physicians continue to have lower rates of promotion compared with male physicians. Teaching evaluations play a role in physician advancement. Few studies have investigated gender disparity in resident evaluations of pediatric faculty. We hypothesized that gender disparities in resident evaluations of faculty exist and vary across subspecialties and primary work environments.

METHODS

Pediatric faculty institution-specific evaluations completed by residents from January 1, 2015, to March 9, 2020, were obtained from a single academic center. Mean ratings of faculty performance were compared by gender using a Wilcoxon 2-sample test.

RESULTS

Fifteen-thousand one-hundred and forty-two evaluations (5091 of male faculty and 10 051 of female faculty) were included. Female faculty were rated higher in overall teaching ability (female = 4.67 versus male = 4.65; P = .004). There was no statistical difference in the mean ratings of male and female faculty in the inpatient setting, whereas outpatient female faculty were rated higher in overall teaching ability (female = 4.79 versus male = 4.73; P = .005). For general pediatric faculty, females received higher ratings for overall teaching ability (female = 4.75 versus male = 4.70; P < .001). By contrast, there was no difference in ratings of subspecialty pediatric faculty.

CONCLUSIONS

Pediatric female faculty were statistically rated higher than male faculty in overall teaching ability, although these findings may not be educationally significant. The difference was driven by evaluations in the outpatient setting and for general pediatricians. This study is one of the first in pediatrics adding to the continued investigation of gender disparities in academic medicine.

"LEARN", a novel teaching method for Chinese clinical clerkship: A cross-sectional study

Background

Despite the clerkship being crucial in the training of a future doctor, no widely accepted education model has been proposed. This study devised a new model for clinical clerkship rotations, titled "LEARN" for Lecture, English-video, Advisor, Real-case and Notion, and evaluated whether the LEARN model is appropriate for medical education in China.

Methods

A cross-sectional study was performed among 101 fourth-year students from the Xiangya School of Medicine during an Orthopaedic Surgery clerkship rotation in the Third Xiangya Hospital. They were divided into seven groups and took clerkship based on the LEARN model. A questionnaire was collected at the conclusion to measure learning outcomes.

Results

The LEARN model was highly accepted with the acceptance of five sessions being 95.92% (94/98), 93.88% (92/98), 96.98% (97/98), 100% (98/98) and 96.94% (95/98). The outcomes of two genders were comparable, whereas a difference was observed in the test score among groups (group 3 scored 93.93 ± 5.20, higher than others). Quantitative analysis showed that positive correlations existed in participation in the Notion (Notion means students' case discussion) section with leadership (r = 0.84, 95% CI: 0.72-0.94, p < 0.001), participation in the Real-case section with leadership (r = 0.66, 95% CI: 0.50-0.80, p < 0.001), participation in the Real-case section with mastery of inquiring skills (r = 0.57, 95% CI: 0.40-0.71, p < 0.001) and participation in the Notion section with mastery of physical examination skills (r = 0.56, 95% CI: 0.40-0.69, p < 0.001). Further qualitative analysis demonstrated that high-level participation in the English-video section indicated better outcomes in mastery of inquiring (p < 0.01), physical examination (p < 0.001), film reading (p < 0.01) and clinical reasoning (p < 0.01) skills.

Conclusion

Our results support the LEARN model is a promising method for medical clerkship in China. Further research involving more participants and more meticulous design is planned to test its efficacy. For refinement, educators may try to promote students' participation in the English-video session.

A Competency-based Tool for Resident Evaluation of Pediatric Emergency Department Faculty

n/a

Characteristics of good clinical teachers in anesthesiology from medical students' perspective: a qualitative descriptive study

PURPOSE

Learning needs are influenced by the stage of learning and medical specialty. We sought to investigate the characteristics of a good clinical teacher in anesthesiology from the medical students' perspective.

METHODS

We conducted a qualitative descriptive study to analyze written comments of medical students about their clinical teachers' performances. Our analysis strategy was the inductive content analysis method. The results are reported as a descriptive summary with major themes as the final product.

RESULTS

Our study identified four themes. The first theme, teachers' individual characteristics, includes characteristics that are usually more related to students' subjective experiences and feelings. The second theme, teachers' characteristics that advance student learning, seems to be one of the most important contributions to learning because it increases the practice of procedural skills. The third theme, teachers' characteristics that prepare students for success, shows characteristics that facilitate students' learning by promoting a healthy and safe environment. Lastly, the fourth theme, characteristics related to teaching approaches, includes characteristics that can guide clinical teachers more objectively.

CONCLUSION

Our analysis of the written comments of medical students identified many characteristics of a good clinical teacher that were organized in four different themes. These themes contribute to expand on existing understandings of clinical teaching in the anesthesiology clerkship environment, and add new interpretations that can be reflected upon and explored by other clinical educators.

El profesor de cirugía, más que un cirujano

Introducción. La educación médica en cirugía se encuentra en transformación, producto de la incorporación de estrategias pedagógicas en sus procesos docente-asistenciales. Sin embargo, existe una limitación de los docentes para responder a las necesidades educativas actuales propias de un programa de especialización en cirugía. Estudios sobre el tema han identificado puntos de intervención en las competencias docentes, con una nueva concepción de un profesor de cirugía. Métodos. Se analizó, con una posición crítica y desde la perspectiva del residente, los atributos esperados en un profesor de cirugía de alta calidad. Discusión. Se caracterizaron los roles de un profesor de cirugía, acorde con la actividad clínica y quirúrgica habitual, sus cualidades e impacto esperado como generador de un cambio en las competencias, técnicas y no técnicas, de un profesional médico en especialización. Asimismo, se efectuó una aproximación teórica al proceso de enseñanza-aprendizaje de la cirugía, así como de la construcción de su identidad docente, y los alcances pedagógicos de un profesor de cirugía en la actualidad. Conclusiones. El perfil del cirujano general ha cambiado y los profesores de cirugía deben estar a la altura de este reto. Los nuevos modelos educativos favorecen un impacto positivo en los postgrados y en la percepción del residente sobre su proceso formativo. Sin embargo, es necesario una reconfiguración del docente, dentro de un contexto propio del ejercicio de la cirugía y su responsabilidad social. Los programas académicos están obligados a propiciar el desarrollo profesoral en pro de elevar el nivel del futuro cirujano.

Gendered Expectations: the Impact of Gender, Evaluation Language, and Clinical Setting on Resident Trainee Assessment of Faculty Performance

BACKGROUND

Gender inequity is pervasive in academic medicine. Factors contributing to these gender disparities must be examined. A significant body of literature indicates men and women are assessed differently in teaching evaluations. However, limited data exist on how faculty gender affects resident evaluation of faculty performance based on the skill being assessed or the clinical practice settings in which the trainee-faculty interaction occurs.

OBJECTIVE

Evaluate for gender-based differences in the assessment of general internal medicine (GIM) faculty physicians by trainees in inpatient and outpatient settings.

DESIGN

Retrospective cohort study SUBJECTS: Inpatient and outpatient GIM faculty physicians in an Internal Medicine residency training program from July 1, 2015, to December 31, 2018.

MAIN MEASURES

Faculty scores on trainee teaching evaluations including overall teaching ability and Accreditation Council for Graduate Medical Education (ACGME) competencies (medical knowledge [MK], patient care [PC], professionalism [PROF], interpersonal and communication skills [ICS], practice-based learning and improvement [PBLI], and systems-based practice [SBP]) based on the institutional faculty assessment form.

KEY RESULTS

In total, 3581 evaluations by 445 trainees (55.1% men, 44.9% women) assessing 161 GIM faculty physicians (50.3% men, 49.7% women) were included. Male faculty were rated higher in overall teaching ability (male=4.69 vs. female=4.63, p=0.003) and in four of the six ACGME competencies (MK, PROF, PBLI, and SBP) based on our institutional evaluation form. In the inpatient setting, male faculty were rated more favorably for overall teaching (male = 4.70, female = 4.53, p=<0.001) and across all ACGME competencies. The only observed gender difference in the outpatient setting favored female faculty in PC (male = 4.65, female = 4.71, p=0.01).

CONCLUSIONS

Male and female GIM faculty performance was assessed differently by trainees. Gender-based differences were impacted by the setting of evaluation, with the greatest difference by gender noted in the inpatient setting.

Creating work environments where people of all genders in gynecologic oncology can thrive: An SGO evidence-based review

Equality, equity, and parity in the workplace are necessary to optimize patient care across all aspects of medicine. Gender-based inequities remain an obstacle to quality of care, including within the now majority women subspecialty of gynecologic oncology. The results of the 2020 SGO State of the Society Survey prompted this evidence-based review. Evidence related to relevant aspects of the clinical care model by which women with malignancies are cared for is summarized. Recommendations are made that include ways to create work environments where all members of a gynecologic oncology clinical care team, regardless of gender, can thrive. These recommendations aim to improve equality and equity within the specialty and, in doing so, elevate the care that our patients receive.

The Effect of Salary Compensation for Time Spent Teaching in an Orthopaedic Residency Program: An Analysis of Teaching Performance Reviews

INTRODUCTION

Although there has been a recent emphasis on standardized resident assessments within Accrediation Council for Graduate Medical Education programs, assessments of faculty teaching performance and effectiveness are less frequent. Our purpose was to compare the teaching performance of orthopaedic surgery faculty receiving compensation for time spent teaching with faculty without compensation.

METHODS

For this prospective investigation, we collected anonymous resident reviews of 23 orthopaedic faculty within a rural, academic orthopaedic residency program over 2 academic years. Performance reviews of the faculty used a validated assessment of clinical teaching effectiveness with nine domains (faculty knowledge, organization, enthusiasm, rapport, involvement in learning experiences, feedback, clinical skill, accessibility, and overall effectiveness). A composite teaching effectiveness score was determined by adding each of the scores from the individual domains. We compared reviews for faculty members with and without compensation for time spent teaching.

RESULTS

A total of 202 performance reviews for 23 orthopaedic faculty were analyzed. Most of the faculty were male (91%), and 61% received compensation for teaching. No demographic differences were observed between the two faculty groups. Notable differences between the groups were noted in three domains: enthusiasm, ability to establish rapport as well as direction, and feedback. Faculty compensated for teaching demonstrated a markedly higher composite teaching effectiveness score than those without compensation.

DISCUSSION

These data suggest that orthopaedic faculty compensated for teaching responsibilities provide a better educational experience for resident trainees compared with faculty without compensation for teaching. Future studies should aim to assess varying compensation models for teaching responsibilities across different departments.

Clinical educator self-efficacy, self-evaluation and its relationship with student evaluations of clinical teaching

BACKGROUND

In a whole-of-system approach to evaluation of teaching across any degree, multiple sources of information can help develop an educators' understanding of their teaching quality. In the health professions, student evaluations of clinical teaching are commonplace. However, self-evaluation of teaching is less common, and exploration of clinical educators' self-efficacy even less so. The aim of the study was to evaluate how a clinical educator's self-evaluation of teaching intersects with their self-efficacy, to ascertain if that matches student evaluation of their teaching. This information may assist in facilitating targeted professional development to improve teaching quality.

METHODS

Clinical educators in the osteopathy program at Victoria University (VU) were invited to complete: a) self-evaluation version of the Osteopathy Clinical Teaching Questionnaire (OCTQ); and b) the Self-Efficacy in Clinical Teaching (SECT) questionnaire. Students in the VU program completed the OCTQ for each of the clinical educators they worked with during semester 2, 2017.

RESULTS

Completed OCTQ and SECT were received from 37 clinical educators. These were matched with 308 student evaluations (mean of 6 student ratings per educator). Three possible educator cohorts were identified: a) high clinical eductor self-OCTQ with low student evaluation; b) low clinical educator self-evaluation and high student evaluations; and, c) no difference between self- and student evaulations. Clinical educators in the first cohort demonstrated significantly higher SECT subscale scores (effect size > 0.42) than their colleagues. Age, gender, teaching qualification, and years practicing or years as a clinical educator were not associated with clinical educator OCTQ scores or the SECT subscales.

CONCLUSIONS

Targeted professional development directed towards fostering self-efficacy may provide an avenue for engaging those clinical educators whose self-efficacy is low and/or those who did not receive high student evaluations. Given there is no gold standard measure of clinical teaching quality, educators should engage with multiple sources of feedback to benchmark their current performance level, and identify opportunities to improve. Student and self-evaluations using the OCTQ and evaluation of self-efficacy using the SECT, are useful tools for inclusion in a whole-of-system approach to evaluation of the clinical learning environment.

Team up! Linking teamwork effectiveness of clinical teaching teams to residents' experienced learning climate

Background: Supportive learning climates are key to ensure high-quality residency training. Clinical teachers, collaborating as teaching team, have an important role in maintaining such climates since they are responsible for residency training. Successful residency training is dependent on effective teamwork within teaching teams. Still, it remains unclear whether this team effort benefits residents' perceptions of the learning climate. We, therefore, investigated to what extent teamwork effectiveness within teaching teams is associated with (1) the overall learning climate, and (2) its affective, cognitive and instrumental facets?Methods: This study used a web-based platform to collect data in clinical departments in the Netherlands from January 2014 to May 2017. Teamwork effectiveness was measured with the TeamQ questionnaire, administered amongst clinical teachers. The learning climate was measured with the D-RECT, applied amongst residents. Associations were analyzed using multilevel models and multivariate general linear models.Results: Teamwork effectiveness was positively associated with the overall learning climate as well as with the affective and the instrumental facets of the learning climate. No significant associations were found with the cognitive facet.Conclusion: Effective teamwork within teaching teams benefits learning climates in postgraduate medical education. Therefore, departments aiming to improve their learning climate should target teamwork within teaching teams.

Educational roles as a continuum of mentoring's role in medicine - a systematic review and thematic analysis of educational studies from 2000 to 2018

BACKGROUND

Recent studies have gone to great lengths to differentiate mentoring from teaching, tutoring, role modelling, coaching and supervision in efforts to better understand mentoring processes. This review seeks to evaluate the notion that teaching, tutoring, role modelling, coaching and supervision may in fact all be part of the mentoring process. To evaluate this theory, this review scrutinizes current literature on teaching, tutoring, role modelling, coaching and supervision to evaluate their commonalities with prevailing concepts of novice mentoring.

METHODS

A three staged approach is adopted to evaluate this premise. Stage one involves four systematic reviews on one-to-one learning interactions in teaching, tutoring, role modelling, coaching and supervision within Internal Medicine, published between 1st January 2000 and 31st December 2018. Braun and Clarke's (2006) approach to thematic analysis was used to identify key elements within these approaches and facilitate comparisons between them. Stage two provides an updated view of one-to-one mentoring between a senior physician and a medical student or junior doctor to contextualise the discussion. Stage three infuses mentoring into the findings delineated in stage one.

RESULTS

Seventeen thousand four hundred ninety-nine citations were reviewed, 235 full-text articles were reviewed, and 104 articles were thematically analysed. Four themes were identified - characteristics, processes, nature of relationship, and problems faced in each of the four educational roles.

CONCLUSIONS

Role modelling, teaching and tutoring, coaching and supervision lie within a mentoring spectrum of increasingly structured interactions, assisted by assessments, feedback and personalised support that culminate with a mentoring approach. Still requiring validation, these findings necessitate a reconceptualization of mentoring and changes to mentor training programs and how mentoring is assessed and supported.

Exploring Validity Evidence Associated With Questionnaire-Based Tools for Assessing the Professional Performance of Physicians: A Systematic Review

PURPOSE

To collect and examine-using an argument-based validity approach-validity evidence of questionnaire-based tools used to assess physicians' clinical, teaching, and research performance.

METHOD

In October 2016, the authors conducted a systematic search of the literature seeking articles about questionnaire-based tools for assessing physicians' professional performance published from inception to October 2016. They included studies reporting on the validity evidence of tools used to assess physicians' clinical, teaching, and research performance. Using Kane's validity framework, they conducted data extraction based on four inferences in the validity argument: scoring, generalization, extrapolation, and implications.

RESULTS

They included 46 articles on 15 tools assessing clinical performance and 72 articles on 38 tools assessing teaching performance. They found no studies on research performance tools. Only 12 of the tools (23%) gathered evidence on all four components of Kane's validity argument. Validity evidence focused mostly on generalization and extrapolation inferences. Scoring evidence showed mixed results. Evidence on implications was generally missing.

CONCLUSIONS

Based on the argument-based approach to validity, not all questionnaire-based tools seem to support their intended use. Evidence concerning implications of questionnaire-based tools is mostly lacking, thus weakening the argument to use these tools for formative and, especially, for summative assessments of physicians' clinical and teaching performance. More research on implications is needed to strengthen the argument and to provide support for decisions based on these tools, particularly for high-stakes, summative decisions. To meaningfully assess academic physicians in their tripartite role as doctor, teacher, and researcher, additional assessment tools are needed.

Role modelling in the training of hospital-based medical specialists: a validation study of the Role Model Apperception Tool (RoMAT)

INTRODUCTION

Role modelling is a key component in the training of doctors that influences professional behaviour, identity and career choices. Clinical teachers and residents are often unaware of this, thereby risking transmission of negative behaviour. On the other hand, awareness positively affects role model behaviour. To assess role model behaviour, the Role Model Apperception Tool (RoMAT) was developed and validated in general practice training. The aim of the current study was to validate the RoMAT in the hospital-based training setting.

METHODS

The authors asked first to last year residents, regardless of their specialty, to participate after written approval from their clinical teachers. The tool was completed online in 2017. The authors performed a principal component analysis and investigated internal consistency, construct validity, inter-rater reliability, known-groups comparisons and floor and ceiling effects.

RESULTS

Of the 473 residents contacted, 187 (40%) completed the questionnaire. As in the primary validation study, the authors extracted two components: 'Caring Attitude' and 'Effectiveness', explaining 67% of the variation with a Cronbach's alpha of 0.94 and 0.93 respectively. Evidence for construct validity was found and there were no floor or ceiling effects, but inter-rater reliability was low.

DISCUSSION

The RoMAT was internally consistent and valid to assess role model behaviour of the clinical teacher towards the resident in the hospital-based training of medical specialists. The poor inter-rater reliability, most likely due to homogeneous RoMAT responses, should be borne in mind when evaluating RoMAT scores on individual clinical teachers.

The Association Between Pediatric Faculty Factors and Resident Physician Ratings of Teaching Effectiveness

Background. Faculty factors not inherently related to teaching effectiveness can influence teaching ratings. No studies have focused on pediatric faculty who possess unique differences from general medical faculty. Methods. We designed a retrospective observational study to compare faculty teaching ratings with measured factors across 3 academic pediatric institutions. Results. Our study included 196 faculty members. The majority (76%) of variation in teaching effectiveness ratings was not accounted for by any measured variable, but 24% was attributed to measurable factors. Increased resident exposure (sequential r 2 = .10, P < .0001) significantly affected teaching effectiveness. Variation between resident ratings of pediatric faculty teaching can be partially explained by measured factors not necessarily related to teaching effectiveness. Conclusions. The identification of faculty factors that significantly contribute to rating variation can enhance interpretation of these rating.

Changes in Perceived Supervision Quality After Introduction of Competency-Based Orthopedic Residency Training: A National 6-Year Follow-Up Study

OBJECTIVE

To evaluate the perceived quality of the learning environment, before and after introduction of competency-based postgraduate orthopedic education.

DESIGN

From 2009 to 2014, we conducted annual surveys among Dutch orthopedic residents. The validated Dutch Residency Educational Climate Test (D-RECT, 50 items on 11 subscales) was used to assess the quality of the learning environment. Scores range from 1 (poor) to 5 (excellent).

SETTING

Dynamic cohort follow-up study.

PARTICIPANTS

All Dutch orthopedic residents were surveyed during annual compulsory courses.

RESULTS

Over the 6-year period, 641 responses were obtained (response rate 92%). Scores for "supervision" (95% CI for difference 0.06-0.28, p = 0.002) and "coaching and assessment" (95% CI 0.11-0.35, p < 0.001) improved significantly after introduction of competency-based training. There was no significant change in score on the other subscales of the D-RECT.

CONCLUSIONS

After the introduction of some of the core components of competency-based postgraduate orthopedic education the perceived quality of "supervision" and "coaching and assessment" improved significantly.

Evaluation of clinical teaching quality in competency-based residency training in Lithuania

BACKGROUND AND AIM

In 2013, all residency programs at the Lithuanian University of Health Sciences were renewed into the competency-based medical education curriculum (CBME). In 2015, we implemented the validated EFFECT questionnaire together with the EFFECT-System for quality assessment of clinical teaching in residency training. The aim of this study was to investigate the influence of characteristics of the resident (year of training) and clinical teacher (gender, age, and type of academic position) on teaching quality, as well as to assess areas for teaching quality improvement.

MATERIALS AND METHODS

Residents from 7 different residency study programs filled out 333 EFFECT questionnaires evaluating 146 clinical teachers. We received 143 self-evaluations of clinical teachers using the same questionnaire. Items were scored on a 6-point Likert scale. Main outcome measures were residents' mean overall (MOS), mean subdomain (MSS) and clinical teachers' self-evaluation scores. The overall comparisons of MOS and MSS across study groups and subgroups were done using Student's t test and ANOVA for trend. The intraclass correlation coefficient (ICC) was calculated in order to see how residents' evaluations match with self-evaluations for every particular teacher. To indicate areas for quality improvement items were analyzed subtracting their mean score from the respective (sub)domain score.

RESULTS

MOS for domains of "role modeling", "task allocation", "feedback", "teaching methodology" and "assessment" valued by residents were significantly higher than those valued by teachers (P<0.01). Teachers who filled out self-evaluation questionnaires were rated significantly higher by residents in role modeling subdomains (P<0.05). Male teachers in (sub)domains "role modeling: CanMEDS roles and reflection", "task allocation", "planning" and "personal support" were rated significantly higher than the female teachers (P<0.05). Teachers aged 40 years or younger were rated higher (P<0.01). Residents ratings by type of teachers' academic position almost in all (sub)domains differed significantly (P<0.05). No correlation observed between MOS of a particular teacher and MOS as rated by residents (ICC=0.055, P=0.399). The main areas for improvement were "feedback" and "assessment".

CONCLUSIONS

Resident evaluations of clinical teachers are influenced by teachers' age, gender, year of residency training, type of teachers' academic position and whether or not a clinical teacher performed self-evaluation. Development of CBME should be focused on the continuous evaluation of quality, clinical teachers educational support and the implementation of e-portfolio.

Redesign of the System for Evaluation of Teaching Qualities in Anesthesiology Residency Training (SETQ Smart)

BACKGROUND

Given the increasing international recognition of clinical teaching as a competency and regulation of residency training, evaluation of anesthesiology faculty teaching is needed. The System for Evaluating Teaching Qualities (SETQ) Smart questionnaires were developed for assessing teaching performance of faculty in residency training programs in different countries. This study investigated (1) the structure, (2) the psychometric qualities of the new tools, and (3) the number of residents' evaluations needed per anesthesiology faculty to use the instruments reliably.

METHODS

Two SETQ Smart questionnaires-for faculty self-evaluation and for resident evaluation of faculty-were developed. A multicenter survey was conducted among 399 anesthesiology faculty and 430 residents in six countries. Statistical analyses included exploratory factor analysis, reliability analysis using Cronbach α, item-total scale correlations, interscale correlations, comparison of composite scales to global ratings, and generalizability analysis to assess residents' evaluations needed per faculty.

RESULTS

In total, 240 residents completed 1,622 evaluations of 247 faculty. The SETQ Smart questionnaires revealed six teaching qualities consisting of 25 items. Cronbach α's were very high (greater than 0.95) for the overall SETQ Smart questionnaires and high (greater than 0.80) for the separate teaching qualities. Interscale correlations were all within the acceptable range of moderate correlation. Overall, questionnaire and scale scores correlated moderately to highly with the global ratings. For reliable feedback to individual faculty, three to five resident evaluations are needed.

CONCLUSIONS

The first internationally piloted questionnaires for evaluating individual anesthesiology faculty teaching performance can be reliably, validly, and feasibly used for formative purposes in residency training.

Predictive Relationships Between Students' Evaluation Ratings and Course Satisfaction

Objective. To assess the reliability and validity of course evaluation data. Methods. A correlation study was conducted using archival data from pharmacy student course evaluations. Bivariate relationships between eight course-rating items and overall rating item and the extent to which course type, level, and grade point average moderated these relationships were analyzed. Results. Significant bivariate relationships were found between the eight course evaluation rating variables and the overall course rating variable. Pharmacy practice course type significantly moderated the relationship between all predictor and criterion variables. Conclusion. Pharmacy school administrators should consider individual course evaluation item ratings when making decisions regarding course offerings or faculty promotion and tenure.

Repaying in Kind: Examination of the Reciprocity Effect in Faculty and Resident Evaluations

BACKGROUND

Although the reciprocity hypothesis (that trainees have a tendency to modify evaluations based on the grades they receive from instructors) has been documented in other fields, very little work has examined this phenomenon in the surgical residency environment. The purpose of this study was to investigate the extent to which lenient-grading faculty receive higher evaluations from surgery residents.

METHODS

Evaluation data from 2 consecutive academic years were collected retrospectively at a large university-based General Surgery residency program. Monthly faculty evaluations of residents (15 items) and resident evaluations of faculty (8 items; 1 = never demonstrates, 10 = always demonstrates) were included. Correlation and regression analyses were conducted with SPSS version 22 (IBM; Chicago, IL).

RESULTS

A total of 2274 faculty assessments and 1480 resident assessments were included in this study, representing 2 years of evaluations for 32 core faculty members responsible for completing all resident evaluations and 68 PGY1-5 general surgery residents. Faculty (63% men, 13.5 ± 9.8 years out of training) represented 5 different divisions (general surgery, surgical oncology, transplant, trauma critical care, and vascular) within the general surgery department. Faculty received an average of 71.1 ± 33.9 evaluations from residents over the course of 2 years. The average rating of faculty teaching by residents was 9.5 ± 0.4. Residents received an average of 21.8 ± 0.5 evaluations with average ratings of 4.2 ± 0.4. Correlation analyses indicated a positive relationship between the average rating received from residents and the number of years since faculty completed training (r = 0.44, p = 0.01). Additionally, a significant relationship emerged between ratings received from residents and ratings given to residents (r = 0.40, p = 0.04). Regression analyses indicated that when both variables (years since training, ratings given to residents) were included in the model, only ratings given to residents remained a significant predictor of evaluation ratings received from residents (F_(1,32) = 4.40, p = 0.04), with an R² of 0.16. Sex or division affiliation did not account for any unique variance.

CONCLUSIONS

These findings suggest that a reciprocity effect exists between surgery faculty and resident evaluations. This effect warrants further exploration, such that efforts to mitigate the risks of providing inaccurate assessments may be developed. Providing trainees with accurate assessments is particularly important given the high-stakes use of these data for milestones, promotion, and graduation purposes, which currently do not account for this reciprocity effect.

SUMMARY

Results suggest that there is a reciprocity effect in the faculty and resident evaluation process.

Student Evaluation of Faculty Physicians: Gender Differences in Teaching Evaluations

PURPOSE

To investigate whether there is a difference in medical student teaching evaluations for male and female clinical physician faculty.

METHODS

The authors examined all teaching evaluations completed by clinical students at one North American medical school in the surgery, obstetrics and gynecology, pediatrics, and internal medicine clinical rotations from 2008 to 2012. The authors focused on how students rated physician faculty on their "overall quality of teaching" using a 5-point response scale (1 = Poor to 5 = Excellent). Linear mixed-effects models provided estimated mean differences in evaluation outcomes by faculty gender.

RESULTS

There were 14,107 teaching evaluations of 965 physician faculty. Of these evaluations, 7688 (54%) were for male physician faculty and 6419 (46%) were for female physician faculty. Female physicians received significantly lower mean evaluation scores in all four rotations. The discrepancy was largest in the surgery rotation (males = 4.23, females = 4.01, p = 0.003). Pediatrics showed the next greatest difference (males = 4.44, females = 4.29, p = 0.009), followed by obstetrics and gynecology (males = 4.38, females = 4.26, p = 0.026), and internal medicine (males = 4.35, females = 4.27, p = 0.043).

CONCLUSIONS

Female physicians received lower teaching evaluations in all four core clinical rotations. This comprehensive examination adds to the medical literature by illuminating subtle differences in evaluations based on physician gender, and provides further evidence of disparities for women in academic medicine.

Evaluations of Neurologists by Their Patients and Residents Are Inversely Correlated

OBJECTIVE AND BACKGROUND

We hypothesized that evaluation scores for attending neurologists by patients and residents would parallel one another. Additionally, we hypothesized that provider productivity would be also be associated with performance evaluations by patients and residents.

METHODS

In a university neurology department, we collected individual Clinician and Group Consumer Assessment of Healthcare Providers and Systems patient satisfaction scores and standardized resident evaluation scores (n = 22 faculty members). We performed bivariate analysis of doctor-patient satisfaction versus resident evaluation scores.

RESULTS

Attending neurologists with higher patient satisfaction received lower resident evaluation scores (P < .05). There seem to be disproportionate neurologists with low evaluations not meeting clinical productivity targets.

CONCLUSION

Finding a significant inverse correlation was surprising. Perhaps what is valued by patients in their physician is not what residents value in teachers. That deserves further study. Maybe attending physicians who spend their energy on the patient experience do not have sufficient time to devote to teaching and vice versa. That neurologists with low evaluation scores appear more likely to not meet productivity targets supports this idea.

ORL residency in France: Satisfaction and training quality in 2013

OBJECTIVES

To assess the quality of ORL training in France and hierarchical relationships, working conditions and quality of life, based on a questionnaire answered by ORL residents and fellows.

MATERIAL AND METHODS

The anonymous questionnaire covered the following topics (106 questions): structure of training, supervision by superiors, involvement in teaching medical students, work for publication, working conditions, quality of life and practical and surgical skills.

RESULTS

One hundred and seventeen questionnaires were included, with 56% female respondents and 78% resident respondents. A total of 80% of respondents would choose the same training again, with overall satisfaction of 70%. In total, 71% felt appreciated, with responsibilities matching their level of training (85%). Supervisors were supportive (85%) and showed good teaching ability (92%). Residents were mainly guided by fellows (70%), or consultants (60%). Providing good training was an important aim for the training structures (78%). Respondents worked 11 hours per day, with a median of 4 duties per month; the post-duty rest-day was fully respected in the case of 38% of respondents. Surgical procedures were taught and delegated step by step, shared fairly between residents, in a large majority of cases. Training progression was in line with seniority.

CONCLUSIONS

Overall satisfaction, quality and supervision of ORL training were good in France. There remain geographical differences in terms of theoretical and surgical training.

Understanding resident ratings of teaching in the workplace: a multi-centre study

Providing clinical teachers with feedback about their teaching skills is a powerful tool to improve teaching. Evaluations are mostly based on questionnaires completed by residents. We investigated to what extent characteristics of residents, clinical teachers, and the clinical environment influenced these evaluations, and the relation between residents' scores and their teachers' self-scores. The evaluation and feedback for effective clinical teaching questionnaire (EFFECT) was used to (self)assess clinical teachers from 12 disciplines (15 departments, four hospitals). Items were scored on a five-point Likert scale. Main outcome measures were residents' mean overall scores (MOSs), specific scale scores (MSSs), and clinical teachers' self-evaluation scores. Multilevel regression analysis was used to identify predictors. Residents' scores and self-evaluations were compared. Residents filled in 1,013 questionnaires, evaluating 230 clinical teachers. We received 160 self-evaluations. 'Planning Teaching' and 'Personal Support' (4.52, SD .61 and 4.53, SD .59) were rated highest, 'Feedback Content' (CanMEDS related) (4.12, SD .71) was rated lowest. Teachers in affiliated hospitals showed highest MOS and MSS. Medical specialty did not influence MOS. Female clinical teachers were rated higher for most MSS, achieving statistical significance. Residents in year 1-2 were most positive about their teachers. Residents' gender did not affect the mean scores, except for role modeling. At group level, self-evaluations and residents' ratings correlated highly (Kendall's τ 0.859). Resident evaluations of clinical teachers are influenced by teacher's gender, year of residency training, type of hospital, and to a lesser extent teachers' gender. Clinical teachers and residents agree on strong and weak points of clinical teaching.

Residents' engagement and empathy associated with their perception of faculty's teaching performance

BACKGROUND

Faculty members rely on residents' feedback about their teaching performance. The influence of residents' characteristics on evaluations of faculty is relatively unexplored. We aimed to evaluate the levels of work engagement and empathy among residents and the association of both characteristics with their evaluation of the faculty's teaching performance.

METHODS

A multicenter questionnaire study among 271 surgery and gynecology residents was performed from September 2012 to February 2013. Residents' ratings of the faculty's teaching performance were collected using the system for evaluation of teaching quality (SETQ). Residents were also invited to fill out standardized measures of work engagement and empathy using the short Utrecht Work Engagement Scale and the Jefferson Scale of Physician Empathy, respectively. Linear regression analysis using generalized estimating equations to evaluate the association of residents' engagement and empathy with residents' evaluations of teaching performance.

RESULTS

Overall, 204 (75.3 %) residents completed 1814 SETQ evaluations of 302 faculty, and 143 (52.8 %) and 140 (51.7 %) residents, respectively, completed the engagement and empathy measurements. The median scores of residents' engagement and empathy were 4.56 (scale 0-6) and 5.55 (scale 1-7), respectively. Higher levels of residents' engagement (regression coefficient b = 0.128; 95 % confidence interval (CI) 0.072-0.184; p < 0.001) and empathy (b = 0.113; 95 % CI 0.063-0.164; p < 0.001) were associated with higher faculty teaching performance scores.

CONCLUSIONS

Residents' engagement and empathy appear to be positively associated with their evaluation of the faculty's performance. A possible explanation is that residents who are more engaged and can understand and share others' perspectives stimulate and experience faculty's teaching better than others.

In the eyes of residents good supervisors need to be more than engaged physicians: the relevance of teacher work engagement in residency training

During their development into competent medical specialists, residents benefit from their attending physicians' excellence in teaching and role modelling. Work engagement increases overall job performance, but it is unknown whether this also applies to attending physicians' teaching performance and role modelling. Attending physicians in clinical teaching practice take on roles as doctors and teachers. Therefore, this study (a) examined levels of attending physicians' work engagement in both roles, and (b) quantified the relationships of both work engagement roles to their teaching performance and role model status. In this multicenter survey, residents evaluated attending physicians' teaching performance and role model status using the validated System for Evaluation of Teaching Qualities. Attending physicians self-reported their work engagement on a 7-point scale, separately for their roles as doctors and teachers, using the validated 9-item Utrecht Work Engagement Scale. In total, 549 (68 %) residents filled out 4,305 attending physician evaluations and 627 (78 %) attending physicians participated. Attending physicians reported higher work engagement in their doctor than in their teacher roles (mean difference: 0.95; 95 % CI 0.86-1.04; p < 0.001). Teacher work engagement was positively related to teaching performance (regression coefficient, B: 0.11; 95 % CI 0.08-0.14; p < 0.001), which in turn was positively associated to role model status (B: 1.08; 95 % CI 0.10-1.18; p < 0.001). In the eyes of residents, good supervisors need to be more than engaged physicians, as attending physicians with high teacher work engagement were evaluated as better teachers.

Hospitalist workload and resident evaluations

BACKGROUND

Most academic hospitalists fulfil the role of clinician educator and have many opportunities for the bedside clinical teaching of resident physicians; however, hospitalists are promoted at lower rates than traditional internal medicine faculty staff. The conflict between the demands of clinical productivity and time to teach may be central to understanding the lower rates of academic promotion seen in hospitalists. This investigation explores the relationship between clinical productivity and learner evaluations of hospitalist clinician educators.

METHODS

A retrospective review of clinical productivity and learner evaluations of hospitalists by residents was collected during the 2009-2012 academic years at Southern Illinois University School of Medicine. Correlation analysis between annual work relative value units (wRVUs), patient encounters and duty days with resident evaluations of faculty staff in the Accreditation Council for Graduate Medical Education core competencies was performed.

RESULTS

Forty-one annual data sets, representing 18 individual hospitalists, were analysed. No significant correlations between clinical productivity, in terms of annual work RVUs, patient encounters and duty days, and resident learner evaluation scores was found. This investigation explores the relationship between clinical productivity and learner evaluations of hospitalist clinician educators

DISCUSSION

This study found no significant influence of measures of annual clinical service workload on resident learner evaluations of hospitalist clinical educators. These results are consistent with data reported for emergency medicine doctors and anaesthesiologists. These results may have significant implications for the staffing requirements for academic hospitalists.

An instrument for evaluating clinical teaching in Japan: content validity and cultural sensitivity

BACKGROUND

Many instruments for evaluating clinical teaching have been developed but almost all in Western countries. None of these instruments have been validated for the Asian culture, and a literature search yielded no instruments that were developed specifically for that culture. A key element that influences content validity in developing instruments for evaluating the quality of teaching is culture. The aim of this study was to develop a culture-specific instrument with strong content validity for evaluating clinical teaching in initial medical postgraduate training in Japan.

METHODS

Based on data from a literature search and an earlier study we prepared a draft evaluation instrument. To ensure a good cultural fit of the instrument with the Asian context we conducted a modified Delphi procedure among three groups of stakeholders (five education experts, twelve clinical teachers and ten residents) to establish content validity, as this factor is particularly susceptible to cultural factors.

RESULTS

Two rounds of Delphi were conducted. Through the procedure, 52 prospective items were reworded, combined or eliminated, resulting in a 25-item instrument validated for the Japanese setting.

CONCLUSIONS

This is the first study describing the development and content validation of an instrument for evaluating clinical teaching specifically tailored to an East Asian setting. The instrument has similarities and differences compared with instruments of Western origin. Our findings suggest that designers of evaluation instruments should consider the probability that the content validity of instruments for evaluating clinical teachers can be influenced by cultural aspects.

Confounding factors in using upward feedback to assess the quality of medical training: a systematic review

PURPOSE

Upward feedback is becoming more widely used in medical training as a means of quality control. Multiple biases exist, thus the accuracy of upward feedback is debatable. This study aims to identify factors that could influence upward feedback, especially in medical training.

METHODS

A systematic review using a structured search strategy was performed. Thirty-five databases were searched. Results were reviewed and relevant abstracts were shortlisted. All studies in English, both medical and non-medical literature, were included. A simple pro-forma was used initially to identify the pertinent areas of upward feedback, so that a focused pro-forma could be designed for data extraction.

RESULTS

A total of 204 articles were reviewed. Most studies on upward feedback bias were evaluative studies and only covered Kirkpatrick level 1-reaction. Most studies evaluated trainers or training, were used for formative purposes and presented quantitative data. Accountability and confidentiality were the most common overt biases, whereas method of feedback was the most commonly implied bias within articles.

CONCLUSION

Although different types of bias do exist, upward feedback does have a role in evaluating medical training. Accountability and confidentiality were the most common biases. Further research is required to evaluate which types of bias are associated with specific survey characteristics and which are potentially modifiable.

Personality traits affect teaching performance of attending physicians: results of a multi-center observational study

Background

Worldwide, attending physicians train residents to become competent providers of patient care. To assess adequate training, attending physicians are increasingly evaluated on their teaching performance. Research suggests that personality traits affect teaching performance, consistent with studied effects of personality traits on job performance and academic performance in medicine. However, up till date, research in clinical teaching practice did not use quantitative methods and did not account for specialty differences. We empirically studied the relationship of attending physicians' personality traits with their teaching performance across surgical and non-surgical specialties.

Method

We conducted a survey across surgical and non-surgical specialties in eighteen medical centers in the Netherlands. Residents evaluated attending physicians' overall teaching performance, as well as the specific domains learning climate, professional attitude, communication, evaluation, and feedback, using the validated 21-item System for Evaluation of Teaching Qualities (SETQ). Attending physicians self-evaluated their personality traits on a 5-point scale using the validated 10-item Big Five Inventory (BFI), yielding the Five Factor model: extraversion, conscientiousness, neuroticism, agreeableness and openness.

Results

Overall, 622 (77%) attending physicians and 549 (68%) residents participated. Extraversion positively related to overall teaching performance (regression coefficient, B: 0.05, 95% CI: 0.01 to 0.10, P = 0.02). Openness was negatively associated with scores on feedback for surgical specialties only (B: −0.10, 95% CI: −0.15 to −0.05, P<0.001) and conscientiousness was positively related to evaluation of residents for non-surgical specialties only (B: 0.13, 95% CI: 0.03 to 0.22, p = 0.01).

Conclusions

Extraverted attending physicians were consistently evaluated as better supervisors. Surgical attending physicians who display high levels of openness were evaluated as less adequate feedback-givers. Non-surgical attending physicians who were conscientious seem to be good at evaluating residents. These insights could contribute to future work on development paths of attending physicians in medical education.

Training satisfaction and work environment in Otorhinolaryngology, Head and Neck surgery: a comparison between France and Germany

With the coalescing of Europe, increased mobility of professionals emerges. Initiatives to harmonize medical education were launched. In Otolaryngology, Head and Neck surgery (ORL) an European board examination was created to ensure standards. Quality of training, satisfaction and quality of life of residents and recent ORL specialists were compared to assess different aspects of work and hierarchical relationships in France (FRA) and Germany (GER) by means of an anonymous questionnaire. 120 FRA and 125 GER questionnaires were included. 78 % of respondents were residents. 86 % would choose the same training again. In both countries, a majority felt well considered with responsibilities adapted to their level of training and with supportive supervisors. Germans reported average daily work hours of 9.6 versus 11 in FRA with compensated overtime (76 %) and a possibility of part-time work (62 %), both nearly inexistent in FRA. In GER, the day-off after duty was more often respected. French attributed their seniors better pedagogic skills, taking time for explanations and providing better teaching. Offering a good training was a more important objective in French training centers (77 vs. 51 %). In both countries, surgical training relied on coaching. Research activities were comparable. The overall satisfaction with ORL training was high. Differences concerned structure of training, guidance by senior doctors and the working conditions. The study results provide guidance before choosing a program and may help to improve current training by identifying positive aspects that, if combined could lead to a convergence of programs. However, present high standards of education must be maintained.

Effect of the learning climate of residency programs on faculty's teaching performance as evaluated by residents

Background

To understand teaching performance of individual faculty, the climate in which residents’ learning takes place, the learning climate, may be important. There is emerging evidence that specific climates do predict specific outcomes. Until now, the effect of learning climate on the performance of the individual faculty who actually do the teaching was unknown.

Objectives

This study: (i) tested the hypothesis that a positive learning climate was associated with better teaching performance of individual faculty as evaluated by residents, and (ii) explored which dimensions of learning climate were associated with faculty’s teaching performance.

Methods and Materials

We conducted two cross-sectional questionnaire surveys amongst residents from 45 residency training programs and multiple specialties in 17 hospitals in the Netherlands. Residents evaluated the teaching performance of individual faculty using the robust System for Evaluating Teaching Qualities (SETQ) and evaluated the learning climate of residency programs using the Dutch Residency Educational Climate Test (D-RECT). The validated D-RECT questionnaire consisted of 11 subscales of learning climate. Main outcome measure was faculty’s overall teaching (SETQ) score. We used multivariable adjusted linear mixed models to estimate the separate associations of overall learning climate and each of its subscales with faculty’s teaching performance.

Results

In total 451 residents completed 3569 SETQ evaluations of 502 faculty. Residents also evaluated the learning climate of 45 residency programs in 17 hospitals in the Netherlands. Overall learning climate was positively associated with faculty’s teaching performance (regression coefficient 0.54, 95% confidence interval: 0.37 to 0.71; P<0.001). Three out of 11 learning climate subscales were substantially associated with better teaching performance: ‘coaching and assessment’, ‘work is adapted to residents’ competence’, and ‘formal education’.

Conclusions

Individual faculty’s teaching performance evaluations are positively affected by better learning climate of residency programs.

Frequency and determinants of residents' narrative feedback on the teaching performance of faculty: narratives in numbers

PURPOSE

Physicians involved in residency training often receive feedback from residents on their teaching. Research shows that learners value narrative feedback, but knowledge of the frequency and determinants of narrative feedback in teaching performance evaluation is lacking. This study aims to identifythe frequency with which residentsgave positive comments and suggestions for improvement to faculty, and the factors influencing that frequency.

METHOD

From September 2008 through May 2010, the authors collected data, using a validated formative feedback system (System for Evaluation of Teaching Qualities). The authors used univariate and multivariable analysis to investigate the associations between participants' characteristics, including faculty members' teaching performance, and the frequency of the two types of narrative comments.

RESULTS

In total, 659 residents (79% of 839) completed 6,216 evaluations on 917 faculty (95% of 964), resulting in 11,574 positive comments and 4,870 suggestions for improvement. On average, faculty members received 13 positive comments and 5 suggestions for improvement. Multivariable analysis showed that higher teaching performance was associated with higher numbers of positive comments (regression coefficient 0.538; 95% confidence interval: 0.464 to 0.613) and with lower numbers of suggestions for improvement (-0.802; -0.911 to -0.692), both P < .0001. Nonacademic hospitals, participation in teacher training, and female residents' evaluation were statistically significant determinants of receiving more narrative feedback.

CONCLUSIONS

Residents provided narrative feedback that paralleled and elaborated on quantitative evaluations they provided; therefore, faculty would be wise to attend to narrative feedback. Analysis of the quality of narrative feedback is needed to understand its effectiveness.

Residents provide feedback to their clinical teachers: reflection through dialogue

BACKGROUND

Physicians play a crucial role in teaching residents in clinical practice. Feedback on their teaching performance to support this role needs to be provided in a carefully designed and constructive way.

AIMS

We investigated an evaluation system for evaluating supervisors and providing formative feedback.

METHOD

In a design based research approach, the 'Evaluation and Feedback For Effective Clinical Teaching System' (EFFECT-S) was examined by conducting semi-structured interviews with residents and supervisors of five departments in five different hospitals about feedback conditions, acceptance and its effects. Interviews were analysed by three researchers, using qualitative research software (ATLAS-Ti).

RESULTS

Principles and characteristics of the design are supported by evaluating EFFECT-S. All steps of EFFECT-S appear necessary. A new step, team evaluation, was added. Supervisors perceived the feedback as instructive; residents felt capable of providing feedback. Creating safety and honesty require different actions for residents and supervisors. Outcomes include awareness of clinical teaching, residents learning feedback skills, reduced hierarchy and an improved learning climate.

CONCLUSIONS

EFFECT-S appeared useful for evaluating supervisors. Key mechanism was creating a safe environment for residents to provide honest and constructive feedback. Residents learned providing feedback, being part of the CanMEDS and ACGME competencies of medical education programmes.

The teacher, the physician and the person: exploring causal connections between teaching performance and role model types using directed acyclic graphs

Background

In fledgling areas of research, evidence supporting causal assumptions is often scarce due to the small number of empirical studies conducted. In many studies it remains unclear what impact explicit and implicit causal assumptions have on the research findings; only the primary assumptions of the researchers are often presented. This is particularly true for research on the effect of faculty’s teaching performance on their role modeling. Therefore, there is a need for robust frameworks and methods for transparent formal presentation of the underlying causal assumptions used in assessing the causal effects of teaching performance on role modeling. This study explores the effects of different (plausible) causal assumptions on research outcomes.

Methods

This study revisits a previously published study about the influence of faculty’s teaching performance on their role modeling (as teacher-supervisor, physician and person). We drew eight directed acyclic graphs (DAGs) to visually represent different plausible causal relationships between the variables under study. These DAGs were subsequently translated into corresponding statistical models, and regression analyses were performed to estimate the associations between teaching performance and role modeling.

Results

The different causal models were compatible with major differences in the magnitude of the relationship between faculty’s teaching performance and their role modeling. Odds ratios for the associations between teaching performance and the three role model types ranged from 31.1 to 73.6 for the teacher-supervisor role, from 3.7 to 15.5 for the physician role, and from 2.8 to 13.8 for the person role.

Conclusions

Different sets of assumptions about causal relationships in role modeling research can be visually depicted using DAGs, which are then used to guide both statistical analysis and interpretation of results. Since study conclusions can be sensitive to different causal assumptions, results should be interpreted in the light of causal assumptions made in each study.

Qualitative study about the ways teachers react to feedback from resident evaluations

BACKGROUND

Currently, one of the main interventions that are widely expected to contribute to teachers' professional development is confronting teachers with feedback from resident evaluations of their teaching performance. Receiving feedback, however, is a double edged sword. Teachers see themselves confronted with information about themselves and are, at the same time, expected to be role models in the way they respond to feedback. Knowledge about the teachers' responses could be not only of benefit for their professional development, but also for supporting their role modeling. Therefore, research about professional development should include the way teachers respond to feedback.

METHOD

We designed a qualitative study with semi-structured individual conversations about feedback reports, gained from resident evaluations. Two researchers carried out a systematic analysis using qualitative research software. The analysis focused on what happened in the conversations and structured the data in three main themes: conversation process, acceptance and coping strategies.

RESULTS

The result section describes the conversation patterns and atmosphere. Teachers accepted their results calmly, stating that, although they recognised some points of interest, they could not meet with every standard. Most used coping strategies were explaining the results from their personal beliefs about good teaching and attributing poor results to external factors and good results to themselves. However, some teachers admitted that they had poor results because of the fact that they were not "sharp enough" in their resident group, implying that they did not do their best.

CONCLUSIONS

Our study not only confirms that the effects of feedback depend first and foremost on the recipient but also enlightens the meaning and role of acceptance and being a role model. We think that the results justify the conclusion that teachers who are responsible for the day release programmes in the three departments tend to respond to the evaluation results just like human beings do and, at the time of the conversation, are initially not aware of the fact that they are role models in the way they respond to feedback.

The teacher, the physician and the person: how faculty's teaching performance influences their role modelling

Objective

Previous studies identified different typologies of role models (as teacher/supervisor, physician and person) and explored which of faculty's characteristics could distinguish good role models. The aim of this study was to explore how and to which extent clinical faculty's teaching performance influences residents' evaluations of faculty's different role modelling statuses, especially across different specialties.

Methods

In a prospective multicenter multispecialty study of faculty's teaching performance, we used web-based questionnaires to gather empirical data from residents. The main outcome measures were the different typologies of role modelling. The predictors were faculty's overall teaching performance and faculty's teaching performance on specific domains of teaching. The data were analyzed using multilevel regression equations.

Results

In total 219 (69% response rate) residents filled out 2111 questionnaires about 423 (96% response rate) faculty. Faculty's overall teaching performance influenced all role model typologies (OR: from 8.0 to 166.2). For the specific domains of teaching, overall, all three role model typologies were strongly associated with “professional attitude towards residents” (OR: 3.28 for teacher/supervisor, 2.72 for physician and 7.20 for the person role). Further, the teacher/supervisor role was strongly associated with “feedback” and “learning climate” (OR: 3.23 and 2.70). However, the associations of the specific domains of teaching with faculty's role modelling varied widely across specialties.

Conclusion

This study suggests that faculty can substantially enhance their role modelling by improving their teaching performance. The amount of influence that the specific domains of teaching have on role modelling differs across specialties.