Assessment of Gender-based Qualitative Differences within Trainee Evaluations of Faculty

Gender Disparity in Teaching Evaluations of Pediatric Faculty by Residents

OBJECTIVES

Despite their overrepresentation, female physicians continue to have lower rates of promotion compared with male physicians. Teaching evaluations play a role in physician advancement. Few studies have investigated gender disparity in resident evaluations of pediatric faculty. We hypothesized that gender disparities in resident evaluations of faculty exist and vary across subspecialties and primary work environments.

METHODS

Pediatric faculty institution-specific evaluations completed by residents from January 1, 2015, to March 9, 2020, were obtained from a single academic center. Mean ratings of faculty performance were compared by gender using a Wilcoxon 2-sample test.

RESULTS

Fifteen-thousand one-hundred and forty-two evaluations (5091 of male faculty and 10 051 of female faculty) were included. Female faculty were rated higher in overall teaching ability (female = 4.67 versus male = 4.65; P = .004). There was no statistical difference in the mean ratings of male and female faculty in the inpatient setting, whereas outpatient female faculty were rated higher in overall teaching ability (female = 4.79 versus male = 4.73; P = .005). For general pediatric faculty, females received higher ratings for overall teaching ability (female = 4.75 versus male = 4.70; P < .001). By contrast, there was no difference in ratings of subspecialty pediatric faculty.

CONCLUSIONS

Pediatric female faculty were statistically rated higher than male faculty in overall teaching ability, although these findings may not be educationally significant. The difference was driven by evaluations in the outpatient setting and for general pediatricians. This study is one of the first in pediatrics adding to the continued investigation of gender disparities in academic medicine.

Better Together: Development and Implementation of Fellow Group Evaluations of Faculty

Background

High-quality trainee evaluations of faculty are essential for meaningful faculty development and for improving the clinical learning environment. However, concerns about anonymity can limit usefulness of trainee evaluations, particularly in smaller programs, such as subspecialty fellowships.

Objective

To develop and implement a fellow-driven group evaluation process to enhance trainee confidentiality and generate high-quality feedback for pulmonary and critical care medicine faculty.

Methods

A novel process was developed for faculty evaluation and feedback consisting of quarterly, structured, fellow-led group evaluation sessions focused on collecting confidential, behaviorally oriented, actionable feedback for faculty. Upper-year fellow moderators utilized a standard format to structure discussion, generating strengths and areas for growth for each faculty member while explicitly asking for input from fellows with divergent perspectives. Moderators compiled anonymized session notes for the program director, who delivered feedback to individual faculty. After the first six sessions, an electronic survey was distributed to assess fellow perceptions of the group evaluation model.

Results

Thirty-seven faculty members were evaluated in 11 group sessions over 42 months. Fellows rated group-generated feedback as more confidential, more specific, more accurate, more efficient, more actionable, and less biased when compared with individual written evaluations (P < 0.01 for all categories).

Conclusion

The authors successfully developed and implemented a process for fellow-led group evaluation of faculty, designed to facilitate fellow confidentiality and enrich the quality of feedback. Fellows preferred the group evaluation process and perceived group-generated feedback more favorably compared with individual written evaluations.

Gender Disparities in Critical Care Procedure Training of Internal Medicine Residents

Background

Procedural training is a required competency in internal medicine (IM) residency, yet limited data exist on residents' experience of procedural training.

Objectives

We sought to understand how gender impacts access to procedural training among IM residents.

Methods

A mixed-methods, explanatory sequential study was performed. Procedure volume for IM residents between 2016 and 2020 was assessed at two large academic residencies (Program A and Program B: 399 residents and 4,020 procedures). Procedural rates and actual versus expected procedure volume by gender were compared, with separate analyses by clinical environment (intensive care unit [ICU] or structured procedural service). Semistructured gender-congruent focus groups were conducted. Topics included identity formation as a proceduralist and the resident procedural learning experience, including perceived gender bias in procedure allocation.

Results

Compared with men, women residents performed disproportionately fewer ICU procedures per month at Program A (1.4 vs. 2.7; P < 0.05) but not at Program B (0.36 vs. 0.54; P = 0.23). At Program A, women performed only 47% of ICU procedures, significantly fewer than the 54% they were expected to perform on the basis of their time on ICU rotations (P < 0.001). For equal gender distribution of procedural volume at Program A, 11% of the procedures performed by men would have needed to have been performed by women instead. Gender was not associated with differences in the Program A structured procedural service (53% observed vs. 52% expected; P = 0.935), Program B structured procedural service (40% observed vs. 43% expected; P = 0.174), or in Program B ICUs (33% observed vs. 34% expected; P = 0.656). Focus group analysis identified that women from both residencies perceived that assertiveness was required for procedural training in unstructured learning environments. Residents felt that gender influenced access to procedural opportunities, ability to self-advocate for procedural experience, identity formation as a proceduralist, and confidence in acquiring procedural skills.

Conclusion

Gender disparities in access to procedural training during ICU rotations were seen at one institution but not another. There were ubiquitous perceptions that assertiveness was important to access procedural opportunities. We hypothesize that structured allocation of procedures would mitigate disparities by allowing all residents to access procedural training regardless of self-advocacy. Residency programs should adopt structured procedural training programs to counteract inequities.

Necessary but Insufficient and Possibly Counterproductive: The Complex Problem of Teaching Evaluations

The evaluation of clinical teachers' performance has long been a subject of research and debate, yet teaching evaluations (TEs) by students remain problematic. Despite their intuitive appeal, there is little evidence that TEs are associated with students' learning in the classroom or clinical setting. TEs are also subject to many forms of bias and are confounded by construct-irrelevant factors, such as the teacher's physical attractiveness or personality. Yet they are used almost exclusively as evaluations of and feedback to teachers. In this commentary, the authors review the literature on what TEs are meant to do, what they actually do in the real world, and their overall impact. The authors also consider productive ways forward. While TEs are certainly necessary to provide the crucial student voice, they are insufficient as the sole way to assess teachers. Further, they are often counterproductive. TEs carry so much weight for faculty that they can act as a disincentive for teachers to challenge learners and provide them with the critical feedback they often need, lest students give them poor ratings. To address these challenges, changes are needed, including embedding TEs in a programmatic assessment framework. For example, TEs might be used for formative feedback only, while other sources of data, such as peer assessments, learning outcomes, 360-degree feedback, and teacher reflections, could be collated into a portfolio to provide a more meaningful evaluation for teachers. Robust, transparent systems should be in place that dictate how TE data are used and to ensure they are not misused. Clinical teachers who do not "fail to fail" learners but instead take the time and effort to identify and support learners in difficulty should be recognized and rewarded. Learners need this support to succeed and the obligation to protect patients demands it.

Analysis of Narrative Text in Evaluations of Continuing Medical Education Faculty by Gender

This mixed-methods study examines the performance ratings and feedback received by female and male lecturers at national internal medicine conferences.

Beyond the ratings: gender effects in written comments from clinical teaching assessments

Assessment of clinical teachers by learners is problematic. Construct-irrelevant factors influence ratings, and women teachers often receive lower ratings than men. However, most studies focus only on numeric scores. Therefore, the authors analyzed written comments on 4032 teacher assessments, representing 282 women and 448 men teachers in one Department of Medicine, to explore for gender differences. NVivo was used to search for 61 evidence- and theoretically-based terms purported to reflect teaching excellence, which were analyzed using 2 × 2 chi-squared tests. The Linguistic Index and Word Count (LIWC) was used to categorize comment data, which were analyzed using linear regressions. The only significant difference in NVivo was that men were more likely than women to have the word "available" in a comment (OR 1.4, p < .05). A subset of LIWC variables showed significant gender differences, but all effects were modest. Men teachers had more positive emotion words written about them, while negative emotion words appeared equally. Significant differences occurred more often between the men and women residents who wrote the comments, rather than those attributed to the gender of the teachers. For example, women residents used more social and gender-related words (β 1.87, p < 0.001) and fewer words related to power or achievement (β -3.78, p < 0.001) than men residents. Profound gender differences were not found in teacher assessment comments in this large, diverse academic department of medicine, which differs from other studies. The authors explore possible reasons including differences in departmental culture and issues related to the methods used.