Gender Bias in the Evaluation of Surgical Performance: Results of a Prospective Randomized Trial

Patient satisfaction with women vs men surgical interns and senior residents

BACKGROUND

Patient satisfaction is critical for referrals and reimbursement of surgical faculty but remains poorly characterized for residents. We investigated whether patient evaluations of surgical trainees vary by resident gender.

METHODS

Surgical inpatients evaluated surgical resident care postoperatively after positively identifying trainees. Evaluations (Consumer Assessment of Healthcare Providers and Systems Surgical Care Surveys (S-CAHPS)) were scored by the "top-box" method, stratified by training level, and compared between women and men residents.

RESULTS

Ninety-one percent of patients participated (n ​= ​324/357). Patients recognized women interns less than men (75.0 ​% vs 87.2 ​%, p ​= ​0.01). S-CAHPS scores for women vs men interns were equivalent except for spending sufficient time with patients (75.6 ​% vs 88.0 ​%, p ​= ​0.02). For senior residents, there was no difference in patient recognition of women vs men (83.9 ​% vs 85.2 ​%, p ​= ​0.91) or in any S-CAHPS scores (p ​> ​0.05).

CONCLUSIONS

Gendered differences in patient evaluations of surgical trainees exist for interns but resolve by senior years. Future work should explore how patient evaluations can support trainee development while ensuring patients recognize the role of surgical residents regardless of gender.

Gender differences in autonomy and performance assessments in a national cohort of vascular surgery trainees

OBJECTIVE

Gender disparities in surgical training and assessment are described in the general surgery literature. Assessment disparities have not been explored in vascular surgery. We sought to investigate gender disparities in operative assessment in a national cohort of vascular surgery integrated residents (VIRs) and fellows (VSFs).

METHODS

Operative performance and autonomy ratings from the Society for Improving Medical Professional Learning (SIMPL) application database were collected for all vascular surgery participating institutions from 2018 to 2023. Logistic generalized linear mixed models were conducted to examine the association of faculty and trainee gender on faculty and self-assessment of autonomy and performance. Data were adjusted for post-graduate year and case complexity. Random effects were included to account for clustering effects due to participant, program, and procedure.

RESULTS

One hundred three trainees (n = 63 VIRs; n = 40 VSFs; 63.1% men) and 99 faculty (73.7% men) from 17 institutions (n = 12 VIR and n = 13 VSF programs) contributed 4951 total assessments (44.4% by faculty, 55.6% by trainees) across 235 unique procedures. Faculty and trainee gender were not associated with faculty ratings of performance (faculty gender: odds ratio [OR], 0.78; 95% confidence interval [CI], 0.27-2.29; trainee gender: OR, 1.80; 95% CI, 0.76-0.43) or autonomy (faculty gender: OR, 0.99; 95% CI, 0.41-2.39; trainee gender: OR, 1.23; 95% CI, 0.62-2.45) of trainees. All trainees self-assessed at lower performance and autonomy ratings as compared with faculty assessments. However, women trainees rated themselves significantly lower than men for both autonomy (OR, 0.57; 95% CI, 0.43-0.74) and performance (OR, 0.40; 95% CI, 0.30-0.54).

CONCLUSIONS

Although gender was not associated with differences in faculty assessment of performance or autonomy among vascular surgery trainees, women trainees perceive themselves as performing with lower competency and less autonomy than their male colleagues. These findings suggest utility for exploring gender differences in real-time feedback delivered to and received by trainees and targeted interventions to align trainee self-perception with actual operative performance and autonomy to optimize surgical skill acquisition.

Differential attainment in assessment of postgraduate surgical trainees: a scoping review

INTRODUCTION

Solving disparities in assessments is crucial to a successful surgical training programme. The first step in levelling these inequalities is recognising in what contexts they occur, and what protected characteristics are potentially implicated.

METHODS

This scoping review was based on Arksey & O'Malley's guiding principles. OVID and Embase were used to identify articles, which were then screened by three reviewers.

RESULTS

From an initial 358 articles, 53 reported on the presence of differential attainment in postgraduate surgical assessments. The majority were quantitative studies (77.4%), using retrospective designs. 11.3% were qualitative. Differential attainment affects a varied range of protected characteristics. The characteristics most likely to be investigated were gender (85%), ethnicity (37%) and socioeconomic background (7.5%). Evidence of inequalities are present in many types of assessment, including: academic achievements, assessments of progression in training, workplace-based assessments, logs of surgical experience and tests of technical skills.

CONCLUSION

Attainment gaps have been demonstrated in many types of assessment, including supposedly "objective" written assessments and at revalidation. Further research is necessary to delineate the most effective methods to eliminate bias in higher surgical training. Surgical curriculum providers should be informed by the available literature on inequalities in surgical training, as well as other neighbouring specialties such as medicine or general practice, when designing assessments and considering how to mitigate for potential causes of differential attainment.

Authorship Trends and Disparities in Cardiothoracic Surgery

BACKGROUND

Previous investigations have revealed significant gender disparities in the academic arenas of cardiothoracic surgery. However, the status of gender representation in cardiothoracic publications has not been well described. This study aimed to evaluate authorship trends by gender in two high-impact cardiothoracic surgical journals.

METHODS

In this bibliometric analysis, PubMed was searched for articles published in The Annals of Thoracic Surgery and the Journal of Thoracic and Cardiovascular Surgery from 2010 to 2021. The web-based application Genderize.io was used to classify names of first and last authors as men vs women. The Cochran-Armitage trend test and multivariable logistic regression were used to evaluate authorship per year and the association of first and last author gender, respectively.

RESULTS

Among 14,443 articles, 16.7% had women first authors and 8.1% had women last authors. The proportion of articles written by women authors increased, rising from 12.6% to 21.1% (P < .0001) for first and 5.4% to 11.5% (P < .0001) for last authors. Papers written with women as first author were associated with 2.0 higher odds of having a woman as last author (95% CI, 1.7-2.3; P < .0001). The mean number of last author publications was higher for men than for women (2.4 vs 1.7, P < .0001).

CONCLUSIONS

Over the past decade, despite a welcomed increase in women authorship in high-impact journals in cardiothoracic surgery, women represent a small proportion of published authors. Women first authors are more likely to publish with women last authors, demonstrating the impact of same-gender collaborations while emphasizing a need for cross-gender mentorship.

Gender and Language in Letters of Recommendation for Obstetrics and Gynecology Fellowship Applications

OBJECTIVE

To delineate the use of gender-biased language in letters of recommendation for Obstetrics and Gynecology fellowships and its impact on applicants.

DESIGN

Fellowship letters of recommendation from 4 Obstetrics and Gynecology specialties at a single institution in 2020 were included.

PRIMARY OUTCOME

frequency of agentic and communal language in letters of recommendation using Linguistics Inquiry Word Count software.

SECONDARY OUTCOMES

letter of recommendation length and language utilization by author gender and applicant success measured by interviews and match success. Marginal models were fit to determine if language varied by applicant and writer gender and subspecialty. Modified Poisson regression models were used to determine associations between language and interview receipt.

SETTING

Single academic institution (Duke University); 2020 OB/GYN fellowship application cycle.

PARTICIPANTS

A total of 1216 letters of recommendation submitted by 326 unique applicants for OB/GYN subspecialty fellowships at our institution.

RESULTS

Rates of gender-biased language were low (Agentic:1.3%; communal: 0.8%). Agentic term use did not vary by applicant or author gender (p = 0.78 and 0.16) Male authors utilized 19% fewer communal terms than females (p < 0.001). Each 0.25% increase in agentic language was associated with an 18% reduction in the probability of interview invitation at our institution (p = 0.004). Percentage of agentic or communal language was not associated with successful matching into any subspecialty.

CONCLUSIONS

No differences in agentic vs communal language based on applicant gender were found in this cohort, though female letter writers wrote longer letters with more communal terms. Increasing agentic terms negatively impacted interview invitation but did not affect successful matching.

Bias in Surgical Residency Evaluations: A Scoping Review

OBJECTIVE

Given widespread disparities in the surgical workforce and the advent of competency-based training models that rely on objective evaluations of resident performance, this review aims to describe the landscape of bias in the evaluation methods of residents in surgical training programs in the United States.

DESIGN

A scoping review was conducted within PubMed, Embase, Web of Science, and ERIC in May 2022, without a date restriction. Studies were screened and reviewed in duplicate by 3 reviewers. Data were described descriptively.

SETTING/PARTICIPANTS

English-language studies conducted in the United States that assessed bias in the evaluation of surgical residents were included.

RESULTS

The search yielded 1641 studies, of which 53 met inclusion criteria. Of the included studies, 26 (49.1%) were retrospective cohort studies, 25 (47.2%) were cross-sectional studies, and 2 (3.8%) were prospective cohort studies. The majority included general surgery residents (n = 30, 56.6%) and nonstandardized examination modalities (n = 38, 71.7%), such as video-based skills evaluations (n = 5, 13.2%). The most common performance metric evaluated was operative skill (n = 22, 41.5%). Overall, the majority of studies demonstrated bias (n = 38, 73.6%) and most investigated gender bias (n = 46, 86.8%). Most studies reported disadvantages for female trainees regarding standardized examinations (80.0%), self-evaluations (73.7%), and program-level evaluations (71.4%). Four studies (7.6%) assessed racial bias, of which all reported disadvantages for trainees underrepresented in surgery.

CONCLUSIONS

Evaluation methods for surgery residents may be prone to bias, particularly with regard to female trainees. Research is warranted regarding other implicit and explicit biases, such as racial bias, as well as for nongeneral surgery subspecialties.

Blinded intraoperative skill evaluations avoid gender-based bias

INTRODUCTION

Gender bias has been identified consistently in written performance evaluations. Qualitative tools may provide a standardized way to evaluate surgical skill and minimize gender bias. We hypothesized that there is no difference in operative time or GEARS scores in robotic hysterectomy for men vs women surgeons.

METHODS

Patients undergoing robotic hysterectomies performed between June 2019 and March 2020 at 8 hospitals within the same hospital system were captured into a prospective database. GEARS scores were assigned by crowd-sourced evaluators by a third party blinded to any surgeon- or patient-identifying information. One-way ANOVA was used to compare the mean operative time and GEARS scores for each group, and significant variables were included in a one-way ANCOVA to control for confounders. Two-tailed p-value < 0.05 was considered significant.

RESULTS

Seventeen women and 13 men performed a total of 188 hysterectomies; women performed 34 (18%) and men performed 153 (81%). Women surgeons had a higher mean operative time (133 ± 58 vs 86.3 ± 46 min, p = 0.024); after adjustment, there were no significant differences in operative time (p = 0.607). There was no significant difference between the genders in total GEARS score (20.0 ± 0.77 vs 20.2 ± 0.70, p = 0.415) or GEARS subcomponent scores: bimanual dexterity (3.98 ± 0.03 vs 4.00 ± 0.03, p = 0.705); depth perception (4.04 ± 0.04 vs 4.05 ± 0.02, p = 0.799); efficiency (3.79 ± 0.02 vs 3.82 ± 0.02, p = 0.437); force sensitivity (4.01 ± 0.04 vs 4.05 ± 0.05, p = 0.533); or robotic control (4.16 ± 0.03 vs 4.26 ± 0.01, p = 0.079).

CONCLUSION

There was no difference in GEARS score between men vs women surgeons performing robotic hysterectomies. Video-based blinded assessment of skills may minimize gender biases when evaluating surgical skill for competency evaluation and credentialing.

An investigation into gender bias in the evaluation of orthopedic trainee arthroscopic skills

BACKGROUND

Women surgeons receive lower compensation, hold fewer academic positions, and hold fewer leadership positions than men, particularly in orthopedic surgery. Gender bias at the trainee level has been demonstrated in various surgical subspecialties, but there is a lack of information on gender bias within the orthopedic training environment. This study investigated whether implicit gender bias is present in the subjective evaluation of orthopedic trainee arthroscopic skills.

METHODS

After institutional review board approval, a web-based survey was sent to American Shoulder and Elbow Surgeons (ASES) members via the society's email listserve. Study participants were informed that the study was being done to develop a systematic evaluation method for trainees. The survey randomized participants to view and evaluate a prefellowship and a postfellowship video of "Rachel" (she/her) or "Charles" (he/him) performing a 15-point diagnostic shoulder arthroscopy. The videos for Rachel and Charles were identical except for the pronouns used in the video. Participants evaluated the trainee's skill level using questions from the Arthroscopic Surgical Skill Evaluation Tool (ASSET). Blinded and deidentified additional comments regarding the trainee's skill were classified as positive, negative, or neutral. Statistical analyses were used to compare scores and comments between Rachel and Charles.

RESULTS

Of 1115 active ASES members, 181 ASES members started the survey and 106 watched both videos and were included in the analysis. Of the 106 participants completing the survey, 96 (91%) were men and 10 (9%) were women with a median (interquartile range) age of 44 (38-51). A teaching role was reported by 84 of 106 participants (79%). There was no significant difference between prefellowship scores (P = .87) or between postfellowship scores (P = .84) for the woman and man fellow. The numbers of comments classified as positive, negative, or neutral were not significantly different between the man and woman fellow (P = .19). Participants in teaching roles gave significantly lower scores to both fellows at both time points (P = .04), and participants who had fellow trainees were more likely to give negative comments to both fellows (P = .02).

DISCUSSION

Trainee gender did not influence the ratings and comments participants gave for trainee arthroscopic skills, suggesting that gender bias may not play a major role in the evaluation of arthroscopic skill during orthopedic training.

Dr. Olga Jonasson: A Pioneer for Women in Surgery

The advancement of women in surgery has seen tremendous progress over the past century. Among the many physicians who paved the way for women in surgery is Dr. Olga Jonasson, a Chicago-based transplant surgeon who performed the first kidney transplantation in the state of Illinois in 1969. Her passion for service and drive for greater female representation in medicine was massively influential to the field. Aspiring female physicians are forever indebted to the efforts of Dr. Jonasson and the bold women who came before us.