Objective Test Scores Throughout Orthopedic Surgery Residency Suggest Disparities in Training Experience

Differential attainment in assessment of postgraduate surgical trainees: a scoping review

INTRODUCTION

Solving disparities in assessments is crucial to a successful surgical training programme. The first step in levelling these inequalities is recognising in what contexts they occur, and what protected characteristics are potentially implicated.

METHODS

This scoping review was based on Arksey & O'Malley's guiding principles. OVID and Embase were used to identify articles, which were then screened by three reviewers.

RESULTS

From an initial 358 articles, 53 reported on the presence of differential attainment in postgraduate surgical assessments. The majority were quantitative studies (77.4%), using retrospective designs. 11.3% were qualitative. Differential attainment affects a varied range of protected characteristics. The characteristics most likely to be investigated were gender (85%), ethnicity (37%) and socioeconomic background (7.5%). Evidence of inequalities are present in many types of assessment, including: academic achievements, assessments of progression in training, workplace-based assessments, logs of surgical experience and tests of technical skills.

CONCLUSION

Attainment gaps have been demonstrated in many types of assessment, including supposedly "objective" written assessments and at revalidation. Further research is necessary to delineate the most effective methods to eliminate bias in higher surgical training. Surgical curriculum providers should be informed by the available literature on inequalities in surgical training, as well as other neighbouring specialties such as medicine or general practice, when designing assessments and considering how to mitigate for potential causes of differential attainment.

Acquisition of Medical Knowledge Varies During Allopathic and Osteopathic Orthopedic Surgery Residency Training

Currently, little is known about the differences in medical knowledge acquisition between osteopathic and allopathic orthopedic surgery residents. The purpose of this study was to determine the relationship between Orthopaedic In-Training Examination (OITE) performance and training pathway in orthopedic surgery. This was a retrospective cohort study of all orthopedic surgery residents taking the OITE during the 2019-2020 academic year. Comparisons in OITE performance were made with parametric tests. A total of 4407 orthopedic surgery residents were in allopathic (86%) and osteopathic (14%) training programs. There was significant improvement in OITE performance between subsequent postgraduate year (PGY) levels among allopathic residents (P<.001). Among osteopathic residents, OITE performance increased between PGY1 and PGY4 (P<.001) but plateaued between PGY4 and PGY5 (P>.05). At the PGY1 level, osteopathic residents had higher OITE performance than allopathic residents (P<.001), but scores were equivalent at the PGY2 to PGY4 levels (P>.05). At the PGY5 level, allopathic residents had higher OITE performance than osteopathic residents (P<.001). Allopathic medical students scored higher on the Step 1 (248±19 vs 242±17, P<.001) and Step 2 (255±16 vs 250±15, P<.001) board examinations. Medical knowledge increases during orthopedic surgery residency. Disparities exist by training pathway, with osteopathic residents outperforming allopathic residents at the PGY1 level but then underperforming at the PGY5 level. Ultimately, this study provides insights into how resident promotion and training pathway impacts the acquisition of medical knowledge during orthopedic surgery residency. [Orthopedics. 2023;46(6):379-383.].

Bias in Surgical Residency Evaluations: A Scoping Review

OBJECTIVE

Given widespread disparities in the surgical workforce and the advent of competency-based training models that rely on objective evaluations of resident performance, this review aims to describe the landscape of bias in the evaluation methods of residents in surgical training programs in the United States.

DESIGN

A scoping review was conducted within PubMed, Embase, Web of Science, and ERIC in May 2022, without a date restriction. Studies were screened and reviewed in duplicate by 3 reviewers. Data were described descriptively.

SETTING/PARTICIPANTS

English-language studies conducted in the United States that assessed bias in the evaluation of surgical residents were included.

RESULTS

The search yielded 1641 studies, of which 53 met inclusion criteria. Of the included studies, 26 (49.1%) were retrospective cohort studies, 25 (47.2%) were cross-sectional studies, and 2 (3.8%) were prospective cohort studies. The majority included general surgery residents (n = 30, 56.6%) and nonstandardized examination modalities (n = 38, 71.7%), such as video-based skills evaluations (n = 5, 13.2%). The most common performance metric evaluated was operative skill (n = 22, 41.5%). Overall, the majority of studies demonstrated bias (n = 38, 73.6%) and most investigated gender bias (n = 46, 86.8%). Most studies reported disadvantages for female trainees regarding standardized examinations (80.0%), self-evaluations (73.7%), and program-level evaluations (71.4%). Four studies (7.6%) assessed racial bias, of which all reported disadvantages for trainees underrepresented in surgery.

CONCLUSIONS

Evaluation methods for surgery residents may be prone to bias, particularly with regard to female trainees. Research is warranted regarding other implicit and explicit biases, such as racial bias, as well as for nongeneral surgery subspecialties.

Bullying, harassment, and discrimination of musculoskeletal researchers and the impact of the COVID-19 pandemic: an international study

PURPOSE

Bullying, harassment, and discrimination (BHD) are prevalent in academic, scientific, and clinical departments, particularly orthopedic surgery, and can have lasting effects on victims. As it is unclear how BHD affects musculoskeletal (MSK) researchers, the following study assessed BHD in the MSK research community and whether the COVID-19 pandemic, which caused hardships in other industries, had an impact.

METHODS

A web-based anonymous survey was developed in English by ORS Spine Section members to assess the impact of COVID-19 on MSK researchers in North America, Europe, and Asia, which included questions to evaluate the personal experience of researchers regarding BHD.

RESULTS

116 MSK researchers completed the survey. Of respondents, 34.5% (n = 40) focused on spine, 30.2% (n = 35) had multiple areas of interest, and 35.3% (n = 41) represented other areas of MSK research. BHD was observed by 26.7% (n = 31) of respondents and personally experienced by 11.2% (n = 13), with mid-career faculty both observing and experiencing the most BHD. Most who experienced BHD (53.8%, n = 7) experienced multiple forms. 32.8% (n = 38) of respondents were not able to speak out about BHD without fear of repercussions, with 13.8% (n = 16) being unsure about this. Of those who observed BHD, 54.8% (n = 17) noted that the COVID-19 pandemic had no impact on their observations.

CONCLUSIONS

To our knowledge, this is the first study to address the prevalence and determinants of BHD among MSK researchers. MSK researchers experienced and observed BHD, while many were not comfortable reporting and discussing violations to their institution. The COVID-19 pandemic had mixed-effects on BHD. Awareness and proactive policy changes may be warranted to reduce/eliminate the occurrence of BHD in this community.

Minority Resident Physicians' Perspectives on the Role of Race/Ethnicity, Culture, and Gender in Their Surgical Training Experiences

BACKGROUND

Female and racial/ethnic minority representation in surgical programs continues to trail behind other medical specialties. Various structural and perceived obstacles which contribute to a difficult path for underrepresented minority (URM) trainees have been identified, and efforts to reduce these hurdles are underway. Gaining perspective and insight from current surgical minority trainees may add valuable insight to aid with improving and innovating strategies to recruit and retain URM surgeons.

OBJECTIVE

To characterize how race/ethnicity, cultural background, and gender affect the surgical training experience of URM surgical residents in all areas of surgery a focus on the field of Orthopedic Surgery, given its particularly poor rates of diversity.

METHODS

Authors conducted semi-structured video interviews on current surgical residents or fellows who were members of underrepresented populations including Female, African-American/Black, Latino, Asian, Native American, and First or Second-generation immigrant status. Recruitment was achieved through a combination of voluntary, convenience, and snowball sampling procedures. Interview transcripts were then coded using conventional thematic analysis. Themes were iteratively expanded into subthemes and subsequently categorized utilizing a pile-sorting methodology.

RESULTS

Among 23 surgical trainees 12 self-identified as Black (60.9%), 5 as Asian (17.4%), 1 as Hispanic (4.4%), and 5 as Caucasian (17.4%). Twelve residents identified as male (52%) and 11 as female (48%). Six surgical specialties were represented with the majority of participants (83%) being trainees in surgical subspecialties, among those orthopedic surgery was most strongly represented (57%). Analysis of their responses revealed 4 major themes: positive experiences, problems related to minority status, coping strategies, and participant suggested interventions. Themes were distilled further to sub-themes. Positive experiences' sub-themes included finding a supportive community, pride in minority status, and being able to better relate to patients. Negative experiences related to minority status' subthemes included perceived microaggressions and additional pressures, such as greater scrutiny and harsher punishments relative to their nonminority counterparts, which negatively impacted their surgical training. Most respondents did not feel there were dedicated resources to help alleviate these additional burdens, so some sought help outside of their training programs while others tried to assimilate, and others felt isolated. Recommended proposed interventions included validating the URM resident experience, providing education/training, and creating opportunities for mentorship.

IMPLICATIONS/CONCLUSIONS

URM surgical trainees face numerous challenges related to their minority status. Recruitment and retention of URM in medicine would benefit from individual early and longitudinal mentorship, mitigating imposter syndrome, acknowledging the challenges faced by residents, and seeking feedback from both past and current residents.

The Impact of Post-graduate Year and Program Accreditation Status on In-Training Examination Performance in Orthopaedic Surgery

Introduction  The progression of medical knowledge competency during surgical residency training is poorly understood. This study measures the acquisition of medical knowledge as orthopedic surgery residents advance during training and the impact of accreditation status on orthopedic in-training examination (OITE) performance. Methods  Orthopedic surgery residents taking the OITE during 2020 and 2021 were included. Residents were grouped into cohorts by post-graduate year (PGY) and Accreditation Council for Graduate Medical Education (ACGME) accreditation status. Comparisons were made with parametric tests. Results  Eight thousand eight hundred and seventy-one ACGME-accredited residents (89%) and 1,057 non-ACGME-accredited residents (11%) were evenly distributed by the PGY level (range, 19-21%). Residents in both ACGME- and non-ACGME-accredited residency programs had a significant increase in OITE performance at each PGY level (P<0.001). At ACGME-accredited programs, OITE performance increased from PGY1 (51%), PGY2 (59%), PGY3 (65%), PGY4 (68%), and PGY5 (70%) (P<0.001). There were progressively smaller percentage increases in OITE performance during accredited residency training (range, 2-8%), but this increase was linear in non-accredited residency training (range, 4%). At each PGY level, residents at accredited programs outperformed their counterparts at non-accredited programs (P<0.001). Conclusion OITE performance increases during residency training. Among ACGME-accredited residents, performance on the OITE progresses rapidly during junior years and plateaus during senior years. Residents in ACGME-accredited residency programs outperform their counterparts in non-accredited residency programs. More research is needed to understand optimal training environments that promote medical knowledge acquisition during orthopedic surgery residency.