The dawn of a new competency-based training era

The correlation between trainee gender and operative autonomy during trauma and orthopaedic training in Ireland and the UK

Aims

The aim of this study was to explore differences in operative autonomy by trainee gender during orthopaedic training in Ireland and the UK, and to explore differences in operative autonomy by trainee gender with regard to training year, case complexity, index procedures, and speciality area.

Methods

This retrospective cohort study examined all operations recorded by orthopaedic trainees in Ireland and the UK between July 2012 and July 2022. The primary outcome was operative autonomy, which was defined as the trainee performing the case without the supervising trainer scrubbed.

Results

A total of 3,533,223 operations were included for analysis. Overall, male trainees performed 5% more operations with autonomy than female trainees (30.5% vs 25.5%; 95% CI 4.85 to 5.09). Female trainees assisted for 3% more operations (35% vs 32%; 95% CI 2.91 to 3.17) and performed 2% more operations with a supervising trainer scrubbed (39% vs 37%; 95% CI 1.79 to 2.06). Male trainees performed more operations with autonomy than female trainees in every year of training, in each category of case complexity, for each orthopaedic speciality area, and for every index procedure except nerve decompression. When adjusting for year, training level, case complexity, speciality area, and urgency, male trainees had 145% (95% CI 2.18 to 2.76) increased odds of performing an operation with autonomy and 35% (95% CI 1.25 to 1.45) increased odds of performing an operation under trainer supervision, than assisting, compared to female trainees.

Conclusion

Male trainees perform more operations with autonomy during orthopaedic training than female trainees. Female orthopaedic trainees assist for a greater proportion of cases than their male counterparts. A comprehensive review of trauma and orthopaedic training is needed to identify any additional differences in training opportunities between female and male trainees, particularly with regard to progression through training.

A comparison of orthopaedic surgery training across five English-speaking countries

INTRODUCTION

There is undocumented and unjustified variability in orthopaedic surgery training between countries. This study compares and contrasts the main features of orthopaedic training in Australia, New Zealand, the United Kingdom, United States, and Canada.

METHODS

Comparisons included: competition for, and selection into, training; training pathway structures; training requirements, and; training length.

RESULTS

Selection into orthopaedic surgery training is competitive in all countries assessed with acceptance rates ranging from 22%-26% in Australia and New Zealand to 85% in Canada. Minimum length of post-medical school training varies from 5 years in the USA and Canada, to 8 years in Australia, 9 years in New Zealand, and 10 years in the United Kingdom. All countries encourage participation in research during training, although there are varying requirements. Significant bottlenecks characterize selection into training in Australia, New Zealand, and the United Kingdom, meaning the majority of doctors take more than a decade from medical school graduation to obtaining their specialty surgery qualification.

CONCLUSIONS

There is high variability between the orthopaedic training programs of the studied countries. An awareness of these differences and similarities may help improve training, or provide solutions for identified gaps in each country.

Development and validation of a simulation-based assessment of operative competence for higher specialist trainees in general surgery

BACKGROUND

Simulation is increasingly being explored as an assessment modality. This study sought to develop and collate validity evidence for a novel simulation-based assessment of operative competence. We describe the approach to assessment design, development, pilot testing, and validity investigation.

METHODS

Eight procedural stations were generated using both virtual reality and bio-hybrid models. Content was identified from a previously conducted Delphi consensus study of trainers. Trainee performance was scored using an equally weighted Objective Structured Assessment of Technical Skills (OSATS) tool and a modified Procedure-Based Assessment (PBA) tool. Validity evidence was analyzed in accordance with Messick's validity framework. Both 'junior' (ST2-ST4) and 'senior' trainees (ST 5-ST8) were included to allow for comparative analysis.

RESULTS

Thirteen trainees were assessed by ten assessors across eight stations. Inter-station reliability was high (α = 0.81), and inter-rater reliability was acceptable (inter-class correlation coefficient 0.77). A significant difference in mean station score was observed between junior and senior trainees (44.82 vs 58.18, p = .004), while overall mean scores were moderately correlated with increasing training year (rs = .74, p = .004, Kendall's tau-b .57, p = 0.009). A pass-fail score generated using borderline regression methodology resulted in all 'senior' trainees passing and 4/6 of junior trainees failing the assessment.

CONCLUSION

This study reports validity evidence for a novel simulation-based assessment, designed to assess the operative competence of higher specialist trainees in general surgery.

Modern Trends in Otologic Surgery and Implications for Residency Training

OBJECTIVE

Placing a middle ear prosthesis is considered a key competency for the general otolaryngologist, but surgeons struggle to obtain and maintain this skill. The current study aims to characterize pre-coronavirus disease 2019 trends in stapedectomy and ossiculoplasty.

STUDY DESIGN

Database review.

SETTING

Tricare beneficiaries are treated at civilian and military facilities.

METHODS

The Department of Defense beneficiary population of more than nine million persons per year was reviewed for patients undergoing either stapedectomy or ossiculoplasty between 2010 and 2019, identified by the current procedural terminology code.

RESULTS

A total of 3052 stapedectomies and 7197 ossiculoplasties were performed. Over the 10-year study period, stapedectomy decreased by 23%, with an average annual rate of -2.7% per year (Pearson r = -.91, P = .0003). Ossiculoplasties declined by 18%, an average annual rate of -1.9% (r = -.8, P = .006). In combination, cases declined by 20%, an average annual rate of -2.2% (r = -.87, P = .001).

CONCLUSION

While declines in stapedectomy surgery have been well reported, here we show steady declines in ossiculoplasty as well. If these trends continue, more cochlear implantations may be performed annually than stapedectomy and ossiculoplasty combined, with cochlear implantation likely to overtake ossicular chain surgery in the near future. These changes in surgical volume have a direct implication on resident education and general otolaryngology expectations after graduation. Strong consideration should be made to replace "Stapedectomy/Ossiculoplasty" as resident key indicator with "Cochlear Implantation," a more professionally meaningful skill.

Predicting Resident Competence for Otolaryngology Key Indicator Procedures

OBJECTIVE

Competency-based surgical education requires practical assessments and meaningful benchmarks. In otolaryngology, key indicator procedure (KIP) minima are indicators of surgical exposure during training, yet it remains unknown how many times trainees must be evaluated on KIPs to ensure operative competence. Herein, we used Bayesian mixed effects models to compute predicted performance expectations for KIPs.

METHODS

From November 2017 to September 2021, a smartphone application (SIMPL OR) was used by attendings at five otolaryngology training programs to rate resident operative performance after each case on a five-level scale. Bayesian mixed effects models were used to estimate the probability that postgraduate year (PGY) 3, 4, or 5 trainees would earn a "practice-ready" (PR) rating on a subsequent evaluation based on their previously earned PR ratings for each KIP. Probabilities of earning a subsequent PR rating were examined for interpretability, and cross-validation was used to assess predictive validity.

RESULTS

A total of 842 assessments of KIPs were submitted by 72 attendings for 92 residents PGY 2-5. The predictive model had an average Area Under the Receiver Operating Curve of 0.77. The number of prior PR ratings that senior residents needed to attain a 95% probability of earning a PR rating on a subsequent evaluation was estimated for each KIP. For example, for mastoidectomies, PGY4 residents needed to earn 10 PR ratings whereas PGY5 residents needed 4 PR ratings on average to have a 95% probability of attaining a PR rating on a subsequent evaluation.

CONCLUSION

Predictive modeling can inform assessment benchmarks for competency-based surgical education.

LEVEL OF EVIDENCE

NA Laryngoscope, 133:3341-3345, 2023.

A Deep Learning Approach to Classify Surgical Skill in Microsurgery Using Force Data from a Novel Sensorised Surgical Glove

Microsurgery serves as the foundation for numerous operative procedures. Given its highly technical nature, the assessment of surgical skill becomes an essential component of clinical practice and microsurgery education. The interaction forces between surgical tools and tissues play a pivotal role in surgical success, making them a valuable indicator of surgical skill. In this study, we employ six distinct deep learning architectures (LSTM, GRU, Bi-LSTM, CLDNN, TCN, Transformer) specifically designed for the classification of surgical skill levels. We use force data obtained from a novel sensorized surgical glove utilized during a microsurgical task. To enhance the performance of our models, we propose six data augmentation techniques. The proposed frameworks are accompanied by a comprehensive analysis, both quantitative and qualitative, including experiments conducted with two cross-validation schemes and interpretable visualizations of the network's decision-making process. Our experimental results show that CLDNN and TCN are the top-performing models, achieving impressive accuracy rates of 96.16% and 97.45%, respectively. This not only underscores the effectiveness of our proposed architectures, but also serves as compelling evidence that the force data obtained through the sensorized surgical glove contains valuable information regarding surgical skill.

Ankle fracture internal fixation performed by cadaveric simulation-trained versus standard-trained orthopaedic trainees: a preliminary, multicentre randomized controlled trial

Aims

Ankle fracture fixation is commonly performed by junior trainees. Simulation training using cadavers may shorten the learning curve and result in a technically superior surgical performance.

Methods

We undertook a preliminary, pragmatic, single-blinded, multicentre, randomized controlled trial of cadaveric simulation versus standard training. Primary outcome was fracture reduction on postoperative radiographs.

Results

Overall, 139 ankle fractures were fixed by 28 postgraduate year three to five trainee surgeons (mean age 29.4 years; 71% males) during ten months' follow-up. Under the intention-to-treat principle, a technically superior fixation was performed by the cadaveric-trained group compared to the standard-trained group, as measured on the first postoperative radiograph against predefined acceptability thresholds. The cadaveric-trained group used a lower intraoperative dose of radiation than the standard-trained group (mean difference 0.011 Gym2, 95% confidence interval 0.003 to 0.019; p = 0.009). There was no difference in procedure time.

Conclusion

Trainees randomized to cadaveric training performed better ankle fracture fixations and irradiated patients less during surgery compared to standard-trained trainees. This effect, which was previously unknown, is likely to be a consequence of the intervention. Further study is required.

The role of Gender in Operative Autonomy in orthopaedic Surgical Trainees (GOAST)

Aims

Global literature suggests that female surgical trainees have lower rates of independent operating (operative autonomy) than their male counterparts. The objective of this study was to identify any association between gender and lead/independent operating in speciality orthopaedic trainees within the UK national training programme.

Methods

This was a retrospective case-control study using electronic surgical logbook data from 2009 to 2021 for 274 UK orthopaedic trainees. Total operative numbers and level of supervision were compared between male and female trainees, with correction for less than full-time training (LTFT), prior experience, and time out during training (OOP). The primary outcome was the percentage of cases undertaken as lead surgeon (supervised and unsupervised) by UK orthopaedic trainees by gender.

Results

All participants gave permission for their data to be used. In total, 274 UK orthopaedic trainees submitted data (65% men (n = 177) and 33% women (n = 91)), with a total of 285,915 surgical procedures logged over 1,364 trainee-years. Males were lead surgeon (under supervision) on 3% more cases than females (61% (115,948/189,378) to 58% (50,285/86,375), respectively; p < 0.001), and independent operator (unsupervised) on 1% more cases. A similar trend of higher operative numbers in male trainees was seen for senior (ST6 to 8) trainees (+5% and +1%; p < 0.001), those with no time OOP (+6% and +8%; p < 0.001), and those with orthopaedic experience prior to orthopaedic specialty training (+7% and +3% for lead surgeon and independent operator, respectively; p < 0.001). The gender difference was less marked for those on LTFT training, those who took time OOP, and those with no prior orthopaedic experience.

Conclusion

This study showed that males perform 3% more cases as the lead surgeon than females during UK orthopaedic training (p < 0.001). This may be due to differences in how cases are recorded, but must engender further research to ensure that all surgeons are treated equitably during their training.

Lithuanian physicians practising abroad: Reasons to leave and conditions to return to Lithuania. A survey

This study aimed to assess the factors of emigration and return among Lithuanian doctors practicing abroad. A call for participation in an online survey was distributed via social media. Questions covered demographics, reasons to emigrate and factors that would favor returning to practice in Lithuania. Survey data were analysed with appropriate statistical methods. Out of 465 respondents, the majority (453/465, 97.4%) work in Europe. The majority (334/465, 71.8%) were women, and 304/465 (65.4%) were trainees (residents). The top three factors to emigrate from and come back to Lithuania were: economic reasons, perceived corruption and the work environment. Most respondents listed more than two factors to emigrate and return (>70% in each category). Out of all respondents, 230 (49.5%) reported an attractive net monthly salary for a full-time post in Lithuania to be >3500 EUR; 173/465 (37.2%) respondents declared intention of not returning to practice in Lithuania. There were statistically significant gender- and training level-related differences in emigration factors. In conclusion, doctors' reasons for leaving and returning to practice in Lithuania are multifactorial, with economic and non-economic circumstances prompting them to choose to work in another country.

The implementation of the European Working Time Directive in Europe and its impact on training in obstetrics and gynaecology: A ten year follow-up

OBJECTIVE

To reassess the compliance with the European Working Time Directive (EWTD) in the member countries of the European Network of Trainees in Obstetrics and Gynaecology (ENTOG) and to investigate the impact of the EWTD on training.

STUDY DESIGN

In this observational, cross-sectional study, an online questionnaire, containing multiple-choice questions and open questions, was distributed among Obstetrics and Gynaecology trainees in 33 ENTOG member countries. The questionnaire was designed as a follow-up of a similar survey conducted by ENTOG in 2009 and assessed the overall compliance with the EWTD, the adaptations needed to achieve this compliance, the impact of the EWTD on the quality of training and the well-being of trainees. The answers were analysed using descriptive statistics in Microsoft Excel.

RESULTS

59 responses from 28/33 (84.8%) ENTOG member countries were collected. Only 5 out of 28 (17.9%) countries were found to be nationally compliant with EWTD. There were no clear differences in the compliance between different types of the hospitals (university/teaching/district), but a trend was observed towards higher rate of implementation in smaller hospitals (<1500 deliveries per year). Regarding the changes needed to become EWTD-compliant and yet maintain high-quality training, the most common suggestions were: hiring extra junior doctors, restructuring training, having less doctors on duty simultaneously, consultants performing more hands on work, dedicated training sessions, reduction of administrative tasks and simulation training for surgical skills. The majority of trainees, 7 out of 12, (58.3%) in the EWTD-compliant hospitals experienced a positive effect on their training, whereas the majority of trainees in non-compliant hospitals, 31 out of 47, (66%) were uncertain about the impact of the EWTD on the quality of training. Among the positive changes, better work-life balance and more consultants available out of the daily working hours were listed.

CONCLUSIONS

Despite the introduction and implementation of the EWTD over two decades ago, the compliance rates across Europe remain low and seem not to have altered in the last ten years. In order to ensure the quality of training and, most importantly patient safety, we suggest that European nations keep striving to implement the EWTD for doctors in training. We also suggest for nations that have yet to implement this directive to use the strategies as an exemplar in countries that follow EWTD.

Undergraduate education of trauma and orthopaedic surgery in the UK

Aims

Evidence exists of a consistent decline in the value and time that medical schools place upon their undergraduate orthopaedic placements. This limited exposure to trauma and orthopaedics (T&O) during medical school will be the only experience in the speciality for the majority of doctors. This review aims to provide an overview of undergraduate orthopaedic training in the UK.

Methods

This review summarizes the relevant literature from the last 20 years in the UK. Articles were selected from database searches using MEDLINE, EMBASE, ERIC, Cochrane, and Web of Science. A total of 16 papers met the inclusion criteria.

Results

The length of exposure to T&O is declining; the mean total placement duration of two to three weeks is significantly less than the four- to six-week minimum advised by most relevant sources. The main teaching methods described in the literature included didactic lectures, bedside teaching, and small group case-based discussions. Students preferred interactive, blended learning teaching styles over didactic methods. This improvement in satisfaction was reflected in improvements in student assessment scores. However, studies failed to assess competencies in clinical skills and examinations, which is consistent with the opinions of UK foundation year doctors, approximately 40% of whom report a “poor” understanding of orthopaedics. Furthermore, the majority of UK doctors are not exposed to orthopaedics at the postgraduate level, which only serves to amplify the disparity between junior and generalist knowledge, and the standards expected by senior colleagues and professional bodies.

Conclusion

There is a deficit in undergraduate orthopaedic training within the UK which has only worsened in the last 20 years, leaving medical students and foundation doctors with a potentially significant lack of orthopaedic knowledge.

Cite this article: Bone Jt Open 2022;3(7):549–556.

Assessing technical skill in ankle fracture surgery from the postoperative radiograph

Aims

To identify a core outcome set of postoperative radiographic measurements to assess technical skill in ankle fracture open reduction internal fixation (ORIF), and to validate these against Van der Vleuten’s criteria for effective assessment.

Methods

An e-Delphi exercise was undertaken at a major trauma centre (n = 39) to identify relevant parameters. Feasibility was tested by two authors. Reliability and validity was tested using postoperative radiographs of ankle fracture operations performed by trainees enrolled in an educational trial (IRCTN 20431944). To determine construct validity, trainees were divided into novice (performed < ten cases at baseline) and intermediate groups (performed ≥ ten cases at baseline). To assess concurrent validity, the procedure-based assessment (PBA) was considered the gold standard. The inter-rater and intrarater reliability was tested using a randomly selected subset of 25 cases.

Results

Overall, 235 ankle ORIFs were performed by 24 postgraduate year three to five trainees during ten months at nine NHS hospitals in England, UK. Overall, 42 PBAs were completed. The e-Delphi panel identified five ‘final product analysis’ parameters and defined acceptability thresholds: medial clear space (MCS); medial malleolar displacement (MMD); lateral malleolar displacement (LMD); tibiofibular clear space (TFCS) (all in mm); and talocrural angle (TCA) in degrees. Face validity, content validity, and feasibility were excellent. PBA global rating scale scores in this population showed excellent construct validity as continuous (p < 0.001) and categorical (p = 0.001) variables. Concurrent validity of all metrics was poor against PBA score. Intrarater reliability was substantial for all parameters (intraclass correlation coefficient (ICC) > 0.8), and inter-rater reliability was substantial for LMD, MMD, TCA, and moderate (ICC 0.61 to 0.80) for MCS and TFCS. Assessment was time efficient compared to PBA.

Conclusion

Assessment of technical skill in ankle fracture surgery using the first postoperative radiograph satisfies the tested Van der Vleuten’s utility criteria for effective assessment. 'Final product analysis' assessment may be useful to assess skill transfer in the simulation-based research setting. Cite this article: Bone Jt Open 2022;3(6):502–509.

Training to proficiency in surgery using simulation: is there a moral obligation?

A deontological approach to surgical ethics advocates that patients have the right to receive the best care that can be provided. The 'learning curve' in surgical skill is an observable and measurable phenomenon. Surgical training may therefore carry risk to patients. This can occur directly, through inadvertent harm, or indirectly through theatre inefficiency and associated costs. Trainee surgeon operating, however, is necessary from a utilitarian perspective, with potential risk balanced by the greater societal need to train future independent surgeons.New technology means that the surgical learning curve could take place, at least in part, outside of the operating theatre. Simulation-based deliberate practice could be used to obtain a predetermined level of proficiency in a safe environment, followed by simulation-based assessment of operative competence. Such an approach would require an overhaul of the current training paradigm and significant investment in simulator technology. This may increasingly be viewed as necessary in light of well-discussed pressures on surgical trainees and trainers.This article discusses the obligations to trainees, trainers and training bodies raised by simulation technology, and outlines the current arguments both against and in favour of a simulation-based training-to-proficiency model in surgery. The significant changes to the current training paradigm that would be required to implement such a model are also discussed.