The knee arthroscopy learning curve: quantitative assessment of surgical skills

Learning curve of total ankle arthroplasty: a systematic review

INTRODUCTION

Together with ankle arthrodesis, total ankle arthroplasty is now accepted as a first-line intervention in the management of end-stage arthritis of the ankle. The evidence regarding how outcomes are affected by surgeon experience is inconsistent; we performed a systematic review to evaluate the effect of a learning curve in total ankle arthroplasty outcomes.

METHODS

An electronic database search was performed in PubMed, Embase, ISI Web of Science and Cochrane trials. Two reviewers independently conducted a two-stage title/abstract and full text screening. English-language original research studies comparing patient-reported outcome measures (PROMs), complication/revision rates, operative time, length of stay or radiation exposure according to surgeon experience were included. Quality assessment was performed using the methodological index for non-randomised studies.

RESULTS

All but one included study report either improved PROMs, reduced complication/revision rate, reduced hospital stay length/operative time or reduced radiation exposure with increasing surgeon experience. However, the majority of these findings lack statistical significance. Two studies assessing the plateau of the learning curve report a wide range of plateau thresholds between 9 and 39 cases.

CONCLUSION

This review finds a largely non-significant trend towards improvements in PROMs, complication, and revision rates with improved surgeon experience. The lack of statistical significance in a number of studies may be partially explained by methodological flaws, with more suitably designed studies reporting significant improvements. Future research into the effect of advancements in implant design and insertion guides is required to further characterise the magnitude of the learning curve and guide both mitigation and learning strategies.

An Overview of Pediatric Knee Arthroscopy With Use of the NeedleScope

As sports-related injuries are becoming more common among children, there has been an increased need for knee arthroscopies in the pediatric population. Nevertheless, pediatric knee arthroscopy exposure is varied among orthopaedic surgeons, especially during residency training. There is a considerable difference in knee arthroscopy case volume between the adult and the pediatric population among orthopaedic residents. Although the fundamentals of adult knee arthroscopy have been well delineated, there is a paucity of literature surrounding basic pediatric knee arthroscopy. Our goal is to further explain the fundamentals needed to perform successful arthroscopic surgery in the pediatric knee. Through discussing proper patient positioning, instrumentation, including the utility of the 1.9-mm small-bore needle arthroscope (NanoScope; Arthrex, Naples, FL), subtle tips and tricks, as well as advantages and disadvantages of arthroscopy in children, this Technical Note along with the corresponding video are designed to help elucidate the intricacies of the pediatric knee. It will serve as educational material targeted to the resident trainee or any orthopaedic practitioner interested in gaining an introductory foundation to pediatric knee arthroscopy.

Arthroscopic Training: Historical Insights and Future Directions

Arthroscopy is an orthopaedic technique that provides surgical solutions in a minimally invasive fashion. Since its introduction, arthroscopy has become a preferred surgical approach for treating various orthopaedic pathologies, such as meniscal tears, anterior cruciate ligament ruptures, rotator cuff tears, and wrist, elbow, ankle, and hip conditions. Despite its ubiquity, surgical training in arthroscopy poses several challenges for educators and trainees. Arthroscopy involves neuromotor skills which differ from those of open surgery, such as the principles of triangulation, bimanual dexterity, and the ability to navigate a three-dimensional space on a two-dimensional screen. There remains no universally implemented curriculum for arthroscopic education within orthopaedic residency or fellowship training programs, permitting the potential for highly variable training experiences from institution to institution. Therefore, the current review seeks to highlight the history of arthroscopic education, strategies and current teaching modalities in modern arthroscopic education, and avenues for future educational pathways.

Can artificial intelligence help decision-making in arthroscopy? Part 2: The IA-RTRHO model - a decision-making aid for long head of the biceps diagnoses in small rotator cuff tears

INTRODUCTION

The possible applications of artificial intelligence (AI) in orthopedic surgery are promising. Deep learning can be utilized in arthroscopic surgery due to the video signal used by computer vision. The intraoperative management of the long head of biceps (LHB) tendon is the subject of a long-standing controversy. The main objective of this study was to model a diagnostic AI capable of determining the healthy or pathological state of the LHB on arthroscopic images. The secondary objective was to create a second diagnostic AI model based on arthroscopic images and the medical, clinical and imaging data of each patient, to determine the healthy or pathological state of the LHB.

HYPOTHESIS

The hypothesis of this study was that it was possible to construct an AI model from operative arthroscopic images to aid in the diagnosis of the healthy or pathological state of the LHB, and its analysis would be superior to a human analysis.

MATERIALS AND METHODS

Prospective clinical and imaging data from 199 patients were collected and associated with images from a validated protocoled arthroscopic video analysis, called "ground truth", made by the operating surgeon. A model based on a convolutional neural network (CNN) modeled via transfer learning on the Inception V3 model was built for the analysis of arthroscopic images. This model was then coupled to MultiLayer Perceptron (MLP), integrating clinical and imaging data. Each model was trained and tested using supervised learning.

RESULTS

The accuracy of the CNN in diagnosing the healthy or pathological state of the LHB was 93.7% in learning and 80.66% in generalization. Coupled with the clinical data of each patient, the accuracy of the model assembling the CNN and MLP were respectively 77% and 58% in learning and in generalization.

CONCLUSION

The AI model built from a CNN manages to determine the healthy or pathological state of the LHB with an accuracy rate of 80.66%. An increase in input data to limit overfitting, and the automation of the detection phase by a Mask-R-CNN are ways of improving the model. This study is the first to assess the ability of an AI to analyze arthroscopic images, and its results need to be confirmed by further studies on this subject.

LEVEL OF EVIDENCE

III Diagnostic study.

Impact of COVID-19 on the Reported Case Volume of Orthopaedic Sports Medicine Fellows in the United States

Background

The impact of the coronavirus 2019 (COVID-19) pandemic on reported case volume during orthopaedic sports medicine fellowship training remains poorly understood from a national perspective.

Hypothesis

It was hypothesized that reported case volume during orthopaedic sports medicine fellowship training would decrease during the 2019-2020 academic year, which corresponded to the COVID-19 outbreak. It was also hypothesized that there would be a subsequent rebound in case volume during the 2020-2021 academic year.

Study Design

Cohort study.

Methods

Reported mean case volumes were extracted for 4 academic years (2017-2018, 2018-2019, 2019-2020, and 2020-2021), and year-over-year percentage changes were calculated. Parametric tests were used for interyear comparisons.

Results

There was a 7% decrease in mean ± SD case volume between the 2018-2019 and 2019-2020 academic years (339 ± 123 vs 316 ± 108; P = .042). Case categories with the greatest percentage declines were foot and ankle (-20%), knee instability (-11%), meniscus (-8%), and glenohumeral instability (-7%). There was a 13% year-over-year increase in case volume between the 2019-2020 and 2020-2021 academic years (316 ± 108 vs 357 ± 117; P < .001).

Conclusion

There was a slight decrease in total orthopaedic sports medicine case volume during the 2019-2020 academic year, corresponding to widespread lockdowns during the COVID-19 outbreak. Certain case categories experienced the greatest negative impact. Results from this study may inform accrediting bodies and surgical educators on the impact of future disruptions to health care delivery.

Defining operative experience targets in surgical training: A systematic review

BACKGROUND

The surgical learning curve is an observable and measurable phenomenon. Operative experience targets are well established as a proxy measure for operative competence in surgical training across jurisdictions. The aim of this study was to critique the available evidence regarding the relationship between operative experience in surgical training and trainee competence.

METHODS

A systematic review of the PubMed, Embase, Web of Science, and Cochrane library databases was conducted in accordance with the Preferred Items for Systematic Reviews and Meta-Analyses guidelines. Articles were sought that defined the relationship between procedural volume in surgical training and trainee competence, proficiency, or mastery. The educational impact of included studies was evaluated using a modified Kirkpatrick model.

RESULTS

Of 3,672 records identified on database searching, 30 papers were ultimately included. Fourteen studies defined operative experience thresholds using operative time as a surrogate measure of competence, whereas another 8 used trainer assessments of operative performance (Kirkpatrick level 3). A further 5 studies were able to determine the relationship between trainee case volumes and subsequent patient outcomes (Kirkpatrick level 4b).

CONCLUSION

Many studies have recorded competent trainee performance in key index procedures after reaching experience threshold numbers in excess of currently mandated targets across jurisdictions. The evidence relating current operative experience targets to patient outcomes across a range of surgical subspecialties of surgical subspecialties is lacking. This review supports a move toward criterion-based referencing of operative performance targets in surgical training.

Computer-Assisted Surgery Enables Beginner Surgeons, Under Expert Guidance, to Achieve Long-Term Clinical Results not Inferior to Those of a Skilled Surgeon in Knee Arthroplasty

PURPOSE

The purpose of this study is to determine whether the use of a surgical navigation system in total knee replacement (TKR) enables beginner and intermediate surgeons to achieve clinical PROM outcomes as good as those conducted by expert surgeons in the long term.

METHODS

We enrolled 100 consecutive patients whose total navigated knee arthroplasty (TKA) was performed in our institution from 2008 to 2010. According to the principal surgeon's surgical experience, the patients were divided into three groups: (1) beginner surgeons, with no more than 30 previous knee replacement performances, (2) intermediate surgeons, with more than 30 but not more than 300, and (3) expert surgeons, with more than 300 knee replacements. Demographic data collected on the cohort included gender, laterality, age, and body mass index (BMI). The outcome measures assessed were Forgotten Joint Score (FJS), implant positioning, limb alignment, and prosthesis survival rate. A margin of equivalence of ± 18.5 points in the FJS scale was prespecified in terms of the minimal clinically important difference (MCID) to compare the FJS results obtained in the long period between the groups of interest.

RESULTS

The mean follow-up was 11.10 ± 0.78, 10.86 ± 0.66, and 11.30 ± 0.74 years, respectively, for each of the groups. The long-term FJS mean score was 80.86 ± 21.88, 81.36 ± 23.87, and 90.48 ± 14.65 for each group. The statistical analysis proved noninferiority and equivalence in terms of the FJS results reported in the long term by patients in Groups 1 or 2 compared to those in Group 3. More specifically, it has been proved that the mean difference between groups is within the interval of equivalence defined in terms of the MCID. The overall prostheses survival rate was 93.7%.

CONCLUSION

Navigated assisted TKA, under expert guidance, can be as effective when performed by beginner or intermediate surgeons as performed by senior surgeons regarding the accuracy of implant positioning, limb alignment, and long-term clinical outcome.

Portal Setup: the Key Point in the Learning Curve for Hip Arthroscopy Technique

OBJECTIVE

To analyze the learning curve experience of hip arthroscopy based on patient demographics, surgical time, portal setup time, and postoperative complications and to find the key point in the learning curve.

METHODS

From May 2016 to February 2019, a prospective study on the learning curve experience of hip arthroscopy was performed in our hospital. We evaluated the first 50 consecutive hip arthroscopy procedures performed by a single surgeon. There were nine females and 41 males with a mean age of 30.8 years. We divide the patients into early group and late group according to the date of their operation, with each group including 25 patients. Data on patient demographics, types of procedure, surgical time, portal setup time, and postoperative complications were collected. Functional scores were assessed with the modified Harris Hip Score (mHHS).

RESULTS

Patients were followed up for 16.4 months on average (range, 13-27 months). The early group of patients had a mean age of 35.2 years and the late group a mean age of 26.5 years. The most common procedures performed for the early group were debridement (17 patients, 68%), and in the late group, most patients underwent labral repair (18 patients, 72%). Mean total surgical time was 168 min for the early group and 143 min for the late group, and there was no statistically significant difference between two groups. The portal setup time in the early group and late group was 40.2 ± 12.4 min and 18.5 ± 6.2 min, respectively (P < 0.001), and the portal setup time was significantly longer in the early group. Further analysis of the learning curve of portal setup showed that the average portal setup time was not statistically significant changed after 30 cases. There were six complications including iatrogenic cartilage injury and iatrogenic labrum injury in the early group and five complications including perineal crush injury and nerve stretch injury in the late group. The functional score of patients in the late group was significantly higher than that in the early group during follow-up.

CONCLUSION

The steep learning curve of hip arthroscopy is mainly caused by the challenge of portal setup and portalrelated complications were more common in the early group than in the late group. Surgical time is not an effective indicator for evaluating progress on the learning curve of hip arthroscopy.

Analysis of Tools Used in Assessing Technical Skills and Operative Competence in Trauma and Orthopaedic Surgical Training: A Systematic Review

BACKGROUND

Robust assessment of skills acquisition and surgical performance during training is vital to ensuring operative competence among orthopaedic surgeons. A move to competency-based surgical training requires the use of tools that can assess surgical skills objectively and systematically. The aim of this systematic review was to describe the evidence for the utility of assessment tools used in evaluating operative performance in trauma and orthopaedic surgical training.

METHODS

We performed a comprehensive literature search of MEDLINE, Embase, and Google Scholar databases to June 2019. From eligible studies we abstracted data on study aim, assessment format (live theater or simulated setting), skills assessed, and tools or metrics used to assess surgical performance. The strengths, limitations, and psychometric properties of the assessment tools are reported on the basis of previously defined utility criteria.

RESULTS

One hundred and five studies published between 1990 and 2019 were included. Forty-two studies involved open orthopaedic surgical procedures, and 63 involved arthroscopy. The majority (85%) were used in the simulated environment. There was wide variation in the type of assessment tools in used, the strengths and weaknesses of which are assessor and setting-dependent.

CONCLUSIONS

Current technical skills-assessment tools in trauma and orthopaedic surgery are largely procedure-specific and limited to research use in the simulated environment. An objective technical skills-assessment tool that is suitable for use in the live operative theater requires development and validation, to ensure proper competency-based assessment of surgical performance and readiness for unsupervised clinical practice.

CLINICAL RELEVANCE

Trainers and trainees can gain further insight into the technical skills assessment tools that they use in practice through the utility evidence provided.

Surgery Performed by Supervised Trainees Does not Affect Outcome in First Metatarsophalangeal Joint (MTPJ) Arthrodesis

First metatarsophalangeal joint (MTPJ) arthrodesis is an index procedure for orthopedic trainees in the United Kingdom. There is an absence of evidence as to the outcome of surgery when performed by supervised trainees. The aim of this study is to compare the incidence of fusion, complication rate and radiographic outcomes in first MTPJ arthrodesis performed by supervised orthopedic trainees with consultants. This is a retrospective cohort study of 117 patients undergoing first MTPJ arthrodesis from August 2015 to December 2017 in our institute for hallux valgus (HV) deformity with first MPTJ arthrosis or for hallux rigidus. Patients were followed to a minimum of 1-year postsurgery and were given an open appointment thereafter. In the hallux rigidus group, there was no significant difference between the complication rate (p = .477), incidence of fusion (p = .663), postoperatively HVA (p = .763), and postoperative intermetatarsal angle (p = .539) between trainees and consultants. There was a significant difference in mean tourniquet time (p = .563) between trainees and consultants. In the hallux valgus group, there was no significant difference in the complication rate (p = .519), incidence of fusion (p = .786), tourniquet time (p = .075), postoperative HVA (p = .423), and postoperative intermetatarsal angle (p = .143) between the trainees and consultants. This is the first MTPJ fusion series that demonstrates good results can be achieved by supervised trainees performing the procedure. This would suggest that first MTPJ arthrodesis remains an opportunity for trainees to continue learning to perform under supervision whilst ensuring no change in outcome for the patient.

Current surgical practice for septic arthritis of the knee in the United States

For septic arthritis of the knee, we attempted to determine: the preferred surgical technique in the United-States (US), the believed "gold-standard" treatment among others. This was performed by an electronic-survey distributed to all academic orthopaedic faculty throughout the US. The preferred method was arthroscopy (69.8%). Arthroscopy is believed to be the gold-standard in 27.0%, arthrotomy in 29.4%, while 43.5% believe no gold-standard exists. In conclusion the majority of surgeons prefer arthroscopy when managing a native, septic knee in an adult patient. However, there is no national consensus on a gold-standard treatment or the role of synovectomy.

ACGME Case Volume Minimums Decrease the Number of Shoulder and Knee Arthroscopies Performed by Residents

Purpose

The purpose of this study was to examine how the implementation of Accreditation Council of Graduate Medical Education (ACGME) case minimums in 2013 has affected the number of shoulder and knee arthroscopies performed by orthopaedic surgery residents during their clinical training.

Methods

The ACGME orthopaedic surgery case log data from graduation years 2007 to 2019 was used to evaluate the number of shoulder and knee arthroscopies performed. The mean and median number of cases performed per resident were compared for the years prior to implementation of the ACGME case minimum (2007-2012) and after (2013-2019).

Results

The ACGME orthopaedic surgery case minimums resulted in a significant decrease in the mean number of shoulder and knee arthroscopies performed. The mean number of shoulder arthroscopies performed in the years before and after the case minimum requirement were 109.8 and 82.0 (P = .025), respectively. The mean number of knee arthroscopies performed in the years before and after the case minimum requirement were 178.6 and 124.8 (P = .006), respectively. Residents in the tenth percentile of cases performed still met the required ACGME case minimums each year. The mean total of all cases performed in the years before and after the case minimum requirement were 2045.5 and 1699.3 (P = .038), respectively.

Conclusions

The number of shoulder and knee arthroscopies performed by orthopaedic surgery residents significantly decreased after the implementation of ACGME case minimums, which may be due to underreporting of cases.

Clinical Relevance

This study may demonstrate the effect of the implementation of the ACGME case minimums on the number of shoulder and knee arthroscopies performed by orthopaedic surgery residents.

Impact of cadaveric practices on arthroscopic knee training for Orthopaedic Surgery and Traumatology residents

BACKGROUND AND OBJECTIVES

There currently is a considerable variability in the content of training programs and courses for orthopaedic surgeons in training. Besides this, there is still no consensus regarding which is the most effective methodology for improving learning of the technique or for its qualitative evaluation. A study was carried out to assess the influence of carrying out a cadaveric practical course on the acquisition of arthroscopic technical skills in fifth-year residents.

MATERIAL AND METHODS

A knee arthroscopy cadaveric course was performed in which 48 students participated. Different technical aspects during a knee arthroscopy, as well as the residents skill and the execution time were evaluated before and after the course using a specific questionnaire.

RESULTS

There was a significant statistic improvement between the initial and final evaluation, increasing from 11,7 till 17,1 (maximum 20) points. Also, a significant improvement in all the parameters analysed was observed; the identification of anatomic structures (p = 0.001), the palpation of specific structures (p = 0.032), and the time necessary for the arthroscopic knee check (p = 0.015).

CONCLUSIONS

A knee arthroscopy cadaveric course improves significantly the technical skills of surgeons in training.

Innovation in Orthopaedic Surgery Education: Novel Tools for Modern Times

Orthopaedic residency training has and will continue to evolve with a wide variety of changes. Hands-on surgical simulation "boot camps," computerized simulation of surgical process, and even virtual reality simulators, all can help trainees acquire surgical experience without compromising patient care. Low-cost training modules help remedy the rising costs associated with teaching complex orthopaedic surgery skills. Motion tracking and checklists help refine standardization of assessment. As technology and healthcare systems continue to grow, we encourage training programs to keep pace by considering engagement of these tools.

Global Rating Scales for the Assessment of Arthroscopic Surgical Skills: A Systematic Review

PURPOSE

To evaluate whether sufficient validity and reliability evidence exists to support the use of global rating scales (GRS) as evaluation tools in both formative assessment and competency assessment of arthroscopic procedures.

METHODS

A search of PubMed, Embase, and Scopus was conducted for articles published between 1990 and 2018. Studies reporting measures of validity and reliability of GRS relating to arthroscopic skills were included. Procedural checklists and other assessment tools were excluded.

RESULTS

A total of 39 articles met the inclusion criteria. In total, 7 de novo GRS specific for arthroscopic education and 3 pre-existing GRS repurposed 4 times for arthroscopic education were identified in the literature. The 11 GRS were used to assess 1175 surgeons 3890 times. Three GRS tools explicitly defined an arbitrary minimum competency threshold, 6 of 11 tools demonstrated construct validity-the ability to significantly discriminate between groups of differing experience-and 5 of 11 tools assessed inter-rater reliability, but only the Arthroscopic Surgical Skills Evaluation Tool demonstrated excellent inter-rater reliability. The Arthroscopic Surgical Skills Evaluation Tool was validated by 16 articles for a total of 537 surgeons for hip, knee, shoulder, and ankle arthroscopy in both simulated and clinical environments but was found to be invalid in wrist arthroscopy. The Basic Arthroscopic Knee Skill Scoring System was validated by 15 articles for a total of 497 surgeons for knee, hip, and shoulder in both clinical and simulated environments. The remaining 9 GRS were validated by 2 or fewer studies.

CONCLUSIONS

Overall, GRS have contributed to training, feedback, and formative assessment practices. The GRS reviewed demonstrate both construct and concurrent validity as well as reliability in multiple arthroscopic procedures in multiple joints. Currently, there is sufficient evidence to use GRS as a feedback tool. However, there is insufficient evidence for its use in high-stakes examinations or as a minimum competency assessment.

LEVEL OF EVIDENCE

Level III, systematic review of level I to III studies.

Quantifying the Opportunity Cost of Resident Involvement in Academic Orthopaedic Sports Medicine: A Matched-Pair Analysis

PURPOSE

To quantify the cost of resident involvement in academic sports medicine by examining differences in operative time, relative value units (RVUs) per case, and RVUs per hour between attending-only cases and cases with resident involvement.

METHODS

A retrospective analysis of common sports medicine procedures identified by Current Procedural Terminology code was performed using data from the American College of Surgeons National Surgical Quality Improvement Program database from 2006 to 2015. Matched cohorts were generated based on demographic variables, comorbidities, preoperative laboratory values, and surgical procedures. Bivariate analysis examined mean differences in operative time, RVUs per case, and RVUs per hour between attending-only cases and cases with resident involvement. A cost analysis was performed to quantify differences in RVUs generated per hour in terms of dollars per case.

RESULTS

A total of 14,840 attending-only cases and 2,230 resident-involved cases were used to generate 2 matched cohorts (N = 4,460). Resident cases had greater mean operative times than attending-only cases, with operative time increasing as residents became more senior (P < .01). Residents participated in cases with larger mean RVUs per case (P < .01). Cases with lone attendings showed greater RVUs per hour (P < .01). The cost of resident involvement increased nearly 8-fold from postgraduate year 1 to postgraduate year 6 residents ($25.70 vs $200.07).

CONCLUSIONS

In academic sports medicine, the involvement of resident physicians increases operative time. The associated decrease in attending physician efficiency in RVUs per hour equates to an average cost per case of $159.18, with costs increasing as residents become more senior.

LEVEL OF EVIDENCE

Level III, retrospective comparative trial.

Three days of training with a low-fidelity arthroscopy triangulation simulator box improves task performance in a virtual reality high-fidelity virtual knee arthroscopy simulator

PURPOSE

The aim of this work is to evaluate the effectiveness of training with the low-fidelity ArthroBox^® regarding performance of different basic arthroscopy tasks using a validated high-fidelity virtual reality simulator of the knee.

METHODS

Nineteen volunteers (14 females and 5 males) without any previous experience in arthroscopy were randomly assigned either to the ArthroBox^® training group (n =10) or the non-training group (n =9). The training group underwent a supervised ArthroBox ^® training consisting of a daily 60-min session for three consecutive days. Both groups completed the basic and the final assessment using a validated virtual reality-based passive haptic knee arthroscopy simulator (ArthroS, VirtaMed™). The following three factors were measured in different exercises (explained in "Materials and methods"): amount of time to finish the task, length of camera and scope path within the joint. Furthermore, the volunteers' demographics (age, sex, dexterity, video game experience, sport activities and profession) was assessed but showed no differences between the groups.

RESULTS

There were no significant differences between the training and non-training group regarding the above-mentioned demographic factors. However, the training group showed significant improvement from baseline to follow-up in most activities (e.g. task performance time in seconds, intra-articular camera and grasp distance in centimetres; see Table 1) in comparison to the non-training group.

CONCLUSIONS

The results from this study demonstrate that training for three consecutive days using a portable and versatile low-fidelity simulator significantly improves arthroscopy performance when using a validated high-fidelity virtual knee simulator. Arthroscopic triangulation training outside the operating theatre with a portable, low-cost simulator has proven to be a valuable educational tool to improve the arthroscopic skills of trainee surgeons.

LEVEL OF EVIDENCE

Diagnostic study, Level II.

Arthroscopic Rotator Cuff Repair Metrics: Establishing Face, Content, and Construct Validity in a Cadaveric Model

PURPOSE

To create and determine face validity and content validity of arthroscopic rotator cuff repair (ARCR) performance metrics, to confirm construct validity of the metrics coupled with a cadaveric shoulder, and to establish a performance benchmark for the procedure on a cadaveric shoulder.

METHODS

Five experienced arthroscopic shoulder surgeons created step, error, and sentinel error metrics for an ARCR. Fourteen shoulder arthroscopy faculty members from the Arthroscopy Association of North America formed the modified Delphi panel to assess face and content validity. Eight Arthroscopy Association of North America shoulder arthroscopy faculty members (experienced group) were compared with 9 postgraduate year 4 or 5 orthopaedic residents (novice group) in their ability to perform an ARCR. Instructions were given to perform a diagnostic arthroscopy and a 2-anchor, 4-simple suture repair of a 2-cm supraspinatus tear. The procedure was videotaped in its entirety and independently scored in blinded fashion by trained, paired reviewers.

RESULTS

Delphi panel consensus for 42 steps and 66 potential errors was obtained. Overall performance assessment showed a mean inter-rater reliability of 0.93. Novice surgeons completed 17% fewer steps (32.1 vs 37.5, P = .001) and enacted 2.5 times more errors than the experienced group (6.21 vs 2.5, P = .012). Fifty percent of the experienced group members and none of the novice group members achieved the proficiency benchmark of a minimum of 37 steps completed with 3 or fewer errors.

CONCLUSIONS

Face validity and content validity for the ARCR metrics, along with construct validity for the metrics and cadaveric shoulder, were verified. A proficiency benchmark was established based on the mean performance of an experienced group of arthroscopic shoulder surgeons.

CLINICAL RELEVANCE

Validated procedural metrics combined with the use of a cadaveric shoulder can be used to accurately assess the performance of an ARCR.

Editorial Commentary: Harnessing Surgical Simulation-Saddle up for a Long and Rigorous Ride!

Arthroscopic simulation tools will assist in the training of skills that are transferrable to the operating room. Success in that endeavor requires the following: relevant outcome metrics rather than surrogate measures of performance, evaluation tools that enable scoring of step and error metrics rather than global rating scales, construct validation of simulators and metrics based on actual performance, development of simulators that not only assess performance but also train additional skills and provide feedback, and a focus first on effective training as opposed to high-fidelity realism.

The Learning Curve for Biplane Medial Open Wedge High Tibial Osteotomy in 100 Consecutive Cases Assessed Using the Cumulative Summation Method

Purpose

The purpose of this study was to investigate whether surgical experience could improve surgical competency in medial open wedge high tibial osteotomy (MOWHTO).

Materials and Methods

One hundred consecutive cases of MOWHTO were performed with preoperative planning using the Miniaci method. Surgical errors were defined as under- or overcorrection, excessive posterior slope change, or the presence of a lateral hinge fracture. Each of these treatment failures was separately evaluated using the cumulative summation test for learning curve (LC-CUSUM).

Results

The LC-CUSUM showed competency in prevention of undercorrection, excessive posterior slope change, and lateral hinge fracture after 27, 47, and 42 procedures, respectively. However, the LC-CUSUM did not signal achievement of competency in prevention of overcorrection after 100 procedures. Furthermore, the failure rate for overcorrection showed an increasing tendency as surgical experience increased.

Conclusions

Surgical experience may improve the surgeon’s competency in prevention of undercorrection, excessive posterior slope change, and lateral hinge fracture. However, it may not help reduce the incidence of overcorrection even after performance of 100 cases of MOWHTO over a period of 6 years.

Assessment of Technical Skills Competence in the Operating Room: A Systematic and Scoping Review

PURPOSE

While academic accreditation bodies continue to promote competency-based medical education (CBME), the feasibility of conducting regular CBME assessments remains challenging. The purpose of this study was to identify evidence pertaining to the practical application of assessments that aim to measure technical competence for surgical trainees in a nonsimulated, operative setting.

METHOD

In August 2016, the authors systematically searched Medline, Embase, and the Cochrane Database of Systematic Reviews for English-language, peer-reviewed articles published in or after 1996. The title, abstract, and full text of identified articles were screened. Data regarding study characteristics, psychometric and measurement properties, implementation of assessment, competency definitions, and faculty training were extracted. The findings from the systematic review were supplemented by a scoping review to identify key strategies related to faculty uptake and implementation of CBME assessments.

RESULTS

A total of 32 studies were included. The majority of studies reported reasonable scores of interrater reliability and internal consistency. Seven articles identified minimum scores required to establish competence. Twenty-five articles mentioned faculty training. Many of the faculty training interventions focused on timely completion of assessments or scale calibration.

CONCLUSIONS

There are a number of diverse tools used to assess competence for intraoperative technical skills and a lack of consensus regarding the definition of technical competence within and across surgical specialties. Further work is required to identify when and how often trainees should be assessed and to identify strategies to train faculty to ensure timely and accurate assessment.

Value of the cumulative sum test for the assessment of a learning curve: Application to the introduction of patient-specific instrumentation for total knee arthroplasty in an academic department

BACKGROUND

The purpose of the study was to use the cumulative summation (CUSUM) test to assess the learning curve during the introduction of a new surgical technique (patient-specific instrumentation) in total knee arthroplasty (TKA) in an academic department.

METHODS

The first 50TKAs operated on at an academic department using patient-specific templates (PSTs) were scheduled to enter the study. All patients had a preoperative computed tomography scan evaluation to plan bone resections. The PSTs were positioned intraoperatively according to the best-fit technique and their three-dimensional orientation was recorded by a navigation system. The position of the femur and tibia PST was compared to the planned position for four items for each component: coronal and sagittal orientation, medial and lateral height of resection. Items were summarized to obtain knee, femur and tibia PST scores, respectively. These scores were plotted according to chronological order and included in a CUSUM analysis. The tested hypothesis was that the PST process for TKA was immediately under control after its introduction.

RESULTS

CUSUM test showed that positioning of the PST significantly differed from the target throughout the study. There was a significant difference between all scores and the maximal score. No case obtained the maximal score of eight points. The study was interrupted after 20 cases because of this negative evaluation.

CONCLUSION

The CUSUM test is effective in monitoring the learning curve when introducing a new surgical procedure. Introducing PST for TKA in an academic department may be associated with a long-lasting learning curve. The study was registered on the clinical.gov website (Identifier NCT02429245).

Use of an Objective Structured Assessment of Technical Skill After a Sports Medicine Rotation

PURPOSE

The purpose of this study was to determine if the use of an Objective Structured Assessment of Technical skill (OSATS), using dry models, would be a valid method of assessing residents' ability to perform sports medicine procedures after training in a competency-based model.

METHODS

Over 18 months, 27 residents (19 junior [postgraduate year (PGY) 1-3] and 8 senior [PGY 4-5]) sat the OSATS after their rotation, in addition to 14 sports medicine staff and fellows. Each resident was provided a list of 10 procedures in which they were expected to show competence. At the end of the rotation, each resident undertook an OSATS composed of 6 stations sampled from the 10 procedures using dry models-faculty used the Arthroscopic Surgical Skill Evaluation Tool (ASSET), task-specific checklists, as well as an overall 5-point global rating scale (GRS) to score each resident. Each procedure was videotaped for blinded review.

RESULTS

The overall reliability of the OSATS (0.9) and the inter-rater reliability (0.9) were both high. A significant difference by year in training was seen for the overall GRS, the total ASSET score, and the total checklist score, as well as for each technical procedure (P < .001). Further analysis revealed a significant difference in the total ASSET score between junior (mean 18.4, 95% confidence interval [CI] 16.8 to 19.9) and senior residents (24.2, 95% CI 22.7 to 25.6), senior residents and fellows (30.1, 95% CI 28.2 to 31.9), as well as between fellows and faculty (37, 95% CI 36.1 to 27.8) (P < .05).

CONCLUSIONS

The results of this study show that an OSATS using dry models shows evidence of validity when used to assess performance of technical procedures after a sports medicine rotation. However, junior residents were not able to perform as well as senior residents, suggesting that overall surgical experience is as important as intensive teaching.

CLINICAL RELEVANCE

As postgraduate medical training shifts to a competency-based model, methods of assessing performance of technical procedures become necessary.

Cumulative sum: An individualized proficiency metric for laparoscopic fundamentals

BACKGROUND

A reliable metric of technical proficiency is indispensable to the training of fellows and residents. The purpose of this study was to determine whether cumulative sum (Cusum) has predictive validity in laparoscopic training. We hypothesized that Cusum would be a better predictor of technical ability in fundamentals of laparoscopic surgery (FLS) than traditional practice volume metrics.

METHODS

Twenty medical students were recruited to practice three FLS tasks: peg transfer, circle cut, and intracorporeal knot tie. Up to 7hours of self-directed practice was allotted to each participant. Practice attempts were scored by standard FLS criteria and monitored via Cusum. Each participant's terminal Cusum performance was analyzed retrospectively. Posttests were conducted by faculty blinded to practice performance.

RESULTS

Eighteen participants completed the study (90%). Median adjusted posttest scores were 102.3, 84.1, and 78.6 for peg transfer, circle cut, and knot tie, respectively. For the knot tie task, participants who exceeded the Cusum decision interval during their final practice attempts performed significantly higher on posttesting (81.2 vs 71.5, p=0.015). Knot tie terminal Cusum score was positively associated with posttest performance after adjusting for practice volume (p=0.031). Total practice volume and practice time were not significantly associated with posttest performance for any FLS task.

CONCLUSION

Cusum score is a more valid representation of FLS proficiency than practice volume or practice time. Incorporating Cusum in a clinical setting may promote more efficient allocation of time resources and operative volume.

Effectiveness of Virtual Reality Training in Orthopaedic Surgery

PURPOSE

The purpose of this study was to conduct a systematic review to determine the effectiveness of virtual reality (VR) training in orthopaedic surgery.

METHODS

A comprehensive systematic review was performed of articles of VR training in orthopaedic surgery published up to November 2014 from MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials databases.

RESULTS

We included 10 relevant trials of 91 identified articles, which all reported on training in arthroscopic surgery (shoulder, n = 5; knee, n = 4; undefined, n = 1). A total of 303 participants were involved. Assessment after training was made on a simulator in 9 of the 10 studies, and in one study it took place in the operating room (OR) on a real patient. A total of 32 different outcomes were extracted; 29 of them were about skills assessment. None involved a patient-related outcome. One study focused on anatomic learning, and the other evaluated technical task performance before and after training on a VR simulator. Five studies established construct validity. Three studies reported a statistically significant improvement in technical skills after training on a VR simulator.

CONCLUSIONS

VR training leads to an improvement of technical skills in orthopaedic surgery. Before its widespread use, additional trials are needed to clarify the transfer of VR training to the OR.

LEVEL OF EVIDENCE

Systematic review of Level I through Level IV studies.

Adaptive simulation training using cumulative sum: a randomized prospective trial

BACKGROUND

Cumulative sum (Cusum) is a novel tool that can facilitate adaptive, individualized training curricula. The purpose of this study was to use Cusum to streamline simulation-based training.

METHODS

Preclinical medical students were randomized to Cusum or control arms and practiced suturing, intubation, and central venous catheterization in simulation. Control participants practiced between 8 and 9 hours each. Cusum participants practiced until Cusum proficient in all tasks. Group comparisons of blinded post-test evaluations were performed using Wilcoxon rank sum.

RESULTS

Forty-eight participants completed the study. Average post-test composite score was 92.1% for Cusum and 93.5% for control (P = .71). Cusum participants practiced 19% fewer hours than control group participants (7.12 vs 8.75 hours, P < .001). Cusum detected proficiency relapses during practice among 7 (29%) participants for suturing and 10 (40%) for intubation.

CONCLUSIONS

In this comparison between adaptive and volume-based curricula in surgical training, Cusum promoted more efficient time utilization while maintaining excellent results.

Expert Surgeons Can Be Distinguished From Trainees, and Surgical Proficiency Can Be Defined, Using Validated Metrics and Shoulder Models

Experts at performing arthroscopic shoulder Bankart procedures can be distinguished from surgical trainees using validated metrics and a cadaveric or simulator model. The combination of performance metrics plus models generates a tool that can be used to precisely and accurately define a performance threshold and assess whether or not a surgeon is proficient at performing a procedure. A tool that can be used to judge surgical expertise has implications for training and credentialing. Experienced surgeons make fewer mistakes and are faster than novices.

Editorial Commentary: Arthroscopic Training and Proficiency Assessment Are Hot Topics

Methods to measure operative skill are improving in quality. Arthroscopic surgical training and arthroscopic proficiency assessment are hot topics.

Learning the Language of Copernicus

The Copernicus Initiative was a bold and important undertaking by the Arthroscopy Association of North America to help further our learning the art of arthroscopy in a controlled setting. Understanding arthroscopic learning, training, and simulation research requires mastery of a lexicon of new terms, which AANA Copernicus researchers define in a glossary. Learning requires practice to develop proficiency. Developing new ability is a rewarding challenge. Metrics may be used to quantitatively measure objective performance, and is a key component of the Copernicus Initiative. A dedicated group of AANA researchers and educators have taken on an important and challenging task to help us improve in the realm of surgical education.