Shoulder Arthroscopy Simulator Training Improves Surgical Procedure Performance: A Controlled Laboratory Study

Arthroscopic Shoulder Simulation May Improve Short-Term Speed, Accuracy, and Efficiency of Surgical Movements in Orthopaedic Residents and Fellows: A Systematic Review

PURPOSE

To systematically review the effectiveness and validity of orthopaedic surgery training using simulation technologies including augmented reality, virtual reality, and/or mixed reality within arthroscopic shoulder surgery.

METHODS

A literature search was conducted of the EMBASE and PubMed databases from inception to January 2024 per the 2020 Preferred Reporting Items for Systematic Review and Meta-Analysis guidelines. Included studies described arthroscopic shoulder surgery simulation training among orthopaedic surgery trainees. Exclusion criteria included studies assessing nonarthroscopic and nonshoulder simulations, non-English-language studies, case reports, animal studies, studies with overlapping cohorts, and review articles. Simulation characteristics, performance measurements, and validity assessed were extracted. The Cochrane risk-of-bias tool and Newcastle-Ottawa Scale assessed study quality. Simulation type, validation type, and simulation outcomes were assessed.

RESULTS

A total of 15 included articles, published from 2011 to 2021, evaluated 421 residents or fellows and 17 medical students. Virtual reality was used in 40% of studies and mixed reality in 60%. The most common outcomes assessed were time to completion (80%), visualizing and probing task performance (60%), and the Arthroscopic Surgery Skill Evaluation Tool (33.3%). Construct validity was assessed in 46.7% of studies, transfer validity in 26.7%, face validity in 20%, and content validity in 6.7%. Three studies demonstrated improved performance in those undergoing simulation training compared with nonsimulation groups. Two studies (13.3%) demonstrated improved time-to-task completion and decreased camera distance traveled when using simulation training. One study demonstrated that postgraduate year 1 and postgraduate year 5 residents derived the greatest benefit from simulation training.

CONCLUSIONS

Arthroscopic shoulder simulation training may benefit the surgical skills of orthopaedic residents of all levels of experience as measured by time-to-completion, accuracy, and efficiency of surgical movements. Simulation training exhibits differences in operative time between more- and less-experienced orthopaedic surgeons and trainees. Virtual reality simulation training may result in more-efficient orthopaedic surgical techniques.

LEVEL OF EVIDENCE

Level III, systematic review of level I-III studies.

Arthroscopic Shoulder Simulation Studies Reveal Improvements in Performance Metrics Without Proven Transferability to the Operating Room: A Systematic Review

PURPOSE

To evaluate the use of shoulder arthroscopic simulation in orthopaedic surgery trainees.

METHODS

A literature search was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, using PubMed, Medline (Ovid), and EMBASE library databases. Inclusion criteria were experimental studies reporting pre- and post-test results of shoulder arthroscopic simulation in orthopaedic trainees (studies reporting results of comparison between groups not within the groups were excluded). Participant demographics, type of simulator training, simulator tasks assessed, and performance outcome measures were systematically reviewed. Each performance outcome measure was graphically represented in a Forest plot with point estimates of the incidence of performance outcomes with corresponding 95% confidence intervals and I2.

RESULTS

Fifteen studies met inclusion criteria with a total of 353 participants. The most common procedures simulated were diagnostic shoulder arthroscopy (n = 9 [60%]), arthroscopic Bankart repairs (n = 3 [20%]), and rotator cuff repairs (n = 2 [13%]). Simulations primarily used virtual reality (60%) and benchtop models (40%). The primary outcomes measured were time to task completion and Arthroscopic Surgical Skill Evaluation Tool scores. Time to task completion improved significantly with training (range 13-439 seconds pretest to 8-253.29 seconds post-test), with substantial heterogeneity across studies (I2 = 87%). ASSET scores improved in 60% of the studies (ranging from 14-20.9 pretest to 17.9-28.5 post-test), with low heterogeneity (I2 = 20%). In addition, both camera and probe distances decreased after simulation use, whereas the 14-point anatomic checklist showed no pre- to post-test differences.

CONCLUSIONS

Arthroscopic simulation training benefits technical skills in shoulder arthroscopy, but the quality, assessment, and validity of these protocols vary. The translation of simulation training into the operating room has yet to be conclusively demonstrated.

LEVEL OF EVIDENCE

Level IV, systematic review of Level I-IV studies.

Virtual and Augmented Reality Simulators Show Intraoperative, Surgical Training, and Athletic Training Applications: A Scoping Review

PURPOSE

To review published literature to identify and evaluate the effect of virtual reality (complete immersion) and augmented reality (overlay of digital information onto the physical world) simulators on intraoperative use for orthopaedic surgeons, orthopaedic surgical education, and athletic training.

METHODS

A systematic review was performed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to identify studies, published since 2014, that evaluated the role of augmented/virtual reality on intraoperative use for orthopaedic surgeons, orthopaedic surgical education, and athletic training.

RESULTS

Virtual reality (VR) simulators provide 3-dimensional graphical simulation of the physical world, and augmented reality (AR) simulators overlay digital information onto the physical world. Simulators can include interactive features (i.e., replication of intraoperative bleeding), haptic feedback, and unrestricted task repetition, and they can record, compare, and analyze performance while being easily accessible and eliminating the need for the presence of a mentor or coach. Four studies reported on VR for intraoperative use, 47 studies on surgical education, and 10 studies on athletic training. Two studies revealed the advantages of using VR simulation during intraoperative procedures, specifically showcasing its benefits for elbow arthroscopy, while 2 studies demonstrated similar positive outcomes for hip arthroscopy. Seventeen studies demonstrated that a VR simulator could be a beneficial tool to assist in surgical education for the knee, while 12 studies found that VR simulation is a valuable tool for aiding in surgical education of shoulder arthroscopy. Ten studies demonstrated that VR simulation improves skills in the operating room. Three studies revealed that individuals with more experience exhibit superior performance on these simulators compared to those with less experience. In the realm of athletic training, 10 studies showcased the potential of VR simulation to play a significant role in athletic performance and injury rehabilitation.

CONCLUSIONS

VR simulation shows benefits in the operating room, is a valuable tool for surgical education resulting in improved skills, and can be used to enhance athletic performance and injury rehabilitation.

CLINICAL RELEVANCE

Understanding that VR simulators can improve surgical outcomes, surgical skill training, and athletic training and rehabilitation could facilitate development and adoption of this advanced technology.

Validity Evidence for Procedure-specific Competence Assessment Tools in Orthopaedic Surgery: A Scoping Review

INTRODUCTION

Competency-based training requires frequent assessment of residents' skills to determine clinical competence. This study reviews existing literature on procedure-specific competence assessment tools in orthopaedic surgery.

METHODS

A systematic search of eight databases up to May 2023 was conducted. Two reviewers independently assessed validity evidence and educational utility of each assessment tool and evaluated studies' methodological quality.

RESULTS

Database searching identified 2,556 unique studies for title and abstract screening. Full texts of 290 studies were reviewed; 17 studies met the inclusion criteria. Bibliography review identified another five studies, totaling 22 studies examining 24 assessment tools included in the analysis. These tools assessed various orthopaedic surgery procedures within trauma, sports medicine, spine, and upper extremity. Overall validity evidence was low across all studies, and was lowest for consequences and highest for content. Methodological quality of studies was moderate. Educational utility assessment was not explicitly done for most tools.

DISCUSSION

The paucity of current procedure-specific assessment tools in orthopaedic surgery lacks the validity evidence required to be used reliably in high-stake summative assessments. Study strengths include robust methodology and use of an evidence-based validity evidence framework. Poor-quality existing evidence is a limitation and highlights the need for evidence-based tools across more subspecialties.

Virtual Reality and Surgical Simulation Training for Orthopaedic Surgery Residents: A Qualitative Assessment of Trainee Perspectives

Background

The demonstrated benefits of virtual reality (VR) in orthopaedic surgical training are numerous. However, it is relatively unknown how best to implement VR into an already established orthopaedic resident education curriculum and how trainees will engage and use these technologies longitudinally.

Methods

This was an exploratory, qualitative research study performed in accordance with Consolidated Criteria for Reporting Qualitative Research guidelines. Orthopaedic surgery residents at a single institution were recruited during the 2022 to 2023 academic year. Semistructured interviews were conducted. Data were analyzed through grounded theory methodology, beginning with open coding, followed by axial coding, and concluding with selective coding that describes orthopaedic surgery residents' current perceptions of VR as a training tool.

Results

Six residents participated in interviews before thematic saturation was achieved. Average interview length was 13:27 (±2:59) minutes. Residents felt that currently, VR is most useful for interns and junior residents as an educational adjunct for learning anatomy, surgical exposures, and the steps of a procedure in a risk- and judgment-free arena. There seems to be a "ceiling effect" with VR given current technological limitations, and residents remarked that there is an associated "opportunity cost" with using VR technology. Some residents may find it more time-efficient to study texts, videos, or surgical guides rather than use VR. Cost (limited number of headsets) and technological barriers (i.e., hardware, software, and Wi-Fi issues) were some of the described barriers to VR utilization. Residents felt that there needs to be dedicated technological support to help with these issues. At this time, given these limitations of VR, many preferred VR as an optional educational adjunct rather than as a required curricular tool or assessment of surgical competency.

Conclusions

There is current utility for VR in orthopaedic surgical training. Future technological advances may make VR more central to resident education. This study describes resident perceptions about the technology and best use practices for the technology.

Level of Evidence

Qualitative Study, Level V Evidence.

Virtual reality simulation training improve diagnostic knee arthroscopy and meniscectomy skills: a prospective transfer validity study

PURPOSE

Limited data exist on the actual transfer of skills learned using a virtual reality (VR) simulator for arthroscopy training because studies mainly focused on VR performance improvement and not on transfer to real word (transfer validity). The purpose of this single-blinded, controlled trial was to objectively investigate transfer validity in the context of initial knee arthroscopy training.

METHODS

For this study, 36 junior resident orthopaedic surgeons (postgraduate year one and year two) without prior experience in arthroscopic surgery were enrolled to receive standard knee arthroscopy surgery training (NON-VR group) or standard training plus training on a hybrid virtual reality knee arthroscopy simulator (1 h/month) (VR group). At inclusion, all participants completed a questionnaire on their current arthroscopic technical skills. After 6 months of training, both groups performed three exercises that were evaluated independently by two blinded trainers: i) arthroscopic partial meniscectomy on a bench-top knee simulator; ii) supervised diagnostic knee arthroscopy on a cadaveric knee; and iii) supervised knee partial meniscectomy on a cadaveric knee. Training level was determined with the Arthroscopic Surgical Skill Evaluation Tool (ASSET) score.

RESULTS

Overall, performance (ASSET scores) was better in the VR group than NON-VR group (difference in the global scores: p < 0.001, in bench-top meniscectomy scores: p = 0.03, in diagnostic knee arthroscopy on a cadaveric knee scores: p = 0.04, and in partial meniscectomy on a cadaveric knee scores: p = 0.02). Subgroup analysis by postgraduate year showed that the year-one NON-VR subgroup performed worse than the other subgroups, regardless of the exercise.

CONCLUSION

This study showed the transferability of the technical skills acquired by novice residents on a hybrid virtual reality simulator to the bench-top and cadaveric models. Surgical skill acquired with a VR arthroscopy surgical simulator might safely improve arthroscopy competences in the operating room, also helping to standardise resident training and follow their progress.

LEVEL OF EVIDENCE: 2

Which, how, and what? Using digital tools to train surgical skills; a systematic review and meta-analysis

Background

Digital tools like digital box trainers and VR seem promising in delivering safe and tailored practice opportunities outside of the surgical clinic, yet understanding their efficacy and limitations is essential. This study investigated Which digital tools are available to train surgical skills, How these tools are used, How effective they are, and What skills they are intended to teach.

Methods

Medline, Embase, and Cochrane libraries were systematically reviewed for randomized trials, evaluating digital skill-training tools based on objective outcomes (skills scores and completion time) in surgical residents. Digital tools effectiveness were compared against controls, wet/dry lab training, and other digital tools. Tool and training factors subgroups were analysed, and studies were assessed on their primary outcomes: technical and/or non-technical.

Results

The 33 included studies involved 927 residents and six digital tools; digital box trainers, (immersive) virtual reality (VR) trainers, robot surgery trainers, coaching and feedback, and serious games. Digital tools outperformed controls in skill scores (SMD 1.66 [1.06, 2.25], P < 0.00001, I2 = 83 %) and completion time (SMD -1.05 [-1.72, -0.38], P = 0.0001, I2 = 71 %). There were no significant differences between digital tools and lab training, between tools, or in other subgroups. Only two studies focussed on non-technical skills.

Conclusion

While the efficacy of digital tools in enhancing technical surgical skills is evident - especially for VR-trainers -, there is a lack of evidence regarding non-technical skills, and need to improve methodological robustness of research on new (digital) tools before they are implemented in curricula.

Key message

This study provides critical insight into the increasing presence of digital tools in surgical training, demonstrating their usefulness while identifying current challenges, especially regarding methodological robustness and inattention to non-technical skills.

The number of arthroscopies performed by trainees does not deduce the level of their arthroscopic proficiency

PURPOSE

It is reasonable to question whether the case volume is a suitable proxy for the manual competence of an arthroscopic surgeon. The aim of this study was to evaluate the correlation between the number of arthroscopies previously performed and the arthroscopic skills acquired using a standardized simulator test.

METHODS

A total of 97 resident and early orthopaedic surgeons who participated in arthroscopic simulator training courses were divided into five groups based on their self-reported number of arthroscopic surgeries: (1) none, (2) < 10, (3) 10 to 19, (4) 20 to 39 and (5) 40 to 100. Arthroscopic manual skills were evaluated with a simulator by means of the diagnostic arthroscopy skill score (DASS) before and after training. Seventy-five points out of 100 must be achieved to pass the test.

RESULTS

In the pretest, only three trainees in group 5 passed the arthroscopic skill test, and all other participants failed. Group 5 (57 ± 17 points; n = 17) scored significantly higher than the other groups (group 1: 30 ± 14, n = 20; group 2: 35 ± 14, n = 24; group 3: 35 ± 18, n = 23; and group 4: 33 ± 17, n = 13). After a two-day simulator training, trainees showed a significant increase in performance. In group 5, participants scored 81 ± 17 points, which was significantly higher than the other groups (group 1: 75 ± 16; group 2: 75 ± 14; group 3: 69 ± 15; and group 4: 73 ± 13). While self-reported arthroscopic procedures were n.s. associated with higher log odds of passing the test (p = 0.423), the points scored in the pretest were found to be a good predictor of whether a trainee would pass the test (p < 0.05). A positive correlation was observed between the points scored in the pretest and the posttest (p < 0.05, r = 0.59, r2 = 0.34).

CONCLUSIONS

The number of previously performed arthroscopies is not a reliable indicator of the skills level of orthopaedic residents. A reasonable alternative in the future would be to verify arthroscopic proficiency on the simulator by means of a score as a pass-fail examination.

LEVEL OF EVIDENCE

III.

Current status of virtual reality simulation education for orthopedic residents: the need for a change in focus

Introduction

Advances in technology are changing surgical education. Simulation provides an important adjunct to operative experience. This pedagogy has arguably become more important in light of the COVID-19 pandemic, with resultant reduction in operative exposure for trainees. Virtual reality (VR) simulators may provide significant contribution to experiential learning; however, much of the investigative focus to date has, correctly, been on establishing validity evidence for these constructs. The aim of this work was to perform a scoping review to assess the current status of VR simulation education to determine curricular development efforts for orthopedic residents.

Methods

With a trained medical librarian, searches of PubMed, EMBASE, and Web of Science were conducted for all articles in the last 10 years (September 2011-September 2021). Controlled vocabulary Medical Subject Headings (MeSH) terms and natural language developed with subject matter experts describing virtual reality or VR simulation and orthopedic training were used. Two trained reviewers evaluated all abstracts for inclusion. Exclusion criteria were all articles that did not assess VR simulation education involving orthopedic residents. Data were extracted from the included full-text articles including: study design, type of participants, type of VR simulation, simulated orthopedic skill, type of educational event, learner assessment including Kirkpatrick's level, assessment of quality using the Medical Education Research Study Quality Instrument (MERSQI), and level of effectiveness (LoE).

Results

Initial search identified 1,394 articles, of which 61 were included in the final qualitative synthesis. The majority (54%) were published in 2019- 2021, 49% in Europe. The commonest VR simulator was ArthroS (23%) and the commonest simulated skill was knee arthroscopy (33%). The majority of studies (70%) focused on simulator validation. Twenty-three studies described an educational module or curriculum, and of the 21 (34%) educational modules, 43% were one-off events. Most modules (18/21, 86%) assessed learners at Kirkpatrick level 2. With regard to methodological quality, 44% of studies had MERSQI 11.5-15 and 89% of studies had LoE of 2. Two studies had LoE of 3.

Conclusion

Current literature pertaining to VR training for orthopedic residents is focused on establishing validity and rarely forms part of a curriculum. Where the focus is education, the majority are discrete educational modules and do not teach a comprehensive amalgam of orthopedic skills. This suggests focus is needed to embed VR simulation training within formal curricula efforts guided by the work of Kern, and assess the efficacy of these against patient outcomes.

The frequency of assessment tools in arthroscopic training: a systematic review

BACKGROUND

Multiple assessment tools are used in arthroscopic training and play an important role in feedback. However, it is not fully recognized as to the standard way to apply these tools. Our study aimed to investigate the use of assessment tools in arthroscopic training and determine whether there is an optimal way to apply various assessment tools in arthroscopic training.

METHODS

A search was performed using PubMed, Embase and Cochrane Library electronic databases for articles published in English from January 2000 to July 2021. Eligible for inclusion were primary research articles related to using assessment tools for the evaluation of arthroscopic skills and training environments. Studies that focussed only on therapeutic cases, did not report outcome measures of technical skills, or did not mention arthroscopic skills training were excluded.

RESULTS

A total of 28 studies were included for review. Multiple assessment tools were used in arthroscopic training. The most common objective metric was completion time, reported in 21 studies. Technical parameters based on simulator or external equipment, such as instrument path length, hand movement, visual parameters and injury, were also widely used. Subjective assessment tools included checklists and global rating scales (GRS). Among these, the most commonly used GRS was the Arthroscopic Surgical Skill Evaluation Tool (ASSET). Most of the studies combined objective metrics and subjective assessment scales in the evaluation of arthroscopic skill training.

CONCLUSIONS

Overall, both subjective and objective assessment tools can be used as feedback for basic arthroscopic skill training, but there are still differences in the frequency of application in different contexts. Despite this, combined use of subjective and objective assessment tools can be applied to more situations and skills and can be the optimal way for assessment.

LEVEL OF EVIDENCE

Level III, systematic review of level I to III studies. Key messagesBoth subjective and objective assessment tools can be used as feedback for basic arthroscopic skill training.Combined use of subjective and objective assessment tools can be applied to more situations and skills and can be the optimal way for assessment.

Surgical education in the post-COVID era: an EAES DELPHI-study

BACKGROUNDS

To date, it is unclear what the educational response to the restrictions on minimally invasive surgery imposed by the COVID-19 pandemic have been, and how MIS-surgeons see the post-pandemic future of surgical education. Using a modified Delphi-methodology, this study aims to assess the effects of COVID on MIS-training and to develop a consensus on the educational response to the pandemic.

METHODS

A three-part Delphi study was performed among the membership of the European Association of Endoscopic Surgery (EAES). The first survey aimed to survey participants on the educational response in four educational components: training in the operating room (OR), wet lab and dry lab training, assessment and accreditation, and use of digital resources. The second and third survey aimed to formulate and achieve consensus on statements on, and resources in, response to the pandemic and in post-pandemic MIS surgery.

RESULTS

Over 247 EAES members participated in the three rounds of this Delphi survey. MIS-training decreased by 35.6-55.6%, alternatives were introduced in 14.7-32.2% of respondents, and these alternatives compensated for 32.2-43.2% of missed training. OR-training and assessments were most often affected due to the cancellation of elective cases (80.7%, and 73.8% affected, respectively). Consensus was achieved on 13 statements. Although digital resources were deemed valuable alternatives for OR-training and skills assessments, face-to-face resources were preferred. Videos and hands-on training-wet labs, dry labs, and virtual reality (VR) simulation-were the best appreciated resources.

CONCLUSIONS

COVID-19 has severely affected surgical training opportunities for minimally invasive surgery. Face-to-face training remains the preferred training method, although digital and remote training resources are believed to be valuable additions to the training palette. Organizations such as the EAES are encouraged to support surgical educators in implementing these resources. Insights from this Delphi can guide (inter)national governing training bodies and hospitals in shaping surgical resident curricula in post pandemic times.

Use of a Dry Surgical Simulator Improves Orthopaedic Residents’ Competency and Technical Skills for Arthroscopic Rotator Cuff Repair

Purpose

Methods

Results

Clinical Relevance