The Arthroscopic Surgery Skill Evaluation Tool Global Rating Scale is a Valid and Reliable Adjunct Measure of Performance on a Virtual Reality Simulator for Hip Arthroscopy

Can a Low-Fidelity Arthroscopic Simulator Improve Technical Expertise in Performing Temporomandibular Joint Arthroscopy?

BACKGROUND

Challenges in temporomandibular joint (TMJ) arthroscopy training include the cost of operating room time, surgical risks, and the accessibility of cadavers and high-fidelity simulators. A low-fidelity simulator (LFS) was developed for initial TMJ arthroscopy training.

PURPOSE

The aim of this study was to evaluate improvement in TMJ arthroscopic skills after training with an LFS using the Arthroscopic Surgical Skill Evaluation Tool (ASSET) global rating scale.

STUDY DESIGN, SETTING, SAMPLE

A prospective randomized controlled study was conducted. Subjects included Oral and Maxillofacial Surgery residents in postgraduate year 1-5 at Thomas Jefferson University. Chief residents actively performing TMJ arthroscopy were excluded.

PREDICTOR VARIABLE

The predictor variable was participation in LFS training. Subjects were randomly assigned to the LFS training or no LFS training group.

MAIN OUTCOME VARIABLES

The primary outcome variable was simulated surgical skill measured by total ASSET score (maximum score of 35). The secondary outcome variables were simulated surgical skill measured by each ASSET domain score and time to completion.

COVARIATES

The covariates included sex, age, handedness, postgraduate year, endoscopic experience, open TMJ surgery experience, and experience with musical instruments and sports.

ANALYSES

Data analyses included paired T-tests to determine changes in outcome variables after TMJ arthroscopy training between experimental groups, and ANOVA and χ2 tests to identify associations between covariates. Inter-rater reliability of the blinded examiners was evaluated using Cronbach's alpha correlation. A P-value < .05 was considered significant.

RESULTS

The sample was composed of 10 residents: 5 (100%) males in the no LFS group, 3 (60%) males, and 2 (40%) females in the LFS group (P = .4). There was no significant difference between the groups in pre-TMJ arthroscopy training ASSET scores (P = .3). After training, the total ASSET score improved by 3.40 ± 6.87 in the no LFS group and by 6.27 ± 2.68 in the LFS group (P = .03).

CONCLUSIONS AND RELEVANCE

Study results showed that low-fidelity simulation can improve fundamental arthroscopic skills. Future research should be conducted on a larger scale to further validate this model and show the transfer of skill improvements to cadaver and live patient settings.

Neuromechanisms of simulation-based arthroscopic skills assessment: a fNIRS study

BACKGROUND

The neural mechanisms underlying differences in the performance of simulated arthroscopic skills across various skill levels remain unclear. Our primary objective is to investigate the learning mechanisms of simulated arthroscopic skills using functional near-infrared spectroscopy (fNIRS).

METHODS

We recruited 27 participants, divided into three groups: novices (n = 9), intermediates (n = 9), and experts (n = 9). Participants completed seven arthroscopic tasks on a simulator, including diagnostic navigation, triangulation, grasping stars, diagnostic exploration, meniscectomy, synovial membrane cleaning, and loose body removal. All tasks were videotaped and assessed via the simulator system and the Arthroscopic Surgical Skill Evaluation Tool (ASSET), while cortical activation data were collected using fNIRS. Simulator scores and ASSET scores were analyzed to identify different level of performance of all participants. Brain region activation and functional connectivity (FC) of different types of participants were analyzed from fNIRS data.

RESULTS

Both the expert and intermediate groups scored significantly higher than the novice group (p < 0.001). There were significant differences in ASSET scores between experts and intermediates, experts and novices, and intermediates and novices (p = 0.0047, p < 0.0001, p < 0.0001), with the trend being experts > intermediates > novices. The intermediate group exhibited significantly greater activation in the left primary motor cortex (LPMC) and left prefrontal cortex (LPFC) compared to the novice group (p = 0.0152, p = 0.0021). Compared to experts, the intermediate group demonstrated significantly increased FC between the presupplementary motor area (preSMA) and the right prefrontal cortex (RPFC; p < 0.001). Additionally, the intermediate group showed significantly increased FC between the preSMA and LPFC, RPFC and LPFC, and LPMC and LPFC compared to novices (p = 0.0077, p = 0.0285, p = 0.0446).

CONCLUSION

Cortical activation and functional connectivity reveal varying levels of activation intensity in the PFC, PMC, and preSMA among novices, intermediates, and experts. The intermediate group exhibited the highest activation intensity.

The influence of virtual reality simulation on surgical residents' heart rate during an assessment of arthroscopic technical skills: A prospective, paired observational study

HYPOTHESIS

To demonstrate that a virtual reality (VR) simulation training program reduces heart rate variability during an assessment of surgical trainees' technical skills in arthroscopy.

STUDY DESIGN

Prospective observational matched study.

MATERIALS & METHODS

Thirty-six orthopaedic surgery residents, new to arthroscopy, received standard training in arthroscopic knee surgery, supplemented by additional monthly training for 6months on a VR simulator for 16 of them. At inclusion, the 2 groups (VR and NON-VR) answered a questionnaire and performed a meniscectomy on a VR simulator. After 6months of training, two independent trainers blinded to the inclusion arms evaluated the technical skills of the two groups during meniscectomies on a model and on an anatomical subject. Heart rate variability (HRV) was measured using a wireless heart rate monitor during baseline, VR training, and assessment.

RESULTS

After removing incomplete data, the analysis focused on 10 VR residents matched at inclusion with 10 NON-VR residents. The VR group had a significantly lower heart rate at the final assessment (p=0.02) and lower overall HRV (p=0.05). The low/high frequency ratio (LF/HF) was not significantly different between the groups (1.84 vs 2.05, p=0.66) but the before-after training comparison showed a greater decrease in this ratio in the VR group compared to the NON-VR group -0.76 (-41%) vs -0.08 (-4%).

CONCLUSION

This study demonstrates a significant difference in heart rate variability between trained residents versus untrained residents during the final assessment of their technical skills at 6months. It appears that improving stress management should be an integral part of training programs in arthroscopic surgery.

CLINICAL INTEREST

VR simulators in arthroscopy could improve non-technical skills such as heart rate variability, from the perspective of accountability.

LEVEL OF EVIDENCE

III.

Hip Arthroscopy Simulator Training With Immersive Virtual Reality Has Similar Effectiveness to Nonimmersive Virtual Reality

PURPOSE

To (1) compare the efficacy of immersive virtual reality (iVR) to nonimmersive virtual reality (non-iVR) training in hip arthroscopy on procedural and knowledge-based skills acquisition and (2) evaluate the relative cost of each platform.

METHODS

Fourteen orthopaedic surgery residents were randomized to simulation training utilizing an iVR Hip Arthroscopy Simulator (n = 7; PrecisionOS) or non-iVR simulator (n = 7; ArthroS Hip VR; VirtaMed). After training, performance was assessed on a cadaver by 4 expert hip arthroscopists through arthroscopic video review of a diagnostic hip arthroscopy. Performance was assessed using the Objective Structured Assessment of Technical Skills (OSATS) and Arthroscopic Surgery Skill Evaluation Tool (ASSET) scores. A cost analysis was performed using the transfer effectiveness ratio (TER) and a direct cost comparison of iVR to non-iVR.

RESULTS

Demographic characteristics did not differ between treatment arms or by training level, hip arthroscopy experience, or prior simulator use. No significant differences were observed in OSATS and ASSET scores between iVR and non-iVR cohorts (OSATS: iVR 19.6 ± 4.4, non-iVR 21.0 ± 4.1, P = .55; ASSET: iVR 23.7 ± 4.5, non-iVR 25.8 ± 4.8, P = .43). The absolute TER was 0.06 and there was a 132-fold cost difference of iVR to non-iVR.

CONCLUSIONS

Hip arthroscopy simulator training with iVR had similar performance results to non-iVR for technical skill and procedural knowledge acquisition after expert arthroscopic video assessment. The iVR platform had similar effectiveness in transfer of skill compared to non-iVR with a 132 times cost differential. CLINICAL RELEVANCE: Due to the accessibility, effectiveness, and relative affordability, iVR training may be beneficial in the future of safe arthroscopic hip training.

Virtual and Augmented Reality Simulators Show Intraoperative, Surgical Training, and Athletic Training Applications: A Scoping Review

PURPOSE

To review published literature to identify and evaluate the effect of virtual reality (complete immersion) and augmented reality (overlay of digital information onto the physical world) simulators on intraoperative use for orthopaedic surgeons, orthopaedic surgical education, and athletic training.

METHODS

A systematic review was performed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to identify studies, published since 2014, that evaluated the role of augmented/virtual reality on intraoperative use for orthopaedic surgeons, orthopaedic surgical education, and athletic training.

RESULTS

Virtual reality (VR) simulators provide 3-dimensional graphical simulation of the physical world, and augmented reality (AR) simulators overlay digital information onto the physical world. Simulators can include interactive features (i.e., replication of intraoperative bleeding), haptic feedback, and unrestricted task repetition, and they can record, compare, and analyze performance while being easily accessible and eliminating the need for the presence of a mentor or coach. Four studies reported on VR for intraoperative use, 47 studies on surgical education, and 10 studies on athletic training. Two studies revealed the advantages of using VR simulation during intraoperative procedures, specifically showcasing its benefits for elbow arthroscopy, while 2 studies demonstrated similar positive outcomes for hip arthroscopy. Seventeen studies demonstrated that a VR simulator could be a beneficial tool to assist in surgical education for the knee, while 12 studies found that VR simulation is a valuable tool for aiding in surgical education of shoulder arthroscopy. Ten studies demonstrated that VR simulation improves skills in the operating room. Three studies revealed that individuals with more experience exhibit superior performance on these simulators compared to those with less experience. In the realm of athletic training, 10 studies showcased the potential of VR simulation to play a significant role in athletic performance and injury rehabilitation.

CONCLUSIONS

VR simulation shows benefits in the operating room, is a valuable tool for surgical education resulting in improved skills, and can be used to enhance athletic performance and injury rehabilitation.

CLINICAL RELEVANCE

Understanding that VR simulators can improve surgical outcomes, surgical skill training, and athletic training and rehabilitation could facilitate development and adoption of this advanced technology.

Variation in Reported Learning Outcomes and Measurement Instruments in Hip Arthroscopy Simulation Training: A Systematic Review

PURPOSE

To systematically review the current literature on the effectiveness of hip arthroscopy simulation training and to determine the consistency of reporting and validation of simulation used in hip arthroscopy.

METHODS

Three databases (PubMed, EMBase, and CINAHL) were screened using PRISMA guidelines in January 2022 for published literature on virtual simulation in hip arthroscopy. Studies reporting on the use of hip arthroscopy simulation training in orthopedic surgical trainees were included and assessed for quality and risk of bias using MINORS criteria. The number of participants, participant education level, experience, simulator type, validation type, method of assessment, and simulation outcomes were extracted from included studies.

RESULTS

Of the 286 articles screened, 11 met inclusion criteria for review evaluating 323 orthopedic trainees with a mean of 29.36 participants per study published between 2012 and 2021, most commonly in the United Kingdom (55%). The four most reported surgical skills evaluated were visualization and probing tasks (82%), mean time to perform the task (73%), number of cartilage and soft tissue collisions (73%), and number of hand movements (73%). The most described measurement instruments included a simulation built-in scoring system (55%), Arthroscopic Surgical Skill Evaluation Tool (ASSET) Global Rating Scale (GRS) (27%), and motion analysis system (18%). Construct validity was the most reported overall type of validity (82%), followed by face validity (36%), transfer validity (18%) and content validity (18%). Construct validity was also the most reported validity for the simulator and measurement instrument (55% and 89%, respectively).

CONCLUSIONS

There is significant variation in reported learning outcomes and measurement instruments for evaluating the effectiveness of hip arthroscopic-based education. This study highlights that simulation training may be an effective tool for evaluation of hip arthroscopy skills.

LEVEL OF EVIDENCE

Level III, systematic review of level I to III studies.

Effectiveness of immersive virtual reality in orthognathic surgical education: A randomized controlled trial

PURPOSE

To evaluate the efficacy of an iVR surgical training system for orthognathic surgery training in medical students.

METHODS

This study comprised 20 fifth year medical students who were randomly assigned to the VR or traditional group for orthognathic surgical education. All participants were initially provided a lecture on orthognathic surgery. The VR group then received 10 educational sessions using the self-developed iVR training system, whereas the traditional group received 10 sessions using technical manuals and annotated operation videos. These sessions were 40-min long in both the groups. Before the evaluation, the traditional group completed one session using the training and assessment modes to become familiar with the iVR training system. The score in the assessment mode, time to complete the procedure, number of instrument selection errors, number of prompts given by the system, number of positional and angular errors, and number of timeouts during each step were recorded to evaluate the learning effect.

RESULTS

The VR group achieved higher scores than the traditional group (94.67 vs. 87.65). Compared with the control group, the VR group completed the procedure more quickly, with fewer instrument selection and angular errors. No difference in the number of prompts given by the system was observed between the two groups.

CONCLUSIONS

The iVR surgical training system showed a better learning effect than the traditional learning method for orthognathic surgery. The iVR surgical training system may have utility as a supplement and potential substitute for the traditional surgical training method.

Virtual reality simulation training improve diagnostic knee arthroscopy and meniscectomy skills: a prospective transfer validity study

PURPOSE

Limited data exist on the actual transfer of skills learned using a virtual reality (VR) simulator for arthroscopy training because studies mainly focused on VR performance improvement and not on transfer to real word (transfer validity). The purpose of this single-blinded, controlled trial was to objectively investigate transfer validity in the context of initial knee arthroscopy training.

METHODS

For this study, 36 junior resident orthopaedic surgeons (postgraduate year one and year two) without prior experience in arthroscopic surgery were enrolled to receive standard knee arthroscopy surgery training (NON-VR group) or standard training plus training on a hybrid virtual reality knee arthroscopy simulator (1 h/month) (VR group). At inclusion, all participants completed a questionnaire on their current arthroscopic technical skills. After 6 months of training, both groups performed three exercises that were evaluated independently by two blinded trainers: i) arthroscopic partial meniscectomy on a bench-top knee simulator; ii) supervised diagnostic knee arthroscopy on a cadaveric knee; and iii) supervised knee partial meniscectomy on a cadaveric knee. Training level was determined with the Arthroscopic Surgical Skill Evaluation Tool (ASSET) score.

RESULTS

Overall, performance (ASSET scores) was better in the VR group than NON-VR group (difference in the global scores: p < 0.001, in bench-top meniscectomy scores: p = 0.03, in diagnostic knee arthroscopy on a cadaveric knee scores: p = 0.04, and in partial meniscectomy on a cadaveric knee scores: p = 0.02). Subgroup analysis by postgraduate year showed that the year-one NON-VR subgroup performed worse than the other subgroups, regardless of the exercise.

CONCLUSION

This study showed the transferability of the technical skills acquired by novice residents on a hybrid virtual reality simulator to the bench-top and cadaveric models. Surgical skill acquired with a VR arthroscopy surgical simulator might safely improve arthroscopy competences in the operating room, also helping to standardise resident training and follow their progress.

LEVEL OF EVIDENCE: 2

The status of virtual simulation experiments in medical education in China: based on the national virtual simulation experiment teaching Center (iLAB-X)

BACKGROUND

Virtual simulation experiments have been rapidly applied to medical education curricula in recent years. China constructed a national virtual simulation experimental teaching center (iLAB-X), and this platform covered almost all of the virtual simulation experiment curricula of domestic colleges or universities. We aimed to comprehensively assess the characteristics and usages of virtual simulation experiments in medical education based on iLAB-X.

METHODS

A total of 480 virtual simulation experiment courses had been constructed on iLAB-X (https://www.ilab-x.com/) by December 20, 2022, and the curriculum level, type and design were all searched in this platform. We also conducted an evaluation of curriculum usage and online tests, including the page view, frequency of participation, number of participants, duration of experimental learning and passing rate of the experimental test.

RESULTS

The national and provincial high-quality virtual simulation experiment curricula accounted for 33.5% (161/480) and 35.8% (172/480), respectively. The curricula were mainly set as basic practice experiments (46.5%) and synthetic designing experiments (48.8%). Significantly, forensic medicine (100%), public health and preventive medicine (83%) and basic medical sciences (66%) focused on synthetic design experiments. In terms of usage experiments, the average duration of experimental learning was 25 minutes per course, and the average number of participants was just 1257. The average passing (score ≥60) rate of online tests was 80.6%, but the average rate of score ≥ 85 was only 58.5%. In particular, the average page views, the number of participants, the duration of learning and the test passing rate of clinical medicine were relatively low.

CONCLUSIONS

The curriculum design features, construction level and utilization rate varied in different medical majors. Virtual simulation experiments are particularly underutilized in clinical medicine. There is a long way for virtual simulation experiments to go to become a supplement or alternative for traditional medical education in the future.

Arthroscopic approach in initial training: Study of a novice cohort using inverse direct and indirect approaches and its implication in the development of training programs

INTRODUCTION

Arthroscopic training includes successive stages of observation, reproduction and then repetition. Learning through simulation in 2D virtual reality makes it possible to repeat these different stages to enhance the learner's experience in complete safety and a shorter timeframe. Some procedures require inversion of the optical and instrumental approaches in the axial plane, disrupting the existing psychomotor and technical skills. The objective of this study was to compare the degree of difficulty and the distribution of results for the same exercise carried out alternately in classical holding and inverted holding of the instruments in a cohort of novice learners.

MATERIALS AND METHODS

Twenty-two medical students, novices in arthroscopic surgery, participated in the study. Each performed an exercise consisting of grasping ten targets with arthroscopic forceps and placing them in a basket on the VirtaMed ArthroS™ simulator. The exercise was performed with the scope and grasping instrument pointed away from the operator, "catch the stars front" (CTSF), then directed towards the operator, "catch the stars back" (CTSB). The simulator recorded several parameters making up an overall composite score ("overall performance score", OPS) out of 120 points. Voluntary abandonment of the exercise was also collected.

RESULTS

All students completed the CTSF exercise but 6 dropped out of the CTSB exercise (27%, p=0.01). In the CTSF exercise, the average OPS was higher with 45.9 points versus 22.8 points in the CTSB exercise (p<0.001). By detailing the components of the OPS score, the parameters of interest on the Fundamentals of Arthroscopic Training (FAST) module of the simulator included: the distance traveled by the scope and the grasping forceps was significantly greater in the CTSB group (p<0.001), the duration of the exercise was significantly greater in the CTSB group (p<0.001), the time spent with the instruments in the videoscopic field was significantly lower in the CTSB group (p=0.001) and finally the absence of a significant difference in the camera alignment compared to the horizontal plane between the two groups.

CONCLUSION

The exercise with the instruments directed towards the operator is more difficult with a greater distribution for all the secondary criteria except for the camera alignment, which suggests that it could be more discriminating. The dropout rate is also higher. It would therefore be interesting to introduce CTSB type training in initial training programs in arthroscopy.

LEVEL OF EVIDENCE

III, comparative prospective study.

Editorial Commentary: The Hip Arthroscopy Volume-to-Competency Learning Curve Debate Is Complicated: Learning When "Enough Is Enough" Is a Lifetime Discipline

As hip arthroscopy use grows, so does resident and fellow exposure, providing more "hands-on" learning opportunities. Nevertheless, hip arthroscopy is technically challenging, particularly noting that improved patient-reported outcomes and survivorship are reported after labral repair or reconstruction (vs debridement) as well as routine capsular closure. Undoubtedly a requisite number of cases is required to reach the saturation point of the "learning curve." A recent review shows that traction time, complication rates, and reoperation rates decrease with increasing case volumes, but there is a wide range of cases reported after which the learning curve "plateaus," ranging from 30 to 520 cases. A large database study shows that hip arthroscopy readmissions and complications are significantly lower in high-volume centers. However, large database studies may include biases requiring attention. First, the rates are relatively low across the entire cohort. Second, more younger patients were treated in the greater-volume centers, which may contribute to the difference in outcomes observed. Finally, older patients (often >50 years old) with concomitant osteoarthritis are also associated with greater complication, readmission, and reoperation rates. Such patients may not be selected as candidates for hip arthroscopy by greater-volume surgeons. The hip arthroscopy volume to competency learning curve debate is complicated. Learning when "enough is enough" is a lifetime discipline.

The number of arthroscopies performed by trainees does not deduce the level of their arthroscopic proficiency

PURPOSE

It is reasonable to question whether the case volume is a suitable proxy for the manual competence of an arthroscopic surgeon. The aim of this study was to evaluate the correlation between the number of arthroscopies previously performed and the arthroscopic skills acquired using a standardized simulator test.

METHODS

A total of 97 resident and early orthopaedic surgeons who participated in arthroscopic simulator training courses were divided into five groups based on their self-reported number of arthroscopic surgeries: (1) none, (2) < 10, (3) 10 to 19, (4) 20 to 39 and (5) 40 to 100. Arthroscopic manual skills were evaluated with a simulator by means of the diagnostic arthroscopy skill score (DASS) before and after training. Seventy-five points out of 100 must be achieved to pass the test.

RESULTS

In the pretest, only three trainees in group 5 passed the arthroscopic skill test, and all other participants failed. Group 5 (57 ± 17 points; n = 17) scored significantly higher than the other groups (group 1: 30 ± 14, n = 20; group 2: 35 ± 14, n = 24; group 3: 35 ± 18, n = 23; and group 4: 33 ± 17, n = 13). After a two-day simulator training, trainees showed a significant increase in performance. In group 5, participants scored 81 ± 17 points, which was significantly higher than the other groups (group 1: 75 ± 16; group 2: 75 ± 14; group 3: 69 ± 15; and group 4: 73 ± 13). While self-reported arthroscopic procedures were n.s. associated with higher log odds of passing the test (p = 0.423), the points scored in the pretest were found to be a good predictor of whether a trainee would pass the test (p < 0.05). A positive correlation was observed between the points scored in the pretest and the posttest (p < 0.05, r = 0.59, r2 = 0.34).

CONCLUSIONS

The number of previously performed arthroscopies is not a reliable indicator of the skills level of orthopaedic residents. A reasonable alternative in the future would be to verify arthroscopic proficiency on the simulator by means of a score as a pass-fail examination.

LEVEL OF EVIDENCE

III.

Simulation training to retool practicing orthopedic surgeons is rare

Purpose

Modern surgical practice is continuously changing as technology develops. New techniques are often implemented after a surgeon has made the transition to independent clinical practice. There is therefore a need to 'retool' technical skills. Additionally, practicing surgeons must maintain and develop skills such as leadership, communication, critical thinking, teaching, and mentoring. Our aim was to perform a scoping review to assess the current status of simulation education for practicing Orthopedic Surgeons (OS).

Methods

A 10 year search of PubMed, ERIC, and Web of Science was performed with a medical librarian. Controlled vocabulary Medical Subject Headings terms and natural language were developed with subject matter experts describing simulation, training and OS. Two trained reviewers evaluated all abstracts for inclusion. Exclusion criteria were articles that did not assess simulation education involving practicing OS. Data were extracted from the included full text articles by two reviewers: details of study design, type of participants, type of simulation and role of OS in the educational event.

Results

Initial search identified 1824 articles of which 443 were duplicates, and 1381 articles were further screened. Of these, 1155 were excluded, 226 full text articles were assessed for eligibility and 80 included in analysis. Most were published in the last 6 years and from the United States. The majority (99%) described technical skill simulations (arthroscopy 56%, screw placement 23%, ligament reconstruction 19%). OS were rarely the only learners with 91% studies also having residents participate. OS were the targeted learner in 6% studies. OS provided content validity for 15 (19%) and construct validity in 59 (74%) studies.

Conclusions

Simulation training to educate practicing OS is rare. OS are often used to validate work rather than being the center of an educational endeavor. A refocusing is needed to provide adequate training for practicing surgeons to retool skills as new techniques become available.

Current status of virtual reality simulation education for orthopedic residents: the need for a change in focus

Introduction

Advances in technology are changing surgical education. Simulation provides an important adjunct to operative experience. This pedagogy has arguably become more important in light of the COVID-19 pandemic, with resultant reduction in operative exposure for trainees. Virtual reality (VR) simulators may provide significant contribution to experiential learning; however, much of the investigative focus to date has, correctly, been on establishing validity evidence for these constructs. The aim of this work was to perform a scoping review to assess the current status of VR simulation education to determine curricular development efforts for orthopedic residents.

Methods

With a trained medical librarian, searches of PubMed, EMBASE, and Web of Science were conducted for all articles in the last 10 years (September 2011-September 2021). Controlled vocabulary Medical Subject Headings (MeSH) terms and natural language developed with subject matter experts describing virtual reality or VR simulation and orthopedic training were used. Two trained reviewers evaluated all abstracts for inclusion. Exclusion criteria were all articles that did not assess VR simulation education involving orthopedic residents. Data were extracted from the included full-text articles including: study design, type of participants, type of VR simulation, simulated orthopedic skill, type of educational event, learner assessment including Kirkpatrick's level, assessment of quality using the Medical Education Research Study Quality Instrument (MERSQI), and level of effectiveness (LoE).

Results

Initial search identified 1,394 articles, of which 61 were included in the final qualitative synthesis. The majority (54%) were published in 2019- 2021, 49% in Europe. The commonest VR simulator was ArthroS (23%) and the commonest simulated skill was knee arthroscopy (33%). The majority of studies (70%) focused on simulator validation. Twenty-three studies described an educational module or curriculum, and of the 21 (34%) educational modules, 43% were one-off events. Most modules (18/21, 86%) assessed learners at Kirkpatrick level 2. With regard to methodological quality, 44% of studies had MERSQI 11.5-15 and 89% of studies had LoE of 2. Two studies had LoE of 3.

Conclusion

Current literature pertaining to VR training for orthopedic residents is focused on establishing validity and rarely forms part of a curriculum. Where the focus is education, the majority are discrete educational modules and do not teach a comprehensive amalgam of orthopedic skills. This suggests focus is needed to embed VR simulation training within formal curricula efforts guided by the work of Kern, and assess the efficacy of these against patient outcomes.

THE STATE OF HIP ARTHROSCOPY IN RUSSIA: ASSESSMENT BASED ON A SOCIOLOGICAL SURVEY OF DOCTORS

Background.Hip arthroscopy is a minimally invasive surgical technique most commonly performed to correct femoroacetabular impingement (FAI). The number of hip arthroscopy performed in Russia is unknown. Information about this surgical technique is also very limited in our country.The aim of the study was to assess the level of hip arthroscopy in Russia.Methods.A sociological survey of orthopaedic surgeon with experience in performing hip arthroscopy was conducted. The questionnaire was posted on the Google Forms platform. The survey included 54 surgeons from Russian clinics, 45 people filled out the questionnaire.Results.The majority of specialists (73,30,06%) were trained in hip arthroscopy. More than half of the respondents (68,80,06%) are currently engaged in this surgical area, however, in 45,160,07% of them, the number of operations does not exceed 5 per year. For surgeons who have completed two or more training courses, the volume of operations performed is higher (p0,05). 51,110,07% of doctors perform arthroscopy using an alternative technique. There are 2,5 time more specialists doing just bone resection, than surgeons who apply any kind of reconstruction technique while treating FAI (p0,05). Fifteen respondents (48,390,08%) perform debridement as an attempt to delay arthroplasty. The most common difficulties faced by surgeon are problems with the diagnosis of FAI (400,02%), lack of the necessary tools (400,02%) and sufficient time to master the technique (33,30,07%). Only three (6,60,07%) respondents believed to achieve planned results, 93,30,03% of surgeons said that it is not always possible to achieve the desired outcomes.Conclusion. Hip arthroscopy in Russia is not very common, the volume of such interventions is insignificant. Doctors not skilled in arthroscopic hip surgery predominate. Factors impeding the development of this area in our country are related to teaching methods, FAI diagnostics, the lack of necessary instruments for performing operations, and the lack of time for specialists to master surgical techniques.

Learning and Short-Term Retention of Simulation-Based Arthroscopic Skills

OBJECTIVE

The study aimed to examine the learning curve and short-term retention of arthroscopic skills acquired on a simulator.

DESIGN

Cohort study.

SETTING

Clinical Skills Training Center of Zhujiang Hospital of Southern Medical University PARTICIPANT AND METHODS: Orthopaedic residents (n = 14) without previous arthroscopy experience were included. After basic information was collected and an initial arthroscopy knowledge level test was administered, the subjects received standardised training on the simulator (day 1); then, they completed tasks on the simulator, including guided diagnostics (4 times), triangulation (5 times) and loose body removal (7 times). A learning curve for each skill was generated based on the total scores. The score of the last repetition of each task was the training level.

RESULTS

A total of 14 orthopedic residents were enrolled. All participants completed the training and testing. There was a learning curve over the course of training for all 3 arthroscopic skills (p < 0.001). On day 8 after the training, the mean score for guided diagnostics decreased from 49.9 to 48.9 (p = 0.001), and the retention rate was 97.8%. For triangulation, the mean total score decreased from 58.9 to 53.6 (p < 0.001), and the retention rate was 90.8%. For loose body removal, the mean total score decreased from 87.1 to 80.7 (p < 0.001), and the retention rate was 92.7%.

CONCLUSIONS

Orthopaedic residents' arthroscopic skills learned through simulator training declined significantly in 1 week after the training, especially more difficult skills.

The frequency of assessment tools in arthroscopic training: a systematic review

BACKGROUND

Multiple assessment tools are used in arthroscopic training and play an important role in feedback. However, it is not fully recognized as to the standard way to apply these tools. Our study aimed to investigate the use of assessment tools in arthroscopic training and determine whether there is an optimal way to apply various assessment tools in arthroscopic training.

METHODS

A search was performed using PubMed, Embase and Cochrane Library electronic databases for articles published in English from January 2000 to July 2021. Eligible for inclusion were primary research articles related to using assessment tools for the evaluation of arthroscopic skills and training environments. Studies that focussed only on therapeutic cases, did not report outcome measures of technical skills, or did not mention arthroscopic skills training were excluded.

RESULTS

A total of 28 studies were included for review. Multiple assessment tools were used in arthroscopic training. The most common objective metric was completion time, reported in 21 studies. Technical parameters based on simulator or external equipment, such as instrument path length, hand movement, visual parameters and injury, were also widely used. Subjective assessment tools included checklists and global rating scales (GRS). Among these, the most commonly used GRS was the Arthroscopic Surgical Skill Evaluation Tool (ASSET). Most of the studies combined objective metrics and subjective assessment scales in the evaluation of arthroscopic skill training.

CONCLUSIONS

Overall, both subjective and objective assessment tools can be used as feedback for basic arthroscopic skill training, but there are still differences in the frequency of application in different contexts. Despite this, combined use of subjective and objective assessment tools can be applied to more situations and skills and can be the optimal way for assessment.

LEVEL OF EVIDENCE

Level III, systematic review of level I to III studies. Key messagesBoth subjective and objective assessment tools can be used as feedback for basic arthroscopic skill training.Combined use of subjective and objective assessment tools can be applied to more situations and skills and can be the optimal way for assessment.

A three-dimensional (3D) printed simulator as a feasible assessment tool for evaluating hip arthroscopy skills

PURPOSE

The aims of this study were (1) to develop a three-dimensional (3D) printed simulator that facilitates the simulation of surgical skills for portal placement, intra-articular identification of anatomical structures and arthroscope navigation for hip arthroscopy and (2) to concurrently examine the feasibility of using this simulator as an assessment tool to evaluate trainees' surgical competencies.

METHODS

A simulator was developed using a combination of medical imaging, computer-aided design, and 3D printing. A cross-sectional study was conducted with 29 participants divided into 3 subgroups (novice, intermediate and experienced). All participants performed related skills on the simulator, and their performance was evaluated using different assessment parameters. The participants' qualitative feedback regarding the simulator was also collected. The data collated from each group of participants were subsequently compared.

RESULTS

Significant differences were observed between the three subgroups of participants with regard to the total checklist score (F_2,26 = 11.3), total Arthroscopic Surgical Skill Evaluation score (F_2,26 = 92.1), overall final global rating scale score (F_2,26 = 49), number of times the participants used fluoroscopy (F_2,26 = 7.4), and task completion times (F_2,26 = 23.5). The participants' performance in the simulated operation was correlated with their prior clinical experience. There was mainly positive feedback with regard to the fidelity and utility of the simulator in relation to the surgeons' prior clinical experience.

CONCLUSIONS

This study demonstrated that a reliable hip arthroscopic simulator can be developed for use by orthopedic surgeons to evaluate their hip arthroscopic skills before performing actual surgical operations.

LEVEL OF EVIDENCE

Level III.

Virtual Simulation in Undergraduate Medical Education: A Scoping Review of Recent Practice

Virtual simulation (VS) as an emerging interactive pedagogical strategy has been paid more and more attentions in the undergraduate medical education. Because of the fast development of modern computer simulation technologies, more and more advanced and emerging VS-based instructional practices are constantly increasing to promote medical education in diverse forms. In order to describe an overview of the current trends in VS-based medical teaching and learning, this scoping review presented a worldwide analysis of 92 recently published articles of VS in the undergraduate medical teaching and learning. The results indicated that 98% of included articles were from Europe, North America, and Asia, suggesting a possible inequity in digital medical education. Half (52%) studies reported the immersive virtual reality (VR) application. Evidence for educational effectiveness of VS in medical students’ knowledge or skills was sufficient as per Kirkpatrick’s model of outcome evaluation. Recently, VS has been widely integrated in surgical procedural training, emergency and pediatric emergency medicine training, teaching of basic medical sciences, medical radiation and imaging, puncture or catheterization training, interprofessional medical education, and other case-based learning experiences. Some challenges, such as accessibility of VS instructional resources, lack of infrastructure, “decoupling” users from reality, as well as how to increase students’ motivation and engagement, should be addressed.

A Portable Hip Arthroscopy Simulator Demonstrates Good Face and Content Validity with Incomplete Construct Validity

Purpose

We evaluate the face, content, and construct validity of a portable hip arthroscopy module in a regional orthopaedic unit.

Methods

Participants were recruited from a regional orthopaedic centre, and categorized into novice (0 arthroscopies), intermediate (1-29 arthroscopies), and expert (>50 arthroscopies) groups based on reported experience in arthroscopy. Face and content validity was evaluated by feedback from users immediately following completion of modules. Objective measurements, including time taken and subjective measurements consisting of simulation software metrics including, cam lesion locations attempts, scope strikes on bone, healthy bone burred, and cam lesion burred. Scores achieved by experts were recorded, and the median score was set at the level at which proficiency was demonstrated. Participant feedback on perceived educational use was collected following completion.

Results

In total, 20 participant results were included for analysis. Good face and content validity was expressed by participants with previous arthroscopic experience. Number of scope strikes within the simulator-derived metrics accurately discerned between levels of experience. Novices had a mean of 5 strikes per attempt (SD ± 5 ) ,  intermediates a mean of 5.8 strikes (SD ± 4.1). There was a significant difference between expert and novice groups (P = .01), and expert and intermediate groups (P = .002). No significant difference between overall performance scores achieved by participants in expert, intermediate, and novice groups (62% ± 19 vs 55% ± 22 vs 50% ± 23 , P = .15). This demonstrates incomplete construct validity of the simulator software-derived metrics.

Conclusions

This hip arthroscopy simulator demonstrated acceptable face and content validity, with incomplete construct validity of simulator software metrics. Participants reported high levels of satisfaction with the module, highlighting that the addition of haptic feedback would be beneficial to improve procedural steps. Incorporation of tactile feedback to the modulator components would likely enable the software to accurately delineate between levels of experience.

Clinical Relevance

This study demonstrates good face and content validity. The addition of haptic feedback in a hip arthroscopy simulator may improve learning.

Standard Setting in Simulation-Based Training of Surgical Procedures: A Systematic Review

OBJECTIVE

This systematic review aims to examine the use of standard-setting methods in the context of simulation-based training of surgical procedures.

SUMMARY OF BACKGROUND

Simulation-based training is increasingly used in surgical education. However, it is important to determine which level of competency trainees must reach during simulation-based training before operating on patients. Therefore, pass/fail standards must be established using systematic, transparent, and valid methods.

METHODS

Systematic literature search was done in four databases (Ovid MEDLINE, Embase, Web of Science, and Cochrane Library). Original studies investigating simulation-based assessment of surgical procedures with application of a standard setting were included. Quality of evidence was appraised using GRADE.

RESULTS

Of 24,299 studies identified by searches, 232 studies met the inclusion criteria. Publications using already established standard settings were excluded (N = 70), resulting in 162 original studies included in the final analyses. Most studies described how the standard setting was determined (N = 147, 91%) and most used the mean or median performance score of experienced surgeons (n = 65, 40%) for standard setting. We found considerable differences across most of the studies regarding study design, set-up, and expert level classification. The studies were appraised as having low and moderate evidence.

CONCLUSION

Surgical education is shifting towards competency-based education, and simulation-based training is increasingly used for acquiring skills and assessment. Most studies consider and describe how standard settings are established using more or less structured methods but for current and future educational programs, a critical approach is needed so that the learners receive a fair, valid and reliable assessment.

Editorial Commentary: Virtual Reality Simulation Can Help Arthroscopic Hip Preservation Surgeons at All Levels of Training and Practice-This is How

Virtual reality (VR) simulation has enormous potential utility in technically demanding manual activities. Hip arthroscopy is a perfect example of a challenging surgical technique with an extensive learning curve. The literature has recently consistently demonstrated that both career and annual maintenance case volume significantly influences patient-reported outcomes and risk of revision surgery and complications. Current residency and fellowship programs do not sufficiently prepare trainees to meet or exceed experience thresholds, so augmentation of training is necessary. A significant strength of VR simulation includes its ability to practice without limits. Unfortunately, hip models are limited to simple tasks, without full surgery models yet available simulating routine arthroscopic hip preservation procedures like labral repair, cam and pincer correction, capsular repair. Advanced techniques like labral reconstruction or augmentation, protrusio acetabulae, extensive cam morphology, revision surgery, peritrochanteric space endoscopy, and deep gluteal space endoscopy are not yet available for simulation. VR simulation can probably achieve competence for most, if not all, surgeons; possibly achieve proficiency; and unlikely to achieve mastery. The use of machine learning and artificial intelligence can process vast quantities of photo and video data to generate high-fidelity, lifelike surgical simulation. The near future will incorporate and assimilate these technologies cost-effectively for training programs and surgeons. Our patients will benefit.