Advancing Simulation-Based Orthopaedic Surgical Skills Training: An Analysis of the Challenges to Implementation

Using neuroimaging to assess brain activity and areas associated with surgical skills: a systematic review

BACKGROUND

Surgical skills acquisition is under continuous development due to the emergence of new technologies, and there is a need for assessment tools to develop along with these. A range of neuroimaging modalities has been used to map the functional activation of brain networks while surgeons acquire novel surgical skills. These have been proposed as a method to provide a deeper understanding of surgical expertise and offer new possibilities for the personalized training of future surgeons. With studies differing in modalities, outcomes, and surgical skills there is a need for a systematic review of the evidence. This systematic review aims to summarize the current knowledge on the topic and evaluate the potential use of neuroimaging in surgical education.

METHODS

We conducted a systematic review of neuroimaging studies that mapped functional brain activation while surgeons with different levels of expertise learned and performed technical and non-technical surgical tasks. We included all studies published before July 1st, 2023, in MEDLINE, EMBASE and WEB OF SCIENCE.

RESULTS

38 task-based brain mapping studies were identified, consisting of randomized controlled trials, case-control studies, and observational cohort or cross-sectional studies. The studies employed a wide range of brain mapping modalities, including electroencephalography, functional magnetic resonance imaging, positron emission tomography, and functional near-infrared spectroscopy, activating brain areas involved in the execution and sensorimotor or cognitive control of surgical skills, especially the prefrontal cortex, supplementary motor area, and primary motor area, showing significant changes between novices and experts.

CONCLUSION

Functional neuroimaging can reveal how task-related brain activity reflects technical and non-technical surgical skills. The existing body of work highlights the potential of neuroimaging to link task-related brain activity patterns with the individual level of competency or improvement in performance after training surgical skills. More research is needed to establish its validity and usefulness as an assessment tool.

Comparison of virtual reality and computed tomography in the preoperative planning of complex tibial plateau fractures

INTRODUCTION

Preoperative planning is a critical step in the success of any complex surgery. The pur-pose of this study is to evaluate the advantage of VR glasses in surgical planning of complex tibial plateau fractures compared to CT planning.

MATERIALS AND METHODS

Five orthopedic surgeons performed preoperative planning for 30 fractures using either conventional CT slices or VR visualization with a VR headset. Planning was performed in a randomized order with a 3-month interval between planning sessions. A standardized questionnaire assessed planned operative time, planning time, fracture classification and understanding, and surgeons' subjective confidence in surgical planning.

RESULTS

The mean planned operative time of 156 (SD 47) minutes was significantly lower (p < 0.001) in the VR group than in the CT group (172 min; SD 44). The mean planning time in the VR group was 3.48 min (SD 2.4), 17% longer than in the CT group (2.98 min, SD 1.9; p = 0.027). Relevant parameters influencing planning time were surgeon experience (-0.61 min) and estimated complexity of fracture treatment (+ 0.65 min).

CONCLUSION

The use of virtual reality for surgical planning of complex tibial plateau fractures resulted in significantly shorter planned operative time, while planning time was longer compared to CT planning. After VR planning, more surgeons felt (very) well prepared for surgery.

Use of Virtual Reality in the Education of Orthopaedic Procedures: A Randomised Control Study in Early Validation of a Novel Virtual Reality Simulator

Background Virtual reality (VR) simulation is a potential solution to the barriers surgical trainees are facing. There needs to be validation for its implementation within current training. We aimed to compare VR simulation to traditional methods in acquiring surgical skills for a TFN-ADVANCED™ Proximal Femoral Nailing System (TFNA; DePuy Synthes, Auckland, New Zealand) femoral nailing system. Methods Thirty-one surgical trainees were randomised to two groups: traditional-training group (control group) and a VR-training group (intervention group) for insertion of a short cephalomedullary TFNA nail. Both groups then inserted the same TFNA system into saw-bone femurs. Surveys evaluated validity of the relevant activities, perception of simulation, confidence, stress and anxiety. The primary outcomes were tip-apex distance (TAD) and user anxiety/confidence levels. Secondary outcomes included number of screw- and nail-guidewire insertion attempts, the time taken to complete and user validity of the VR system. Results There was no statistical difference in TAD between the intervention and control groups (9mm vs 15mm, p=0.0734). The only TAD at risk of cut-out was in the control group (25mm). There was no statistical difference in time taken (2547.5ss vs 2395ss, p=0.668), nail guide-wire attempts (two for both groups, p=0.355) and screw guide-wire attempts (one for both groups, p=0.702). The control group versus intervention had higher anxiety levels (50% vs 33%) and had lower confidence (61% vs 84%). Interpretation There was no objective difference in performance on a saw-bone model between groups. However, this VR simulator resulted in more confidence and lower anxiety levels whilst performing a simulated TFNA. Whilst further studies with larger sample sizes and exploration of transfer validity to the operating theatre are required, this study does indicate potential benefits of VR within surgical training.

Application of Virtual Reality Systems in Bone Trauma Procedures

Background and Objectives: Bone fractures contribute significantly to the global disease and disability burden and are associated with a high and escalating incidence and tremendous economic consequences. The increasingly challenging climate of orthopaedic training and practice re-echoes the established potential of leveraging computer-based reality technologies to support patient-specific simulations for procedural teaching and surgical precision. Unfortunately, despite the recognised potential of virtual reality technologies in orthopaedic surgery, its adoption and integration, particularly in fracture procedures, have lagged behind other surgical specialities. We aimed to review the available virtual reality systems adapted for orthopaedic trauma procedures. Materials and Methods: We performed an extensive literature search in Medline (PubMed), Science Direct, SpringerLink, and Google Scholar and presented a narrative synthesis of the state of the art on virtual reality systems for bone trauma procedures. Results: We categorised existing simulation modalities into those for fracture fixation techniques, drilling procedures, and prosthetic design and implantation and described the important technical features, as well as their clinical validity and applications. Conclusions: Over the past decade, an increasing number of high- and low-fidelity virtual reality systems for bone trauma procedures have been introduced, demonstrating important benefits with regard to improving procedural teaching and learning, preoperative planning and rehearsal, intraoperative precision and efficiency, and postoperative outcomes. However, further technical developments in line with industry benchmarks and metrics are needed in addition to more standardised and rigorous clinical validation.

Simulation-based summative assessment in healthcare: an overview of key principles for practice

BACKGROUND

Healthcare curricula need summative assessments relevant to and representative of clinical situations to best select and train learners. Simulation provides multiple benefits with a growing literature base proving its utility for training in a formative context. Advancing to the next step, "the use of simulation for summative assessment" requires rigorous and evidence-based development because any summative assessment is high stakes for participants, trainers, and programs. The first step of this process is to identify the baseline from which we can start.

METHODS

First, using a modified nominal group technique, a task force of 34 panelists defined topics to clarify the why, how, what, when, and who for using simulation-based summative assessment (SBSA). Second, each topic was explored by a group of panelists based on state-of-the-art literature reviews technique with a snowball method to identify further references. Our goal was to identify current knowledge and potential recommendations for future directions. Results were cross-checked among groups and reviewed by an independent expert committee.

RESULTS

Seven topics were selected by the task force: "What can be assessed in simulation?", "Assessment tools for SBSA", "Consequences of undergoing the SBSA process", "Scenarios for SBSA", "Debriefing, video, and research for SBSA", "Trainers for SBSA", and "Implementation of SBSA in healthcare". Together, these seven explorations provide an overview of what is known and can be done with relative certainty, and what is unknown and probably needs further investigation. Based on this work, we highlighted the trustworthiness of different summative assessment-related conclusions, the remaining important problems and questions, and their consequences for participants and institutions of how SBSA is conducted.

CONCLUSION

Our results identified among the seven topics one area with robust evidence in the literature ("What can be assessed in simulation?"), three areas with evidence that require guidance by expert opinion ("Assessment tools for SBSA", "Scenarios for SBSA", "Implementation of SBSA in healthcare"), and three areas with weak or emerging evidence ("Consequences of undergoing the SBSA process", "Debriefing for SBSA", "Trainers for SBSA"). Using SBSA holds much promise, with increasing demand for this application. Due to the important stakes involved, it must be rigorously conducted and supervised. Guidelines for good practice should be formalized to help with conduct and implementation. We believe this baseline can direct future investigation and the development of guidelines.

Virtual Simulation for Interactive Visualization of 3D Fracture Fixation Biomechanics

INTRODUCTION

In the surgical fixation of fractures, proper biomechanical stability is key in preventing clinical complications including poor fracture healing, residual deformity, loss of fixation, or implant failure. Stability is largely influenced by treatment decisions made by the surgeon. The interplay of surgeon-controlled variables and their effect on the three-dimensional (3D) biomechanics of a fracture fixation construct are often not intuitive, and current training methods do not facilitate a deep understanding of these interactions.

METHODS

A simulation software interface, FracSim, was developed. FracSim is built on a large precomputed library of finite element simulations. The software allows a surgeon to make adjustments to a virtual fracture fixation construct/weight-bearing plan and immediately visualize how these changes affect 3D biomechanics, including implant stress and fracture gap strain, important for clinical success. Twenty-one orthopaedic residents completed an instructor-led educational session with FracSim focused on bridge plating. Subjects completed pretests and posttests of knowledge of biomechanical concepts and a questionnaire.

RESULTS

Subjects scored a mean of 5.6/10 on the pretest of biomechanical knowledge. Senior residents scored better than junior residents (P = 0.04). After the educational session with FracSim, residents improved their test scores to a mean of 8.0/10, with a significant improvement (P < 0.001). Questionnaire scores indicated that subjects believed that FracSim had realistic implants, constructs, and motions and that training with FracSim was purposeful, desirable, efficient, fun, and useful for enhancing the understanding of fracture fixation biomechanics.

DISCUSSION

This new type of simulation software enables interactive visualization of 3D fracture fixation biomechanics. Limitations of this study include lack of a control group undergoing traditional education and lack of a delayed posttest to assess retention. FracSim may provide an effective and engaging way to promote a deeper understanding of biomechanical concepts in the orthopaedic learner.

Cost-effective and adaptable cataract surgery simulation with basic technology

Purpose

To assess the subjective validity of a cost-effective and adaptable cataract surgery simulation technique using basic technology.

Methods

We devised and filmed a range of simulation techniques that mimic steps of phacoemulsification cataract surgery using various “everyday” basic materials. This video was combined in a “parallel” fashion with live cataract surgery so that all steps of surgery were simulated. Subsequently, we distributed an online subjective validation questionnaire on Google Forms with the embedded simulation video in a generic invitation that was forwarded via email and/or text messages/WhatsApp messenger amongst Ophthalmologists of all grades within our regions (Kent, Surrey and Sussex, London and Wales Postgraduate Deaneries).

Results

Face validity: 66 (99%) participants agreed that the explanations in the video were clear and 53 (79%) concurred with the realistic feel of simulated technique. Instrumentation and adaptations demonstrated were deemed user friendly and conducive to replicate by 99% participants. Content validity: 60 (90%) of participants agreed the techniques described in the video reflected the technical skills required to train cataract surgeons. Forty-nine (74%) agreed that the simulation techniques were relevant for acquiring other generic and transferable microsurgical and manual dexterity skills.

Conclusions

We demonstrated subjective validity of our cost-effective cataract simulation technique. Our model can be used as an adjunct to intraocular and virtual reality training for cataract surgery by removing the barrier of cost and improved exposure to real instruments used in cataract surgery.

Surgical Skill Can be Objectively Measured From Fluoroscopic Images Using a Novel Image-based Decision Error Analysis (IDEA) Score

BACKGROUND

To advance orthopaedic surgical skills training and assessment, more rigorous and objective performance measures are needed. In hip fracture repair, the tip-apex distance is a commonly used summative performance metric with clear clinical relevance, but it does not capture the skill exercised during the process of achieving the final implant position. This study introduces and evaluates a novel Image-based Decision Error Analysis (IDEA) score that better captures performance during fluoroscopically-assisted wire navigation.

QUESTIONS/PURPOSES

(1) Can wire navigation skill be objectively measured from a sequence of fluoroscopic images? (2) Are skill behaviors observed in a simulated environment also exhibited in the operating room? Additionally, we sought to define an objective skill metric that demonstrates improvement associated with accumulated surgical experience.

METHODS

Performance was evaluated both on a hip fracture wire navigation simulator and in the operating room during actual fracture surgery. After examining fluoroscopic image sequences from 176 consecutive simulator trials (performed by 58 first-year orthopaedic residents) and 21 consecutive surgical procedures (performed by 19 different orthopaedic residents and one attending orthopaedic surgeon), three main categories of erroneous skill behavior were identified: off-target wire adjustments, out-of-plane wire adjustments, and off-target drilling. Skill behaviors were measured by comparing wire adjustments made between consecutive images against the goal of targeting the apex of the femoral head as part of our new IDEA scoring methodology. Decision error metrics (frequency, magnitude) were correlated with other measures (image count and tip-apex distance) to characterize factors related to surgical performance on both the simulator and in the operating room. An IDEA composite score integrating decision errors (off-target wire adjustments, out-of-plane wire adjustments, and off-target drilling) and the final tip-apex distance to produce a single metric of overall performance was created and compared with the number of hip wire navigation cases previously completed (such as surgeon experience levels).

RESULTS

The IDEA methodology objectively analyzed 37,000 images from the simulator and 688 images from the operating room. The number of decision errors (7 ± 5 in the operating room and 4 ± 3 on the simulator) correlated with fluoroscopic image count (33 ± 14 in the operating room and 20 ± 11 on the simulator) in both the simulator and operating room environments (R2 = 0.76; p < 0.001 and R2 = 0.71; p < 0.001, respectively). Decision error counts did not correlate with the tip-apex distance (16 ± 4 mm in the operating room and 12 ± 5 mm on the simulator) for either the simulator or the operating room (R2 = 0.08; p = 0.15 and R2 = 0.03; p = 0.47, respectively), indicating that the tip-apex distance is independent of decision errors. The IDEA composite score correlated with surgical experience (R2 = 0.66; p < 0.001).

CONCLUSION

The fluoroscopic images obtained in the course of placing a guide wire contain a rich amount of information related to surgical skill. This points the way to an objective measure of skill that also has potential as an educational tool for residents. Future studies should expand this analysis to the wide variety of procedures that rely on fluoroscopic images.

CLINICAL RELEVANCE

This study has shown how resident skill development can be objectively assessed from fluoroscopic image sequences. The IDEA scoring provides a basis for evaluating the competence of a resident. The score can be used to assess skill at key timepoints throughout residency, such as when rotating onto/off of a new surgical service and before performing certain procedures in the operating room, or as a tool for debriefing/providing feedback after a procedure is completed.

A review of participant recruitment transparency for sound validation of hip surgery simulators: a novel umbrella approach

Malposition of implants is associated with complications, higher wear and increased revision rates in total hip replacement (THR) along with surgeon inexperience. Training THR residents to reach expert proficiency is affected by the high cost and resource limitations of traditional training techniques. Research in extended reality (XR) technologies can overcome such barriers. These offer a platform for learning, objective skill-monitoring and, potentially, for automated certification. Prior to their incorporation into curricula however, thorough validation must be undertaken. As validity is heavily dependent on the participants recruited, there is a need to review, scrutinise and define recruitment criteria in the absence of pre-defined standards, for sound simulator validation. A systematic review on PubMed and IEEE databases was conducted. Training simulator validation research in fracture, arthroscopy and arthroplasty relating to the hip was included. 46 validation studies were reviewed. It was observed that there was no uniformity in reporting or recruitment criteria, rendering cross-comparison challenging. This work developed Umbrella categories to help prioritise recruitment, and has formulated a detailed template of fields and guidelines for reporting criteria so that, in future, research may come to a consensus as to recruitment criteria for a hip "expert" or "novice".

Validation of an endoscopic flavectomy training model

OBJECTIVE

to validate a lumbar spine endoscopic flavectomy simulator using the construct method and to assess the acceptability of the simulator in medical education.

METHODS

thirty medical students and ten video-assisted surgery experienced orthopedists performed an endoscopic flavectomy procedure in the simulator. Time, look-downs, lost instruments, respect for the stipulated edge of the ligamentum flavum, regularity of the incision, GOALS checklist (Global Operative Assessment of Laparoscopic Skills), and responses to the Likert Scale adapted for this study were analyzed.

RESULTS

all variables differed between groups. Procedure time was shorter in the physician group (p < 0.001). Look-downs and instrument losses were seven times greater among students than physicians. Half of the students respected the designated incision limits, compared to 80% of the physicians. In the student group, about 30% of the incisions were regular, compared to 100% in the physician group (p < 0.001). The physicians performed better in all GOALS checklist domains. All the physicians and more than 96% of the students considered the activity enjoyable, and approximately 90% believed that the model was realistic and could contribute to medical education.

CONCLUSIONS

the simulator could differentiate the groups' experience level, indicating construct validity, and both groups reported high acceptability.

Editorial Commentary: Simulation-Based Training in Orthopaedic Surgery: Current Evidence and Limitations

Simulation-based training has been widely adopted by surgical educators and is now an essential component of the modern resident's skills acquisition pathway and career progression. The challenges faced by residents because of lack of exposure as a result of working-time directives-and now the COVID-19 (coronavirus disease 2019) pandemic limiting nonurgent and elective operating-reinforce the need for evidence-based simulation training. Although a wide range of training platforms have been developed, very few have shown transfer of skills. Simulation is thought to enhance the initial phase of the procedural learning curve; however, this hypothesis is yet to be tested in a high-quality study. Nevertheless, in light of the current evidence, simulation-based procedural curricula should be developed using the strengths of multiple different training platforms while incorporating the essential concept of nontechnical skills.

Hybrid Spine Simulator Prototype for X-ray Free Pedicle Screws Fixation Training

Simulation for surgical training is increasingly being considered a valuable addition to traditional teaching methods. 3D-printed physical simulators can be used for preoperative planning and rehearsal in spine surgery to improve surgical workflows and postoperative patient outcomes. This paper proposes an innovative strategy to build a hybrid simulation platform for training of pedicle screws fixation: the proposed method combines 3D-printed patient-specific spine models with augmented reality functionalities and virtual X-ray visualization, thus avoiding any exposure to harmful radiation during the simulation. Software functionalities are implemented by using a low-cost tracking strategy based on fiducial marker detection. Quantitative tests demonstrate the accuracy of the method to track the vertebral model and surgical tools, and to coherently visualize them in either the augmented reality or virtual fluoroscopic modalities. The obtained results encourage further research and clinical validation towards the use of the simulator as an effective tool for training in pedicle screws insertion in lumbar vertebrae.

HipSim - hip fracture surgery simulation utilizing the Learning Curve-Cumulative Summation test (LC-CUSUM)

Background and purpose - Virtual reality simulation of hip fracture surgery is available for orthopedic residents nationwide in Denmark. Summative assessment of learning applying the learning curve cumulative summation test (LC-CUSUM) has not been utilized in orthopedic simulation training. The strength of the LC-CUSUM is that it assumes incompetency and signals competency based on solid statistics. We investigated the LC-CUSUM characteristics of novices stepwise mastering the simulated dynamic hip screw (DHS) procedure. Material and methods - 32 1st-year orthopedic residents participated in HipSim and its 3 subsequent LC-CUSUM evaluations: placing a Kirschner wire, placing a Kirschner wire in different patients, and performing the entire DHS procedure in different patients. The career status of the participants, i.e., still working in orthopedics or in another specialty was recorded ≥ 2 years after participation and associated with the simulation performance (passed/failed). Results - 13/14 participants passing HipSim according to LC-CUSUM were still working in orthopedics, while 9/18 participants failing HipSim had quit orthopedics at ≥ 2 years follow-up. The simulator-generated feedback did not statistically significantly differ between the groups. Interpretation - LC-CUSUM and its summative pass/fail assessment of each simulation was feasible in this formative simulation program. Clinical educators can be reassured that participants passing HipSim are likely to continue to 2nd-5th year of residency, while failing HipSim should raise concerns and trigger career counselling and clinical supervision. The motivational aspect of LC-CUSUM pass/fail assessment when designing formative simulation training warrants further research.