Surgical Simulation: Markers of Proficiency

Simulating success: Applying EPA assessments to simulation in surgical education

Entrustable professional activities are a competency-based evaluation framework which was deployed by the American Board of Surgery in 2023 to evaluate general surgical residents and provide a path to independent practice. Entrustable professional activity microassessments are based on 18 conditions which are core to being a practicing general surgeon, and most include multiple phases of care, such as preoperative care, intraoperative care, and postoperative care. These evaluations are an amalgam of all the clinical competencies, including medical knowledge and patient care skills. Many concerns have arisen about declining practice readiness of general surgery trainees over the past couple of decades, while simulation in surgical education has emerged over a similar time frame. Simulation has the potential to boost trainee confidence, prepare trainees for specific clinical scenarios, and provide a means to practice core operative maneuvers. Traditional evaluation of trainees participating in simulated scenarios is variable, and the entrustable professional activity framework has the potential to lend structure to simulated scenarios such that they may more closely model a path to operative autonomy and independence while familiarizing early-stage trainees with the entrustable professional activity evaluation format. However, these simulations are just that-a simulated environment whose efficacy hinges upon its similarity to actual clinical care. Entrustable professional activity assessments of simulated scenarios, while potentially valuable, should not be used as actual resident evaluations, as they cannot replace authentic clinical training.

The Future of Artificial Intelligence in Surgery

Until recently, innovations in surgery were largely represented by extensions or augmentations of the surgeon's perception. This includes advancements such as the operating microscope, tumor fluorescence, intraoperative ultrasound, and minimally invasive surgical instrumentation. However, introducing artificial intelligence (AI) into the surgical disciplines represents a transformational event. Not only does AI contribute substantively to enhancing a surgeon's perception with such methodologies as three-dimensional anatomic overlays with augmented reality, AI-improved visualization for tumor resection, and AI-formatted endoscopic and robotic surgery guidance. What truly makes AI so different is that it also provides ways to augment the surgeon's cognition. By analyzing enormous databases, AI can offer new insights that can transform the operative environment in several ways. It can enable preoperative risk assessment and allow a better selection of candidates for procedures such as organ transplantation. AI can also increase the efficiency and throughput of operating rooms and staff and coordinate the utilization of critical resources such as intensive care unit beds and ventilators. Furthermore, AI is revolutionizing intraoperative guidance, improving the detection of cancers, permitting endovascular navigation, and ensuring the reduction in collateral damage to adjacent tissues during surgery (e.g., identification of parathyroid glands during thyroidectomy). AI is also transforming how we evaluate and assess surgical proficiency and trainees in postgraduate programs. It offers the potential for multiple, serial evaluations, using various scoring systems while remaining free from the biases that can plague human supervisors. The future of AI-driven surgery holds promising trends, including the globalization of surgical education, the miniaturization of instrumentation, and the increasing success of autonomous surgical robots. These advancements raise the prospect of deploying fully autonomous surgical robots in the near future into challenging environments such as the battlefield, disaster areas, and even extraplanetary exploration. In light of these transformative developments, it is clear that the future of surgery will belong to those who can most readily embrace and harness the power of AI.

French translation and validation of the OSATS tool for the assessment of surgical skill

AIM OF THE STUDY

The OSATS rating scale is one of the tools most widely used in the scientific literature for assessing technical surgical skill. The aim of this study was to translate this scale into French (OSATS-FR) and validate it.

METHODS

OSATS was translated into French by two independent translators. The two translations were pooled and back-translated by a third translator into English to produce a consensus version (OSATS-FR). Three independent assessors then used it to assess surgical skill in tracheostomy videos. The reproducibility of the scoring was measured.

RESULTS

The OSATS-FR version was easily understood and used by all three assessors. In all, 18 procedures were assessed, performed by 14 interns and 4 senior surgeons. The mean OSATS-FR score was 18.6±4.08 for the interns and 31.6±1.62 for the seniors (p<0.0001). The Pearson correlation coefficient was 0.937 CI95% [0.867-1.269] between assessors 1 and 2, 0.977 CI95% [0.899-1.149] between assessors 1 and 3, and 0.965 CI95% [0.876-1.196] between assessors 2 and 3. Cronbach's alpha was greater than 0.9 for each assessor.

CONCLUSION

OSATS-FR can distinguish between two groups with different surgical expertise and assess surgical skill with an inter-assessor reproducibility and internal consistency comparable to those of the English version.

Design and validation of a hemispherectomy simulator for neurosurgical education

OBJECTIVE

Early adaptors of surgical simulation have documented a translation to improved intraoperative surgical performance. Similar progress would boost neurosurgical education, especially in highly nuanced epilepsy surgeries. This study introduces a hands-on cerebral hemispheric surgery simulator and evaluates its usefulness in teaching epilepsy surgeries.

METHODS

Initially, the anatomical realism of the simulator and its perceived effectiveness as a training tool were evaluated by two epilepsy neurosurgeons. The surgeons independently simulated hemispherotomy procedures and provided questionnaire feedback. Both surgeons agreed on the anatomical realism and effectiveness of this training tool. Next, construct validity was evaluated by modeling the proficiency (task-completion time) of 13 participants, who spanned the experience range from novice to expert.

RESULTS

Poisson regression yielded a significant whole-model fit (χ2 = 30.11, p < 0.0001). The association between proficiency when using the training tool and the combined effect of prior exposure to hemispherotomy surgery and career span was statistically significant (χ2 = 7.30, p = 0.007); in isolation, pre-simulation exposure to hemispherotomy surgery (χ2 = 6.71, p = 0.009) and career length (χ2 = 14.21, p < 0.001) were also significant. The mean (± SD) task-completion time was 25.59 ± 9.75 minutes. Plotting career length against task-completion time provided insights on learning curves of epilepsy surgery. Prediction formulae estimated that 10 real-life hemispherotomy cases would be needed to approach the proficiency seen in experts.

CONCLUSIONS

The cerebral hemispheric surgery simulator is a reasonable epilepsy surgery training tool in the quest to increase preoperative practice opportunities for neurosurgical education.

Subjective vs. objective assessment of simulation performance on laparoscopic cholecystectomy: are we evaluating the right things?

BACKGROUND

Simulation using virtual reality (VR) simulators is an important tool in surgical training. VR laparoscopic simulators can provide immediate objective performance assessment without observer evaluation. This study aims to explore the correlation between subjective observer evaluation and VR laparoscopic simulator performance metrics in a laparoscopic cholecystectomy (LC) simulation module.

METHODS

A LC simulation module using a VR laparoscopic simulator was completed by PGY2-3 general surgery residents at a single institution. Simulation performance was recorded and evaluated by a trained evaluator using the validated Global Operative Assessment of Laparoscopic Skills (GOALS) form, the Objective Structured Assessment of Technical Skills (OSATS) form, and a LC-specific simulation assessment form (LC-SIM). Objective performance metrics were also obtained from the simulator system. Performance before the curriculum (pre-test) and after the curriculum (post-test) were compared.

RESULTS

Fourteen residents were included in the study. There were significant improvements from pre-test to post-test on each component of GOALS, OSATS, and LC-SIM scores (all p values < 0.05). In terms of objective simulator metrics, significant improvements were noted in time to extract gallbladder (481 ± 221 vs 909 ± 366 min, p = 0.019), total number of movements (475 ± 264 vs 839 ± 324 min, p = 0.012), and total path length (955 ± 475 vs 1775 ± 632 cm, p = 0.012) from pre-test to post-test. While number of movements and total path lengths of both hands decreased, speed of right instrument also decreased from 4.1 + 2.7 to 3.0 ± 0.7 cm/sec (p = 0.007). Average speed of left instrument was associated with respect for tissue (r = 0.60, p < 0.05) and depth perception (r = 0.68, p < 0.05) on post-test evaluations.

CONCLUSION

Our study demonstrated significant improvement in technical skills based on subjective evaluator assessment as well as objective simulator metrics after simulation. The few correlations identified between the subjective evaluator and the objective simulator assessments suggest the two evaluation modalities were measuring different aspects of the technical skills and should both be considered in the evaluation process.

The impact of virtual reality simulation training on operative performance in laparoscopic cholecystectomy: meta-analysis of randomized clinical trials

BACKGROUND

Simulation training can improve the learning curve of surgical trainees. This research aimed to systematically review randomized clinical trials (RCT) evaluating the performance of junior surgical trainees following virtual reality training (VRT) and other training methods in laparoscopic cholecystectomy.

METHODS

MEDLINE (PubMed), Embase (Ovid SP), Web of Science, Scopus and LILACS were searched for trials randomizing participants to VRT or no additional training (NAT) or simulation training (ST). Outcomes of interest were the reported performance using global rating scores (GRS), the Objective Structured Assessment of Technical Skill (OSATS) and Global Operative Assessment of Laparoscopic Skills (GOALS), error counts and time to completion of task during laparoscopic cholecystectomy on either porcine models or humans. Study quality was assessed using the Cochrane Risk of Bias Tool. PROSPERO ID: CRD42020208499.

RESULTS

A total of 351 titles/abstracts were screened and 96 full texts were reviewed. Eighteen RCT were included and 15 manuscripts had data available for meta-analysis. Thirteen studies compared VRT and NAT, and 4 studies compared VRT and ST. One study compared VRT with NAT and ST and reported GRS only. Meta-analysis showed OSATS score (mean difference (MD) 6.22, 95%CI 3.81 to 8.36, P < 0.001) and time to completion of task (MD -8.35 min, 95%CI 13.10 to 3.60, P = <0.001) significantly improved after VRT compared with NAT. No significant difference was found in GOALS score. No significant differences were found between VRT and ST groups. Intraoperative errors were reported as reduced in VRT groups compared with NAT but were not suitable for meta-analysis.

CONCLUSION

Meta-analysis suggests that performance measured by OSATS and time to completion of task is improved with VRT compared with NAT for junior trainee in laparoscopic cholecystectomy. However, conclusions are limited by methodological heterogeneity and more research is needed to quantify the potential benefit to surgical training.

Validation of a Cost-effective Cast Saw Simulation-based Educational Module to Improve Cast Removal Safety

BACKGROUND

Inexperience in cast removal in the pediatric population can lead to a range of cast saw-related injuries. The purpose of this study is to validate a simple simulation-based wax model that is both reproducible and economical while providing a valuable tool that can be used to grade cast saw use performance in trainees.

METHODS

Cylindrical wax models were used as an analog for a pediatric upper extremity. The wax models were casted in a proscribed reproducible fashion for consistency. Two groups, the first consisting of 15 experienced cast saw users and the second consisting of 15 inexperienced individuals, completed 4 sequential longitudinal cuts in the casted wax models. After removal of the cast material, marks left by the cast saw in the wax were counted and measured. Indentation length, maximum depth, and maximum width were measured on each wax model. The total length of the cast saw indentations per cast saw user was also calculated.

RESULTS

For the inexperienced cast saw users, the average total length of the cast saw indentations was 526.56 mm, average maximum depth was 1.91 mm, and average maximum width was 3.24 mm. For experienced cast saw users, the average total length of the cast saw indentations was 156.57 mm with an average maximum depth of 1.06 mm and average maximum width of 2.19 mm. Receiver operating characteristic curves of the total number of errors, total error length, maximum error depth, and maximum error width show effective discrimination of experienced from inexperienced trainees.

CONCLUSIONS

This study provides valid evidence supporting a cost-effective, time-efficient, and easily reproducible educational simulation module that can objectively measure cast saw the performance in trainees. This model demonstrates construct validity and can distinguish novice from experienced cast saw users. It is sensitive enough to identify mistakes even in the most experienced cast saw users, creating a platform that can provide performance-based feedback to cast saw users of all experience levels.

LEVEL OF EVIDENCE

Level III-diagnostic test.

Low-Fidelity Arthroscopic Simulation Training in Trauma and Orthopaedic Surgery: A Systematic Review of Experimental Studies

PURPOSE

To identify and appraise evidence assessing the effectiveness of low-fidelity arthroscopic simulation in the acquisition of arthroscopic surgical skills in a novice population.

METHODS

Four databases were electronically searched in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) algorithm. Studies from any year that described the use of orthopaedic, low-fidelity arthroscopic training models in novice populations were included. Questionnaires, case studies, and review studies were excluded. Risk of bias assessment was conducted using the Cochrane Collaboration's Risk of Bias Tool or the Cochrane Risk of Bias in Non-Randomised Studies of Interventions (ROBINS-I) checklist.

RESULTS

Sixteen studies were identified. Using the PRISMA algorithm, 6 studies were included with a total of 131 novice participants. Individual studies ranged from 8 to 40 novices and were of Level I to II evidence. Outcome measurements varied between studies (total 16 different outcomes used). Various outcome measures used for assessing arthroscopic surgical skills within all 6 studies demonstrated significant improvement. A cross-study subjective outcome synthesis revealed low-fidelity arthroscopic simulators reduced time to completion outcomes (2 studies, P < .05), increased Arthroscopic Surgical Skill Evaluation Tool scores (2 studies, P < .01), and confirmed face validity (2 studies) and transfer of skills to cadavers (2 studies) or live patients (1 study). Cost data were under-reported in all studies apart from one.

CONCLUSIONS

Arthroscopic training using low-fidelity simulators likely improves the performance of novice participants in completing basic arthroscopic procedures. These simulators may also be more cost effective and thus more implementable than their high-fidelity counterparts.

LEVEL OF EVIDENCE

Level II, systematic review of Level I-II studies.

[Pedagogical impact of a MOOC on surgical technique of kidney transplantation]

OBJECTIVE

To evaluate the educational impact of a pilot MOOC (Massive Open Online Course), validated by the French College of Urology Teachers (FCUT), on the surgical technique of kidney transplantation.

MATERIALS AND METHODS

We developed a MOOC on the surgical technique of kidney transplantation, based on a video of a surgical procedure, performed by an expert surgeon. The MOOC has been validated by the FCUT. We have created 2 student groups: 1) MOOC-pre-QCM group: visualization of the MOOC then answer to the MCQs and satisfaction questions; 2) MOOC-post-QCM group: answer to the MCQs then visualization of the MOOC then answers to the satisfaction questions. In total, 20 MCQs on the kidney transplantation technique were completed by the 2 groups. The answers were anonymous.

RESULTS

A total of 142 people answered the MCQs (MOOC-pre-QCM group (n=66) and MOOC-post-QCM group (n=76)). Twenty-nine percent (41/142) of the participants were fellows and 71 % (101/142) were residents. The proportion of fellows and residents was identical between the 2 groups. The rate of correct answers to the 20 MCQs was statistically higher in the MOOC-pre-QCM group, compared to the MOOC-post-QCM group (88.6 % versus 73.3 %, P<0.0001). Ninety-one percent of students found the MOOC "Very Useful" or "Useful". The median MOOC rating, given by students, was 8/10.

CONCLUSION

This study showed a positive impact of the MOOC on theoretical knowledge of kidney transplantation surgical technique. This MOOC could serve as a pilot project for the development of other MOOCs on urological surgery.

LEVEL

3.

A cost-effective IoT learning environment for the training and assessment of surgical technical skills with visual learning analytics

BACKGROUND

Surgeons need to train and certify their technical skills. This is usually done with the intervention of experts who monitor and assess trainees. Nevertheless, this is a time-consuming task that is subject to variations among evaluators. In recent decades, subjectivity has been significantly reduced through 1) the introduction of standard curricula, such as the Fundamentals of Laparoscopic Surgery (FLS) program, which measures students' performance in specific exercises, and 2) rubrics, which are widely accepted in the literature and serve to provide feedback about the overall technical skills of the trainees. Although these two elements reduce subjectivity, they do not, however, eliminate the figure of the expert evaluator, and so the process remains time consuming.

OBJECTIVES

The objective of this work is to automate those parts of the work of the expert evaluator that the technology can measure objectively, using sensors to collect evidence, and visualizations to provide feedback. We designed and developed 1) a cost-effective IoT (Internet of Things) learning environment for the training and assessment of surgical technical skills and 2) visualizations supported by the literature on visual learning analytics (VLA) to provide feedback about the exercises (in real time) and overall performance (at the end of the training) of the trainee.

METHODS

A hybrid approach was followed based on previous research for the design of the sensor based IoT learning environment. Previous studies were used as the basis for getting best practices on the tracking of surgical instruments and on the detection of the force applied to the tissue, with a focus on reducing the costs of data collection. The monitoring of the specific exercises required the design of sensors and collection mechanisms from scratch as there is little existing research on this subject. Moreover, it was necessary to design the overall architecture to collect, process, synchronize and communicate the data coming from the different sensors to provide high-level information relevant to the end user. The information to be presented was already validated by the literature and the focus was on how to visualize this information and the optimal time for its presentation to end users. The visualizations were validated with 18 VLA experts assessing the technical aspects of the visualizations and 4 medical experts assessing their functional aspects.

RESULTS

This IoT learning environment amplifies the evaluation mechanisms already validated by the literature, allowing automatic data collection. First, it uses IoT sensors to automatically correct two of the exercises defined in the FLS (peg transfer and precision cutting), providing real-time visualizations. Second it monitors the movement of the surgical instruments and the force applied to the tissues during the exercise, computing 6 of the high-level indicators used by expert evaluators in their rubrics (efficiency, economy of movement, hand tremor, depth perception, bimanual dexterity, and respect for tissue), providing feedback about the technical skills of the trainee using a radar chart with these six indicators at the end of the training (summative visualizations).

CONCLUSIONS

The proposed IoT learning environment is a promising and cost-effective alternative to help in the training and assessment of surgical technical skills. The system shows the trainees' progress and presents new indicators about the correctness of each specific exercise through real-time visualizations, as well as their general technical skills through summative visualizations, aligned with the 6 more frequent indicators in standardized scales. Early results suggest that although both types of visualizations are useful, it is necessary to reduce the cognitive load of the graphs presented in real time during training. Nevertheless, an additional evaluation is needed to confirm these results.

Alternatives in Education-Rat and Mouse Simulators Evaluated from Course Trainers' and Supervisors' Perspective

Simulators allow the inexperienced to practice their skills prior to exercise on live animals. Therefore, they bear great potential in overcoming the dilemma between the present demand for high-quality practical training involving live animals whilst implementing the 3R principle according to the Directive 2010/63/EU. Currently, one mouse and six rat simulators are commercially available. As data on their impact are lacking, this project aimed at providing an overview of the awareness, implementation, and methodical and practical satisfaction provided by 35 course trainers and supervisors of laboratory animal training courses for mice and rats regarding the simulators available. Although simulators facilitate training of relevant techniques and relatively high awareness of them seemed to be present, their implementation is currently very low, possibly due to lack of meeting the respondents' demands. Thus, this study revealed the overall approval of simulator training and general demand for user-optimized, realistic, and financially affordable simulators and, hence, indicates a strong impulse for new developments strengthening the 3Rs as a benefit to all animals used in research.

The Efficacy of Arthroscopic Simulation Training on Clinical Ability: A Systematic Review

PURPOSE

To examine the effect of arthroscopic simulator training on technical performance in a human model.

METHODS

A systematic review was conducted in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Literature searches of PubMed, Embase, and Cochrane Library were conducted using combinations of the terms virtual, digital, computer, reality, simulation, arthroscopy, training, learning, and education. Studies were considered for inclusion if they tested the effect of arthroscopic simulator training in a randomized controlled fashion, performed testing in a cadaver or live patient, and used explicit outcome measures of technical skill. Data from studies were extracted and study characteristics and outcomes were reviewed. The primary outcome measure was the number of studies in which the simulation trained group had significantly improved performance results relative to the control group in ≥50% of all measured outcomes. Risk of bias was assessed with Cochrane's Collaboration Tool.

RESULTS

Twelve studies, including 340 total study participants, were included for review. Eight studies showed improved performance of the simulation trained group relative to the control group in ≥50% of assessed outcomes. Six of ten studies reporting completion time, three of six studies reporting task checklist completion, 3 of 7 studies reporting global rating scales, and 1 of 4 studies reporting Arthroscopic Surgical Skill Evaluation Tool scores showed improved performance of the simulation group relative to the control group for the respective outcome measures.

CONCLUSIONS

The literature is limited due to heterogeneity, both in type and merit, of the outcome measures that have been used to assess the transfer validity of arthroscopic simulator training to clinical performance. Despite the limitations of the literature, this review demonstrates that arthroscopic simulator training has potential to improve clinical performance.

LEVEL OF EVIDENCE

II, systematic review of Level II studies.

Global versus task-specific postoperative feedback in surgical procedure learning

BACKGROUND

Task-specific checklists and global rating scales are both recommended assessment tools to provide constructive feedback on surgical performance. This study evaluated the most effective feedback tool by comparing the effects of the Observational Clinical Human Reliability Analysis (OCHRA) and the Objective Structured Assessment of Technical Skills (OSATS) on surgical performance in relation to the visual-spatial ability of the learners.

METHODS

In a randomized controlled trial, medical students were allocated to either the OCHRA (n = 25) or OSATS (n = 25) feedback group. Visual-spatial ability was measured by a Mental Rotation Test. Participants performed an open inguinal hernia repair procedure on a simulation model twice. Feedback was provided after the first procedure. Improvement in performance was evaluated blindly using a global rating scale (performance score) and hand-motion analysis (time and path length).

RESULTS

Mean improvement in performance score was not significantly different between the OCHRA and OSATS feedback groups (P = .100). However, mean improvement in time (371.0 ± 223.4 vs 274.6 ± 341.6; P = .027) and path length (53.5 ± 42.4 vs 34.7 ± 39.0; P = .046) was significantly greater in the OCHRA feedback group. When stratified by mental rotation test scores, the greater improvement in time (P = .032) and path length (P = .053) was observed only among individuals with low visual-spatial abilities.

CONCLUSION

A task-specific (OCHRA) feedback is more effective in improving surgical skills in terms of time and path length in novices compared to a global rating scale (OSATS). The effects of a task-specific feedback are present mostly in individuals with lower visual-spatial abilities.

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

The assessment of surgical technical skills to be acquired by novice surgeons has been traditionally done by an expert surgeon and is therefore of a subjective nature. Nevertheless, the recent advances on IoT (Internet of Things), the possibility of incorporating sensors into objects and environments in order to collect large amounts of data, and the progress on machine learning are facilitating a more objective and automated assessment of surgical technical skills. This paper presents a systematic literature review of papers published after 2013 discussing the objective and automated assessment of surgical technical skills. 101 out of an initial list of 537 papers were analyzed to identify: 1) the sensors used; 2) the data collected by these sensors and the relationship between these data, surgical technical skills and surgeons' levels of expertise; 3) the statistical methods and algorithms used to process these data; and 4) the feedback provided based on the outputs of these statistical methods and algorithms. Particularly, 1) mechanical and electromagnetic sensors are widely used for tool tracking, while inertial measurement units are widely used for body tracking; 2) path length, number of sub-movements, smoothness, fixation, saccade and total time are the main indicators obtained from raw data and serve to assess surgical technical skills such as economy, efficiency, hand tremor, or mind control, and distinguish between two or three levels of expertise (novice/intermediate/advanced surgeons); 3) SVM (Support Vector Machines) and Neural Networks are the preferred statistical methods and algorithms for processing the data collected, while new opportunities are opened up to combine various algorithms and use deep learning; and 4) feedback is provided by matching performance indicators and a lexicon of words and visualizations, although there is considerable room for research in the context of feedback and visualizations, taking, for example, ideas from learning analytics.

Developing and Assessing the Feasibility of Implementing a Surgical Objective Structured Clinical Skills Examination (S-OSCE)

OBJECTIVE

To1 describe the development and evaluate the feasibility of a surgical objective structured clinical examination (OSCE) for the purpose of competency assessment based on the Royal College of Canada's CanMEDS framework.

DESIGN

A unique surgical OSCE was developed to evaluate the clinical and surgical management of common orthopaedic problems using simulated patients and cadaveric specimens. Cases were graded by degree of difficulty (less complex, complex, more complex) Developing an assessment tool with significant resource utilization and good correlation with traditional methods is challenging. The feasibility of an OSCE that evaluates independent clinical and surgical decision making was evaluated. In addition, as part of establishing construct validity, correlation of OSCE scores with previously validated O-scores was performed.

SETTING

A tertiary level academic teaching hospital.

PARTICIPANTS

Thirty-four Postgraduate year 3-5 trainees of a 5-year Canadian orthopedic residency program creating 96 operative case performances available for final review.

RESULTS

The development of the OSCE cases involved a multistep process with attending surgeons, residents and a surgical education consultant. There were 4 different OSCE days, over a 3 year period (2016-2018) encompassing a variety of less complex and more complex procedures. Performance on the OSCE correlated strongly with the (O-SCORE, 0.89) and a linear regression analysis correlated moderately with year of training (r² = 0.5737). The feasibility analysis demonstrated good financial practicality with solid programmatic integration.

CONCLUSIONS

The unique surgical OSCE scores correlate strongly with an established entrustability scale. Administering this OSCE to evaluate preoperative and intraoperative decision making to complement other forms of assessment is feasible. The financial burden to training programs is modest in comparison to the insight gained by both residents and faculty.

Fully Immersive Virtual Reality for Total Hip Arthroplasty: Objective Measurement of Skills and Transfer of Visuospatial Performance After a Competency-Based Simulation Curriculum

BACKGROUND

Fully immersive virtual reality (VR) uses headsets to situate a surgeon in a virtual operating room to perform open surgical procedures. The aims of this study were to determine (1) if a VR curriculum for training residents to perform anterior approach total hip replacement (AA-THR) was feasible, (2) if VR enabled residents' performance to be measured objectively, and (3) if cognitive and motor skills that were learned with use of VR were transferred to the physical world.

METHODS

The performance of 32 orthopaedic residents (surgical postgraduate years [PGY]-1 through 4) with no prior experience with AA-THR was measured during 5 consecutive VR training and assessment sessions. Outcome measures were related to procedural sequence, efficiency of movement, duration of surgery, and visuospatial precision in acetabular component positioning and femoral neck osteotomy, and were compared with the performance of 4 expert hip surgeons to establish competency-based criteria. Pretraining and post-training assessments on dry bone models were used to assess the transfer of visuospatial skills from VR to the physical world.

RESULTS

Residents progressively developed surgical skills in VR on a learning curve through repeated practice, plateauing, on average, after 4 sessions (4.1 ± 0.6 hours); they reached expert VR levels for 9 of 10 metrics (except femoral osteotomy angle). Procedural errors were reduced by 79%, assistive prompts were reduced by 70%, and procedural duration was reduced by 28%. Dominant and nondominant hand movements were reduced by 35% and 36%, respectively, and head movement was reduced by 44%. Femoral osteotomy was performed more accurately, and acetabular implant orientation improved in VR assessments. In the physical world assessments, experts were more accurate than residents prior to simulation, but were matched by residents after simulation for all of the metrics except femoral osteotomy angle. The residents who performed best in VR were the most accurate in the physical world, while 2 residents were unable to achieve competence despite sustained practice.

CONCLUSIONS

For novice surgeons learning AA-THR skills, fully immersive VR technology can objectively measure progress in the acquisition of surgical skills as measured by procedural sequence, efficiency of movement, and visuospatial accuracy. Skills learned in this environment are transferred to the physical environment.