Identifying orthopaedic surgeons of the future

Movement-based cognitive training does not significantly shorten the learning curve for acquiring arthroscopic basic skills

PURPOSE

Skilful arthroscopy requires an aboveaverage level of manual dexterity. It is evident that particular motor skills can be learned and trained before arthroscopic training. The aim of this prospective cohort study was to investigate the impact of movement-related cognitive training on the learning curve during arthroscopic basic training.

METHODS

Fifty right-handed participants without arthroscopic experience were matched to an intervention group (n = 25) and a control group (n = 25). Prior to basic arthroscopic skill training with a simulator, the intervention group underwent 12 weeks of movement-related cognitive training. Cognitive and motor skills were assessed in both groups by using standardised tests (CogniFit test, angle reproduction test, two-arm coordination test) as a pretest and, for the intervention group, again before arthroscopic training as a posttest. For arthroscopic simulator training, three tasks ('Telescoping', 'Periscoping', 'Triangulation') from the Fundamentals of Arthroscopic Surgery Training module were selected and practiced 10 times with the camera in the right and left hands. The learning progress was quantified by exercise time, camera path length and hook path length.

RESULTS

No significant differences in sex distribution, age distribution or the results of the pretests between the intervention group (n = 21) and the control group (n = 25) were found (n.s.). The intervention group improved significantly from the pretest to the posttest in the CogniFit (p = 0.003) and two-arm coordination test in terms of time (p < 0.001) and errors (p = 0.002) but not in the angle reproduction test. No significant differences were found between the groups for the three arthroscopic tasks.

CONCLUSION

The hypothesis that movement-related cognitive training shortens the learning curve for acquiring arthroscopic basic skills cannot be confirmed. Other factors influencing the learning curve such as talent, teaching method and motivation have a greater impact on the acquisition of complex motor skills.

LEVEL OF EVIDENCE

Level II.

Performance in Irish Selection and Future Performance in Surgical Training

Importance

Selection processes for surgical training should aim to identify candidates who will become competent independent practitioners and should aspire to high standards of reliability and validity.

Objective

To determine the association between measured candidate factors at the time of an Irish selection and assessment outcomes in surgical training, examined via rate of progression to Higher Specialist Training (HST), attrition rates, and performance as assessed through a multimodal framework of workplace-based and simulation-based assessments.

Design, Setting, and Participants

This retrospective observational cohort study included data from all successful applicants to the Royal College of Surgeons in Ireland (RCSI) national Core Surgical Training (CST) program. Participants included all trainees recruited to dedicated postgraduate surgical training from 2016 to 2020. These data were analyzed from July 11, 2016, through July 10, 2022.

Exposures

Selection decisions were based on a composite score that was derived from technical aptitude assessments, undergraduate academic performance, and a 4-station multiple mini-interview.

Main outcomes and measures

Assessment data, attrition rates, and rates of progression to HST were recorded for each trainee. CST performance was assessed using workplace-based and simulation-based technical and nontechnical skill assessments. Potential associations between selection and assessment measures were explored using Pearson correlation, logistic regression, and multiple linear-regression analyses.

Results

Data were available for 303 trainees. Composite scores were positively associated with progression to HST (odds ratio [OR], 1.09; 95% CI, 1.05-1.13). There was a weak positive correlation, ranging from 0.23 to 0.34, between scores and performance across all CST assessments. Multivariable linear regression analysis showed technical aptitude scores at application were associated with future operative performance assessment scores, both in the workplace (β = 0.31; 95% CI, 0.14-0.48) and simulated environments (β = 0.57; 95% CI, 0.33-0.81). There was evidence that the interpersonal skills interview station was associated with future performance in simulated communication skill assessments (β = 0.55; 95% CI, 0.22-0.87).

Conclusions and Relevance

In this study, performance at the time of Irish national selection, measured across technical and nontechnical domains in a multimodal fashion, was associated with future performance in the workplace and in simulated environments. Future studies will be required to explore the consequential validity of selection, including potential unintended effects of selection and ranking on candidate performance.

The number of arthroscopies performed by trainees does not deduce the level of their arthroscopic proficiency

PURPOSE

It is reasonable to question whether the case volume is a suitable proxy for the manual competence of an arthroscopic surgeon. The aim of this study was to evaluate the correlation between the number of arthroscopies previously performed and the arthroscopic skills acquired using a standardized simulator test.

METHODS

A total of 97 resident and early orthopaedic surgeons who participated in arthroscopic simulator training courses were divided into five groups based on their self-reported number of arthroscopic surgeries: (1) none, (2) < 10, (3) 10 to 19, (4) 20 to 39 and (5) 40 to 100. Arthroscopic manual skills were evaluated with a simulator by means of the diagnostic arthroscopy skill score (DASS) before and after training. Seventy-five points out of 100 must be achieved to pass the test.

RESULTS

In the pretest, only three trainees in group 5 passed the arthroscopic skill test, and all other participants failed. Group 5 (57 ± 17 points; n = 17) scored significantly higher than the other groups (group 1: 30 ± 14, n = 20; group 2: 35 ± 14, n = 24; group 3: 35 ± 18, n = 23; and group 4: 33 ± 17, n = 13). After a two-day simulator training, trainees showed a significant increase in performance. In group 5, participants scored 81 ± 17 points, which was significantly higher than the other groups (group 1: 75 ± 16; group 2: 75 ± 14; group 3: 69 ± 15; and group 4: 73 ± 13). While self-reported arthroscopic procedures were n.s. associated with higher log odds of passing the test (p = 0.423), the points scored in the pretest were found to be a good predictor of whether a trainee would pass the test (p < 0.05). A positive correlation was observed between the points scored in the pretest and the posttest (p < 0.05, r = 0.59, r2 = 0.34).

CONCLUSIONS

The number of previously performed arthroscopies is not a reliable indicator of the skills level of orthopaedic residents. A reasonable alternative in the future would be to verify arthroscopic proficiency on the simulator by means of a score as a pass-fail examination.

LEVEL OF EVIDENCE

III.

Learning and Short-Term Retention of Simulation-Based Arthroscopic Skills

OBJECTIVE

The study aimed to examine the learning curve and short-term retention of arthroscopic skills acquired on a simulator.

DESIGN

Cohort study.

SETTING

Clinical Skills Training Center of Zhujiang Hospital of Southern Medical University PARTICIPANT AND METHODS: Orthopaedic residents (n = 14) without previous arthroscopy experience were included. After basic information was collected and an initial arthroscopy knowledge level test was administered, the subjects received standardised training on the simulator (day 1); then, they completed tasks on the simulator, including guided diagnostics (4 times), triangulation (5 times) and loose body removal (7 times). A learning curve for each skill was generated based on the total scores. The score of the last repetition of each task was the training level.

RESULTS

A total of 14 orthopedic residents were enrolled. All participants completed the training and testing. There was a learning curve over the course of training for all 3 arthroscopic skills (p < 0.001). On day 8 after the training, the mean score for guided diagnostics decreased from 49.9 to 48.9 (p = 0.001), and the retention rate was 97.8%. For triangulation, the mean total score decreased from 58.9 to 53.6 (p < 0.001), and the retention rate was 90.8%. For loose body removal, the mean total score decreased from 87.1 to 80.7 (p < 0.001), and the retention rate was 92.7%.

CONCLUSIONS

Orthopaedic residents' arthroscopic skills learned through simulator training declined significantly in 1 week after the training, especially more difficult skills.

Modeling Technical Skills Learning Curves of Incoming Surgical Trainees

OBJECTIVE

Not all trainees reach technical competency even after completing surgical training. While assessment of technical skill is not part of the residency interview process, identifying under-performers early on may help identify opportunities for individualized, targeted training. The objectives of this study were to (1) create predictive learning curve (LC) models for each of 3 basic laparoscopic tasks to identify performers versus underperformers and (2) evaluate the use of LCs to identify underperformers during selection into surgical training.

DESIGN

Predictive LC models were created for laparoscopic pattern cutting (PC), peg transfer (PT) and intra-corporeal knots (IC) over 40 repetitions by 65 novice trainees in 2014. Trainees were categorized as performers and underperformers. Receiver operator characteristic analysis determined the minimum number of repetitions required to predict individual LCs, which were then used to determine the proportion of underperformers.

SETTING

Technical performance was assessed onsite at the Canadian Residence Matching Service (CaRMS) interviews, after interview completion (January 2015).

PARTICIPANTS

Applicants to general surgery (GS) and gynecology (OBGYN) participated in a skills assessment during.

RESULTS

The PC, PT and IC tasks required a minimum of 8, 10, and 5 repetitions respectively, to predict overall performance. Predictive values for each task had excellent sensitivity and specificity: 1.00, 1.00 (PC); 1.00, 1.00 (PT); and 0.94, 1.00 (IC). Eighty applicants completed 8 PC repetitions; 16% were identified as underperformers.

CONCLUSIONS

Individual LCs for three different laparoscopic tasks can be predicted with excellent sensitivity and specificity based on 10 repetitions or less. This information can be used to identify trainees who may have difficulty with laparoscopic technical skills early on.

The frequency of assessment tools in arthroscopic training: a systematic review

BACKGROUND

Multiple assessment tools are used in arthroscopic training and play an important role in feedback. However, it is not fully recognized as to the standard way to apply these tools. Our study aimed to investigate the use of assessment tools in arthroscopic training and determine whether there is an optimal way to apply various assessment tools in arthroscopic training.

METHODS

A search was performed using PubMed, Embase and Cochrane Library electronic databases for articles published in English from January 2000 to July 2021. Eligible for inclusion were primary research articles related to using assessment tools for the evaluation of arthroscopic skills and training environments. Studies that focussed only on therapeutic cases, did not report outcome measures of technical skills, or did not mention arthroscopic skills training were excluded.

RESULTS

A total of 28 studies were included for review. Multiple assessment tools were used in arthroscopic training. The most common objective metric was completion time, reported in 21 studies. Technical parameters based on simulator or external equipment, such as instrument path length, hand movement, visual parameters and injury, were also widely used. Subjective assessment tools included checklists and global rating scales (GRS). Among these, the most commonly used GRS was the Arthroscopic Surgical Skill Evaluation Tool (ASSET). Most of the studies combined objective metrics and subjective assessment scales in the evaluation of arthroscopic skill training.

CONCLUSIONS

Overall, both subjective and objective assessment tools can be used as feedback for basic arthroscopic skill training, but there are still differences in the frequency of application in different contexts. Despite this, combined use of subjective and objective assessment tools can be applied to more situations and skills and can be the optimal way for assessment.

LEVEL OF EVIDENCE

Level III, systematic review of level I to III studies. Key messagesBoth subjective and objective assessment tools can be used as feedback for basic arthroscopic skill training.Combined use of subjective and objective assessment tools can be applied to more situations and skills and can be the optimal way for assessment.

The Interactive Relationship Between Instructor Perceptions and Learner Personality on Surgical Skills Performance

OBJECTIVE

The objective of this study was to examine the association between learner personality and capacity to be trained (i.e., performance improvement) on a surgical task, and how instructor perceptions of the learners' capacity to be trained interact with learner personality and performance during training and feedback. There is meaningful heterogeneity in the degree of learner surgical skills acquisition despite receiving the same amount of training. While learner personality may independently contribute to skill acquisition, the instructor-learner feedback process is also important to consider. To better understand this interpersonal relationship, it is necessary to also consider instructor factors (i.e., perceptions), and how this may contribute to learner variability in skills training.

DESIGN

This exploratory study employed a prospective two-phase design. Medical and non-medical undergraduate students (N = 62) completed measures of personality and participated in two 20-minute training sessions with expert feedback 2 weeks apart, performing an end-to-side anastomosis on a low-fidelity model. Learner performance and instructors' perceptions of a learner's capacity to be trained were assessed.

PARTICIPANTS

Sixty-two medical and non-medical undergraduate students.

RESULTS

There was a significant interaction between learner Extraversion and instructor's perceptions of learner capacity to be trained. Higher learner Extraversion was only associated with an increase in performance improvement for those who were considered trainable (OR = 4.83, p = 0.017). Post hoc analysis revealed a significant difference in the amount of feedback provided to participants who were considered trainable (M = 9.45) versus not trainable (M = 16.48).

CONCLUSIONS

This study highlights the importance of both individual learner factors and instructor perceptions on surgical skill acquisition.

Is the Virtual Reality Fundamentals of Arthroscopic Surgery Training Program a Valid Platform for Resident Arthroscopy Training?

BACKGROUND

Training in arthroscopy is associated with a steep learning curve for trainees and bears risks for patients. Virtual reality (VR) arthroscopy simulation platforms seek to overcome this and to provide a safe environment for surgical learners. The Fundamentals of Arthroscopic Surgery Training (FAST) program is one such platform. It is currently not known whether the VR FAST program can be employed as a useful teaching or examination tool to assess the basic arthroscopic skills of surgical trainees.

QUESTIONS/PURPOSES

(1) Does the VR FAST program differentiate among novice, intermediate, and expert arthroscopists? (2) Does ambidextrous performance in the VR FAST program correlate with arthroscopic experience?

METHODS

We prospectively recruited orthopaedic interns (novices), residents (intermediates), and fellows and attendings (experts) to complete the VR FAST program over a 1-year period from four major orthopaedic training programs on a voluntary basis. Sixty-six of 156 invited orthopaedic surgeons participated: 26 of 50 novices (16 men and 10 women), 27 of 65 intermediates (20 men and seven women), and 13 of 41 experts (10 men and three women). Surgeons of any arthroscopic experience were included, with only those with prior experience on the VR FAST program being excluded. The program consists of eight modules: three basic camera modules (Image Centering, Horizon Control, and Telescoping), three advanced camera modules (Periscoping, Trace the Line, and Trace the Curve), and two instrumented bimanual-dexterity modules (Probe Triangulation and Gather the Stars). Time taken to complete each task and measures of economy of movement (camera and instrument path length, camera alignment) were used as measures of arthroscopic experience. Every participant completed the modules using their dominant and nondominant hands. Equality in proficiency in completing the tasks using the dominant and nondominant hands were determined to be measures of arthroscopic experience. Due to the large number of outcome variables, only p values < 0.01 were considered to be statistically significant.

RESULTS

Six of eight VR FAST modules did not discriminate among novice, intermediate, and expert arthroscopy participants. However, two did, and the ones that were most effective at distinguishing participants by level of experience were the Periscoping and Gather the Stars modules. For the Periscoping module using the dominant hand, novices required longer to complete the task with a median time of 231 seconds (IQR 149 to 358) and longer camera path length median of 191 cm (IQR 128 to 273) compared with intermediates who needed 127 seconds (IQR 106 to 233) and 125 cm (IQR 92 to 159) and experts who needed 121 seconds (IQR 93 to 157) and 119 cm (IQR 90 to 134) (p = 0.001 and p = 0.003, respectively). When using the nondominant hand, novices took longer to complete the task with a median time of 231 seconds (IQR 170 to 350) and longer camera path length 204 cm (IQR 169 to 273) compared with intermediates who required 132 seconds (IQR 97 to 162) and 111 cm (IQR 88 to 143) and experts who needed 119 seconds (IQR 104 to 183) and 120 cm (IQR 108 to 166) (p < 0.001 and p < 0.001, respectively). For the Gather the Stars module using the nondominant hand, only the novices needed longer to complete the task at a median of 131 seconds (IQR 112 to 157) and needed a longer grasper path length of 290 cm (IQR 254 to 332) compared with intermediates who needed 84 seconds (IQR 72 to 119) and 232 cm (IQR 195 to 254) and experts who needed 98 seconds (IQR 87 to 107) and 244 cm (IQR 215 to 287) (p < 0.001 and p = 0.001, respectively).

CONCLUSION

Six of eight VR FAST modules did not demonstrate construct validity, and we found no correlation between arthroscopic experience and ambidextrous performance. Two modules demonstrated construct validity; however, refinement and expansion of the modules is needed with further validation in large prospective trials so that pass-fail thresholds can be set for use in high-stakes examinations.

CLINICAL RELEVANCE

Most VR FAST modules were not discriminatory; however, they can form essential conceptual and procedural building blocks in an arthroscopic curriculum that are beneficial for novices when developing key psychomotor skills. In their present format, however, they are unsuitable for assessing arthroscopic proficiency.

Contemporary Issues in the Acquisition of Orthopaedic Surgical Skills During Residency: Competency-Based Medical Education and Simulation

➤

Orthopaedic education should produce surgeons who are competent to function independently and can obtain and maintain board certification.

➤

Contemporary orthopaedic training programs exist within a fixed 5-year time frame, which may not be a perfect match for each trainee.

➤

Most modern orthopaedic residencies have not yet fully adopted objective, proficiency-based, surgical skill training methods despite nearly 2 decades of evidence supporting the use of this methodology.

➤

Competency-based medical education backed by surgical simulation rooted in proficiency-based progression has the potential to address surgical skill acquisition challenges in orthopaedic surgery.

Working Towards Competence: A Novel Application of Borderline Regression to a Task-Specific Checklist for Technical Skills in Novices

OBJECTIVE

Demonstrated competence through frequent assessment is an expected goal for progressive development in competency-based medical education curricula. The Objective Structured Assessment of Technical Skill (OSATS) is considered a valid method of formative assessment, but in few instances have standards been set for determining competence. The present study used borderline regression methods to examine standard setting of performance on a complex technical task with novices assessed using an OSATS checklist.

METHODS

This was a single institution prospective single arm experimental design study. Participants were 58 non-medical undergraduate students with no previous surgical experience, who observed a computer-based training module on end-to-side vascular anastomosis. Subsequently, participants were provided two 20-minute training sessions, two weeks apart where they received expert feedback whilst performing the task on a low-fidelity model. After each training session, participants completed the task unaided. Sessions were recorded and assessed using an OSATS checklist retrospectively by experts.

RESULTS

Paired t-test analyses indicate that for both the checklist total score (t(52) = 8.05, p < 0.001) and the global rating score (t(53) = 8.15, p < 0.001), individuals performed significantly better in Phase 2. Borderline regression analyses indicated that in Phase 1 (R² = .60) and Phase 2 (R² = .75), the OSATS checklist could adequately capture variation in performance in novices. Further, the checklist could reliably classify novices at three of the five global rating performance levels. Pass rates determined by regression equations improved from Phase 1 to Phase 2 on all global rating levels.

CONCLUSIONS

With the increasing focus on competency-based medical education, it is imperative that training programs have the capacity to accurately assess outcomes and set minimum performance standards. Borderline regression methods can accurately differentiate novice learners of varying performance levels before and after training on a complex technical skill task using an OSATS checklist.

The FAST Workstation Shows Construct Validity and Participant Endorsement

Purpose

To determine in what way the proposed simulation-based intervention (SBI) is an effective intervention for use in basic arthroscopic skills training.

Methods

Twenty candidates were recruited and grouped according to experience. Performance metrics included the time to activity completion, errors made, and Global Rating Scale score. Qualitative data were collected using a structured questionnaire.

Results

Performance on the SBI differed depending on previous arthroscopic training received. Performance on the simulator differed between groups to a statistically significant level regarding time to completion. A difference was also present between participants with no previous training and those with previous training when assessed using the Global Rating Scale. The SBI was deemed acceptable, user-friendly, and realistic. Participants practicing at the expert level believe that such an SBI would be beneficial in developing basic arthroscopic skills.

Conclusions

The results of this study provide evidence that the use of an SBI consisting of a benchtop workstation, laptop viewing platform, 30° arthroscope, and defined performance metrics can detect differences in the level of arthroscopic experience. This format of SBI has been deemed acceptable and useful to the intended user, increasing the feasibility of introducing it into surgical training.

Clinical Relevance

This study adds to the existing body of evidence supporting the potential benefits of benchtop SBIs in arthroscopic skills training. Improved performance on such an SBI may be beneficial for the purpose of basic arthroscopic skills training, and we would support the inclusion of this system in surgical training programs such as those developed by the Arthroscopy Association of North America and American Board of Orthopaedic Surgery.

A pilot study to evaluate the face & construct validity of an orthopaedic virtual reality simulator

AIMS

This study aimed to identify the face and construct validity of the Precision OS trauma module proximal femoral nail procedure. Secondary outcomes included perceived use of simulation in surgical training, with structured feedback from participants.

METHODS

A comparative interventional study was carried out in a regional orthopaedics trauma unit hospital. Volunteers were stratified into novice, intermediate and expert groups based on self-reported levels of experience. Each participant carried out a simulated proximal femoral nail on an immersive virtual platform following instruction on its use, with objective metrics such as time and x-rays, and novel metrics calculated by the simulation module recorded. Face validity was also assessed.

RESULTS

The proximal femoral nail module demonstrated construct validity. Kruskal Wallis test demonstrated a statistically significant difference across all group's novel performance (p=.018). Intermediate surgeons performed significantly better than novices (P=.022), with shorter procedural times (P=.018) Three of the intermediate group achieved the proficiency level set by the expert group, with no significant difference noted between these two groups (=.06). Time taken to completion for expert surgeons was less than intermediate group, although this did not reach significance (P=.19).

CONCLUSION

The proximal femoral nail module on the Precision OS platform demonstrated good face, and construct validity. Further research evaluating use of virtual platform simulation in surgical trauma training is needed.

Aktueller Stand virtueller Simulatoren für die chirurgische Ausbildung in Orthopädie und Unfallchirurgie

Die praktische chirurgische Ausbildung wird in der Orthopädie und Unfallchirurgie durch den steigenden Kostendruck in den Kliniken sowie die Verlagerung vieler Operationen in den ambulanten Bereich immer schwieriger. Das betrifft besonders die Arthroskopie, die zudem hohe Anforderungen an die Fähigkeiten des Operateurs stellt. In den letzten 20 Jahren wurden daher zahlreiche Arthroskopiesimulatoren entwickelt. Dabei handelt es sich um virtuelle Desktopsysteme, die mit einem Gelenksimulator gekoppelt sind und eine taktile Rückkopplung erlauben. Auf diese Weise können arthroskopische Basisfähigkeiten (z. B. Triangulation) oder auch arthroskopische Operationen (z. B. partielle Meniskektomie) trainiert werden. Die Systeme ermöglichen eine Erfolgskontrolle durch die Erfassung verschiedener Parameter (Operationszeit, Kamera‑/Instrumentenpfadlänge, Knorpelberührungen). Es existieren bereits Simulatoren für alle großen Gelenke. Mittlerweile liegen Daten zur Augenscheinvalidität, Inhaltsvalidität, Konstruktvalidität und zur Transfervalidität dieser Systeme vor. Die erlernten Fähigkeiten lassen sich daher auf den Erfolg im Operationssaal übertragen. Ungelöst ist bisher die Kosteneffizienz dieser Systeme. Von den Desktop-Gelenkmodell-Systemen müssen immersive Virtual-Reality(VR)-Systeme unter Verwendung von 3‑D-Brillen und taktilen Handsensoren abgegrenzt werden. Der Teilnehmende tritt der Applikation dabei meist als Avatar bei. Daher eignen sich immersive Systeme besonders zur Simulation offener Operationen (z. B. Hüftendoprothetik). Erste Studien haben auch für den Einsatz der immersiven virtuellen Realität Evidenz für Inhalts‑, Konstrukt- und Transfervalidität beim Training verschiedener offener Operationen gezeigt. Hier sind in Zukunft weitere Entwicklungen zu erwarten.

Innate Arthroscopic & Laparoscopic Surgical Skills: A Systematic Review of Predictive Performance Indicators Within Novice Surgical Trainees

OBJECTIVE

To query the literature for predictive factors for performance on arthroscopic and laparoscopic surgical simulators in novice surgical trainees. These predictive factors may prove a valuable tool in identifying certain innate potential to becoming a future skilled surgeon that could benefit both surgical residency candidates and program directors alike, while also improving patient care.

DESIGN

Systematic Review.

RESULTS

The initial searches yielded 249 articles. After screening titles/abstracts and implementing inclusion and exclusion criteria, 36 studies were included in the final systematic review.

CONCLUSIONS

Current literature suggests that video game experience/frequency, psychomotor and visuospatial aptitude, and perceptual ability are among the most promising predictive indicators of baseline simulator performance. Study limitations include utilization of different standards for characterizing predictive factors. Future studies should aim to utilize standard guidelines for accurate quantification of innate predictive factors. Future research should also focus on utilizing standardized simulator platforms and aptitude tests to allow for more accurate cross-study comparisons and meta-analyses with larger sample sizes.

Efficacy and Validity of Orthopaedic Simulators in Surgical Training: A Systematic Review and Meta-Analysis of Randomized Controlled Trials

INTRODUCTION

The purpose of this study was to perform a systematic review and meta-analysis of the effects of training simulators on surgical skill measures across randomized controlled trials. The authors hypothesized that simulated training would (1) result in objective improvements in skill acquisition and (2) be heterogeneous regarding the outcomes and types of validity assessed.

METHODS

The Cochrane Database of Systematic Reviews, the Central Register of Controlled Trials, PubMed, EMBASE, and MEDLINE databases were queried for Level I studies on training simulators between 2007 and 2019 in accordance with the 2009 Preferred Reporting Items for Systematic Review and Meta-Analysis statement. Exclusion criteria were studies without discrete assessment of skills acquisition after surgical simulator training and level of evidence II to V. The Jadad scale was used to assess the methodological quality of all included articles. Data pertaining to patient demographics, validity measures, simulator types, and study-specific outcome measures were extracted. Meta-analyses adjusted for random effects and heterogeneity analyses (I) were used to compare pooled time-to-completion and performance outcomes among included studies.

RESULTS

A total of 24 studies with 494 participants were identified. The most common simulator type involved knee arthroscopy (11 studies, 45.8%). Eight studies reporting time-to-task completion and performance scores were included in the meta-analysis. Virtual reality training was favored in time-to-task completion (mean difference = -82.25 seconds, P = 0.002) and improvement in objective performance scores (mean difference = 1.24, P = 0.02) relative to traditional training. Sensitivity analysis of time-to-task completion based on the length of training interval revealed a mean difference of -45.24 (P = 0.07) and -137.74 (P < 0.001) seconds for the short-term and immediate posttesting subgroups, respectively.

CONCLUSION

Overall, improved task efficiency and performance were observed with the use of orthopaedic simulators. However, simulator type, training protocols, and outcome measures were heterogeneous. Future studies are warranted to evaluate financial cost and longitudinal training programs and to standardize outcomes regarding the use of simulators in orthopaedic education.

LEVEL OF EVIDENCE

Level I.

Virtual Reality and Physical Models in Undergraduate Orthopaedic Education: A Modified Randomised Crossover Trial

Background

Orthopaedic surgery is underrepresented in the United Kingdom medical school curriculum, with an average of less than 3 weeks of exposure over the five-year degree. This study evaluates the effectiveness of high-fidelity virtual reality (VR) and physical model simulation in teaching undergraduate orthopaedic concepts.

Methods

A modified randomised crossover trial was used. Forty-nine students were randomly allocated to two groups, with thirty-three finishing the six-week follow-up assessment. All undergraduate medical students were eligible for inclusion. Both groups were given introductory lectures, before completing a pre-test with questions on the principles of fracture fixation and osteotomy. Each group then received a lecture on these topics with the same content, but one was delivered with VR and the other with physical models. Both groups completed the post-course assessments. Knowledge was assessed by way of questionnaire immediately before, immediately after, and six-weeks after.

Results

In the VR group, participants improved their post-training score by 192.1% (U=32; p<0.00001). In the physical models group, participants improved their post-training scores by 163.1% (U=8.5; p<0.00001). Overall, there was no statistically significant difference in the total means of post-training test scores between the VR and the physical models study groups (U=260.5; p=0.4354).

Conclusion

Both VR and physical models represent valuable educational adjuncts for the undergraduate medical curriculum. Both have demonstrated improvements in immediate and long-term knowledge retention of key orthopaedic concepts.

Improvement and Retention of Arthroscopic Skills in Novice Subjects Using Fundamentals of Arthroscopic Surgery Training (FAST) Module

INTRODUCTION

Analysis of the Fundamentals of Arthroscopy Surgery Training (FAST) workstation regarding increased proficiency and retention of basic arthroscopy skills in novice subjects.

METHODS

First-year medical students from a single allopathic medical school performed weekly standardized FAST workstation modules for a consecutive 6 weeks. Primary outcomes evaluated were time to task completion and error rate on specific modules. Scores were analyzed using a one-way repeated measures analysis of variance design for overall trends in time and errors over the 6-week study. Psychomotor retention was analyzed after a 12-week and 24-week interlude.

RESULTS

Across the initial 6-week study, the average time to complete all modules at the workstation decreased significantly (P < 0.001) with a mean reduction in the total workstation time of 21.9 minutes (s = 8.12 minutes). Weekly comparisons showed the most significant improvement from week 1 to week 2 for the total workstation time (P < 0.001). Results after a 12-week and 24-week interval of inactivity demonstrated no significant difference in the mean workstation time or errors when compared with the original 6-week study.

DISCUSSION

The FAST workstation significantly improved the task performance of novice participants over a 6-week period with no significant deterioration in task performance after 12 and 24 weeks of inactivity.

The Effect of Triangulation Simulator Training on Arthroscopy Skills: A Prospective Randomized Controlled Trial

Purpose

To prospectively evaluate the transferability of skills acquired on a low-cost, at-home, nonanatomic triangulation simulation system to cadaveric models.

Methods

We randomized 28 medical students into either a simulator-training group (n = 14) or group with no training (control, n = 14). All subjects were pretested using a standardized checklist of arthroscopic skills on cadaveric knees and shoulders. Training-group subjects practiced on the triangulation simulator for 90 minutes per week for 4 consecutive weeks. Control subjects received no training. All subjects completed a post-test checklist of arthroscopic skills on cadaveric knees and shoulders, as well as 4 training tasks on the simulator. A blinded orthopaedic surgeon evaluated the arthroscopic videos using the Arthroscopic Surgical Skill Evaluation Tool (ASSET) score.

Results

Training-group knee and shoulder ASSET scores increased from 12.2 ± 1.85 to 14.6 ± 2.76 (P = .02) and from 14.6 ± 3.5 to 17.9 ± 4.5 (P = .29), respectively. In the control group, knee and shoulder ASSET scores increased from 14.3 ± 3.12 to 14.25 ± 4.67 (P = .99) and from 14.2 ± 2.7 to 17.07 ± 6.7 (P = .58), respectively. There were no significant differences in the mean post-test ASSET scores between the training group and control group for either knee or shoulder arthroscopy. The post-test ASSET safety subscore during knee arthroscopy was significantly higher in the training group (P = .03). The training group was able to complete significantly more simulator tasks compared with controls (P = .003) at post-testing. A significant positive correlation was found between knee arthroscopy performance and the number of tasks completed during simulation post-testing (P = .043). There was no significant correlation between shoulder arthroscopy performance and simulation performance (P = .532).

Conclusions

Basic triangulation skills may be acquired by training on a low-cost, at-home, nonanatomic triangulation simulation system, although the degree of transferability and universal joint applicability, as well as the existence of an early ceiling effect in skill development, could not be shown.

Level of Evidence

Level II, randomized controlled trial.

Can an Augmented Reality Headset Improve Accuracy of Acetabular Cup Orientation in Simulated THA? A Randomized Trial

BACKGROUND

Accurate implant orientation reduces wear and increases stability in arthroplasty but is a technically demanding skill. Augmented reality (AR) headsets overlay digital information on top of the real world. We have developed an enhanced AR headset capable of tracking bony anatomy in relation to an implant, but it has not yet been assessed for its suitability as a training tool for implant orientation.

QUESTIONS/PURPOSES

(1) In the setting of simulated THA performed by novices, does an AR headset improve the accuracy of acetabular component positioning compared with hands-on training by an expert surgeon? (2) What are trainees' perceptions of the AR headset in terms of realism of the task, acceptability of the technology, and its potential role for surgical training?

METHODS

Twenty-four study participants (medical students in their final year of school, who were applying to surgery residency programs, and who had no prior arthroplasty experience) participated in a randomized simulation trial using an AR headset and a simulated THA. Participants were randomized to two groups completing four once-weekly sessions of baseline assessment, training, and reassessment. One group trained using AR (with live holographic orientation feedback) and the other received one-on-one training from a hip arthroplasty surgeon. Demographics and baseline performance in orienting an acetabular implant to six patient-specific values on the phantom pelvis were collected before training and were comparable. The orientation error in degrees between the planned and achieved orientations was measured and was not different between groups with the numbers available (surgeon group mean error ± SD 16° ± 7° versus AR 14° ± 7°; p = 0.22). Participants trained by AR also completed a validated posttraining questionnaire evaluating their experiences.

RESULTS

During the four training sessions, participants using AR-guidance had smaller mean (± SD) errors in orientation than those receiving guidance from the surgeon: 1° ± 1° versus AR 6° ± 4°, p < 0.001. In the fourth session's assessment, participants in both groups had improved (surgeon group mean improvement 6°, 95% CI, 4-8°; p < 0.001 versus AR group 9°, 95% CI 7-10°; p < 0.001). There was no difference between participants in the surgeon-trained and AR-trained group: mean difference 1.2°, 95% CI, -1.8 to 4.2°; p = 0.281. In posttraining evaluation, 11 of 12 participants would use the AR platform as a training tool for developing visuospatial skills and 10 of 12 for procedure-specific rehearsals. Most participants (11 of 12) stated that a combination of an expert trainer for learning and AR for unsupervised training would be preferred.

CONCLUSIONS

A novel head-mounted AR platform tracked an implant in relation to bony anatomy to a clinically relevant level of accuracy during simulated THA. Learners were equally accurate, whether trained by AR or a surgeon. The platform enabled the use of real instruments and gave live feedback; AR was thus considered a feasible and valuable training tool as an adjunct to expert guidance in the operating room. Although there were no differences in accuracy between the groups trained using AR and those trained by an expert surgeon, we believe the tool may be useful in education because it demonstrates that some motor skills for arthroplasty may be learned in an unsupervised setting. Future studies will evaluate AR-training for arthroplasty skills other than cup orientation and its transfer validity to real surgery.

LEVEL OF EVIDENCE

Level I, therapeutic study.

Fundamentals of Arthroscopic Surgery Training Program Improves Knee Arthroscopy Simulator Performance in Arthroscopic Trainees

PURPOSE

To determine the effectiveness of a nonanatomic simulator in developing basic arthroscopy motor skills transferable to an anatomic model.

METHODS

Forty-three arthroscopy novice individuals currently enrolled in medical school were recruited to perform a diagnostic knee arthroscopy using a high-fidelity virtual reality arthroscopic simulator providing haptic feedback after viewing a video of an expert performing an identical procedure. Students were then randomized into an experimental or control group. The experimental group then completed a series of self-guided training modules using the fundamentals of arthroscopy simulator training nonanatomic modules including camera centering, tracking, periscoping, palpation, and collecting stars in a three-dimensional space. Both groups completed another diagnostic knee arthroscopy between 1 and 2 weeks later. Camera path length, time, tibia and femur cartilage damage, as well as a composite score were recorded by the simulator on each attempt.

RESULTS

The experimental group (n = 22) showed superior performance in composite score (30.09 vs 24, P = .046) and camera path length (71.51 cm vs 109.07 cm, P = .0274) at the time of the second diagnostic knee arthroscope compared with the control group (n = 21). The experimental group also showed significantly greater improvement in composite score between the first and second arthroscopes compared with the control group (14.27 vs 4.95, P < .01). Femoral and tibial cartilage damage were not significantly improved between arthroscopy attempts (-0.86% vs -1.45%, P = .40) and (-1.10 vs -1.27%, P = .83), respectively.

CONCLUSIONS

The virtual reality-based fundamentals of arthroscopy simulator training nonanatomic simulator is beneficial in developing basic motor skills in arthroscopy novice individuals resulting in significantly greater composite performance in an anatomic knee model. Based on the results of this study, it appears that there may be benefit from nonanatomic simulators in general as part of an arthroscopy training program.

LEVEL OF EVIDENCE

Level II, randomized trial.

Prior Generic Arthroscopic Volume Correlates with Hip Arthroscopic Proficiency: A Simulator Study

BACKGROUND

Changing trends in surgical education and patient expectation are leading to proficiency models of progression and the use of simulators. Hip arthroscopy is increasingly performed and has a steep learning curve mainly addressed during fellowship training. The aim of this study was to assess the impact of previous generic arthroscopic experience on performance at a simulated hip arthroscopy task to both estimate the minimum case numbers that correlate with expert proficiency levels and help to guide selection for hip arthroscopy fellowships.

METHODS

Fifty-two participants were recruited to a cross-sectional study. Four consultants (expert hip arthroscopists), 28 trainees (residents and fellows), and 20 novices (interns and medical students) performed a standardized bench-top simulated hip arthroscopy task. A validated global rating scale (GRS) score and motion analysis were used to assess surgical performance. Prior arthroscopic experience was recorded from surgical electronic logbooks. Receiver operating characteristic (ROC) curve analyses were conducted to identify optimum cut-points for task proficiency at both expert and competent GRS levels.

RESULTS

There were significant differences (p < 0.05) between the arthroscopic ability of all experience groups based on GRS assessment and for all motion analysis metrics. There was a significant positive correlation between logbook numbers and GRS scores (p < 0.0001). ROC curve analysis demonstrated that a minimum of 610 prior arthroscopic procedures were necessary to achieve an expert GRS score, and 78 prior arthroscopic procedures were necessary for a competent score.

CONCLUSIONS

Performing a basic hip arthroscopy task competently requires substantial previous generic arthroscopic experience. The numbers identified in this study provide targets for residents. Program directors appointing to hip arthroscopy fellowship training posts may find these results useful as a guide during the selection process.

Cigar Box Arthroscopy: A Randomized Controlled Trial Validates Nonanatomic Simulation Training of Novice Arthroscopy Skills

PURPOSE

The goal of this study was to validate the cigar box arthroscopy trainer (CBAT) as a training tool and then compare its effectiveness to didactic training and to another previously validated low-fidelity but anatomic model, the anatomic knee arthroscopy trainer (AKAT).

METHODS

A nonanatomic knee arthroscopy training module was developed at our institution. Twenty-four medical students with no prior arthroscopic or laparoscopic experience were enrolled as subjects. Eight subjects served as controls. The remaining 16 subjects were randomized to participate in 4 hours of either the CBAT or a previously validated AKAT. Subjects' skills were assessed by 1 of 2 faculty members through repeated attempts at performing a diagnostic knee arthroscopy on a cadaveric specimen. Objective scores were given using a minimally adapted version of the Basic Arthroscopic Knee Skill Scoring System. Total cost differences were calculated.

RESULTS

Seventy-five percent of subjects in the CBAT and AKAT groups succeeded in reaching minimum proficiency in the allotted time compared with 25% in the control group (P < .05). There was no significant difference in the number of attempts to reach proficiency between the CBAT and AKAT groups. The cost to build the CBAT was $44.12, whereas the cost was $324.33 for the AKAT.

CONCLUSIONS

This pilot study suggests the CBAT is an effective knee arthroscopy trainer that may decrease the learning curve of residents without significant cost to a residency program. This study demonstrates the need for an agreed-upon objective scoring system to properly evaluate residents and compare the effectiveness of different training tools.

Early Performance on an Eye Surgery Simulator Predicts Subsequent Resident Surgical Performance

OBJECTIVE

To examine early performance on an eye surgery simulator and its relationship to subsequent live surgical performance in a single large residency program.

DESIGN

Retrospective study.

SETTING

Massachusetts Eye and Ear, Harvard Medical School, Department of Ophthalmology.

METHODS

In a retrospective study, we compared performance of 30 first-year ophthalmology residents on an eye surgery simulator to their surgical skills as third-year residents. Variables collected from the eye surgery simulator included scores on the following modules of the simulator (Eyesi, VRmagic, Mannheim, Germany): antitremor training level 1, bimanual training level 1, capsulorhexis level 1 (configured), forceps training level 1, and navigation training level 1. Subsequent surgical performance was assessed using the total number of phacoemulsification cataract surgery cases for each resident, as well as the number performed as surgeon during residency and scores on global rating assessment of skills in intraocular surgery (GRASIS) scales during the third year of residency. Spearman correlation coefficients were calculated between each of the simulator performance and subsequent surgical performance variables. We also compared variables in a small group of residents who needed extra help in learning cataract surgery to the other residents in the study.

MAIN OUTCOME MEASURES

Relationships between Eyesi scores early in residency and surgical performance measures in the final year of residency.

RESULTS

A total of 30 residents had Eyesi data from their first year of residency and had already graduated so that all subsequent surgical performance data were available. There was a significant correlation between capsulorhexis task score on the simulator and total surgeries (r = 0.745, p = 0.008). There was a significant correlation between antitremor training level 1 (r = 0.554, p = 0.040), and forceps training level 1 (r = 0.622, p = 0.023) with primary surgery numbers. There was a significant correlation between forceps training level 1 (r = 0.811, p = 0.002), and navigation training level 1 (r = 0.692, p = 0.013) with total GRASIS score. There was a significant inverse correlation between total GRASIS score and residents in need of extra help (r = -0.358, p = 0.003).

CONCLUSION

Module scores on an eye surgery simulator early in residency may predict a resident׳s future performance in the operating room. These scores may allow early identification of residents in need of supplemental training in cataract surgery.

Can Surgical Trainees Achieve Arthroscopic Competence at the End of Training Programs? A Cross-sectional Study Highlighting the Impact of Working Time Directives

PURPOSE

To provide training guidance on procedure numbers by assessing how the number of previously performed arthroscopic procedures relate to both competent and expert performance in simulated arthroscopic shoulder tasks.

METHODS

A cross-sectional study that assessed simulated shoulder arthroscopic performance was undertaken. A total of 45 participants of varying experience performed 2 validated tasks: a simple diagnostic task and a more complex Bankart labral repair task. All participants provided logbook numbers for previously performed arthroscopies. Performance was assessed with the Global Rating Scale and motion analysis. Receiver operating characteristic curve analyses were conducted to identify optimum cut points for task proficiency at both "competent" and "expert" levels.

RESULTS

Increasing surgical experience resulted in significantly better performance for both tasks as assessed by Global Rating Scale or motion analysis (P < .0001). Receiver operating characteristic curve analyses demonstrated 52 previous arthroscopies were needed to perform to a competent level at the diagnostic task and 248 to be competent at the complex task. To perform at an expert level, 290 and 476 previous arthroscopies, respectively, were needed.

CONCLUSIONS

This study provides quantified guidance for arthroscopic training and highlights the positive relationship between arthroscopic case load and arthroscopic competency. We have estimated that the number of arthroscopies required to achieve competency in a basic arthroscopic task exceed those recommended in some countries. These estimates provide useful guidance to those responsible for training program.

CLINICAL RELEVANCE

The numbers to achieve competent arthroscopic performance in the assessed simulated tasks exceed what is recommended and what is possible during surgical training programs in some countries.

Practice does not always make perfect: need for selection curricula in modern surgical training

BACKGROUND

It is hypothesized that not all surgical trainees are able to reach technical competence despite ongoing practice. The objectives of the study were to assess a trainees' ability to reach technical competence by assessing learning patterns of the acquisition of surgical skills. Furthermore, it aims to determine whether individuals' learning patterns were consistent across a range of open and laparoscopic tasks of variable difficulty.

METHODS

Sixty-five preclinical medical students participated in a training curriculum with standardized feedback over forty repetitions of the following laparoscopic and open technical tasks: peg transfer (PT), circle cutting (CC), intracorporeal knot tie (IKT), one-handed tie, and simulated laparotomy closure. Data mining techniques were used to analyze the prospectively collected data and stratify the students into four learning clusters. Performance was compared between groups, and learning curve characteristics unique to trainees who have difficulty reaching technical competence were quantified.

RESULTS

Top performers (22-35%) and high performers (32-42%) reached proficiency in all tasks. Moderate performers (25-37%) reached proficiency for all open tasks but not all laparoscopic tasks. Low performers (8-15%) failed to reach proficiency in four of five tasks including all laparoscopic tasks (PT 7.8%; CC 9.4%; IKT 15.6%). Participants in lower performance clusters demonstrated sustained performance disadvantage across tasks, with widely variable learning curves and no evidence of progression towards a plateau phase.

CONCLUSIONS

Most students reached proficiency across a range of surgical tasks, but low-performing trainees failed to reach competence in laparoscopic tasks. With increasing use of laparoscopy in surgical practice, screening potential candidates to identify the lowest performers may be beneficial.

Effect of Mirrored Views on Endoscopic and Arthroscopic Skill Performance

Background:

Arthroscopic procedures may be technically challenging because of impaired vision, limited space, and the 2-dimensional vision of a 3-dimensional structure. Spatial orientation may get more complicated when the camera is pointing toward the surgeon.

Hypothesis:

Spatial orientation and arthroscopic performance may be improved by simply mirroring the image on the monitor in different configurations regarding the position and orientation of camera and instrument.

Study Design:

Descriptive laboratory study.

Methods:

Thirty volunteers from an orthopaedic department were divided into 3 equal groups according to their arthroscopic experience (beginners, intermediates, seniors). All subjects were asked to perform a standardized task in a closed box mimicking an endoscopic space. The same task had to be performed in 4 different configurations regarding camera and instrument position and orientation (pointing toward or away from the subject) with either the original or mirrored image on the monitor. Efficiency (time per stick; TPS), precision (successful completion of the task), and difficulty rating using a visual analog scale (VAS) were analyzed.

Results:

Mirroring the image demonstrated no advantage over the original images in any configuration regarding TPS. Successful completion of the task was significantly better when the image was mirrored in the configuration with the camera pointing toward and the instrument away from the surgeon. There was a positive correlation between TPS and subjective VAS difficulty rating (r = 0.762, P = .000) and a negative correlation between the successful completion of the task and VAS (r = −0.515, P = .000).

Conclusion:

Mirroring the image may have a positive effect on arthroscopic performance of surgeons in certain configurations. A significantly improved performance was seen when the arthroscope was pointing toward and the grasping instrument pointing away from the subject. Mirroring the image may facilitate surgery in such clinical situations.

Clinical Relevance:

Mirroring the image may facilitate arthroscopic procedures in certain clinical situations.

Validation of the updated ArthroS simulator: face and construct validity of a passive haptic virtual reality simulator with novel performance metrics

PURPOSE

To assess the construct and face validity of ArthroS, a passive haptic VR simulator. A secondary aim was to evaluate the novel performance metrics produced by this simulator.

METHODS

Two groups of 30 participants, each divided into novice, intermediate or expert based on arthroscopic experience, completed three separate tasks on either the knee or shoulder module of the simulator. Performance was recorded using 12 automatically generated performance metrics and video footage of the arthroscopic procedures. The videos were blindly assessed using a validated global rating scale (GRS). Participants completed a survey about the simulator's realism and training utility.

RESULTS

This new simulator demonstrated construct validity of its tasks when evaluated against a GRS (p ≤ 0.003 in all cases). Regarding it's automatically generated performance metrics, established outputs such as time taken (p ≤ 0.001) and instrument path length (p ≤ 0.007) also demonstrated good construct validity. However, two-thirds of the proposed 'novel metrics' the simulator reports could not distinguish participants based on arthroscopic experience. Face validity assessment rated the simulator as a realistic and useful tool for trainees, but the passive haptic feedback (a key feature of this simulator) is rated as less realistic.

CONCLUSION

The ArthroS simulator has good task construct validity based on established objective outputs, but some of the novel performance metrics could not distinguish between surgical experience. The passive haptic feedback of the simulator also needs improvement. If simulators could offer automated and validated performance feedback, this would facilitate improvements in the delivery of training by allowing trainees to practise and self-assess.

Newly acquired arthroscopic skills: Are they transferable during simulator training of other joints?

PURPOSE

This randomized study investigates whether novices learning simulation-based arthroscopic skills in one anatomical joint environment can immediately transfer their learnt skills to another joint.

METHODS

Medical students were randomized to a simulated diagnostic knee or shoulder arthroscopic task on benchtop training models. After nine task repetitions over 3 weeks on one model, each participant undertook the simulation task of the other anatomical joint. Performance was objectively measured using a validated electromagnetic motion analysis system and a validated global rating scale (GRS).

RESULTS

Eighteen students participated; eight started the knee task and ten the shoulder task. All participants demonstrated a learning curve in all parameters during task repetition (time taken, hand path length, number of hand movements and GRS scores; p < 0.001) with learning effects >1 SD from initial performance (range 1.1-2.2 SD). When the groups swapped models, there was no immediate evidence of skill transfer, with a significant drop in performance between the final training episode and the transfer task (all parameters p < 0.003). In particular, the transfer task performance was no better than the first episode performance on that model by these novices.

CONCLUSION

This study showed basic arthroscopic skills did not immediately transfer to an unfamiliar anatomical environment within a simulated setting. These findings have important clinical implications with regard to surgical training as they potentially challenge the assumption that arthroscopic skills acquired in one joint are universally transferrable to other joints. Future orthopaedic simulation training should aim to deliver exposure to a greater variety of arthroscopic procedures and joint environments. This would allow trainees to become familiar with the different arthroscopic setting before undertaking real surgery and consequently improve patient safety.

LEVEL OF EVIDENCE

Therapeutic, Level II.

Simulation in shoulder surgery

Simulation is a rapidly developing field in medical education. There is a growing need for trainee surgeons to acquire surgical skills in a cost-effective learning environment to improve patient safety and compensate for a reduction in training time and operative experience. Although simulation is not a replacement for traditional models of surgical training, and robust assessment metrics need to be validated before widespread use for accreditation, it is a useful adjunct that may ultimately lead to improving surgical outcomes for our patients.

Validation of hand motion analysis as an objective assessment tool for the Focused Assessment with Sonography for Trauma examination

BACKGROUND

Point-of-care ultrasonography is a standard part of trauma assessments, but there are no objective tools to assess proficiency and ensure high-quality examinations. Hand motion analysis (HMA) has been validated as a measure of surgical skill but has not previously been applied to ultrasonography. HMA was assessed for construct validity in Focused Assessment with Sonography for Trauma (FAST) performance.

METHODS

Two cohorts of 12 expert and 12 novice ultrasonographers performed a FAST examination on a healthy volunteer. Hand motions were recorded with the trakSTAR 3D electromagnetic motion-tracking device (Ascension Technology) and analyzed using our custom-designed Motion Analysis and Recording System (MARS) software. Data were recorded at 240 Hz. Outcomes included time of examination, number of movements, and path length.

RESULTS

Time of examination was not different between cohorts (expert, 345.9 seconds; novice, 475.7 seconds; p = 0.12). Total path length of travel was shorter, and the number of discreet movements was less in the expert cohort for the left-hand (18.52 m vs. 28.01 m, p = 0.03, and 109.5 vs. 193.9, p = 0.027, respectively) and the right-hand performance (14.25 m vs. 32.09 m, p < 0.01, and 153.5 vs. 258.5, p = 0.03, respectively) versus the novice cohort. Both total path length traveled and total number of discreet movements were associated with expertise level in logistic regression modeling with areas under the receiver operating characteristic curves of 0.8269 and 0.8205, respectively.

CONCLUSION

This is the first study in the medical literature showing HMA as an objective, valid measure of FAST imaging performance. These objective, automated metrics can function as an adjunct measure to assess FAST performance as well as follow progress of and provide feedback to learners to improve future performances.

LEVEL OF EVIDENCE

A "diagnostic criteria"-style test where the "diagnosis" is a determination of competence in a care provider, level II.

Assessing Arthroscopic Skills Using Wireless Elbow-Worn Motion Sensors

BACKGROUND

Assessment of surgical skill is a critical component of surgical training. Approaches to assessment remain predominantly subjective, although more objective measures such as Global Rating Scales are in use. This study aimed to validate the use of elbow-worn, wireless, miniaturized motion sensors to assess the technical skill of trainees performing arthroscopic procedures in a simulated environment.

METHODS

Thirty participants were divided into three groups on the basis of their surgical experience: novices (n = 15), intermediates (n = 10), and experts (n = 5). All participants performed three standardized tasks on an arthroscopic virtual reality simulator while wearing wireless wrist and elbow motion sensors. Video output was recorded and a validated Global Rating Scale was used to assess performance; dexterity metrics were recorded from the simulator. Finally, live motion data were recorded via Bluetooth from the wireless wrist and elbow motion sensors and custom algorithms produced an arthroscopic performance score.

RESULTS

Construct validity was demonstrated for all tasks, with Global Rating Scale scores and virtual reality output metrics showing significant differences between novices, intermediates, and experts (p < 0.001). The correlation of the virtual reality path length to the number of hand movements calculated from the wireless sensors was very high (p < 0.001). A comparison of the arthroscopic performance score levels with virtual reality output metrics also showed highly significant differences (p < 0.01). Comparisons of the arthroscopic performance score levels with the Global Rating Scale scores showed strong and highly significant correlations (p < 0.001) for both sensor locations, but those of the elbow-worn sensors were stronger and more significant (p < 0.001) than those of the wrist-worn sensors.

CONCLUSIONS

A new wireless assessment of surgical performance system for objective assessment of surgical skills has proven valid for assessing arthroscopic skills. The elbow-worn sensors were shown to achieve an accurate assessment of surgical dexterity and performance.

CLINICAL RELEVANCE

The validation of an entirely objective assessment of arthroscopic skill with wireless elbow-worn motion sensors introduces, for the first time, a feasible assessment system for the live operating theater with the added potential to be applied to other surgical and interventional specialties.

Cadaveric surgical training improves surgeon confidence

Is using cadavers the best way to train arthroplastic surgeons?

Advances in arthroscopy-indications and therapeutic applications

Advances in optical technology, instrumentation and implants now enable arthroscopic surgery to be performed on all large joints and most small joints of the limbs. Arthroscopic techniques are usually a development of surgical procedures previously performed through a large open incision, although the critical element of each procedure (for example removal of a torn meniscus) usually remains unchanged. The smaller size of incisions and reduction in tissue damage associated with arthroscopic surgery can reduce morbidity and complications. Therapeutic arthroscopy now encompasses excision, reconstruction and replacement of damaged or abnormal tissue. Improvements in the accuracy of MRI, CT and high-definition ultrasonography have limited the use of diagnostic arthroscopy to rare indications, but in the past 10 years the rates of some arthroscopic surgeries have increased by over 7-fold. Considerable variation in the type and utilization of arthroscopic procedures exists in practice, partly explained by the slow diffusion of new techniques and technology, but also by differences in clinician and patient beliefs and expectations. This Review reflects on both the success of arthroscopy and the general lack of evidence-based assessment of the efficacy and cost-effectiveness of arthroscopic procedures-a clear sign that more clinical trials in this field are required.

The role of simulation in developing surgical skills

Surgical training has followed the master-apprentice model for centuries but is currently undergoing a paradigm shift. The traditional model is inefficient with no guarantee of case mix, quality, or quantity. There is a growing focus on competency-based medical education in response to restrictions on doctors' working hours and the traditional mantra of "see one, do one, teach one" is being increasingly questioned. The medical profession is subject to more scrutiny than ever before and is facing mounting financial, clinical, and political pressures. Simulation may be a means of addressing these challenges. It provides a way for trainees to practice technical tasks in a protected environment without putting patients at risk and helps to shorten the learning curve. The evidence for simulation-based training in orthopedic surgery using synthetic models, cadavers, and virtual reality simulators is constantly developing, though further work is needed to ensure the transfer of skills to the operating theatre.

The knee arthroscopy learning curve: quantitative assessment of surgical skills

PURPOSE

To assess orthopaedic trainees performing diagnostic knee arthroscopies and evaluate procedural competence using a novel statistical method, the Cumulative Summation Test for Learning Curve (LC-CUSUM).

METHODS

Twenty orthopaedic trainees in postgraduate year (PGY) 1 through 5 performed diagnostic knee arthroscopy and were evaluated intraoperatively with a validated 10-point knee task-specific checklist (TSCL) and 50-point global rating scale (GRS). A score of 40 points or greater (of 50 points) for the GRS and 8 points or greater (of 10 points) for the knee TSCL was considered to indicate a successful procedure. For the LC-CUSUM analysis, adequate performance was defined as a 10% failure rate, inadequate performance was defined as a 30% failure rate, and an acceptable deviation from adequate performance was defined as 10%. A limit h equal to 1.6 was selected to give a true-discovery rate of 90% and a false-discovery rate of 10% over 50 procedures.

RESULTS

A total of 340 consecutive procedures were performed by 20 trainees during the 12-month study period. The cumulative number of arthroscopic procedures performed by trainees before study start increased with increasing PGY. The median number of arthroscopic procedures performed per trainee was 16.5 (interquartile range, 14 to 21.75). Competency in knee arthroscopy for the TSCL was achieved by 8 trainees (40%), after a median of 16 procedures (interquartile range, 13 to 20), and for the GRS by only a single trainee (5%), after 14 procedures. Threshold-adjusted curves stratified by PGY level enabled multiple trainees to achieve competency for both the TSCL and GRS.

CONCLUSIONS

The LC-CUSUM can be successfully applied to knee arthroscopy to provide an individualized assessment of performance and quantitatively demonstrate competency for basic arthroscopic tasks.

CLINICAL RELEVANCE

The LC-CUSUM is an effective method to evaluate procedure competence in arthroscopic training and can provide objective feedback and benchmarks in the learning phase.

Mechanical thrombectomy with the Solitaire stent: is there a learning curve in achieving rapid recanalization times?

METHODS

In acute ischemic stroke, good outcome following successful recanalization is time dependent. In patients undergoing endovascular therapy at our institution, recanalization times with the Solitaire stent were retrospectively evaluated to assess for the presence of a learning curve in achieving rapid recanalization.

METHODS

We reviewed patients who presented to our stroke center and achieved successful recanalization with the Solitaire stent exclusively. Time intervals were calculated (CT to angiography arrival, angiography arrival to groin puncture, groin puncture to first deployment, and deployment to recanalization) from time stamped images and angiography records. Patients were divided into three sequential groups, with overall CT to recanalization time and subdivided time intervals compared.

RESULTS

83 patients were treated with the Solitaire stent from May 2009 to February 2012. Recanalization (Thrombolyis in Cerebral Infarction score 2A) occurred in 75 (90.4%) patients. CT to recanalization demonstrated significant improvement over time, which was greatest between the first 25 and the most recent 25 cases (161-94 min; p<0.01). The maximal contribution to this was from improvements in first stent deployment to recanalization time (p=0.001 between the first and third groups), with modest contributions from moving patients from CT to the angiography suite faster (p=0.02 between the first and third groups) and from groin puncture to first stent deployment (p=0.02 between the first and third groups).

CONCLUSIONS

There is a learning curve involved in the efficient use of the Solitaire stent in endovascular acute stroke therapy. Along with improvements in patient transfer to angiography and improved efficiency with intracranial access, mastering this device contributed significantly towards reducing recanalization times.

Validating a global rating scale to monitor individual resident learning curves during arthroscopic knee meniscal repair

PURPOSE

To determine whether a global rating scale (GRS) with construct validity can also be used to assess the learning curve of individual orthopaedic trainees during simulated arthroscopic knee meniscal repair.

METHODS

An established arthroscopic GRS was used to evaluate the technical skill of 19 orthopaedic residents performing a standardized arthroscopic meniscal repair in a bioskills laboratory. The residents had diagnostic knee arthroscopy experience but no experience with arthroscopic meniscal repair. Residents were videotaped performing an arthroscopic meniscal repair on 12 separate occasions. Their performance was assessed by use of the GRS and motion analysis objectively measuring the time taken to complete tasks, path length of the subject's hands, and number of hand movements. One author assessed all 228 videos, whereas 2 other authors rated 34 randomly selected videos, testing the interobserver reliability of the GRS. The validity of the GRS was tested against the motion analysis.

RESULTS

Objective assessment with motion analysis defined the surgeon's learning curve, showing significant improvement by each subject over 12 episodes (P < .0001). The GRS also showed a similar learning curve with significant improvements in performance (P < .0001). The median GRS score improved from 15 of 34 (interquartile range, 14 to 17) at baseline to 22 of 34 (interquartile range, 19 to 23) in the final period. There was a moderate correlation (P < .0001, Spearman test) between the GRS and motion analysis parameters (r = -0.58 for time, r = -0.58 for path length, and r = -0.51 for hand movements). The inter-rater reliability among 3 trained assessors using the GRS was excellent (Cronbach α = 0.88).

CONCLUSIONS

When compared with motion analysis, an established arthroscopic GRS, with construct validity, also offers a moderately feasible method to monitor the learning curve of individual residents during simulated knee meniscal repair.

CLINICAL RELEVANCE

An arthroscopic GRS can be used for monitoring skill improvement during knee meniscal repair and has the potential for use as a training and assessment tool in the real operating room.

Arthroscopic proficiency: methods in evaluating competency

BACKGROUND

The current paradigm of arthroscopic training lacks objective evaluation of technical ability and its adequacy is concerning given the accelerating complexity of the field. To combat insufficiencies, emphasis is shifting towards skill acquisition outside the operating room and sophisticated assessment tools. We reviewed (1) the validity of cadaver and surgical simulation in arthroscopic training, (2) the role of psychomotor analysis and arthroscopic technical ability, (3) what validated assessment tools are available to evaluate technical competency, and (4) the quantification of arthroscopic proficiency.

METHODS

The Medline and Embase databases were searched for published articles in the English literature pertaining to arthroscopic competence, arthroscopic assessment and evaluation and objective measures of arthroscopic technical skill. Abstracts were independently evaluated and exclusion criteria included articles outside the scope of knee and shoulder arthroscopy as well as original articles about specific therapies, outcomes and diagnoses leaving 52 articles cited in this review.

RESULTS

Simulated arthroscopic environments exhibit high levels of internal validity and consistency for simple arthroscopic tasks, however the ability to transfer complex skills to the operating room has not yet been established. Instrument and force trajectory data can discriminate between technical ability for basic arthroscopic parameters and may serve as useful adjuncts to more comprehensive techniques. There is a need for arthroscopic assessment tools for standardized evaluation and objective feedback of technical skills, yet few comprehensive instruments exist, especially for the shoulder. Opinion on the required arthroscopic experience to obtain proficiency remains guarded and few governing bodies specify absolute quantities.

CONCLUSIONS

Further validation is required to demonstrate the transfer of complex arthroscopic skills from simulated environments to the operating room and provide objective parameters to base evaluation. There is a deficiency of validated assessment tools for technical competencies and little consensus of what constitutes a sufficient case volume within the arthroscopy community.