Objective psychomotor skills assessment of experienced, junior, and novice laparoscopists with virtual reality

Fluoroscopic image-based behavior analysis can objectively explain subjective expert assessment of wire navigation skill

Psychomotor skill and decision-making efficiency in surgical wire navigation can be objectively evaluated by analysis of intraoperative fluoroscopic image sequences. Prior work suggests that such image-based behavior analysis of operating room (OR) performance can predict performer experience level (R2  = 0.62) and agree with expert opinion (the current standard) on the quality of a final implant construct (R2  = 0.59). However, it is unclear how objective image-based evaluation compares with expert assessments for entire technical OR performances. This study examines the relationships between three key variables: (1) objective image-based criteria, (2) expert opinions, and (3) performing surgeon experience level. A paired-comparison survey of seven experts, based upon eight OR fluoroscopic wire navigation image sequences, shows that the experts' preferences are best explained by objective metrics that reflect psychomotor and decision-making behaviors which are counter-productive to successful implant placement, like image count (R2  = 0.83) and behavior tally (R2  = 0.74). One such behavior, adjustments away from goal, uniquely correlated well with all three key variables: a fluoroscopic image-based analysis composite score (R2  = 0.40), expert consensus (R2  = 0.76), and performer experience (R2  = 0.41). These results confirm that experts view less efficient technical behavior as indicative of lesser technical proficiency. While expert assessments of technical skill were reliable and consistent, neither individual nor consensus expert opinion appears to correlate with performer experience (R2  = 0.11).

Laboratory performance prediction using virtual reality behaviometrics

In this study, we show that virtual reality (VR) behaviometrics can be used for the assessment of compliance and physical laboratory skills. Drawing on approaches from machine learning and classical statistics, significant behavioral predictors were deduced from a logistic regression model that classified students and biopharma company employees as experts or novices on pH meter handling with 77% accuracy. Specifically, the game score and number of interactions in VR tasks requiring practical skills were found to be performance predictors. The study provides biopharma companies and academic institutions the possibility of assessing performance using an automatic, reliable, and simple alternative to traditional in-person assessment methods. Integrating the assessment into the training tool renders such laborious post-training assessments unnecessary.

Discrimination, reliability, sensitivity, and specificity of metric-based assessment of an unstable pertrochanteric 31A2 intramedullary nailing procedure performed by experienced and novice surgeons

INTRODUCTION

Identifying objective performance metrics for surgical training in orthopedic surgery is imperative for effective training and patient safety. The objective of this study was to determine if an internationally agreed, metric-based objective assessment of video recordings of an unstable pertrochanteric 31A2 intramedullary nailing procedure distinguished between the performance of experienced and novice orthopedic surgeons.

MATERIALS AND METHODS

Previously agreed procedure metrics (i.e., 15 phases of the procedure, 75 steps, 88 errors, and 28 sentinel errors) for a closed reduction and standard cephalomedullary nail fixation with a single cephalic element of an unstable pertrochanteric 31A2 fracture. Experienced surgeons trained to assess the performance metrics with an interrater reliability (IRR) > 0.8 assessed 14 videos from 10 novice surgeons (orthopaedic residents/trainees) and 20 videos from 14 experienced surgeons (orthopaedic surgeons) blinded to group and procedure order.

RESULTS

The mean IRR of procedure assessments was 0.97. No statistically significant differences were observed between the two groups for Procedure Steps, Errors, Sentinel Errors, and Total Errors. A small number of Experienced surgeons made a similar number of Total Errors as the weakest performing Novices. When the scores of each group were divided at the median Total Error score, large differences were observed between the Experienced surgeons who made the fewest errors and the Novices making the most errors (p < 0.001). Experienced surgeons who made the most errors made significantly more than their Experienced peers (p < 0.003) and the best performing Novices (p < 0.001). Error metrics assessed with Area Under the Curve demonstrated good to excellent Sensitivity and Specificity (0.807-0.907).

DISCUSSION

Binary performance metrics previously agreed by an international Delphi meeting discriminated between the objectively assessed video-recorded performance of Experienced and Novice orthopedic surgeons when group scores were sub-divided at the median for Total Errors. Error metrics discriminated best and also demonstrated good to excellent Sensitivity and Specificity. Some very experienced surgeons performed similar to the Novice group surgeons that made most errors.

CONCLUSIONS

The procedure metrics used in this study reliably distinguish Novice and Experienced orthopaedic surgeons' performance and will underpin quality-assured novice training.

How can virtual reality (VR) based conversational agents be designed to train clinicians in verbal de-escalation skills: An Exploratory Usability Study (Preprint)

Background

Violence and aggression are significant workplace challenges faced by clinicians worldwide. Traditional methods of training consist of “on-the-job learning” and role-play simulations. Although both approaches can result in improved skill levels, they are not without limitation. Interactive simulations using virtual reality (VR) can complement traditional training processes as a cost-effective, engaging, easily accessible, and flexible training tool.

Objective

In this exploratory study, we aimed to determine the feasibility of and barriers to verbal engagement with a virtual agent in the context of the Code Black VR application. Code Black VR is a new interactive VR-based verbal de-escalation trainer that we developed based on the Clinical Training Through VR Design Framework.

Methods

In total, 28 participants with varying clinical expertise from 4 local hospitals enrolled in the Western Sydney Local Health District Clinical Initiative Nurse program and Transition to Emergency Nursing Programs and participated in 1 of 5 workshops. They completed multiple playthroughs of the Code Black VR verbal de-escalation trainer application and verbally interacted with a virtual agent. We documented observations and poststudy reflection notes. After the playthroughs, the users completed the System Usability Scale and provided written comments on their experience. A thematic analysis was conducted on the results. Data were also obtained through the application itself, which also recorded the total interactions and successfully completed interactions.

Results

The Code Black VR verbal de-escalation training application was well received. The findings reinforced the factors in the existing design framework and identified 3 new factors—motion sickness, perceived value, and privacy—to be considered for future application development.

Conclusions

Verbal interaction with a virtual agent is feasible for training staff in verbal de-escalation skills. It is an effective medium to supplement clinician training in verbal de-escalation skills. We provide broader design considerations to guide further developments in this area.

The role of virtual reality simulation in surgical training in the light of COVID-19 pandemic: Visual spatial ability as a predictor for improved surgical performance: a randomized trial

INTRODUCTION

Due to the current COVID-19 pandemic, surgical training has become increasingly challenging due to required social distancing. Therefore, the use of virtual reality (VR)-simulation could be a helpful tool for imparting surgical skills, especially in minimally invasive environments. Visual spatial ability (VSA) might influence the learning curve for laparoscopic surgical skills. However, little is known about the influence of VSA for surgical novices on VR-simulator training regarding the complexity of different tasks over a long-term training period. Our study evaluated prior VSA and VSA development in surgical trainees during VR-simulator training, and its influence on surgical performance in simulator training.

METHODS

In our single-center prospective two-arm randomized trial, VSA was measured with a tube figure test before curriculum training. After 1:1 randomization, the training group (TG) participated in the entire curriculum training consisting of 48 different VR-simulator tasks with varying difficulty over a continuous nine-day training session. The control group (CG) performed two of these tasks on day 1 and 9. Correlation and regression analyses were used to assess the influence of VSA on VR-related surgical skills and to measure procedural abilities.

RESULTS

Sixty students (33 women) were included. Significant improvements in the TG in surgical performance and faster completion times were observed from days 1 to 9 for the scope orientation 30° right-handed (SOR), and cholecystectomy dissection tasks after the structured 9-day training program. After training, the TG with pre-existing low VSA scores achieved performance levels similar to those with pre-existing high VSA scores for the two VR simulator tasks. Significant correlations between VSA and surgical performance on complex laparoscopic camera navigation SOR tasks were found before training.

CONCLUSIONS

Our study revealed that that all trainees improved their surgical skills irrespective of previous VSA during structured VR simulator training. An increase in VSA resulted in improvements in surgical performance and training progress, which was more distinct in complex simulator tasks. Further, we demonstrated a positive relationship between VSA and surgical performance of the TG, especially at the beginning of training. Our results identified pre-existing levels of VSA as a predictor of surgical performance.

Properties and Characteristics of Three-Dimensional Printed Head Models Used in Simulation of Neurosurgical Procedures: A Scoping Review

BACKGROUND

Intracranial surgery can be complex and high risk. Safety, ethical and financial factors make training in the area challenging. Head model 3-dimensional (3D) printing is a realistic training alternative to patient and traditional means of cadaver and animal model simulation.

OBJECTIVE

To describe important factors relating to the 3D printing of human head models and how such models perform as simulators.

METHODS

Searches were performed in PubMed, the Cochrane Library, Scopus, and Web of Science. Articles were screened independently by 3 reviewers using Covidence software. Data items were collected under 5 categories: study information; printers and processes; head model specifics; simulation and evaluations; and costs and production times.

RESULTS

Forty articles published over the last 10 years were included in the review. A range of printers, printing methods, and substrates were used to create head models and tissue types. Complexity of the models ranged from sections of single tissue type (e.g., bone) to high-fidelity integration of multiple tissue types. Some models incorporated disease (e.g., tumors and aneurysms) and artificial physiology (e.g., pulsatile circulation). Aneurysm clipping, bone drilling, craniotomy, endonasal surgery, and tumor resection were the most commonly practiced procedures. Evaluations completed by those using the models were generally favorable.

CONCLUSIONS

The findings of this review indicate that those who practice surgery and surgical techniques on 3D-printed head models deem them to be valuable assets in cranial surgery training. Understanding how surgical simulation on such models affects surgical performance and patient outcomes, and considering cost-effectiveness, are important future research endeavors.

Objective assessment of intraoperative skills for robot-assisted radical prostatectomy (RARP): results from the ERUS Scientific and Educational Working Groups Metrics Initiative

OBJECTIVE

To develop and seek consensus from procedure experts on the metrics that best characterise a reference robot-assisted radical prostatectomy (RARP) and determine if the metrics distinguished between the objectively assessed RARP performance of experienced and novice urologists, as identifying objective performance metrics for surgical training in robotic surgery is imperative for patient safety.

MATERIALS AND METHODS

In Study 1, the metrics, i.e. 12 phases of the procedure, 81 steps, 245 errors and 110 critical errors for a reference RARP were developed and then presented to an international Delphi panel of 19 experienced urologists. In Study 2, 12 very experienced surgeons (VES) who had performed >500 RARPs and 12 novice urology surgeons performed a RARP, which was video recorded and assessed by two experienced urologists blinded as to subject and group. Percentage agreement between experienced urologists for the Delphi meeting and Mann-Whitney U- and Kruskal-Wallis tests were used for construct validation of the newly identified RARP metrics.

RESULTS

At the Delphi panel, consensus was reached on the appropriateness of the metrics for a reference RARP. In Study 2, the results showed that the VES performed ~4% more procedure steps and made 72% fewer procedure errors than the novices (P = 0.027). Phases VIIa and VIIb (i.e. neurovascular bundle dissection) best discriminated between the VES and novices.

LIMITATIONS

VES whose performance was in the bottom half of their group demonstrated considerable error variability and made five-times as many errors as the other half of the group (P = 0.006).

CONCLUSIONS

The international Delphi panel reached high-level consensus on the RARP metrics that reliably distinguished between the objectively scored procedure performance of VES and novices. Reliable and valid performance metrics of RARP are imperative for effective and quality assured surgical training.

STELA (Smart TV and smartphone-basEd Laparoscopy TrAiner): a no-cost home-based trainer for beginners

BACKGROUND

Proficiency in laparoscopy is gradually achieved. After initial simulation, it is safe to move to real patients. Simulation improves the basic attributes of laparoscopy, and its non-availability hampers training. Virtual reality and commercial simulators are exorbitantly expensive. Cheaper non-commercial latest, mobile phone-based simulators appear ergonomically unsuitable. A need for a no-cost, home-based laparoscopic endotrainer was felt by authors.

METHODS

The authors proposed the concept of smart TV and smart phone-based laparoscopy trainer (STELA), an almost zero cost, lightweight indigenous, cable-less box-type endotrainer, with a smart phone housed on the model, projecting to smart TV via Wi-fi direct. The simulation timings on STELA were compared with Universal Beetel endotrainer by a group of surgeons and residents using identical tasks like object transfer (OT) and knot making (KM).

RESULTS

Data were analysed using SPSS, version 23.There was no significant difference in the mean timings of the residents (p > 0.05) on two endotrainers, for both tasks, and of surgeons for OT. Surgeons took significantly longer time (p < 0.05) in KM on STELA. Highest correlation (r = +.848) (<.05) was seen for KM on both devices by residents.

CONCLUSION

STELA is a viable, technologically advanced, no cost alternative to the non-commercial cumbersome simulators especially for beginners.

Role of Simulation-Based Training in Minimally Invasive and Robotic Colorectal Surgery

Properly performing minimally invasive colorectal procedures requires specific skills. With a focus on patient safety, the training of surgeons on patients is only accepted under exceptionally controlled, expensive, and challenging conditions. Moreover, many new techniques in colorectal surgery have been developed. Therefore, undertaking minimally invasive colorectal surgery in modern times requires specific psychomotor skills that trainee surgeons must gather in less time. In addition, there are not enough proctors with sufficient expertise for such an expressive number of new different techniques likes transanal and robotic procedures. Studies that have demonstrated an improvement in minimally invasive surgery skills to the actual operating room in general surgery and a stepwise approach to surgical simulation with a combination of various training methods appears to be useful in colorectal surgery training programs. However, the scientific evidence on the transfer of skills specifically for colorectal surgery is extremely scarce and very variable. Thus, the evaluation of the results remains quite difficult. In this review, we present the best available evidence on the types of training based on simulation, their characteristics, advantages and disadvantages, and finally the results available on their adoption. Nevertheless, scientific evidence about the benefit of simulation training in minimally invasive colorectal surgery is limited and there is a need to build more robust evidence.

Use of a Low-Cost Portable 3D Virtual Reality Simulator for Psychomotor Skill Training in Minimally Invasive Surgery: Task Metrics and Score Validity

Background

The high cost and low availability of virtual reality simulators in surgical specialty training programs in low- and middle-income countries make it necessary to develop and obtain sources of validity for new models of low-cost portable simulators that enable ubiquitous learning of psychomotor skills in minimally invasive surgery.

Objective

The aim of this study was to obtain validity evidence for relationships to other variables, internal structure, and consequences of testing for the task scores of a new low-cost portable simulator mediated by gestures for learning basic psychomotor skills in minimally invasive surgery. This new simulator is called SIMISGEST-VR (Simulator of Minimally Invasive Surgery mediated by Gestures - Virtual Reality).

Methods

In this prospective observational validity study, the authors looked for multiple sources of evidence (known group construct validity, prior videogaming experience, internal structure, test-retest reliability, and consequences of testing) for the proposed SIMISGEST-VR tasks. Undergraduate students (n=100, reference group), surgical residents (n=20), and experts in minimally invasive surgery (n=28) took part in the study. After answering a demographic questionnaire and watching a video of the tasks to be performed, they individually repeated each task 10 times with each hand. The simulator provided concurrent, immediate, and terminal feedback and obtained the task metrics (time and score). From the reference group, 29 undergraduate students were randomly selected to perform the tasks 6 months later in order to determine test-retest reliability.

Results

Evidence from multiple sources, including strong intrarater reliability and internal consistency, considerable evidence for the hypothesized consequences of testing, and partial confirmation for relations to other variables, supports the validity of the scores and the metrics used to train and teach basic psychomotor skills for minimally invasive surgery via a new low-cost portable simulator that utilizes interaction technology mediated by gestures.

Conclusions

The results obtained provided multiple sources of evidence to validate SIMISGEST-VR tasks aimed at training novices with no prior experience and enabling them to learn basic psychomotor skills for minimally invasive surgery.

Use of a Low-Cost Portable 3D Virtual Reality Gesture-Mediated Simulator for Training and Learning Basic Psychomotor Skills in Minimally Invasive Surgery: Development and Content Validity Study

Background

Simulation in virtual environments has become a new paradigm for surgeon training in minimally invasive surgery (MIS). However, this technology is expensive and difficult to access.

Objective

This study aims first to describe the development of a new gesture-based simulator for learning skills in MIS and, second, to establish its fidelity to the criterion and sources of content-related validity evidence.

Methods

For the development of the gesture-mediated simulator for MIS using virtual reality (SIMISGEST-VR), a design-based research (DBR) paradigm was adopted. For the second objective, 30 participants completed a questionnaire, with responses scored on a 5-point Likert scale. A literature review on the validity of the MIS training-VR (MIST-VR) was conducted. The study of fidelity to the criterion was rated using a 10-item questionnaire, while the sources of content-related validity evidence were assessed using 10 questions about the simulator training capacity and 6 questions about MIS tasks, and an iterative process of instrument pilot testing was performed.

Results

A good enough prototype of a gesture-based simulator was developed with metrics and feedback for learning psychomotor skills in MIS. As per the survey conducted to assess the fidelity to the criterion, all 30 participants felt that most aspects of the simulator were adequately realistic and that it could be used as a tool for teaching basic psychomotor skills in laparoscopic surgery (Likert score: 4.07-4.73). The sources of content-related validity evidence showed that this study’s simulator is a reliable training tool and that the exercises enable learning of the basic psychomotor skills required in MIS (Likert score: 4.28-4.67).

Conclusions

The development of gesture-based 3D virtual environments for training and learning basic psychomotor skills in MIS opens up a new approach to low-cost, portable simulation that allows ubiquitous learning and preoperative warm-up. Fidelity to the criterion was duly evaluated, which allowed a good enough prototype to be achieved. Content-related validity evidence for SIMISGEST-VR was also obtained.

Contribution of Haptic Simulation to Analogic Training Environment in Restorative Dentistry

The aim of this study was to evaluate the contribution of virtual reality to the conventional analogic training environment and show the complementarity of conventional techniques and virtual reality in the learning of dental students. All 88 first-year dental students at a dental school in France in early 2019 were randomly assigned to one of two groups: group 1 (n = 45) was assigned to cavity preparations on a haptic simulator (Virteasy) and group 2 (n = 43) was assigned to conventional practical work on plastic analogue teeth (Kavo). Following three training sessions, the students in group 1 took a final exam on the same plastic analogue teeth exercise. The results showed improvement in the drilling skill of both groups. The simulator-trained group (group 1) had similar results to the plastic analogue-trained group (group 2) in the final test on a plastic analogue tooth. In this study, virtual reality allowed an assessment based on objective criteria and reduced the subjectivity of evaluations conducted on plastic analogue teeth. Considering the saving of supervision and teaching time as well as the material gain offered by virtual reality, the learning methods of haptic simulators are educational options that should be considered by dental educators.

An Epiduroscopy Simulator Based on a Serious Game for Spatial Cognitive Training (EpiduroSIM): User-Centered Design Approach

Background

Performing high-level surgeries with endoscopy is challenging, and hence, an efficient surgical training method or system is required. Serious game–based simulators can provide a trainee-centered educational environment unlike traditional teacher-centered education environments since serious games provide a high level of interaction (feedback that induces learning).

Objective

This study aimed to propose an epiduroscopy simulator, EpiduroSIM, based on a serious game for spatial cognitive training.

Methods

EpiduroSIM was designed based on a serious game. For spatial cognitive training, the virtual environment of EpiduroSIM was modeled based on a cognitive map.

Results

EpiduroSIM was developed considering user accessibility to provide various functions. The experiment for the validation of EpiduroSIM focused on psychological fidelity and repetitive training effects. The experiments were conducted by dividing 16 specialists into 2 groups of 8 surgeons. The group was divided into beginner and expert based on their epiduroscopy experience. The psychological fidelity of EpiduroSIM was confirmed through the training results of the expert group rather than the beginner group. In addition, the repetitive training effect of EpiduroSIM was confirmed by improving the training results in the beginner group.

Conclusions

EpiduroSIM may be useful for training beginner surgeons in epiduroscopy.

Development and application of a multi-modal task analysis to support intelligent tutoring of complex skills

BACKGROUND

Contemporary work in the design and development of intelligent training systems employs task analysis (TA) methods for gathering knowledge that is subsequently encoded into task models. These task models form the basis of intelligent interpretation of student performance within education and training systems. Also referred to as expert models, they represent the optimal way(s) of performing a training task. Within Intelligent Tutoring Systems (ITSs), real-time comparison of trainee task performance against the task model drives automated assessment and interactive support (such as immediate feedback) functionality. However, previous task analysis (TA) methods, including various forms of cognitive task analysis (CTA), may not be sufficient to support identification of the detailed design specifications required for the development of an ITS for a complex training task incorporating multiple underlying skill components, as well as multi-modal information presentation, assessment, and feedback modalities. Our current work seeks to develop an ITS for training Robotic Assisted Laparoscopic Surgery (RALS), a complex task domain that requires a coordinated utilization of integrated cognitive, psychomotor, and perceptual skills.

RESULTS

In this paper, we describe a methodological extension to CTA, referred to as multi-modal task analysis (MMTA) that elicits and captures the nuances of integrated and isolated cognitive, psychomotor, and perceptual skill modalities as they apply to training and performing complex operational tasks. In the current case, we illustrate the application of the MMTA method described here to RALS training tasks. The products of the analysis are quantitatively summarized, and observations from a preliminary qualitative validation are reported.

CONCLUSIONS

We find that iterative use of the described MMTA method leads to sufficiently complete and robust task models to support encoding of cognitive, psychomotor, and perceptual skills requisite to training and performance of complex skills within ITS task models.

Negating the fulcrum effect in manual laparoscopic surgery: Investigating skill acquisition with a haptic simulator

BACKGROUND

Manual laparoscopic surgery requires extensive training and familiarization. It has been suggested that motion inversion caused by the 'fulcrum effect' is key to motor challenges. We investigate the potential of a conceptual semi-robotic handheld tool that negates natural inversion.

METHODS

A custom laparoscopic simulator with haptic feedback was developed to allow interactive evaluation of the conceptual tool via virtual prototyping, prior to fabricating a physical prototype. Two groups of eight participants each used either the conceptual or a regular virtual tool over a ten week study to complete two abstract tasks of motor control and force regulation.

RESULTS

Statistically significant higher rates of skill improvement were demonstrated with the conceptual tool for motion efficiency, task completion time and error reduction. Force regulation increased for both groups but without significant differences.

CONCLUSIONS

The results indicate potential for fulcrum-negating hand tools in reducing the time needed to acquire motor skills.

Performance of High School Students in a Laparoscopic Training Program

BACKGROUND AND OBJECTIVES

We hypothesized that high school students can be subjected to the same laparoscopic surgical training curriculum used by surgeons and successfully complete it. The goal of this study was to evaluate the appropriateness of early training in minimally invasive surgical techniques.

METHODS

Thirteen high school students, ages 15-18, participated in the validated Top Gun Surgeon Laparoscopic Skills and Suturing program. The students performed 3 preparatory drills 10 times each. The students' scores were then compared to a database of 393 surgeons. Performance graphs were prepared to allow comparison of skills acquisition between the 2 training groups.

RESULTS

All 13 students successfully completed the tasks. The Students' performance (expressed as time/percentile range/average percentile) for each task were as follows: rope pass 101.8 seconds/3.8-47.1/11.8; bean drop 149.5 seconds/18.7-96.0/59.4; triangle transfer 303.2 seconds/1.3-16.0/5.8. The students started each drill with slower times, but their average improvement (decreased time to complete tasks) was more rapid than that of the surgeons between the first and second trials for each drill (-83 seconds vs -25 seconds, -120 seconds vs -53 seconds, -100 seconds vs -60 seconds). Average student times compared to average surgeon times during the last trials measured were not significantly different in the triangle transfer and rope pass drills (P = .40 and .18, respectively). Students' times were significantly faster than surgeons' in the last measured trial of the bean drop (P = .039).

CONCLUSIONS

Despite the small sample size, this investigation suggests that high school students can successfully complete skill-building programs in minimally invasive surgery. Further study is needed to evaluate the appropriateness of starting surgical training of future residents at an earlier stage of their careers.

Evaluation of Physical versus Virtual Surgical Training Simulators

With the recent attention on patient safety, there is an increased interest in standardized training for laparoscopic surgeons. Studies have shown that laparoscopic simulators can be used to train surgical skills. A comparison of two popular systems (a real physical model and a virtual model) was conducted to determine the relative effectiveness of the systems for training purposes. Twenty-two medical students and surgical residents were tested on both simulators. Time to task completion and errors committed were recorded and compared. Our results showed that the physical training system was more sensitive to the experience levels of the subjects than the virtual system, and may be more effective as a tool for standardized training. However, as virtual reality technology becomes better developed, and surgeons become more familiar with the technology, we may see a change.

Surgical Education's 100 Most Cited Articles: A Bibliometric Analysis

BACKGROUND

Bibliometric analysis highlights the key topics and publications, which have shaped surgical education. Here, the 100 most cited articles in the arena of surgical education were analyzed.

METHODS

Thomson Reuters Web of Science was interrogated using the keyword search terms "surgery" and ("learning" or "skills" or "competence" or "assessment" or "training" or "procedure-based assessments" or "performance" or "technical skills" or "curriculum" or "education" or "mentoring"] to identify all English language full articles, and the 100 most cited articles were analyzed by topic, journal, author, year, institution, and country of origin.

RESULTS

A total of 403,733 eligible articles were returned and the median citation number was 164 (range: 107-1018). The most cited article (by Seymour, Yale University School of Medicine, Annals of Surgery, 1018 citations) focused on the use of virtual reality surgical simulation training. Annals of Surgery published the highest number of articles and received the most citations (n = 16, 3715 citations). The countries with the greatest number of publications were the USA (n = 45), Canada (n = 19), and the UK (n = 18). The commonest topics included simulation (n = 45) and assessment of clinical competence (n = 40).

CONCLUSION

Surgical skill acquisition and assessment was the area of focus of 85% of the most cited contemporary articles, and this study provides the most cited references, serving as a guide as to what makes a citable published work in the field of surgical education.

The Pretrained Novice: Using Simulation-Based Training to Improve Learning in the Operating Room

Enabling trainees to acquire advanced technical skills before they begin the operating room experience benefits both trainee and patient. Whether medical students who had received exclusively simulation-based training could perform laparoscopic suturing and knot-tying as well as senior surgery residents was determined. Simulators were used to train 11 fourth-year medical students with no previous suturing experience to perform intracorporeal suturing and to successfully tie a free-hand intracorporeal knot. Students’ skills were assessed by the performance of the fundal suturing portion of a Nissen fundoplication in a porcine model. Their operative performance was evaluated for time, needle manipulations, and total errors. Results were compared to those of 11 senior-level surgery residents performing the same task. The study concluded that trainees could learn advanced technical skills such as laparoscopic suturing and knot tying by using simulation exclusively. The trainees and senior level surgery residents had a similar number of needle manipulations.

Night Call Does Not Impair Learning of Laparoscopic Skills

Investigators have looked at the effect of night call on surgical residents but not at learning of laparoscopic skills. The Minimally Invasive Surgical Trainer-Virtual Reality (MIST-VR) tests 6 tasks similar to a laparoscopic cholecystectomy. We hypothesized that night call would impair laparoscopic performance and that skills would not improve but rather deteriorate after night call. Seventeen volunteers were tested before and after night call. Data collected included economy of movement, time, and number of errors for each hand/foot. A paired Student t test was used for statistical analysis. On the first 2 tasks, there was an improvement in all parameters post-call, with significance reached in 5 of 18 parameters (P <.05). In the "running of the bowel," 8 of 9 parameters were significantly improved (P < .05). In the final task, 9 of 11 parameters showed a deterioration post-call, but only economy of movement of the foot was significant (P <.05). Most parameters (16) showed improvement rather than deterioration post-call, which is consistent with learning of laparoscopic skills despite lack of sleep from night call.

Comparison of surgical skills in laparoscopic and robotic tasks between experienced surgeons and novices in laparoscopic surgery: an experimental study

PURPOSE

Robotic surgery is known to provide an improved technical ability as compared to laparoscopic surgery. We aimed to compare the efficiency of surgical skills by performing the same experimental tasks using both laparoscopic and robotic systems in an attempt to determine if a robotic system has an advantage over laparoscopic system.

METHODS

Twenty participants without any robotic experience, 10 laparoscopic novices (LN: medical students) and 10 laparoscopically-experienced surgeons (LE: surgical trainees and fellows), performed 3 laparoscopic and robotic training-box-based tasks. This entire set of tasks was performed twice.

RESULTS

Compared with LN, LEs showed significantly better performances in all laparoscopic tasks and in robotic task 3 during the 2 trials. Within the LN group, better performances were shown in all robotic tasks compared with the same laparoscopic tasks. However, in the LE group, compared with the same laparoscopic tasks, significantly better performance was seen only in robotic task 1. When we compared the 2 sets of trials, in the second trial, LN showed better performances in laparoscopic task 2 and robotic task 3; LE showed significantly better performance only in robotic task 3.

CONCLUSION

Robotic surgery had better performance than laparoscopic surgery in all tasks during the two trials. However, these results were more noticeable for LN. These results suggest that robotic surgery can be easily learned without laparoscopic experience because of its technical advantages. However, further experimental trials are needed to investigate the advantages of robotic surgery in more detail.

Relative contribution of haptic technology to assessment and training in implantology

Background. The teaching of implant surgery, as in other medical disciplines, is currently undergoing a particular evolution. Aim of the Study. To assess the usefulness of haptic device, a simulator for learning and training to accomplish basic acts in implant surgery. Materials and Methods. A total of 60 people including 40 third-year dental students without knowledge in implantology (divided into 2 groups: 20 beginners and 20 experiencing a simulator training course) and 20 experienced practitioners (experience in implantology >15 implants) participated in this study. A basic exercise drill was proposed to the three groups to assess their gestural abilities. Results. The results of the group training with the simulator tended to be significantly close to those of the experienced operators. Conclusion. Haptic simulator brings a real benefit in training for implant surgery. Long-term benefit and more complex exercises should be evaluated.

Testing manual dexterity using a virtual reality simulator: reliability and validity

Virtual reality dental training simulators, unlike traditional human-based assessment, have the potential to enable consistent and reliable assessment. The purpose of this study was to determine whether a haptic simulator (IDEA Dental(®) ) could provide a reliable and valid assessment of manual dexterity. A total of 106 participants were divided into three groups differing in dental manual dexterity experience: (i) 63 dental students, (ii) 28 dentists, (iii) 14 non-dentists. The groups, which were expected to display various performance levels, were required to perform virtual drilling tasks in different geometric shapes. The following task parameters were registered: (i) Time to completion (ii) accuracy (iii) number of trials to successful completion and (iv) score provided by the simulator. The reliability of the tasks was calculated for each parameter. The simulator and its scoring algorithm showed high reliability in all the parameters measured. The simulator was able to differentiate between non-professionals and dental students or non-professionals and dentists. Our study suggests that for improved construct validity, shorter working times and more difficult tasks should be introduced. The device should also be designed to provide greater sensitivity in measuring the accuracy of the task.

Objective simulator-based evaluation of carotid artery stenting proficiency (from Assessment of Operator Performance by the Carotid Stenting Simulator Study [ASSESS])

Studies have suggested that operator proficiency has a substantial effect on complication rates and procedural outcomes. Endovascular simulators have been used for training and have been proposed as an alternative to the conventional assessment of skills. The present study sought to validate simulation as an objective method for proficiency evaluation in carotid artery stenting. Interventional cardiologists classified as novice, intermediate, or experienced practitioners performed 3 simulated, interactive carotid stenting cases on an AngioMentor endovascular simulator. An automated algorithm scored the participants according to the technical performance, medical management, and angiographic results. A total of 33 interventional cardiologists (8 novices, 15 intermediates, and 10 experts) completed 82 simulated procedures. The composite scores for the case simulations varied significantly by operator experience, with better scores for the more experienced groups (p <0.0001). The metrics that discriminated between operator experience groups included fluoroscopy time, crossing the carotid lesion with devices other than a 0.014-in. wire before filter deployment, and incomplete coverage of the lesion by the stent. In conclusion, the results of the present study validate that a simulator with an automated scoring system is able to discriminate between levels of operator proficiency for carotid artery stenting. Simulator-based performance assessment could have a role in initial and ongoing proficiency evaluations and credentialing of interventional operators of high-risk endovascular procedures.

Virtual reality robotic surgery warm-up improves task performance in a dry laboratory environment: a prospective randomized controlled study

BACKGROUND

Preoperative simulation warm-up has been shown to improve performance and reduce errors in novice and experienced surgeons, yet existing studies have only investigated conventional laparoscopy. We hypothesized that a brief virtual reality (VR) robotic warm-up would enhance robotic task performance and reduce errors.

STUDY DESIGN

In a 2-center randomized trial, 51 residents and experienced minimally invasive surgery faculty in General Surgery, Urology, and Gynecology underwent a validated robotic surgery proficiency curriculum on a VR robotic simulator and on the da Vinci surgical robot (Intuitive Surgical Inc). Once they successfully achieved performance benchmarks, surgeons were randomized to either receive a 3- to 5-minute VR simulator warm-up or read a leisure book for 10 minutes before performing similar and dissimilar (intracorporeal suturing) robotic surgery tasks. The primary outcomes compared were task time, tool path length, economy of motion, technical, and cognitive errors.

RESULTS

Task time (-29.29 seconds, p = 0.001; 95% CI, -47.03 to -11.56), path length (-79.87 mm; p = 0.014; 95% CI, -144.48 to -15.25), and cognitive errors were reduced in the warm-up group compared with the control group for similar tasks. Global technical errors in intracorporeal suturing (0.32; p = 0.020; 95% CI, 0.06-0.59) were reduced after the dissimilar VR task. When surgeons were stratified by earlier robotic and laparoscopic clinical experience, the more experienced surgeons (n = 17) demonstrated significant improvements from warm-up in task time (-53.5 seconds; p = 0.001; 95% CI, -83.9 to -23.0) and economy of motion (0.63 mm/s; p = 0.007; 95% CI, 0.18-1.09), and improvement in these metrics was not statistically significantly appreciated in the less-experienced cohort (n = 34).

CONCLUSIONS

We observed significant performance improvement and error reduction rates among surgeons of varying experience after VR warm-up for basic robotic surgery tasks. In addition, the VR warm-up reduced errors on a more complex task (robotic suturing), suggesting the generalizability of the warm-up.

Simulators in surgery

The introduction of minimally invasive surgery has demonstrated the need for training surgical skills outside the operating room using animal models or simulators. As laparoscopic surgery involves displaying images on a screen, virtual reality simulation of the surgical tasks is feasible. Different types of simulators have become available. The existing trainers can be divided into three groups: mechanical, hybrid, and virtual reality. This article aims at giving an overview of the different simulators available and the potential of simulators in the education of surgeons with focus on virtual reality simulators. All simulators aim at training psychomotoric skills and some simulators also allow training in decision-making and anatomical orientation. In the future virtual reality simulators may become a tool for training and validation of surgical skills and monitoring the training progress.

The effect of call on neurosurgery residents' skills: implications for policy regarding resident call periods

Object

Although fatigue and its effects on surgical proficiency have been an actively researched area, previous studies have not examined the effect of fatigue on neurosurgery residents specifically. This study aims to quantify the effect of fatigue on the psychomotor and cognitive skills of neurosurgery residents.

Methods

Seven neurosurgery residents performed a minimum of 3 and a maximum of 4 sessions of 6 surgical exercises precall and postcall. The simulation exercises were designed to measure a surgeon's cognitive abilities, such as memory and attention, while performing simulated surgical tasks and exercises that have been previously validated in several studies, including studies measuring the impact of fatigue on general surgery residents. Each exercise measured tool-movement smoothness, time elapsed, and cognitive errors. The change in surgical skills in precall and postcall conditions was assessed by means of an ANOVA, with p < 0.05 considered statistically significant.

Results

The neurosurgery residents did not show a statistically significant difference in their surgical skills between the pre- and postcall states (p < 0.3, p < 0.4, and p < 0.2 for movement smoothness, time elapsed, and cognitive errors, respectively). The mean decrement for all residents in the postcall condition was 13.1%.

Conclusions

Postcall fatigue is associated with a marginal decrease in proficiency during simulated surgery in neurosurgery residents. In a similar study, general surgery residents showed a statistically significant decrement of 27.3% in the postcall condition. The impact of fatigue on different specialties should be further investigated prior to implementation of a national physician work-hour policy.

Objective measures for longitudinal assessment of robotic surgery training

OBJECTIVES

Current robotic training approaches lack the criteria for automatically assessing and tracking (over time) technical skills separately from clinical proficiency. We describe the development and validation of a novel automated and objective framework for the assessment of training.

METHODS

We are able to record all system variables (stereo instrument video, hand and instrument motion, buttons and pedal events) from the da Vinci surgical systems using a portable archival system integrated with the robotic surgical system. Data can be collected unsupervised, and the archival system does not change system operations in any way. Our open-ended multicenter protocol is collecting surgical skill benchmarking data from 24 trainees to surgical proficiency, subject only to their continued availability. Two independent experts performed structured (objective structured assessment of technical skills) assessments on longitudinal data from 8 novice and 4 expert surgeons to generate baseline data for training and to validate our computerized statistical analysis methods in identifying the ranges of operational and clinical skill measures.

RESULTS

Objective differences in operational and technical skill between known experts and other subjects were quantified. The longitudinal learning curves and statistical analysis for trainee performance measures are reported. Graphic representations of the skills developed for feedback to the trainees are also included.

CONCLUSIONS

We describe an open-ended longitudinal study and automated motion recognition system capable of objectively differentiating between clinical and technical operational skills in robotic surgery. Our results have demonstrated a convergence of trainee skill parameters toward those derived from expert robotic surgeons during the course of our training protocol.

Mapping the maze of minimally invasive surgery simulators

BACKGROUND

Conforming to, among other considerations, legal and ethical concerns for patient safety, there is an increasing demand to assess a surgeon's skills prior to performance in the operating room in pursuit of higher-quality treatment. Training in minimally invasive surgery (MIS) must therefore be intensified, including team training. New methods to train and assess minimally invasive surgical skills are gaining interest. The goal of this review is to provide instructors with an overview of available MIS training tools. In this review, we discuss currently available simulators for MIS training. Applicability, validity, and construction of simulators are reviewed. Also, some of the leading training programs and assessment methods in MIS are reviewed.

METHODS

A literature search was performed on studies evaluating surgical task performance on a simulator, reviewing satisfaction with laparoscopic training programs, or validating simulators or assessment methods.

RESULTS

Simulators may be divided into simple box trainers and computer-based systems, such as virtual and augmented simulators. All have advantages and disadvantages. An overview is provided of currently available training systems, validity, trainee assessment, and the importance of training programs in MIS.

CONCLUSIONS

No simulator yet provides the ability to train the entire set of required psychomotor skills or procedures for MIS. A multiyear training program combining various simulators for multiple-level training, including team training, should be constructed.

Establishing hand preference: why does it matter?

Hand preference has been associated with psychological and physical well-being, risk of injury, pathological irregularities, longevity, and cognitive function. To determine hand preference, individuals are often asked what hand they use to write with, or what hand is used more frequently in activities of daily living. However, relying only on one source of information may be misleading, given the strong evidence to support a disassociation between self-reported hand preference and outcomes of hand performance assessments. This brief communication is intended to highlight the various methods used to determine hand preference, to discuss the relationship between hand preference inventories and performance measures and to present some recent findings associated with hand preference and musculoskeletal disorders.

Approaching time is important for assessment of endoscopic surgical skills

This study aimed to verify whether the approaching time (the time taken to reach the target point from another point, a short distance apart, during point-to-point movement in endoscopic surgery), assessed using the Hiroshima University Endoscopic Surgical Assessment Device (HUESAD), could distinguish the skill level of surgeons. Expert surgeons (who had performed more than 50 endoscopic surgeries) and novice surgeons (who had no experience in performing endoscopic surgery) were tested using the HUESAD. The approaching time, total time, and intermediate time (total time--approaching time) were measured and analyzed using the trajectory of the tip of the instrument. The approaching time and total time were significantly shorter in the expert group than in the novice group (p < 0.0001). The intermediate time did not significantly differ between the groups (p > 0.05). The approaching time, which is a component of the total time, is very mportant in the measurement of the total time to assess endoscopic surgical skills. Further, the approaching time was useful for skill assessment by the HUESAD for evaluating the skill of surgeons performing endoscopic surgery.

Quantitative evaluation of retention of surgical skills learned in simulation

BACKGROUND

While initial results suggest that simulation does promote learning, there is a dearth of studies that define the extent to which skills learned through simulation are retained.

METHODS

Residents skills were measured upon completion of an initial simulation training (baseline scores) and then every month for 6 months. Analysis was also performed to identify the number of iterations of practice required to regain baseline scores.

RESULTS

While skill scores did not deteriorate from baseline after the first 3 months (p = 0.61, p = 0.44, p = 0.2, respectively), all scores (except time elapsed) reflected significant deterioration from the fourth month onward (p < 0.05, p < 0.032, p < 0.02). However the number of practice sessions required to regain baseline scores was significantly less than that required to achieve the baseline skill set (p < 0.0003).

CONCLUSIONS

Skills learned through simulation show significant deterioration over long periods of time, suggesting that periodic retraining of skills may be necessary to maintain surgical proficiency.