Validation and implementation of surgical simulators: a critical review of present, past, and future

A 4-task skills examination for residents for the assessment of technical ability in hand trauma surgery

OBJECTIVES

The aim of this study was to evaluate a 4-task skills examination model for surgical trainees as a method of assessment of the different technical skills essential in hand trauma surgery.

DESIGN

Using established validation methodology, construct validity was assessed by comparing the performance of 2 groups of residents in surgery (group A: residents with any formation in microsurgery and hand surgery and group B: residents with specific formation in hand or microsurgery or both).

PARTICIPANTS AND SETTING

Overall, 19 residents in surgery with different degree of formation in hand and microsurgery participated in the study. All the residents performed 4 tasks on synthetic models consecutively: task 1-Z-plasty, task 2-metacarpal fracture fixation, task 3-tendon repair, and task 4-end-to-end anastomosis. The running order was awarded in a random drawing and 4 independent observers scored each resident.

RESULTS

There was a significant difference in performance in the overall score between groups A and B and particularly for tasks 1 and 4. All participants felt the 4 tasks were good models to learn the procedure and recommended this approach to younger residents.

CONCLUSION

This approach was based on a 4-tasks examination is the first model of evaluation of the different technical skills required for hand trauma surgery for residents. The results show a good differentiation between residents that have microsurgical and hand formation and those who do not have. This easy model can be easily integrated in the curriculum of residents, who want to specialize in hand surgery.

Training of hysteroscopic skills in residency program: the Dutch experience

STUDY OBJECTIVE

To evaluate whether hysteroscopy training in the Dutch gynecological residency program is judged as sufficient in daily practice, by assessment of the opinion on hysteroscopy training and current performance of hysteroscopic procedures. In addition, the extent of progress in comparison with that of the residency program of a decade ago is reviewed.

DESIGN

Survey (Canadian Task Force Classification III).

PARTICIPANTS

Postgraduate years 5 and 6 residents in obstetrics and gynecology and gynecologists who finished residency within 2008 to 2013 in the Netherlands.

INTERVENTION

Subjects received an online survey regarding performance and training of hysteroscopy, self-perceived competence, and hysteroscopic skills acquirement.

RESULTS

Response rate was 65% of the residents and 73% of the gynecologists. Most residents felt adequately prepared for basic hysteroscopic procedures (86.7%), but significantly less share this opinion for advanced procedures (64.5%) (p < 0.01). In comparison with their peers in 2003, the current residents demonstrated a 10% higher appreciation of the training curriculum. However, their self-perceived competence did not increase, except for diagnostic hysteroscopy. Regarding daily practice, not only do more gynecologists perform advanced procedures nowadays but also their competence level received higher scores in comparison with gynecologists in 2003. Lack of simulation training was indicated to be the most important factor during residency that could be enhanced for optimal acquirement of hysteroscopic skills.

CONCLUSION

Implementation of hysteroscopic procedures taught during residency training in the Netherlands has improved since 2003 and is judged as sufficient for basic procedures. The skills of surgical educators have progressed toward a level in which gynecologists feel competent to teach and supervise advanced hysteroscopic procedures. Even though the residency preparation for hysteroscopy is more highly appreciated than a decade ago, this study indicated that simulation training might serve as an additional method to improve hysteroscopic skills acquisition. Future research is needed to determine the value of simulation training in hysteroscopy.

Laparoscopy training in Belgium: results from a nationwide survey, in urology, gynecology, and general surgery residents

BACKGROUND

The purpose of this study was to investigate the exposure of Belgian residents in urology, general surgery, and gynecology to laparoscopic surgery and to training of laparoscopic skills in dedicated training facilities.

METHODS

Three similar specialty-specific questionnaires were used to interrogate trainees in urology, general surgery, and gynecology about their exposure to laparoscopic procedures, their acquired laparoscopic experience, training patterns, training facilities, and motivation. Residents were contacted via their Belgian specialist training organization, using Survey Monkey as an online survey tool. Data were analyzed with descriptive statistics.

RESULTS

The global response rate was 58%. Only 28.8% of gynecology respondents, 26.9% of urology respondents, and 52.2% of general surgery respondents felt they would be able to perform laparoscopy once they had finished their training. A total 47% of urology respondents, 66.7% of general surgery respondents, and 69.2% of gynecology respondents had a surgical skills lab that included laparoscopy within their training hospital or university. Most training programs did not follow the current evidence about proficiency-based structured simulation training with deliberate practice.

CONCLUSION

Belgian resident training facilities for laparoscopic surgery should be optimized.

A review of training research and virtual reality simulators for the da Vinci surgical system

PHENOMENON: Virtual reality simulators are the subject of several recent studies of skills training for robot-assisted surgery. Yet no consensus exists regarding what a core skill set comprises or how to measure skill performance. Defining a core skill set and relevant metrics would help surgical educators evaluate different simulators.

APPROACH

This review draws from published research to propose a core technical skill set for using the da Vinci surgeon console. Publications on three commercial simulators were used to evaluate the simulators' content addressing these skills and associated metrics.

FINDINGS

An analysis of published research suggests that a core technical skill set for operating the surgeon console includes bimanual wristed manipulation, camera control, master clutching to manage hand position, use of third instrument arm, activating energy sources, appropriate depth perception, and awareness of forces applied by instruments. Validity studies of three commercial virtual reality simulators for robot-assisted surgery suggest that all three have comparable content and metrics. However, none have comprehensive content and metrics for all core skills. INSIGHTS: Virtual reality simulation remains a promising tool to support skill training for robot-assisted surgery, yet existing commercial simulator content is inadequate for performing and assessing a comprehensive basic skill set. The results of this evaluation help identify opportunities and challenges that exist for future developments in virtual reality simulation for robot-assisted surgery. Specifically, the inclusion of educational experts in the development cycle alongside clinical and technological experts is recommended.

Description and validation of realistic and structured endourology training model

PURPOSE

The aim of the present study was to validate a model of training, which combines the use of non-biological and ex vivo biological bench models, as well as the modelling of urological injuries for endourological treatment in a porcine animal model.

MATERIAL AND METHODS

A total of 40 participants took part in this study. The duration of the activity was 16 hours. The model of training was divided into 3 levels: level I, concerning the acquisition of basic theoretical knowledge; level II, involving practice with the bench models and level III, concerning practice in the porcine animal model. First, trainees practiced with animals without using a model of injured (ureteroscopy, management of guide wires and catheters under fluoroscopic control) and later practiced in lithiasic animal model. During the activity, an evaluation of the face and content validity was conducted, as well as constructive validation provided by the trainees versus experts. Evolution of the variables during the course within each group was analysed using the Student's t test for paired samples, while comparisons between groups, were performed using the Student's t test for unpaired samples.

RESULTS

The assessments of face and content validity were satisfactory. The constructive validation, "within one trainee" shows that were statistical significant differences between the first time the trainees performed the tasks in the animal model and the last time, mainly in the knowledge of procedure and Holmium laser lithotripsy cathegories. At the beginning of level III, there are also statistical significant differences between trainee's scores and the expert's scores.

CONCLUSIONS

This realistic Endourology training model allows the acquisition of knowledge and technical and non-technical skills as evidenced by the face, content and constructive validity. Structured use of bench models (biological and non biological) and animal model simulators increase the endourological basic skills.

Comparative analysis of the functionality of simulators of the da Vinci surgical robot

BACKGROUND

The implementation of robotic technology in minimally invasive surgery has led to the need to develop more efficient and effective training methods, as well as assessment and skill maintenance tools for surgical education. Multiple simulators and procedures are available for educational and training purposes. A need for comparative evaluations of these simulators exists to aid users in selecting an appropriate device for their purposes.

METHODS

We conducted an objective review and comparison of the design and capabilities of all dedicated simulators of the da Vinci robot, the da Vinci Skill Simulator (DVSS) (Intuitive Surgical Inc., Sunnyvale, CA, USA), dV-Trainer (dVT) (Mimic Technologies Inc., Seattle, WA, USA), and Robotic Surgery Simulator (RoSS) (Simulated Surgical Skills, LLC, Williamsville, NY, USA). This provides base specifications of the hardware and software, with an emphasis on the training capabilities of each system.

RESULTS

Each simulator contains a large number of training exercises, DVSS = 40, dVT = 65, and RoSS = 52 for skills development. All three offer 3D visual images but use different display technologies. The DVSS leverages the real robotic surgeon's console to provide visualization, hand controls, and foot pedals. The dVT and RoSS created simulated versions of all of these control systems. They include systems management services which allow instructors to collect, export, and analyze the scores of students using the simulators.

CONCLUSIONS

This study is the first to provide comparative information of the three simulators functional capabilities with an emphasis on their educational skills. They offer unique advantages and capabilities in training robotic surgeons. Each device has been the subject of multiple validation experiments which have been published in the literature. But those do not provide specific details on the capabilities of the simulators which are necessary for an understanding sufficient to select the one best suited for an organization's needs.

Simulator-based assessment of haptic surgical skill: a comparative study

The aim of this study was to examine if the forces applied by users of a haptic simulator could be used to distinguish expert surgeons from novices. Seven surgeons with significant operating room expertise and 9 novices with no surgical experience participated in this study. The experimental task comprised exploring 4 virtual materials with the haptic device and learning the precise forces required to compress the materials to various depths. The virtual materials differed in their stiffness and force-displacement profiles. The results revealed that for nonlinear virtual materials, surgeons applied significantly greater magnitudes of force than novices. Furthermore, for the softer nonlinear and linear materials, surgeons were significantly more accurate in reproducing forces than novices. The results of this study suggest that the magnitudes of force measured using haptic simulators may be used to objectively differentiate experts' haptic skill from that of novices. This knowledge can inform the design of virtual reality surgical simulators and lead to the future incorporation of haptic skills training in medical school curricula.

Face, content, and construct validation of the Bristol TURP trainer

INTRODUCTION

Validation studies are an important part of simulator evaluation and are considered necessary to establish the effectiveness of simulation-based training. The widely used Bristol transurethral resection of prostate (TURP) simulator has not been formally validated.

OBJECTIVES

Evaluation of the face, content, and construct validities of the Bristol TURP simulator as an endourology training tool.

DESIGN

Using established validation methodology, face, content, and construct validities were evaluated. Face and content validities were assessed using a structured quantitative survey. Construct validity was assessed by comparing the performance of experts and novices using a validated performance scale and resection efficiency.

PARTICIPANTS AND SETTING

Overall, 8 novice urologists and 8 expert urologists participated in the study. The study was conducted in a dedicated surgical simulation training facility.

RESULTS

All 16 participants felt the model was a good training tool and should be used as an essential part of urology training (face validity). Content validity evaluation showed that most aspects of the simulator were adequately realistic (mean Likert scores 3.38-3.57/5); however, the model does not simulate bleeding. Experts significantly outperformed novices (p < 0.001) across all measures of performance, therefore establishing construct validity.

CONCLUSIONS

The Bristol TURP simulator shows face, content, and construct validities, although some aspects of the simulator were not very realistic (e.g., bleeding). This study provides evidence for the continuing use of this simulator in endourology training.

Visual control strategies of surgeons: a novel method of establishing the construct validity of a transurethral resection of the prostate surgical simulator

OBJECTIVE

To examine novice and expert differences in visual control strategies while performing a virtual reality transurethral resection of the prostate (TURP) task and to determine if these differences could provide a novel method for assessing construct validity of the simulator.

SUBJECTS AND METHODS

A total of 11 novices (no TURP experience) and 7 experts (>200 TURPs) completed a virtual reality prostate resection task on the TURPsim (Simbionix USA Corp, Cleveland, OH) while wearing an eye tracker (ASL, Bedford, MA). Performance parameters and the surgeon's visual control strategy were measured and compared between the 2 groups.

RESULTS

Experts resected a greater percentage of prostate than novices (p < 0.01) and had less active diathermy time without tissue contact (p < 0.01). Experts adopted a target-locking visual strategy, employing fewer visual fixations (p < 0.05) with longer mean fixation duration (p < 0.005). With multiple learning trials, novices' performance improved and the adoption of a more expertlike gaze strategy was observed.

CONCLUSION

Significant differences between experts and novices in both performance and visual control strategy were observed. The study of visual control strategies may be a useful adjunct, alongside measurements of motor performance, providing a novel method of assessing the construct validity of surgical simulators.

Effects of robotic manipulators on movements of novices and surgeons

BACKGROUND

Robot-assisted surgery is widely adopted for many procedures but has not realized its full potential to date. Based on human motor control theories, the authors hypothesized that the dynamics of the master manipulators impose challenges on the motor system of the user and may impair performance and slow down learning. Although studies have shown that robotic outcomes are correlated with the case experience of the surgeon, the relative contribution of cognitive versus motor skill is unknown. This study quantified the effects of da Vinci Si master manipulator dynamics on movements of novice users and experienced surgeons and suggests possible implications for training and robot design.

METHODS

In the reported study, six experienced robotic surgeons and ten novice nonmedical users performed movements under two conditions: teleoperation of a da Vinci Si Surgical system and freehand. A linear mixed model was applied to nine kinematic metrics (including endpoint error, movement time, peak speed, initial jerk, and deviation from a straight line) to assess the effects of teleoperation and expertise. To assess learning effects, t tests between the first and last movements of each type were used.

RESULTS

All the users moved slower during teleoperation than during freehand movements (F(1,9343) = 345; p < 0.001). The experienced surgeons had smaller errors than the novices (F(1,14) = 36.8; p < 0.001). The straightness of movements depended on their direction (F(7,9343) = 117; p < 0.001). Learning effects were observed in all conditions. Novice users first learned the task and then the dynamics of the manipulator.

CONCLUSIONS

The findings showed differences between the novices and the experienced surgeons for extremely simple point-to-point movements. The study demonstrated that manipulator dynamics affect user movements, suggesting that these dynamics could be improved in future robot designs. The authors showed the partial adaptation of novice users to the dynamics. Future studies are needed to evaluate whether it will be beneficial to include early training sessions dedicated to learning the dynamics of the manipulator.

Assessing visual control during simulated and live operations: gathering evidence for the content validity of simulation using eye movement metrics

BACKGROUND

Although virtual reality (VR) simulators serve an important role in the training and assessment of surgeons, they need to be evaluated for evidence of validity. Eye-tracking technology and measures of visual control have been used as an adjunct to the performance parameters produced by VR simulators to help in objectively establishing the construct validity (experts vs. novices) of VR simulators. However, determining the extent to which VR simulators represent the real procedure and environment (content validity) has largely been a subjective process undertaken by experienced surgeons. This study aimed to examine the content validity of a VR transurethral resection of the prostate (TURP) simulator by comparing visual control metrics taken during simulated and real TURP procedures.

METHODS

Eye-tracking data were collected from seven surgeons performing 14 simulated TURP operations and three surgeons performing 15 real TURP operations on live patients. The data were analyzed offline, and visual control metrics (number and duration of fixations, percentage of time the surgeons fixated on the screen) were calculated.

RESULTS

The surgeons displayed more fixations of a shorter duration and spent less time fixating on the video monitor during the real TURP than during the simulated TURP. This could have been due to (1) the increased complexity of the operating room (OR) environment (2) the decreased quality of the image of the urethra and associated anatomy (compared with the VR simulator), or (3) the impairment of visual attentional control due to the increased levels of stress likely experienced in the OR.

CONCLUSIONS

The findings suggest that the complexity of the environment surrounding VR simulators needs to be considered in the design of effective simulated training curricula. The study also provides support for the use of eye-tracking technology to assess the content validity of simulation and to examine psychomotor processes during live operations.

Task and crisis analysis during surgical training

PURPOSE

To design a surgical training environment based on task and crisis analysis of the surgical workflow.

METHOD

The environment consists of: (1) real surgical instruments that are augmented with realistic haptic feedback and VR capabilities, (2) human sensory channels such as tactile, auditory and visual in real time, and (3) the ability to facilitate deliberate exposure to adverse events enabling mediation of error recovery strategies.

VALIDATION

Five surgeons were immersed in our medical simulation environment through task and crisis scenarios of a typical vertebroplasty workflow.

RESULTS

Based on a five-point Likert-scale survey, the face validity of our simulation environment was confirmed by investigating surgeon behavior and workflow response.

CONCLUSIONS

The result of the conducted user-study corroborates our unique medical simulation concept of combining VR and human multisensory responses into surgical workflow.

Development and evaluation of a canine laparoscopic simulator for veterinary clinical training

Human laparoscopic simulators have been used in medical education for minimally invasive surgery (MIS) in the past years. Simulator-based laparoscopic training has attracted much interest because unique skills have to be learned not only by surgeons in training but also by surgeons in practice. MIS forces the surgeon to adapt to monocular vision and decreased tactile sensation and entails training and improving hand-eye and hand-hand coordination. Those skills require a learning curve that could be overcome gradually with use of simulators. The Canine Laparoscopic Simulator (CLS) for laparoscopic training was developed based on the working and optical space obtained from computed tomography (CT) scan images of three Beagle dogs. Thirty veterinarians (expert group, n=7; novice group, n=23) performed basic laparoscopic exercises in one training session on the CLS. During the performance of the exercises, an experienced laparoscopic veterinarian assessed all the tasks. Afterwards, participants were asked to complete an anonymous survey describing their experience. Most participants expressed positive opinions about the design and usability of the CLS. There were no significant differences between the two groups' opinions. The CLS showed good preliminary acceptance in the basic laparoscopy tasks by veterinarians. They perceived it to be a good training tool, and these results suggest that CLS is an engaging tool for education but still has some limitations inherent in training boxes. Further studies would be needed to establish the validity of training programs performed in the CLS.

An update and review of simulation in urological training

Simulation, if appropriately integrated into surgical training, may provide a time efficient, cost effective and safe method of training. The use of simulation in urology training is supported by a growing evidence base for its use, leading many authors to call for it to be integrated into the curriculum. There is growing evidence for the utilisation of part task (technical skills) simulators to shorten the learning curve in an environment that does not compromise patient safety. There is also evidence that non-technical skills affect patient outcomes in the operating room and that high fidelity team based simulation training can improve non-technical skills and surgical team performance. This evidence has strengthened the argument of surgical educators who feel that simulation should be formally incorporated into the urology training curriculum to develop both technical and non-technical skills with the aim of optimising performance and patient safety.

A systematic review on low-cost box models to achieve basic and advanced laparoscopic skills during modern surgical training

INTRODUCTION

Low-cost box models (BMs) are a valuable tool alternative to virtual-reality simulators. We aim to provide surgical trainees with a description of most common BMs and to present their validity to achieve basic and advanced laparoscopic skills.

MATERIALS AND METHODS

A literature search was undertaken for all studies focusing on BMs, excluded were those presenting data on virtual-reality simulators only. Databases were screened up to December 2011.

RESULTS

Numerous studies focused on various BMs to improve generic tasks (ie, instrument navigation, coordination, and cutting). Only fewer articles described models specific for peculiar operations. All studies showed a significant improvement of basic laparoscopic skills after training with BMs. Furthermore, their low costs make them easily available to most surgical trainees.

CONCLUSIONS

BMs should be developed by all surgical trainees during their training. Fields for future improvement regard endoscopy and complex laparoscopic operations for which ad hoc BMs are not available.

Instructional design affects the efficacy of simulation-based training in central venous catheterization

BACKGROUND

Simulation-based learning is a common educational tool in health care training and frequently involves instructional designs based on Experiential Learning Theory (ELT). However, little research explores the effectiveness and efficiency of different instructional design methodologies appropriate for simulations. The aim of this study was to compare 2 instructional design models, ELT and Guided Experiential Learning (GEL), to determine which is more effective for training the central venous catheterization procedure.

METHODS

Using a quasi-experimental randomized block design, nurse anesthetists completed training under 1 of the 2 instructional design models. Performance was assessed using a checklist of central venous catheterization performance, pass rates, and critical action errors.

RESULTS

Participants in the GEL condition performed significantly better than those in the ELT condition on the overall checklist score after controlling for individual practice time (F[1, 29] = 4.021, P = .027, Cohen's d = .71), had higher pass rates (P = .006, Cohen's d = 1.15), and had lower rates of failure due to critical action errors (P = .038, Cohen's d = .81).

CONCLUSIONS

The GEL model of instructional design is significantly more effective than ELT for simulation-based learning of the central venous catheterization procedure, yielding large differences in effect size.

Simulation Training in Health Care

In this chapter, we discuss the application of human factors and ergonomics to developing effective simulation training in health care. Simulation provides a safe, effective method for training and assessing human performance. In aviation, simulation-based training and assessment has been widely used, significantly improving safety. This progress would have been impossible without the involvement of human factors and ergonomics. Although aviation and health care have similarities, there also are differences that complicate the widespread implementation of simulation in health care.

Training in ureteroscopy for urolithiasis

Objectives

To provide an insight into the current status of semi-rigid and flexible ureteroscopy, following new curricula for training methods, including training with models, virtual reality and active mentoring.

Methods

We systematically reviewed previous reports, including articles in English identified using the following strategy: (‘ureteroscopy’[Mesh]) or (‘urolithiasis’[Mesh]) AND (‘education’[Mesh]), or (‘teaching’[Mesh]). Abstracts submitted at congresses were not included. Relevant articles that were identified as references in the retrieved articles were also included.

Results

The terms (‘urolithiasis’[Mesh] AND ‘education’[Mesh]) retrieved 106 articles, of which five were included. The terms (‘urolithiasis’[Mesh] AND ‘teaching’[Mesh]) retrieved six articles, of which three were included. The terms (‘ureteroscopy’[Mesh] AND ‘education’[Mesh]) retrieved 29 articles, of which 21 were included. The terms (‘ureteroscopy’[Mesh] AND ‘teaching’[Mesh]) retrieved eight articles, of which seven were included. Remaining articles were found in the reference section of retrieved articles. Finally, 43 articles were included. Four randomised controlled trials with level 1b evidence were included. Currently there is no standard teaching method for ureteroscopy and the number of cases to reach competence has not yet been defined. However, simulation-based training has been shown to be effective, cost-effective, and to increase patient safety.

Conclusions

Simulators lead to a more rapid acquisition of skills in ureteroscopy than do conventional training methods, and improve the performance of future surgeons. Flexible ureteroscopy simulators are a promising tool for training, and have the advantage of minimising the need for learning the procedures on patients. A didactic and clinical curriculum, including surgical videotape reviews as well as operative mentoring, enables a rapid progression in already experienced endourologists. However, there are few reports specifically addressing the skills necessary for training.

Technology-enhanced simulation to assess health professionals: a systematic review of validity evidence, research methods, and reporting quality

PURPOSE

To summarize the tool characteristics, sources of validity evidence, methodological quality, and reporting quality for studies of technology-enhanced simulation-based assessments for health professions learners.

METHOD

The authors conducted a systematic review, searching MEDLINE, EMBASE, CINAHL, ERIC, PsychINFO, Scopus, key journals, and previous reviews through May 2011. They selected original research in any language evaluating simulation-based assessment of practicing and student physicians, nurses, and other health professionals. Reviewers working in duplicate evaluated validity evidence using Messick's five-source framework; methodological quality using the Medical Education Research Study Quality Instrument and the revised Quality Assessment of Diagnostic Accuracy Studies; and reporting quality using the Standards for Reporting Diagnostic Accuracy and Guidelines for Reporting Reliability and Agreement Studies.

RESULTS

Of 417 studies, 350 (84%) involved physicians at some stage in training. Most focused on procedural skills, including minimally invasive surgery (N=142), open surgery (81), and endoscopy (67). Common elements of validity evidence included relations with trainee experience (N=306), content (142), relations with other measures (128), and interrater reliability (124). Of the 217 studies reporting more than one element of evidence, most were judged as having high or unclear risk of bias due to selective sampling (N=192) or test procedures (132). Only 64% proposed a plan for interpreting the evidence to be presented (validity argument).

CONCLUSIONS

Validity evidence for simulation-based assessments is sparse and is concentrated within specific specialties, tools, and sources of validity evidence. The methodological and reporting quality of assessment studies leaves much room for improvement.

Design and implementation of a proficiency-based, structured endoscopy course for medical students applying for a surgical specialty

BACKGROUND

Surgical simulation is becoming increasingly important in surgical education. Despite the important work done on simulators, simulator model development, and simulator assessment methodologies, there is a need for development of integrated simulators in the curriculum. In this paper, we describe the design of our evidence-based preclinical training program for medical students applying for a surgical career at the Centre for Surgical Technologies.

METHODS

Twenty-two students participated in this training program. During their final months as medical students, they received structured, proficiency-based endoscopy training. The total amount of mentored training was 18 hours and the training was organized into three training blocks. The first block focused on psychomotor training, the second block focused on laparoscopic stitching and suturing, and the third block on laparoscopic dissection techniques and hemostasis. Deliberate practice was allowed and students had to show proficiency before proceeding to the next training block. Students' psychomotor abilities were tested before the course and after each training block. At the beginning of their careers as surgical registrars, their performance on a laparoscopic suturing task was compared with that of registrars from the previous year who did not have this training course. Student opinions about this course were evaluated using a visual analog scale.

RESULTS

All students rated the training course as useful and their psychomotor abilities improved markedly. All students performed deliberate practice, and those who participated in this course scored significantly (P < 0.0001) better on the laparoscopic suturing task than first year registrars who did not participate in this course.

CONCLUSION

Organization of a structured preclinical training program in laparoscopy for final year medical students is feasible, attractive, and successful.

High-fidelity simulation is superior to case-based discussion in teaching the management of shock

BACKGROUND

Case-based discussion (CBD) is an established method for active learning in medical education. High-fidelity simulation has emerged as an important new educational technology. There is limited data from direct comparisons of these modalities.

AIMS

The primary purpose of this study was to compare the effectiveness of high-fidelity medical simulation with CBD in an undergraduate medical curriculum for shock.

METHODS

The subjects were 85 third-year medical students in their required surgery rotation. Scheduling circumstances created two equal groups. One group managed a case of septic shock in simulation and discussed a case of cardiogenic shock, the other group discussed septic shock and experienced cardiogenic shock through simulation. Student comprehension of the assessment and management of shock was then evaluated by oral examination (OE).

RESULTS

Examination scores were superior in all comparisons for the type of shock experienced through simulation. This was true regardless of the shock type. Scores associated with patient evaluation and invasive monitoring, however, showed no difference between groups or in crossover comparison.

CONCLUSIONS

In this study, students demonstrated better understanding of shock following simulation than after CBD. The secondary finding was the effectiveness of an OE with just-in-time deployment in curriculum assessment.

Development and validation of a theoretical test in basic laparoscopy

BACKGROUND

Testing of knowledge is an important component in a successful skills curriculum. Nonetheless, structured testing of basic procedure-relevant knowledge in the surgical domains is not ordinary practice. A regional need assessment showed insufficient knowledge regarding basic laparoscopy for first-year residents in obstetrics and gynecology. This study therefore aimed to develop and validate a framework for a theoretical knowledge test, a multiple-choice test, in basic theory related to laparoscopy.

METHODS

The content of the multiple-choice test was determined by conducting informal conversational interviews with experts in laparoscopy. The subsequent relevance of the test questions was evaluated using the Delphi method involving regional chief physicians. Construct validity was tested by comparing test results from three groups with expected different clinical competence and knowledge levels: senior medical students, first-year residents, and chief physicians.

RESULTS

The four conversational interviews resulted in the development of 47 test questions, which were narrowed down to 37 test questions after two Delphi rounds involving 12 chief physicians. Significant differences were found between the test scores from the senior medical students (n = 14) and the first-year residents (n = 52) (median test scores, 18 vs. 24, respectively; p = 0.001), and between the first-year residents and the chief physicians (n = 12) (median test scores, 24 vs. 33, respectively; p = 0.001). Internal consistency (Cronbach's alpha) was 0.82. There was no evidence of differential item functioning between the three groups tested.

CONCLUSIONS

A newly developed knowledge test in basic laparoscopy proved to have content and construct validity. The formula for the development and validation of a theoretical test could potentially be used for any topics that require structured testing of knowledge.

Evaluation of a novel laparoscopic simulation laboratory curriculum

BACKGROUND

Simulation training may be an invaluable tool in training and assessing surgeons. Questions remain regarding its true impact on acquisition of laparoscopic skills and their transferability to the operating room.

METHODS

Twenty general surgery residents completed 5 exercises: bean drop/rope drill, foam cutting/endoloop, checkerboard, endostitch, and suturing/intracorporeal knot. After baseline scoring, residents had 3 weeks to practice before re-testing. Statistical analysis of objective and subjective scores included the signed rank test or paired t-test, the Kruskal-Walis test, the McNemar test, and the Global Operative Assessment of Laparoscopic Skills (GOALS). P < .05 was considered significant.

RESULTS

Five postgraduate year (PGY) 4 and 15 PGY 1-3 residents completed the curriculum. When compared with baseline, overall scores improved (P < .05) in all categories except for the bean drop/rope drill, which improved on objectively measured tasks only (dropped beads and beads threaded). The foam cutting/endoloop task was mastered by a majority of trainees at baseline leaving no room for improvement.

CONCLUSION

The results of this study reinforce that simulation leads to improvement in laparoscopic skills and that our curriculum is a valid educational tool. Further studies are needed to validate whether this results in improved skills in the operating room.

Program for laparoscopic urologic skills: a newly developed and validated educational program

OBJECTIVE

To develop and evaluate a program for laparoscopic urologic skills (PLUS) to determine the face, content, and construct validation to achieve uniformity and standardization in training residents in urology.

METHODS

The PLUS consists of 5 basic laparoscopic tasks. Three tasks were abstracted from the Fundamentals of Laparoscopic Surgery program, and 2 additional tasks were developed under continuous evaluation by expert urologists. Fifty participants were recruited from different hospitals and performed the final PLUS training. They all completed a questionnaire after performance. Three outcome parameters were measured: performance quality, time, and dropped objects. The relationship between laparoscopic experience and the outcome parameters was investigated.

RESULTS

Of the 50 participants, 13 were students, 20 were residents, and 17 were urologists. Double-log linear regression analysis for all 5 tasks showed a significant effect (effect size range 0.53-0.82; P < .0005) for laparoscopic experience on performance time. Substantial correlations were found between experience and quality ratings (log-linear regression effect size 0.37; P = .012) and the number of dropped objects (Spearman correlation effect size 0.49; P < .01). The usefulness of the PLUS model as a training tool for basic laparoscopic skills was rated 4.55 on a scale from 1 (not useful) to 5 (useful) (standard deviation 0.58; range 3-5).

CONCLUSION

The results of the present study indicated the face, content, and construct validity for the PLUS. The training is considered appropriate for use as a primary training tool for an entry test or as part of a step-wise training program in which basic and procedural laparoscopic skills are integrated.

Cheating experience: Guiding novices to adopt the gaze strategies of experts expedites the learning of technical laparoscopic skills

BACKGROUND

Previous research has demonstrated that trainees can be taught (via explicit verbal instruction) to adopt the gaze strategies of expert laparoscopic surgeons. The current study examined a software template designed to guide trainees to adopt expert gaze control strategies passively, without being provided with explicit instructions.

METHODS

We examined 27 novices (who had no laparoscopic training) performing 50 learning trials of a laparoscopic training task in either a discovery-learning (DL) group or a gaze-training (GT) group while wearing an eye tracker to assess gaze control. The GT group performed trials using a surgery-training template (STT); software that is designed to guide expert-like gaze strategies by highlighting the key locations on the monitor screen. The DL group had a normal, unrestricted view of the scene on the monitor screen. Both groups then took part in a nondelayed retention test (to assess learning) and a stress test (under social evaluative threat) with a normal view of the scene.

RESULTS

The STT was successful in guiding the GT group to adopt an expert-like gaze strategy (displaying more target-locking fixations). Adopting expert gaze strategies led to an improvement in performance for the GT group, which outperformed the DL group in both retention and stress tests (faster completion time and fewer errors).

CONCLUSION

The STT is a practical and cost-effective training interface that automatically promotes an optimal gaze strategy. Trainees who are trained to adopt the efficient target-locking gaze strategy of experts gain a performance advantage over trainees left to discover their own strategies for task completion.

Instructor feedback versus no instructor feedback on performance in a laparoscopic virtual reality simulator: a randomized educational trial

BACKGROUND

Several studies have found a positive effect on the learning curve as well as the improvement of basic psychomotor skills in the operating room after virtual reality training. Despite this, the majority of surgical and gynecological departments encounter hurdles when implementing this form of training. This is mainly due to lack of knowledge concerning the time and human resources needed to train novice surgeons to an adequate level. The purpose of this trial is to investigate the impact of instructor feedback regarding time, repetitions and self-perception when training complex operational tasks on a virtual reality simulator.

METHODS/DESIGN

The study population consists of medical students on their 4th to 6th year without prior laparoscopic experience. The study is conducted in a skills laboratory at a centralized university hospital. Based on a sample size estimation 98 participants will be randomized to an intervention group or a control group. Both groups have to achieve a predefined proficiency level when conducting a laparoscopic salpingectomy using a surgical virtual reality simulator. The intervention group receives standardized instructor feedback of 10 to 12 min a maximum of three times. The control group receives no instructor feedback. Both groups receive the automated feedback generated by the virtual reality simulator. The study follows the CONSORT Statement for randomized trials. Main outcome measures are time and repetitions to reach the predefined proficiency level on the simulator. We include focus on potential sex differences, computer gaming experience and self-perception.

DISCUSSION

The findings will contribute to a better understanding of optimal training methods in surgical education.

TRIAL REGISTRATION

NCT01497782.

Face validity, construct validity and training benefits of a virtual reality TURP simulator

OBJECTIVE

To assess face validity, construct validity and the training benefits of a virtual reality TURP simulator.

METHOD

11 novices (no TURP experience) and 7 experts (>200 TURP's) completed a virtual reality median lobe prostate resection task on the TURPsim™ (Simbionix USA Corp., Cleveland, OH). Performance indicators (percentage of prostate resected (PR), percentage of capsular resection (CR) and time diathermy loop active without tissue contact (TAWC) were recorded via the TURPsim™ and compared between novices and experts to assess construct validity. Verbal comments provided by experts following task completion were used to assess face validity. Repeated attempts of the task by the novices were analysed to assess the training benefits of the TURPsim™.

RESULTS

Experts resected a significantly greater percentage of prostate per minute (p < 0.01) and had significantly less active diathermy time without tissue contact (p < 0.01) than novices. After practice, novices were able to perform the simulation more effectively, with significant improvement in all measured parameters. Improvement in performance was noted in novices following repetitive training, as evidenced by improved TAWC scores that were not significantly different from the expert group (p = 0.18).

CONCLUSIONS

This study has established face and construct validity for the TURPsim™. The potential benefit in using this tool to train novices has also been demonstrated.

Validation of a novel resin-porcine thorax model for chest drain insertion training

Chest drain insertion in inexperienced hands carries a significant morbidity and mortality. The royal colleges, recognising this, stipulated that chest drain insertion be included as one of the core competences for all core medical trainees. However, there is no formal training in chest drain insertion included in training programmes. Simulation training should, in theory, provide a safe and objective method to overcome the obstacles in chest drain insertion training. There have been a number of attempts to find the ideal simulator for chest drain insertion with varying success. This article describes a model which is practical and affordable in all clinical skills labs, using porcine ribs mounted on a resin cast of a human thorax, and the data about the validation of the porcine-thorax model for chest drain insertion presented.

Designing simulator-based training: an approach integrating cognitive task analysis and four-component instructional design

Most studies of simulator-based surgical skills training have focused on the acquisition of psychomotor skills, but surgical procedures are complex tasks requiring both psychomotor and cognitive skills. As skills training is modelled on expert performance consisting partly of unconscious automatic processes that experts are not always able to explicate, simulator developers should collaborate with educational experts and physicians in developing efficient and effective training programmes. This article presents an approach to designing simulator-based skill training comprising cognitive task analysis integrated with instructional design according to the four-component/instructional design model. This theory-driven approach is illustrated by a description of how it was used in the development of simulator-based training for the nephrostomy procedure.

Implicit motor learning promotes neural efficiency during laparoscopy

BACKGROUND

An understanding of differences in expert and novice neural behavior can inform surgical skills training. Outside the surgical domain, electroencephalographic (EEG) coherence analyses have shown that during motor performance, experts display less coactivation between the verbal-analytic and motor planning regions than their less skilled counterparts. Reduced involvement of verbal-analytic processes suggests greater neural efficiency. The authors tested the utility of an implicit motor learning intervention specifically devised to promote neural efficiency by reducing verbal-analytic involvement in laparoscopic performance.

METHODS

In this study, 18 novices practiced a movement pattern on a laparoscopic trainer with either conscious awareness of the movement pattern (explicit motor learning) or suppressed awareness of the movement pattern (implicit motor learning). In a retention test, movement accuracy was compared between the conditions, and coactivation (EEG coherence) was assessed between the motor planning (Fz) region and both the verbal-analytic (T3) and the visuospatial (T4) cortical regions (T3-Fz and T4-Fz, respectively).

RESULTS

Movement accuracy in the conditions was not different in a retention test (P = 0.231). Findings showed that the EEG coherence scores for the T3-Fz regions were lower for the implicit learners than for the explicit learners (P = 0.027), but no differences were apparent for the T4-Fz regions (P = 0.882).

CONCLUSIONS

Implicit motor learning reduced EEG coactivation between verbal-analytic and motor planning regions, suggesting that verbal-analytic processes were less involved in laparoscopic performance. The findings imply that training techniques that discourage nonessential coactivation during motor performance may provide surgeons with more neural resources with which to manage other aspects of surgery.

Perceptual impairment and psychomotor control in virtual laparoscopic surgery

BACKGROUND

It is recognised that one of the major difficulties in performing laparoscopic surgery is the translation of two-dimensional video image information to a three-dimensional working area. However, research has tended to ignore the gaze and eye-hand coordination strategies employed by laparoscopic surgeons as they attempt to overcome these perceptual constraints. This study sought to examine if measures related to tool movements, gaze strategy, and eye-hand coordination (the quiet eye) differentiate between experienced and novice operators performing a two-handed manoeuvres task on a virtual reality laparoscopic surgical simulator (LAP Mentor™).

METHODS

Twenty-five right-handed surgeons were categorised as being either experienced (having led more than 60 laparoscopic procedures) or novice (having performed fewer than 10 procedures) operators. The 10 experienced and 15 novice surgeons completed the "two-hand manoeuvres" task from the LAP Mentor basic skills learning environment while wearing a gaze registration system. Performance, movement, gaze, and eye-hand coordination parameters were recorded and compared between groups.

RESULTS

The experienced surgeons completed the task significantly more quickly than the novices, used significantly fewer movements, and displayed shorter tool paths. Gaze analyses revealed that experienced surgeons spent significantly more time fixating the target locations than novices, who split their time between focusing on the targets and tracking the tools. A more detailed analysis of a difficult subcomponent of the task revealed that experienced operators used a significantly longer aiming fixation (the quiet eye period) to guide precision grasping movements and hence needed fewer grasp attempts.

CONCLUSION

The findings of the study provide further support for the utility of examining strategic gaze behaviour and eye-hand coordination measures to help further our understanding of how experienced surgeons attempt to overcome the perceptual difficulties inherent in the laparoscopic environment.