Towards virtual FLS: development of a peg transfer simulator

Objective metrics for hand-sewn bowel anastomoses can differentiate novice from expert surgeons

BACKGROUND

Assessing performance automatically in a virtual reality trainer or from recorded videos is advantageous but needs validated objective metrics. The purpose of this study is to obtain expert consensus and validate task-specific metrics developed for assessing performance in double-layered end-to-end anastomosis.

MATERIALS AND METHODS

Subjects were recruited into expert (PGY 4-5, colorectal surgery residents, and attendings) and novice (PGY 1-3) groups. Weighted average scores of experts for each metric item, completion time, and the total scores computed using global and task-specific metrics were computed for assessment.

RESULTS

A total of 43 expert surgeons rated our task-specific metric items with weighted averages ranging from 3.33 to 4.5 on a 5-point Likert scale. A total of 20 subjects (10 novices and 10 experts) participated in validation study. The novice group completed the task significantly more slowly than the experienced group (37.67 ± 7.09 vs 25.47 ± 7.82 min, p = 0.001). In addition, both the global rating scale (23.47 ± 4.28 vs 28.3 ± 3.85, p = 0.016) and the task-specific metrics showed a significant difference in performance between the two groups (38.77 ± 2.83 vs 42.58 ± 4.56 p = 0.027) following partial least-squares (PLS) regression. Furthermore, PLS regression showed that only two metric items (Stay suture tension and Tool handling) could reliably differentiate the performance between the groups (20.41 ± 2.42 vs 24.28 ± 4.09 vs, p = 0.037).

CONCLUSIONS

Our study shows that our task-specific metrics have significant discriminant validity and can be used to evaluate the technical skills for this procedure.

Face, content, and construct validity of the virtual immersive operating room simulator for training laparoscopic procedures

BACKGROUND

The aim of this work is to present the face, content, and construct validation of the virtual immersive operating room simulator (VIORS) for procedural training of surgeons' laparoscopic psychomotor skills and evaluate the immersive training experience.

METHODS

The VIORS simulator consists of an HMD Oculus Rift 2016 with a visor on a 1080 × 1200 pixel OLED screen, two positioning sensors with two adapted controls to simulate laparoscopic instruments, and an acrylic base to simulate the conventional laparoscopic setup. The immersion consists of a 360° virtual operating room environment, based on the EndoSuite at Hospital Infantil de Mexico Federico Gomez, which reproduces a configuration of equipment, instruments, and common distractions in the operating room during a laparoscopic cholecystectomy procedure. Forty-five surgeons, residents, and medicine students participated in this study: 27 novices, 13 intermediates, and 5 experts. They completed a questionnaire on the realism and operating room immersion, as well as their capabilities for laparoscopic procedural training, scored in the 5-point Likert scale. The data of instrument movement were recorded and analyzed using 13 movement analysis parameters (MAPs). The experience during training with VIORS was evaluated through NASA-TLX.

RESULTS

The participants were enthusiastic about the immersion and sensation levels of the VIORS simulator, with positive scores on the realism and its capabilities for procedural training using VIORS. The results proved that the VIORS simulator was able to differentiate between surgeons with different skill levels. Statistically significant differences were found in nine MAPs, demonstrating their construct validity for the objective assessment of the procedural laparoscopic performance. At cognitive level, the inversion experience proves a moderate mental workload when the laparoscopic procedure is carried out.

CONCLUSION

The VIORS simulator has been successfully presented and validated. The VIORS simulator is a useful and effective device for the training of procedural laparoscopic psychomotor skills.

Surgeons With Five or More Actual Cricothyrotomies Perform Significantly Better on a Virtual Reality Simulator

BACKGROUND

Discriminating performance of learners with varying experience is essential to developing and validating a surgical simulator. For rare and emergent procedures such as cricothyrotomy (CCT), the criteria to establish such groups are unclear. This study is to investigate the impact of surgeons' actual CCT experience on their virtual reality simulator performance and to determine the minimum number of actual CCTs that significantly discriminates simulator scores. Our hypothesis is that surgeons who performed more actual CCT cases would perform better on a virtual reality CCT simulator.

METHODS

47 clinicians were recruited to participate in this study at the 2018 annual conference of the Society of American Gastrointestinal and Endoscopic Surgeons. We established groups based on three different experience thresholds, that is, the minimal number of CCT cases performed (1, 5, and 10), and compared simulator performance between these groups.

RESULTS

Participants who had performed more clinical cases manifested higher mean scores in completing CCT simulation tasks, and those reporting at least 5 actual CCTs had significantly higher (P = 0.014) simulator scores than those who had performed fewer cases. Another interesting finding was that classifying participants based on experience level, that is, attendings, fellows, and residents, did not yield statistically significant differences in skills related to CCT.

CONCLUSIONS

The simulator was sensitive to prior experience at a threshold of 5 actual CCTs performed.

Towards Virtual VATS, Face, and Construct Evaluation for Peg Transfer Training of Box, VR, AR, and MR Trainer

The aim of this study is to develop and assess the peg transfer training module face, content and construct validation use of the box, virtual reality (VR), cognitive virtual reality (CVR), augmented reality (AR), and mixed reality (MR) trainer, thereby to compare advantages and disadvantages of these simulators. Training system (VatsSim-XR) design includes customized haptic-enabled thoracoscopic instruments, virtual reality helmet set, endoscope kit with navigation, and the patient-specific corresponding training environment. A cohort of 32 trainees comprising 24 novices and 8 experts underwent the real and virtual simulators that were conducted in the department of thoracic surgery of Yunnan First People's Hospital. Both subjective and objective evaluations have been developed to explore the visual and haptic potential promotions in peg transfer education. Experiments and evaluation results conducted by both professional and novice thoracic surgeons show that the surgery skills from experts are better than novices overall, AR trainer is able to provide a more balanced training environments on visuohaptic fidelity and accuracy, box trainer and MR trainer demonstrated the best realism 3D perception and surgical immersive performance, respectively, and CVR trainer shows a better clinic effect that the traditional VR trainer. Combining these in a systematic approach, tuned with specific fidelity requirements, medical simulation systems would be able to provide a more immersive and effective training environment.

Development and face validation of a virtual camera navigation task trainer

BACKGROUND

The fundamentals of laparoscopic surgery (FLS) trainer box, which is now established as a standard for evaluating minimally invasive surgical skills, consists of five tasks: peg transfer, pattern cutting, ligation, intra- and extracorporeal suturing. Virtual simulators of these tasks have been developed and validated as part of the Virtual Basic Laparoscopic Skill Trainer (VBLaST) (Arikatla et al. in Int J Med Robot Comput Assist Surg 10:344-355, 2014; Zhang et al. in Surg Endosc 27(10):3603-3615, 2013; Sankaranarayanan et al. in J Laparoendosc Adv Surg Tech 20(2):153-157, 2010; Qi et al. J Biomed Inform 75:48-62, 2017). The virtual task trainers have many advantages including automatic real-time objective scoring, reduced costs, and eliminating human proctors. In this paper, we extend VBLaST by adding two camera navigation system tasks: (a) pattern matching and (b) path tracing.

METHODS

A comprehensive camera navigation simulator with two virtual tasks, simplified and cheaper hardware interface (compared to the prior version of VBLaST), graphical user interface, and automated metrics has been designed and developed. Face validity of the system is tested with medical students and residents from the University at Buffalo's medical school.

RESULTS

The subjects rated the simulator highly in all aspects including its usefulness in training to center the target and to teach sizing skills. The quality and usefulness of the force feedback scored the lowest at 2.62.

Face and construct validation of a next generation virtual reality (Gen2-VR) surgical simulator

INTRODUCTION

Surgical performance is affected by distractors and interruptions to surgical workflow that exist in the operating room. However, traditional surgical simulators are used to train surgeons in a skills laboratory that does not recreate these conditions. To overcome this limitation, we have developed a novel, immersive virtual reality (Gen2-VR) system to train surgeons in these environments. This study was to establish face and construct validity of our system.

METHODS AND PROCEDURES

The study was a within-subjects design, with subjects repeating a virtual peg transfer task under three different conditions: Case I: traditional VR; Case II: Gen2-VR with no distractions and Case III: Gen2-VR with distractions and interruptions. In Case III, to simulate the effects of distractions and interruptions, music was played intermittently, the camera lens was fogged for 10 s and tools malfunctioned for 15 s at random points in time during the simulation. At the completion of the study subjects filled in a 5-point Likert scale feedback questionnaire. A total of sixteen subjects participated in this study.

RESULTS

Friedman test showed significant difference in scores between the three conditions (p < 0.0001). Post hoc analysis using Wilcoxon signed-rank tests with Bonferroni correction further showed that all the three conditions were significantly different from each other (Case I, Case II, p < 0.0001), (Case I, Case III, p < 0.0001) and (Case II, Case III, p = 0.009). Subjects rated that fog (mean 4.18) and tool malfunction (median 4.56) significantly hindered their performance.

CONCLUSION

The results showed that Gen2-VR simulator has both face and construct validity and that it can accurately and realistically present distractions and interruptions in a simulated OR, in spite of limitations of the current HMD hardware technology.

A simple sensor calibration technique for estimating the 3D pose of endoscopic instruments

INTRODUCTION

The aim of this study was to describe a simple and easy-to-use calibration method that is able to estimate the pose (tip position and orientation) of a rigid endoscopic instrument with respect to an electromagnetic tracking device attached to the handle.

METHODS

A two-step calibration protocol was developed. First, the orientation of the instrument shaft is derived by performing a 360° rotation of the instrument around its shaft using a firmly positioned surgical trocar. Second, the 3D position of the instrument tip is obtained by allowing the tip to come in contact with a planar surface.

RESULTS

The results indicate submillimeter accuracy in the estimation of the tooltip position, and subdegree accuracy in the estimation of the shaft orientation, both with respect to a known reference frame. The assets of the proposed method are also highlighted by illustrating an indicative application in the field of augmented reality simulation.

CONCLUSIONS

The proposed method is simple, inexpensive, does not require employment of special calibration frames, and has potential applications not only in training systems but also in the operating room.

Face and construct validation of a virtual peg transfer simulator

BACKGROUND

The Fundamentals of Laparoscopic Surgery (FLS) trainer box is now established as a standard for evaluating minimally invasive surgical skills. A particularly simple task in this trainer box is the peg transfer task which is aimed at testing the surgeon's bimanual dexterity, hand-eye coordination, speed, and precision. The Virtual Basic Laparoscopic Skill Trainer (VBLaST) is a virtual version of the FLS tasks which allows automatic scoring and real-time, subjective quantification of performance without the need of a human proctor. In this article we report validation studies of the VBLaST peg transfer (VBLaST-PT) simulator.

METHODS

Thirty-five subjects with medical background were divided into two groups: experts (PGY 4-5, fellows, and practicing surgeons) and novices (PGY 1-3). The subjects were asked to perform the peg transfer task on both the FLS trainer box and the VBLaST-PT simulator; their performance was evaluated based on established metrics of error and time. A new length of trajectory (LOT) metric has also been introduced for offline analysis. A questionnaire was used to rate the realism of the virtual system on a 5-point Likert scale.

RESULTS

Preliminary face validation of the VBLaST-PT with 34 subjects rated on a 5-point Likert scale questionnaire revealed high scores for all aspects of simulation, with 3.53 being the lowest mean score across all questions. A two-tailed Mann-Whitney test performed on the total scores showed significant (p = 0.001) difference between the groups. A similar test performed on the task time (p = 0.002) and the LOT (p = 0.004) separately showed statistically significant differences between the experts and the novices (p < 0.05). The experts appear to be traversing shorter overall trajectories in less time than the novices.

CONCLUSION

VBLaST-PT showed both face and construct validity and has promise as a substitute for the FLS for training peg transfer skills.