1
|
Fanizzi C, Carone G, Rocca A, Ayadi R, Petrenko V, Casali C, Rani M, Giachino M, Falsitta LV, Gambatesa E, Galbiati TF, Orena EF, Tramacere I, Riker NI, Mocca A, Schaller K, Meling TR, DiMeco F, Perin A. Simulation to become a better neurosurgeon. An international prospective controlled trial: The Passion study. BRAIN & SPINE 2024; 4:102829. [PMID: 38812880 PMCID: PMC11134543 DOI: 10.1016/j.bas.2024.102829] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/14/2024] [Revised: 04/30/2024] [Accepted: 05/06/2024] [Indexed: 05/31/2024]
Abstract
Introduction Surgical training traditionally adheres to the apprenticeship paradigm, potentially exposing trainees to an increased risk of complications stemming from their limited experience. To mitigate this risk, augmented and virtual reality have been considered, though their effectiveness is difficult to assess. Research question The PASSION study seeks to investigate the improvement of manual dexterity following intensive training with neurosurgical simulators and to discern how surgeons' psychometric characteristics may influence their learning process and surgical performance. Material and methods Seventy-two residents were randomized into the simulation group (SG) and control group (CG). The course spanned five days, commencing with assessment of technical skills in basic procedures within a wet-lab setting on day 1. Over the subsequent core days, the SG engaged in simulated procedures, while the CG carried out routine activities in an OR. On day 5, all residents' technical competencies were evaluated. Psychometric measures of all participants were subjected to analysis. Results The SG demonstrated superior performance (p < 0.0001) in the brain tumour removal compared to the CG. Positive learning curves were evident in the SG across the three days of simulator-based training for all tumour removal tasks (all p-values <0.05). No significant differences were noted in other tasks, and no meaningful correlations were observed between performance and any psychometric parameters. Discussion and conclusion A brief and intensive training regimen utilizing 3D virtual reality simulators enhances residents' microsurgical proficiency in brain tumour removal models. Simulators emerge as a viable tool to expedite the learning curve of in-training neurosurgeons.
Collapse
Affiliation(s)
- Claudia Fanizzi
- Department of Neurosurgery, Fondazione I.R.C.C.S. Istituto Neurologico
“C. Besta”, Milano, Italy
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
| | - Giovanni Carone
- Department of Neurosurgery, Fondazione I.R.C.C.S. Istituto Neurologico
“C. Besta”, Milano, Italy
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
| | - Alessandra Rocca
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
| | - Roberta Ayadi
- Department of Neurosurgery, Fondazione I.R.C.C.S. Istituto Neurologico
“C. Besta”, Milano, Italy
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
| | - Veronika Petrenko
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
| | - Cecilia Casali
- Department of Neurosurgery, Fondazione I.R.C.C.S. Istituto Neurologico
“C. Besta”, Milano, Italy
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
| | - Martina Rani
- Department of Psychology, Università Cattolica del Sacro Cuore, Milan,
Italy
| | - Marta Giachino
- Department of Psychology, Università Cattolica del Sacro Cuore, Milan,
Italy
| | - Lydia Viviana Falsitta
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
| | - Enrico Gambatesa
- Department of Neurosurgery, Fondazione I.R.C.C.S. Istituto Neurologico
“C. Besta”, Milano, Italy
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
| | - Tommaso Francesco Galbiati
- Department of Neurosurgery, Fondazione I.R.C.C.S. Istituto Neurologico
“C. Besta”, Milano, Italy
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
| | - Eleonora Francesca Orena
- Department of Neurosurgery, Fondazione I.R.C.C.S. Istituto Neurologico
“C. Besta”, Milano, Italy
| | - Irene Tramacere
- Department of Neurosurgery, Fondazione I.R.C.C.S. Istituto Neurologico
“C. Besta”, Milano, Italy
| | - Nicole Irene Riker
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
| | - Alessandro Mocca
- Department of Psychology, Università Cattolica del Sacro Cuore, Milan,
Italy
| | - Karl Schaller
- Department of Clinical Neurosciences, Division of Neurosurgery, Geneva
University Hospitals & Faculty of Medicine, Geneva, Switzerland
- Department of Clinical Neurosciences, Division of Neurosurgery, Geneva
University Hospitals & Faculty of Medicine, and SFITS, Geneva,
Switzerland
| | - Torstein Ragnar Meling
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
- Department of Neurosurgery, The National Hospital of Denmark,
Rigshospitalet, Copenhagen, Denmark
| | - Francesco DiMeco
- Department of Neurosurgery, Fondazione I.R.C.C.S. Istituto Neurologico
“C. Besta”, Milano, Italy
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
- Department of Pathophysiology and Transplantation, University of Milano,
Milano, Italy
- Department of Neurological Surgery, Johns Hopkins Medical School,
Baltimore, MD, USA
| | - Alessandro Perin
- Department of Neurosurgery, Fondazione I.R.C.C.S. Istituto Neurologico
“C. Besta”, Milano, Italy
- Besta NeuroSim Center, Fondazione I.R.C.C.S. Istituto Neurologico
Nazionale "C. Besta", Milano, Italy
- Department of Life Sciences, University of Trieste, Trieste,
Italy
| |
Collapse
|
2
|
Yilmaz R, Fazlollahi AM, Winkler-Schwartz A, Wang A, Makhani HH, Alsayegh A, Bakhaidar M, Tran DH, Santaguida C, Del Maestro RF. Effect of Feedback Modality on Simulated Surgical Skills Learning Using Automated Educational Systems- A Four-Arm Randomized Control Trial. JOURNAL OF SURGICAL EDUCATION 2024; 81:275-287. [PMID: 38160107 DOI: 10.1016/j.jsurg.2023.11.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/03/2023] [Revised: 09/05/2023] [Accepted: 11/01/2023] [Indexed: 01/03/2024]
Abstract
OBJECTIVE To explore optimal feedback methodologies to enhance trainee skill acquisition in simulated surgical bimanual skills learning during brain tumor resections. HYPOTHESES (1) Providing feedback results in better learning outcomes in teaching surgical technical skill when compared to practice alone with no tailored performance feedback. (2) Providing more visual and visuospatial feedback results in better learning outcomes when compared to providing numerical feedback. DESIGN A prospective 4-parallel-arm randomized controlled trial. SETTING Neurosurgical Simulation and Artificial Intelligence Learning Centre, McGill University, Canada. PARTICIPANTS Medical students (n = 120) from 4 Quebec medical schools. RESULTS Participants completed a virtually simulated tumor resection task 5 times while receiving 1 of 4 feedback based on their group allocation: (1) practice-alone without feedback, (2) numerical feedback, (3) visual feedback, and (4) visuospatial feedback. Outcome measures were participants' scores on 14-performance metrics and the number of expert benchmarks achieved during each task. There were no significant differences in the first task which determined baseline performance. A statistically significant interaction between feedback allocation and task repetition was found on the number of benchmarks achieved, F (10.558, 408.257)=3.220, p < 0.001. Participants in all feedback groups significantly improved their performance compared to baseline. The visual feedback group achieved significantly higher number of benchmarks than the practice-alone group by the third repetition of the task, p = 0.005, 95%CI [0.42 3.25]. Visual feedback and visuospatial feedback improved performance significantly by the second repetition of the task, p = 0.016, 95%CI [0.19 2.71] and p = 0.003, 95%CI [0.4 2.57], respectively. CONCLUSION Simulations with autonomous visual computer assistance may be effective pedagogical tools in teaching bimanual operative skills via visual and visuospatial feedback information delivery.
Collapse
Affiliation(s)
- Recai Yilmaz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Faculty of Medicine and Health Sciences, McGill University, Montreal, Canada.
| | - Ali M Fazlollahi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Faculty of Medicine and Health Sciences, McGill University, Montreal, Canada
| | - Alexander Winkler-Schwartz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Anna Wang
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | - Hafila Hassan Makhani
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | - Ahmad Alsayegh
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada; Division of Neurosurgery, Department of Surgery, Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Mohamad Bakhaidar
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada; Division of Neurosurgery, Department of Surgery, Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Dan Huy Tran
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | - Carlo Santaguida
- Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Rolando F Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Faculty of Medicine and Health Sciences, McGill University, Montreal, Canada; Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
3
|
Banko L, Patel RV, Nawabi N, Altshuler M, Medeiros L, Cosgrove GR, Bi WL. Strategies to improve surgical technical competency: a systematic review. Acta Neurochir (Wien) 2023; 165:3565-3572. [PMID: 37945995 DOI: 10.1007/s00701-023-05868-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2023] [Accepted: 10/18/2023] [Indexed: 11/12/2023]
Abstract
BACKGROUND A cornerstone of surgical residency training is an educational program that produces highly skilled and effective surgeons. Training structures are constantly being revised due to evolving program structures, shifting workforces, and variability in the clinical environment. This has resulted in significant heterogeneity in all surgical resident education, training tools utilized, and measures of training efficacy. METHODS We systematically reviewed educational interventions for technical skills in neurosurgery published across PubMed, Embase, and Web of Science over four decades. We extracted general characteristics of each surgical training tool while categorizing educational interventions by modality and neurosurgical application. RESULTS We identified 626 studies which developed surgical training tools across eight different training modalities: textbooks and literature (11), online resources (53), didactic teaching and one-on-one instruction (7), laboratory courses (50), cadaveric models (63), animal models (47), mixed reality (166), and physical models (229). While publication volume has grown exponentially, a majority of studies were cited with relatively low frequency. Most training programs were published in the development and validation phase with only 2.1% of tools implemented long-term. Each training modality expressed unique strengths and limitations, with limited data reported on the educational impact connected to each training tool. CONCLUSIONS Numerous surgical training tools have been developed and implemented across residency training programs. Though many creative and cutting-edge tools have been devised, evidence supporting educational efficacy and long-term application is lacking. Increased utilization of novel surgical training tools will require validation of metrics used to assess the training outcomes and optimized integration with clinical practice.
Collapse
Affiliation(s)
- Lauren Banko
- Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, USA
- University of Arkansas for Medical Sciences, Little Rock, AR, USA
| | - Ruchit V Patel
- Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, USA
| | - Noah Nawabi
- Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, USA
| | - Marcelle Altshuler
- Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, USA
| | - Lila Medeiros
- Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, USA
| | - G Rees Cosgrove
- Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, USA
| | - Wenya Linda Bi
- Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, USA.
| |
Collapse
|
4
|
Li Z, Liu PX, Hou W. Modeling fibrous soft tissue dissection with elastic-plastic deformation for simulation of brain tumor removal. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 232:107420. [PMID: 36854236 DOI: 10.1016/j.cmpb.2023.107420] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/12/2022] [Revised: 02/11/2023] [Accepted: 02/12/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE Realistic modeling the dissection of brain tissue is of key importance for simulation of brain tumor removal in virtual neurosurgery systems. However, existing methods are unable to characterize inelastic behaviors of brain tissue, such as plastic deformation and dissection evolution, making it ineffective in simulating brain tumor removal procedures. METHODS In this paper, a model of fibrous soft tissue dissection for the simulation of brain tumor removal is proposed. A dissection variable of representative volume element is used to characterize the dissection state of the fibrous soft tissue. The evolution of dissection with elastic-plastic deformation under the effects of external loads is presented. RESULTS Simulation results show that the proposed model provides realistic, stable and intuitive results in the simulation of fracture in fibrous soft tissues. As the external load increases, the fibrous soft tissue begins to crack, with the cracks growing and multiplying until they eventually merge to form a fracture. The proposed model is incorporated into the simulation of brain tumor removal. CONCLUSIONS The experimental results demonstrate the feasibility of modeling fibrous soft tissue dissection with elastic-plastic deformation. A relative high degree of realistic visual feedback is achieved.
Collapse
Affiliation(s)
- Zimeng Li
- School of Information Science and Engineering, Zhejiang Sci-Tech University, Hangzhou, Zhejiang, China
| | - Peter Xiaoping Liu
- School of Information Science and Engineering, Zhejiang Sci-Tech University, Hangzhou, Zhejiang, China; Department of Systems and Computer Engineering, Carleton University, Ottawa, ON KIS 5B6, Canada.
| | - Wenguo Hou
- School of Information Science and Engineering, Zhejiang Sci-Tech University, Hangzhou, Zhejiang, China.
| |
Collapse
|
5
|
Iop A, El-Hajj VG, Gharios M, de Giorgio A, Monetti FM, Edström E, Elmi-Terander A, Romero M. Extended Reality in Neurosurgical Education: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:6067. [PMID: 36015828 PMCID: PMC9414210 DOI: 10.3390/s22166067] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Revised: 08/06/2022] [Accepted: 08/12/2022] [Indexed: 06/15/2023]
Abstract
Surgical simulation practices have witnessed a rapid expansion as an invaluable approach to resident training in recent years. One emerging way of implementing simulation is the adoption of extended reality (XR) technologies, which enable trainees to hone their skills by allowing interaction with virtual 3D objects placed in either real-world imagery or virtual environments. The goal of the present systematic review is to survey and broach the topic of XR in neurosurgery, with a focus on education. Five databases were investigated, leading to the inclusion of 31 studies after a thorough reviewing process. Focusing on user performance (UP) and user experience (UX), the body of evidence provided by these 31 studies showed that this technology has, in fact, the potential of enhancing neurosurgical education through the use of a wide array of both objective and subjective metrics. Recent research on the topic has so far produced solid results, particularly showing improvements in young residents, compared to other groups and over time. In conclusion, this review not only aids to a better understanding of the use of XR in neurosurgical education, but also highlights the areas where further research is entailed while also providing valuable insight into future applications.
Collapse
Affiliation(s)
- Alessandro Iop
- Department of Neurosurgery, Karolinska University Hospital, 141 86 Stockholm, Sweden
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
- KTH Royal Institute of Technology, 114 28 Stockholm, Sweden
| | - Victor Gabriel El-Hajj
- Department of Neurosurgery, Karolinska University Hospital, 141 86 Stockholm, Sweden
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Maria Gharios
- Department of Neurosurgery, Karolinska University Hospital, 141 86 Stockholm, Sweden
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Andrea de Giorgio
- SnT—Interdisciplinary Center for Security, Reliability and Trust, University of Luxembourg, 4365 Esch-sur-Alzette, Luxembourg
| | | | - Erik Edström
- Department of Neurosurgery, Karolinska University Hospital, 141 86 Stockholm, Sweden
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Adrian Elmi-Terander
- Department of Neurosurgery, Karolinska University Hospital, 141 86 Stockholm, Sweden
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Mario Romero
- KTH Royal Institute of Technology, 114 28 Stockholm, Sweden
| |
Collapse
|
6
|
Cotter T, Mongrain R, Driscoll M. Vacuum curette lumbar discectomy mechanics for use in spine surgical training simulators. Sci Rep 2022; 12:13517. [PMID: 35933556 PMCID: PMC9357010 DOI: 10.1038/s41598-022-17512-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2022] [Accepted: 07/26/2022] [Indexed: 11/19/2022] Open
Abstract
Simulation in surgical training is a growing field and this study aims to understand the force and torque experienced during lumbar spine surgery to design simulator haptic feedback. It was hypothesized that force and torque would differ among lumbar spine levels and the amount of tissue removed by ≥ 7%, which would be detectable to a user. Force and torque profiles were measured during vacuum curette insertion and torsion, respectively, in multiple spinal levels on two cadavers. Multiple tests per level were performed. Linear and torsional resistances of 2.1 ± 1.6 N/mm and 5.6 ± 4.3 N mm/°, respectively, were quantified. Statistically significant differences were found in linear and torsional resistances between all passes through disc tissue (both p = 0.001). Tool depth (p < 0.001) and lumbar level (p < 0.001) impacted torsional resistance while tool speed affected linear resistance (p = 0.022). Average differences in these statistically significant comparisons were ≥ 7% and therefore detectable to a surgeon. The aforementioned factors should be considered when developing haptic force and torque feedback, as they will add to the simulated lumbar discectomy realism. These data can additionally be used inform next generation tool design. Advances in training and tools may help improve future surgeon training.
Collapse
Affiliation(s)
- Trevor Cotter
- Musculoskeletal Biomechanics Research Lab, Department of Mechanical Engineering, McGill University, Montreal, H3A 0C4, Canada.,Orthopedic Research Laboratory, Montreal General Hospital, Montreal, QC, H3H 1V8, Canada
| | - Rosaire Mongrain
- Musculoskeletal Biomechanics Research Lab, Department of Mechanical Engineering, McGill University, Montreal, H3A 0C4, Canada
| | - Mark Driscoll
- Musculoskeletal Biomechanics Research Lab, Department of Mechanical Engineering, McGill University, Montreal, H3A 0C4, Canada. .,Orthopedic Research Laboratory, Montreal General Hospital, Montreal, QC, H3H 1V8, Canada.
| |
Collapse
|
7
|
Reich A, Mirchi N, Yilmaz R, Ledwos N, Bissonnette V, Tran DH, Winkler-Schwartz A, Karlik B, Del Maestro RF. Artificial Neural Network Approach to Competency-Based Training Using a Virtual Reality Neurosurgical Simulation. Oper Neurosurg (Hagerstown) 2022; 23:31-39. [PMID: 35726927 DOI: 10.1227/ons.0000000000000173] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2021] [Accepted: 11/08/2021] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND The methodology of assessment and training of surgical skills is evolving to deal with the emergence of competency-based training. Artificial neural networks (ANNs), a branch of artificial intelligence, can use newly generated metrics not only for assessment performance but also to quantitate individual metric importance and provide new insights into surgical expertise. OBJECTIVE To outline the educational utility of using an ANN in the assessment and quantitation of surgical expertise. A virtual reality vertebral osteophyte removal during a simulated surgical spine procedure is used as a model to outline this methodology. METHODS Twenty-one participants performed a simulated anterior cervical diskectomy and fusion on the Sim-Ortho virtual reality simulator. Participants were divided into 3 groups, including 9 postresidents, 5 senior residents, and 7 junior residents. Data were retrieved from the osteophyte removal component of the scenario, which involved using a simulated burr. The data were manipulated to initially generate 83 performance metrics spanning 3 categories (safety, efficiency, and motion) of which only the most relevant metrics were used to train and test the ANN. RESULTS The ANN model was trained on 6 safety metrics to a testing accuracy of 83.3%. The contributions of these performance metrics to expertise were revealed through connection weight products and outlined 2 identifiable learning patterns of technical skills. CONCLUSION This study outlines the potential utility of ANNs which allows a deeper understanding of the composites of surgical expertise and may contribute to the paradigm shift toward competency-based surgical training.
Collapse
Affiliation(s)
- Aiden Reich
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | | | | | | | | | | | | | | | | |
Collapse
|
8
|
Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation. NPJ Digit Med 2022; 5:54. [PMID: 35473961 PMCID: PMC9042967 DOI: 10.1038/s41746-022-00596-8] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2021] [Accepted: 03/29/2022] [Indexed: 11/22/2022] Open
Abstract
In procedural-based medicine, the technical ability can be a critical determinant of patient outcomes. Psychomotor performance occurs in real-time, hence a continuous assessment is necessary to provide action-oriented feedback and error avoidance guidance. We outline a deep learning application, the Intelligent Continuous Expertise Monitoring System (ICEMS), to assess surgical bimanual performance at 0.2-s intervals. A long-short term memory network was built using neurosurgeon and student performance in 156 virtually simulated tumor resection tasks. Algorithm predictive ability was tested separately on 144 procedures by scoring the performance of neurosurgical trainees who are at different training stages. The ICEMS successfully differentiated between neurosurgeons, senior trainees, junior trainees, and students. Trainee average performance score correlated with the year of training in neurosurgery. Furthermore, coaching and risk assessment for critical metrics were demonstrated. This work presents a comprehensive technical skill monitoring system with predictive validation throughout surgical residency training, with the ability to detect errors.
Collapse
|
9
|
Alkadri S, Ledwos N, Mirchi N, Reich A, Yilmaz R, Driscoll M, Del Maestro RF. Utilizing a multilayer perceptron artificial neural network to assess a virtual reality surgical procedure. Comput Biol Med 2021; 136:104770. [PMID: 34426170 DOI: 10.1016/j.compbiomed.2021.104770] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Revised: 08/12/2021] [Accepted: 08/13/2021] [Indexed: 11/18/2022]
Abstract
BACKGROUND Virtual reality surgical simulators are a safe and efficient technology for the assessment and training of surgical skills. Simulators allow trainees to improve specific surgical techniques in risk-free environments. Recently, machine learning has been coupled to simulators to classify performance. However, most studies fail to extract meaningful observations behind the classifications and the impact of specific surgical metrics on the performance. One benefit from integrating machine learning algorithms, such as Artificial Neural Networks, to simulators is the ability to extract novel insights into the composites of the surgical performance that differentiate levels of expertise. OBJECTIVE This study aims to demonstrate the benefits of artificial neural network algorithms in assessing and analyzing virtual surgical performances. This study applies the algorithm on a virtual reality simulated annulus incision task during an anterior cervical discectomy and fusion scenario. DESIGN An artificial neural network algorithm was developed and integrated. Participants performed the simulated surgical procedure on the Sim-Ortho simulator. Data extracted from the annulus incision task were extracted to generate 157 surgical performance metrics that spanned three categories (motion, safety, and efficiency). SETTING Musculoskeletal Biomechanics Research Lab; Neurosurgical Simulation and Artificial Intelligence Learning Center, McGill University, Montreal, Canada. PARTICIPANTS Twenty-three participants were recruited and divided into 3 groups: 11 post-residents, 5 senior and 7 junior residents. RESULTS An artificial neural network model was trained on nine selected surgical metrics, spanning all three categories and achieved 80% testing accuracy. CONCLUSIONS This study outlines the benefits of integrating artificial neural networks to virtual reality surgical simulators in understanding composites of expertise performance.
Collapse
Affiliation(s)
- Sami Alkadri
- Musculoskeletal Biomechanics Research Lab, Department of Mechanical Engineering, McGill University, Macdonald Engineering Building, 815 Sherbrooke St W, Montreal, H3A 2K7, QC, Canada
| | - Nicole Ledwos
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, 3801 University Street, Room E2.89, H3A 2B4, Montreal, Quebec, Canada
| | - Nykan Mirchi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, 3801 University Street, Room E2.89, H3A 2B4, Montreal, Quebec, Canada
| | - Aiden Reich
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, 3801 University Street, Room E2.89, H3A 2B4, Montreal, Quebec, Canada
| | - Recai Yilmaz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, 3801 University Street, Room E2.89, H3A 2B4, Montreal, Quebec, Canada
| | - Mark Driscoll
- Musculoskeletal Biomechanics Research Lab, Department of Mechanical Engineering, McGill University, Macdonald Engineering Building, 815 Sherbrooke St W, Montreal, H3A 2K7, QC, Canada.
| | - Rolando F Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, 3801 University Street, Room E2.89, H3A 2B4, Montreal, Quebec, Canada
| |
Collapse
|
10
|
Ledwos N, Mirchi N, Bissonnette V, Winkler-Schwartz A, Yilmaz R, Del Maestro RF. Virtual Reality Anterior Cervical Discectomy and Fusion Simulation on the Novel Sim-Ortho Platform: Validation Studies. Oper Neurosurg (Hagerstown) 2020; 20:74-82. [DOI: 10.1093/ons/opaa269] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2020] [Accepted: 06/28/2020] [Indexed: 11/13/2022] Open
Abstract
ABSTRACT
BACKGROUND
Virtual reality spine simulators are emerging as potential educational tools to assess and train surgical procedures in safe environments. Analysis of validity is important in determining the educational utility of these systems.
OBJECTIVE
To assess face, content, and construct validity of a C4-C5 anterior cervical discectomy and fusion simulation on the Sim-Ortho virtual reality platform, developed by OSSimTechTM (Montreal, Canada) and the AO Foundation (Davos, Switzerland).
METHODS
Spine surgeons, spine fellows, along with neurosurgical and orthopedic residents, performed a simulated C4-C5 anterior cervical discectomy and fusion on the Sim-Ortho system. Participants were separated into 3 categories: post-residents (spine surgeons and spine fellows), senior residents, and junior residents. A Likert scale was used to assess face and content validity. Construct validity was evaluated by investigating differences between the 3 groups on metrics derived from simulator data. The Kruskal-Wallis test was employed to compare groups and a post-hoc Dunn's test with a Bonferroni correction was utilized to investigate differences between groups on significant metrics.
RESULTS
A total of 21 individuals were included: 9 post-residents, 5 senior residents, and 7 junior residents. The post-resident group rated face and content validity, median ≥4, for the overall procedure and at least 1 tool in each of the 4 steps. Significant differences (P < .05) were found between the post-resident group and senior and/or junior residents on at least 1 metric for each component of the simulation.
CONCLUSION
The C4-C5 anterior cervical discectomy and fusion simulation on the Sim-Ortho platform demonstrated face, content, and construct validity suggesting its utility as a formative educational tool.
Collapse
Affiliation(s)
- Nicole Ledwos
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| | - Nykan Mirchi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| | - Vincent Bissonnette
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
- Division of Orthopaedic Surgery, Montreal General Hospital, McGill University, Montreal, Canada
| | - Alexander Winkler-Schwartz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| | - Recai Yilmaz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| | - Rolando F Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| |
Collapse
|
11
|
Hou W, Liu PX, Zheng M. Modeling of connective tissue damage for blunt dissection of brain tumor in neurosurgery simulation. Comput Biol Med 2020; 120:103696. [PMID: 32421640 DOI: 10.1016/j.compbiomed.2020.103696] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2019] [Revised: 03/03/2020] [Accepted: 03/03/2020] [Indexed: 10/24/2022]
Abstract
We introduce a new model for connective tissue damage in blunt dissection, which is a very important process in neurosurgery simulation. Specifically, the tool-tissue interaction between the instrument and connective tissue is incorporated into the model of connective tissue damage. This damage develops with the evolution criterion due to the effect of the external load. The tetrahedral mesh in the soft tissue model is removed for the representation of rupture as the damage accumulates to the threshold value. Analysis and experiments show that the connective tissue damage model provides stable, visually realistic results for the simulation of the connective tissue rupture process. The stiffness of the connective tissue decreases as the damage accumulates. The proposed model for connective tissue damage was incorporated into the development of a neurosurgery simulator, in which blunt dissection of a brain tumor was simulated.
Collapse
Affiliation(s)
- Wenguo Hou
- School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing 100044, PR China.
| | - Peter X Liu
- Department of Systems and Computer Engineering, Carleton University, Ottawa, ON, Canada.
| | - Minhua Zheng
- School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing 100044, PR China.
| |
Collapse
|
12
|
Meling TR, Meling TR. The impact of surgical simulation on patient outcomes: a systematic review and meta-analysis. Neurosurg Rev 2020; 44:843-854. [PMID: 32399730 PMCID: PMC8035110 DOI: 10.1007/s10143-020-01314-2] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2020] [Revised: 04/16/2020] [Accepted: 04/29/2020] [Indexed: 12/15/2022]
Abstract
The use of simulation in surgical training is ever growing. Evidence suggests such training may have beneficial clinically relevant effects. The objective of this research is to investigate the effects of surgical simulation training on clinically relevant patient outcomes by evaluating randomized controlled trials (RCT). PubMed was searched using PRISMA guidelines: "surgery" [All Fields] AND "simulation" [All Fields] AND "patient outcome" [All Fields]. Of 119 papers identified, 100 were excluded for various reasons. Meta-analyses were conducted using the inverse-variance random-effects method. Nineteen papers were reviewed using the CASP RCT Checklist. Sixteen studies looked at surgical training, two studies assessed patient-specific simulator practice, and one paper focused on warming-up on a simulator before performing surgery. Median study population size was 22 (range 3-73). Most articles reported outcome measures such as post-intervention Global Rating Scale (GRS) score and/or operative time. On average, the intervention group scored 0.42 (95% confidence interval 0.12 to 0.71, P = 0.005) points higher on a standardized GRS scale of 1-10. On average, the intervention group was 44% (1% to 87%, P = 0.04) faster than the control group. Four papers assessed the impact of simulation training on patient outcomes, with only one finding a significant effect. We found a significant effect of simulation training on operative performance as assessed by GRS, albeit a small one, as well as a significant reduction to operative time. However, there is to date scant evidence from RCTs to suggest a significant effect of surgical simulation training on patient outcomes.
Collapse
Affiliation(s)
- Trym R Meling
- Faculty of Medicine, University of Oslo, Oslo, Norway
| | - Torstein R Meling
- Faculty of Medicine, University of Oslo, Oslo, Norway. .,Department of Clinical Neurosciences, Division of Neurosurgery, Geneva University Hospitals, Rue Gabriel-Perret-Gentil 5, 1205, Geneva, Switzerland. .,Faculty of Medicine, University of Geneva, Geneva, Switzerland.
| |
Collapse
|
13
|
Siyar S, Azarnoush H, Rashidi S, Del Maestro RF. Tremor Assessment during Virtual Reality Brain Tumor Resection. JOURNAL OF SURGICAL EDUCATION 2020; 77:643-651. [PMID: 31822389 DOI: 10.1016/j.jsurg.2019.11.011] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/06/2019] [Revised: 11/25/2019] [Accepted: 11/26/2019] [Indexed: 06/10/2023]
Abstract
OBJECTIVE Assessment of physiological tremor during neurosurgical procedures may provide further insights into the composites of surgical expertise. Virtual reality platforms may provide a mechanism for the quantitative assessment of physiological tremor. In this study, a virtual reality simulator providing haptic feedback was used to study physiological tremor in a simulated tumor resection task with participants from a "skilled" group and a "novice" group. DESIGN The task involved using a virtual ultrasonic aspirator to remove a series of virtual brain tumors with different visual and tactile characteristics without causing injury to surrounding tissue. Power spectral density analysis was employed to quantitate hand tremor during tumor resection. Statistical t test was used to determine tremor differences between the skilled and novice groups obtained from the instrument tip x, y, z coordinates, the instrument roll, pitch, yaw angles, and the instrument haptic force applied during tumor resection. SETTING The study was conducted at the Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada. PARTICIPANTS The skilled group comprised 23 neurosurgeons and senior residents and the novice group comprised 92 junior residents and medical students. RESULTS The spectral analysis allowed quantitation of physiological tremor during virtual reality tumor resection. The skilled group displayed smaller physiological tremor than the novice group in all cases. In 3 out of 7 cases the difference was statistically significant. CONCLUSIONS The first investigation of the application of a virtual reality platform is presented for the quantitation of physiological tremor during a virtual reality tumor resection task. The goal of introducing such methodology to assess tremor is to highlight its potential educational application in neurosurgical resident training and in helping to further define the psychomotor skill set of surgeons.
Collapse
Affiliation(s)
- Samaneh Siyar
- Department of Biomedical Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran; Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| | - Hamed Azarnoush
- Department of Biomedical Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran; Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada.
| | - Saeid Rashidi
- Science and Research Branch, Islamic Azad University, Tehran, Iran
| | - Rolando F Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| |
Collapse
|
14
|
Siyar S, Azarnoush H, Rashidi S, Winkler-Schwartz A, Bissonnette V, Ponnudurai N, Del Maestro RF. Machine learning distinguishes neurosurgical skill levels in a virtual reality tumor resection task. Med Biol Eng Comput 2020; 58:1357-1367. [DOI: 10.1007/s11517-020-02155-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2019] [Accepted: 03/12/2020] [Indexed: 10/24/2022]
|
15
|
Bissonnette V, Mirchi N, Ledwos N, Alsidieri G, Winkler-Schwartz A, Del Maestro RF. Artificial Intelligence Distinguishes Surgical Training Levels in a Virtual Reality Spinal Task. J Bone Joint Surg Am 2019; 101:e127. [PMID: 31800431 PMCID: PMC7406145 DOI: 10.2106/jbjs.18.01197] [Citation(s) in RCA: 51] [Impact Index Per Article: 10.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
BACKGROUND With the emergence of competency-based training, the current evaluation scheme of surgical skills is evolving to include newer methods of assessment and training. Artificial intelligence through machine learning algorithms can utilize extensive data sets to analyze operator performance. This study aimed to address 3 questions: (1) Can artificial intelligence uncover novel metrics of surgical performance? (2) Can support vector machine algorithms be trained to differentiate "senior" and "junior" participants who are executing a virtual reality hemilaminectomy? (3) Can other algorithms achieve a good classification performance? METHODS Participants from 4 Canadian universities were divided into 2 groups according to their training level (senior and junior) and were asked to perform a virtual reality hemilaminectomy. The position, angle, and force application of the simulated burr and suction instruments, along with tissue volumes that were removed, were recorded at 20-ms intervals. Raw data were manipulated to create metrics to train machine learning algorithms. Five algorithms, including a support vector machine, were trained to predict whether the task was performed by a senior or junior participant. The accuracy of each algorithm was assessed through leave-one-out cross-validation. RESULTS Forty-one individuals were enrolled (22 senior and 19 junior participants). Twelve metrics related to safety of the procedure, efficiency, motion of the tools, and coordination were selected. Following cross-validation, the support vector machine achieved a 97.6% accuracy. The other algorithms achieved accuracy of 92.7%, 87.8%, 70.7%, and 65.9%, respectively. CONCLUSIONS Artificial intelligence defined novel metrics of surgical performance and outlined training levels in a virtual reality spinal simulation procedure. CLINICAL RELEVANCE The significance of these results lies in the potential of artificial intelligence to complement current educational paradigms and better prepare residents for surgical procedures.
Collapse
Affiliation(s)
- Vincent Bissonnette
- Neurosurgical Simulation & Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
- Division of Orthopaedic Surgery, Montreal General Hospital, McGill University, Montreal, Quebec, Canada
| | - Nykan Mirchi
- Neurosurgical Simulation & Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Nicole Ledwos
- Neurosurgical Simulation & Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Ghusn Alsidieri
- Neurosurgical Simulation & Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Alexander Winkler-Schwartz
- Neurosurgical Simulation & Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Rolando F Del Maestro
- Neurosurgical Simulation & Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
16
|
Hou W, Liu PX, Zheng M. A new model of soft tissue with constraints for interactive surgical simulation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2019; 175:35-43. [PMID: 31104713 DOI: 10.1016/j.cmpb.2019.03.018] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/28/2019] [Revised: 03/23/2019] [Accepted: 03/27/2019] [Indexed: 06/09/2023]
Abstract
BACKGROUND AND OBJECTIVES An accurate and real-time model of soft tissue is critical for surgical simulation for which a user interacts haptically and visually with simulated patients. This paper focuses on the real-time deformation model of brain tissue for the interactive surgical simulation, such as neurosurgical simulation. METHODS A new Finite Element Method (FEM) based model with constraints is proposed for the brain tissue in neurosurgical simulation. A new energy function of constraints characterizing the interaction between the virtual instrument and the soft tissue is incorporated into the optimization problem derived from the implicit integration scheme. Distance and permanent deformation constraints are introduced to describe the interaction in the convexity meningioma dissection and hemostasis. The proposed model is particularly suitable for GPU-based computing, making it possible to achieve real-time performance. RESULTS AND CONCLUSIONS Simulation results show that the simulated soft tissue exhibits the behaviors of adhesion and permanent deformation under the constraints. Experiments show that the proposed model is able to converge to the exact solution of the implicit Euler method after 96 iterations. The proposed model was implemented in the development of a neurosurgical simulator, in which surgical procedures such as dissection of convexity meningioma and hemostasis were simulated.
Collapse
Affiliation(s)
- Wenguo Hou
- School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing 100044, PR China
| | - Peter X Liu
- School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing 100044, PR China; Department of Systems and Computer Engineering, Carleton University, Ottawa, ON, Canada.
| | - Minhua Zheng
- School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing 100044, PR China.
| |
Collapse
|
17
|
Winkler-Schwartz A, Marwa I, Bajunaid K, Mullah M, Alotaibi FE, Bugdadi A, Sawaya R, Sabbagh AJ, Del Maestro R. A Comparison of Visual Rating Scales and Simulated Virtual Reality Metrics in Neurosurgical Training: A Generalizability Theory Study. World Neurosurg 2019; 127:e230-e235. [PMID: 30880209 DOI: 10.1016/j.wneu.2019.03.059] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2018] [Revised: 03/05/2019] [Accepted: 03/06/2019] [Indexed: 11/27/2022]
Abstract
BACKGROUND Adequate assessment and feedback remains a cornerstone of psychomotor skills acquisition, particularly within neurosurgery where the consequence of adverse operative events is significant. However, a critical appraisal of the reliability of visual rating scales in neurosurgery is lacking. Therefore, we sought to design a study to compare visual rating scales with simulated metrics in a neurosurgical virtual reality task. METHODS Neurosurgical faculty rated anonymized participant video recordings of the removal of simulated brain tumors using a visual rating scale made up of seven composite elements. Scale reliability was evaluated using generalizability theory, and scale subcomponents were compared with simulated metrics using Pearson correlation analysis. RESULTS Four staff neurosurgeons evaluated 16 medical student neurosurgery applicants. Overall scale reliability and internal consistency were 0.73 and 0.90, respectively. Reliability of 0.71 was achieved with two raters. Individual participants, raters, and scale items accounted for 27%, 11%, and 0.6% of the data variability. The hemostasis scale component related to the greatest number of simulated metrics, whereas respect for no-go zones and tissue was correlated with none. Metrics relating to instrument force and patient safety (brain volume removed and blood loss) were captured by the fewest number of rating scale components. CONCLUSIONS To our knowledge, this is the first study comparing participant's ratings with simulated performance. Given rating scales capture less well instrument force, quantity of brain volume removed, and blood loss, we suggest adopting a hybrid educational approach using visual rating scales in an operative environment, supplemented by simulated sessions to uncover potentially problematic surgical technique.
Collapse
Affiliation(s)
- Alexander Winkler-Schwartz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada.
| | - Ibrahim Marwa
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
| | - Khalid Bajunaid
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada; Division of Neurosurgery, Department of Surgery, Faculty of Medicine, University of Jeddah, Jeddah, Saudi Arabia
| | - Muhammad Mullah
- Department of Epidemiology, Biostatistics & Occupational Health, McGill University, Montreal, Quebec, Canada
| | - Fahad E Alotaibi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada; Neurosurgical Department, National Neuroscience Institute, King Fahad Medical City, Riyadh, Saudi Arabia
| | - Abdulgadir Bugdadi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada; Department of Surgery, Faculty of Medicine, Umm Al Qura University, Makkah, Saudi Arabia
| | - Robin Sawaya
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
| | - Abdulrahman J Sabbagh
- Division of Neurosurgery, Department of Surgery, Faculty of Medicine, University of Jeddah, Jeddah, Saudi Arabia; Clinical Skills and Simulation Center, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Rolando Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
18
|
Zhang JF, Paciorkowski AR, Craig PA, Cui F. BioVR: a platform for virtual reality assisted biological data integration and visualization. BMC Bioinformatics 2019; 20:78. [PMID: 30767777 PMCID: PMC6376704 DOI: 10.1186/s12859-019-2666-z] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2018] [Accepted: 01/31/2019] [Indexed: 12/01/2022] Open
Abstract
Background Functional characterization of single nucleotide variants (SNVs) involves two steps, the first step is to convert DNA to protein and the second step is to visualize protein sequences with their structures. As massively parallel sequencing has emerged as a leading technology in genomics, resulting in a significant increase in data volume, direct visualization of SNVs together with associated protein sequences/structures in a new user interface (UI) would be a more effective way to assess their potential effects on protein function. Results We have developed BioVR, an easy-to-use interactive, virtual reality (VR)-assisted platform for integrated visual analysis of DNA/RNA/protein sequences and protein structures using Unity3D and the C# programming language. It utilizes the cutting-edge Oculus Rift, and Leap Motion hand detection, resulting in intuitive navigation and exploration of various types of biological data. Using Gria2 and its associated gene product as an example, we present this proof-of-concept software to integrate protein and nucleic acid data. For any amino acid or nucleotide of interest in the Gria2 sequence, it can be quickly linked to its corresponding location on Gria2 protein structure and visualized within VR. Conclusions Using innovative 3D techniques, we provide a VR-based platform for visualization of DNA/RNA sequences and protein structures in aggregate, which can be extended to view omics data. Electronic supplementary material The online version of this article (10.1186/s12859-019-2666-z) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Jimmy F Zhang
- Thomas H. Gosnell School of Life Sciences, Rochester Institute of Technology, One Lomb Memorial Drive, Rochester, NY, 14623, USA.
| | - Alex R Paciorkowski
- Departments of Neurology, Pediatrics, Biomedical Genetics, and Neuroscience, University of Rochester Medical Center, 601 Elmwood Ave, Rochester, NY, 14642, USA
| | - Paul A Craig
- School of Chemistry and Materials Science, Rochester Institute of Technology, One Lomb Memorial Drive, Rochester, NY, 14623, USA
| | - Feng Cui
- Thomas H. Gosnell School of Life Sciences, Rochester Institute of Technology, One Lomb Memorial Drive, Rochester, NY, 14623, USA.
| |
Collapse
|
19
|
Yao LP, Zhang L, Mei J, Ding FB, Li HM, Ding M, Yang X, Li XM, Sun K. A pilot study of a cardiovascular virtual endoscopy system based on multi-detector computed tomography in diagnosing tetralogy of Fallot in pediatric patients. Exp Ther Med 2018; 15:1552-1559. [PMID: 29434740 PMCID: PMC5776613 DOI: 10.3892/etm.2017.5572] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2016] [Accepted: 04/28/2017] [Indexed: 11/05/2022] Open
Abstract
The present study aimed to investigate the capabilities of the cardiovascular virtual endoscopy (VE) system in diagnosing tetralogy of Fallot (TOF) and performing measurements. A total of 37 patients underwent two-dimensional echocardiography (2-DE) and multi-detector computed tomography (MDCT) examinations. The obtained MDCT images were applied to a cardiovascular VE system. Diagnostic time by VE was first studied and compared with MDCT. Subsequently, with surgical findings as the ground truth, the capabilities of VE, 2-DE and MDCT in diagnosing TOF and its complications were investigated. Additionally, measurements on aorta overriding ratio and diameters for the left pulmonary artery, right pulmonary artery and right ventricular outflow tract by 2-DE and VE were analyzed. Diagnostic time by VE was significantly shorter than MDCT (188±42 vs. 303±42 sec, respectively; P<0.0001). VE, MDCT and 2-DE demonstrated comparable diagnostic rates of TOF (35/37 vs. 34/37 vs. 32/37, respectively; P>0.05). Similar findings were demonstrated in diagnosing complications of the muscular ventricular septal defects, patent ductus arteriosus, vagus subclavian artery, right arch, double superior vena cava and pulmonary artery. Furthermore, in diagnosing the atrial septal defect, 2-DE outperformed MDCT and VE (accuracy, 100 vs. 81 vs. 73%, respectively; all P<0.05). In performing relevant measurements, VE outperformed MDCT and 2-DE, particularly in accessing aorta overriding ratios with no intra-operator difference (P=0.3770) and high consistency (r=0.916). In conclusion, cardiovascular VE was demonstrated to have acceptable accuracy in diagnosing TOF, and possess advantages in shortening the diagnostic time and in performing measurements.
Collapse
Affiliation(s)
- Li-Ping Yao
- Department of Biomedical Engineering, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, P.R. China
- Department of Cardiothoracic Surgery, Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine, Shanghai 200092, P.R. China
| | - Li Zhang
- Department of Cardiothoracic Surgery, Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine, Shanghai 200092, P.R. China
| | - Ju Mei
- Department of Cardiothoracic Surgery, Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine, Shanghai 200092, P.R. China
| | - Fang-Bao Ding
- Department of Cardiothoracic Surgery, Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine, Shanghai 200092, P.R. China
| | - Hui-Ming Li
- Department of Radiology, Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine, Shanghai 200092, P.R. China
| | - Ming Ding
- Department of Radiology, Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine, Shanghai 200092, P.R. China
| | - Xin Yang
- Department of Automation, Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai 200240, P.R. China
| | - Xiao-Ming Li
- Healthcare Department, Philips Research China, Shanghai 200233, P.R. China
| | - Kun Sun
- Department of Biomedical Engineering, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, P.R. China
- Department of Pediatric Cardiology, Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine, Shanghai 200092, P.R. China
| |
Collapse
|
20
|
Competency Assessment in Virtual Reality-Based Simulation in Neurosurgical Training. COMPREHENSIVE HEALTHCARE SIMULATION: NEUROSURGERY 2018. [DOI: 10.1007/978-3-319-75583-0_12] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
21
|
|
22
|
Lemos JD, Hernandez AM, Soto-Romero G. An Instrumented Glove to Assess Manual Dexterity in Simulation-Based Neurosurgical Education. SENSORS 2017; 17:s17050988. [PMID: 28468268 PMCID: PMC5469341 DOI: 10.3390/s17050988] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/31/2016] [Revised: 04/01/2017] [Accepted: 04/07/2017] [Indexed: 12/03/2022]
Abstract
The traditional neurosurgical apprenticeship scheme includes the assessment of trainee’s manual skills carried out by experienced surgeons. However, the introduction of surgical simulation technology presents a new paradigm where residents can refine surgical techniques on a simulator before putting them into practice in real patients. Unfortunately, in this new scheme, an experienced surgeon will not always be available to evaluate trainee’s performance. For this reason, it is necessary to develop automatic mechanisms to estimate metrics for assessing manual dexterity in a quantitative way. Authors have proposed some hardware-software approaches to evaluate manual dexterity on surgical simulators. This paper presents IGlove, a wearable device that uses inertial sensors embedded on an elastic glove to capture hand movements. Metrics to assess manual dexterity are estimated from sensors signals using data processing and information analysis algorithms. It has been designed to be used with a neurosurgical simulator called Daubara NS Trainer, but can be easily adapted to another benchtop- and manikin-based medical simulators. The system was tested with a sample of 14 volunteers who performed a test that was designed to simultaneously evaluate their fine motor skills and the IGlove’s functionalities. Metrics obtained by each of the participants are presented as results in this work; it is also shown how these metrics are used to automatically evaluate the level of manual dexterity of each volunteer.
Collapse
Affiliation(s)
- Juan Diego Lemos
- Bioinstrumentation and Clinical Engineering Research Group-GIBIC, Bioengineering Department, Engineering Faculty, Universidad de Antioquia UdeA, Calle 70 No. 52-21, Medellín 050010, Colombia.
| | - Alher Mauricio Hernandez
- Bioinstrumentation and Clinical Engineering Research Group-GIBIC, Bioengineering Department, Engineering Faculty, Universidad de Antioquia UdeA, Calle 70 No. 52-21, Medellín 050010, Colombia.
| | - Georges Soto-Romero
- LAAS-CNRS, Université de Toulouse, CNRS, Toulouse 31400, France.
- ISIFC, Université de Franche-Comté, Besançon 25000, France.
| |
Collapse
|
23
|
AlZhrani G, Alotaibi F, Azarnoush H, Winkler-Schwartz A, Sabbagh A, Bajunaid K, Lajoie SP, Del Maestro RF. Proficiency performance benchmarks for removal of simulated brain tumors using a virtual reality simulator NeuroTouch. JOURNAL OF SURGICAL EDUCATION 2015; 72:685-696. [PMID: 25687956 DOI: 10.1016/j.jsurg.2014.12.014] [Citation(s) in RCA: 43] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/29/2014] [Revised: 12/10/2014] [Accepted: 12/31/2014] [Indexed: 06/04/2023]
Abstract
OBJECTIVE Assessment of neurosurgical technical skills involved in the resection of cerebral tumors in operative environments is complex. Educators emphasize the need to develop and use objective and meaningful assessment tools that are reliable and valid for assessing trainees' progress in acquiring surgical skills. The purpose of this study was to develop proficiency performance benchmarks for a newly proposed set of objective measures (metrics) of neurosurgical technical skills performance during simulated brain tumor resection using a new virtual reality simulator (NeuroTouch). DESIGN Each participant performed the resection of 18 simulated brain tumors of different complexity using the NeuroTouch platform. Surgical performance was computed using Tier 1 and Tier 2 metrics derived from NeuroTouch simulator data consisting of (1) safety metrics, including (a) volume of surrounding simulated normal brain tissue removed, (b) sum of forces utilized, and (c) maximum force applied during tumor resection; (2) quality of operation metric, which involved the percentage of tumor removed; and (3) efficiency metrics, including (a) instrument total tip path lengths and (b) frequency of pedal activation. SETTING All studies were conducted in the Neurosurgical Simulation Research Centre, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada. PARTICIPANTS A total of 33 participants were recruited, including 17 experts (board-certified neurosurgeons) and 16 novices (7 senior and 9 junior neurosurgery residents). RESULTS The results demonstrated that "expert" neurosurgeons resected less surrounding simulated normal brain tissue and less tumor tissue than residents. These data are consistent with the concept that "experts" focused more on safety of the surgical procedure compared with novices. By analyzing experts' neurosurgical technical skills performance on these different metrics, we were able to establish benchmarks for goal proficiency performance training of neurosurgery residents. CONCLUSION This study furthers our understanding of expert neurosurgical performance during the resection of simulated virtual reality tumors and provides neurosurgical trainees with predefined proficiency performance benchmarks designed to maximize the learning of specific surgical technical skills.
Collapse
Affiliation(s)
- Gmaan AlZhrani
- Neurosurgical Simulation Research Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada; National Neuroscience Institute, King Fahad Medical City, Riyadh, Saudi Arabia.
| | - Fahad Alotaibi
- Neurosurgical Simulation Research Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada; National Neuroscience Institute, King Fahad Medical City, Riyadh, Saudi Arabia
| | - Hamed Azarnoush
- Neurosurgical Simulation Research Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada; Department of Biomedical Engineering, Tehran Polytechnic, Tehran, Iran
| | - Alexander Winkler-Schwartz
- Neurosurgical Simulation Research Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| | - Abdulrahman Sabbagh
- Neurosurgical Simulation Research Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada; National Neuroscience Institute, King Fahad Medical City, Riyadh, Saudi Arabia
| | - Khalid Bajunaid
- Neurosurgical Simulation Research Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada; Division of Neurosurgery, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Susanne P Lajoie
- Department of Educational and Counseling Psychology, McGill University, Montreal, Canada
| | - Rolando F Del Maestro
- Neurosurgical Simulation Research Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| |
Collapse
|