1
|
Bogar PZ, Virag M, Bene M, Hardi P, Matuz A, Schlegl AT, Toth L, Molnar F, Nagy B, Rendeki S, Berner-Juhos K, Ferencz A, Fischer K, Maroti P. Validation of a novel, low-fidelity virtual reality simulator and an artificial intelligence assessment approach for peg transfer laparoscopic training. Sci Rep 2024; 14:16702. [PMID: 39030307 PMCID: PMC11271545 DOI: 10.1038/s41598-024-67435-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Accepted: 07/11/2024] [Indexed: 07/21/2024] Open
Abstract
Simulators are widely used in medical education, but objective and automatic assessment is not feasible with low-fidelity simulators, which can be solved with artificial intelligence (AI) and virtual reality (VR) solutions. The effectiveness of a custom-made VR simulator and an AI-based evaluator of a laparoscopic peg transfer exercise was investigated. Sixty medical students were involved in a single-blinded randomised controlled study to compare the VR simulator with the traditional box trainer. A total of 240 peg transfer exercises from the Fundamentals of Laparoscopic Surgery programme were analysed. The experts and AI-based software used the same criteria for evaluation. The algorithm detected pitfalls and measured exercise duration. Skill improvement showed no significant difference between the VR and control groups. The AI-based evaluator exhibited 95% agreement with the manual assessment. The average difference between the exercise durations measured by the two evaluation methods was 2.61 s. The duration of the algorithmic assessment was 59.47 s faster than the manual assessment. The VR simulator was an effective alternative practice compared with the training box simulator. The AI-based evaluation produced similar results compared with the manual assessment, and it could significantly reduce the evaluation time. AI and VR could improve the effectiveness of basic laparoscopic training.
Collapse
Affiliation(s)
- Peter Zoltan Bogar
- 3D Printing and Visualisation Centre, University of Pecs, Medical School, Boszorkany Str. 2, Pecs, 7624, Hungary
| | - Mark Virag
- 3D Printing and Visualisation Centre, University of Pecs, Medical School, Boszorkany Str. 2, Pecs, 7624, Hungary
- Department of Public Health Medicine, University of Pecs, Szigeti Str. 12, Pecs, 7624, Hungary
| | - Matyas Bene
- 3D Printing and Visualisation Centre, University of Pecs, Medical School, Boszorkany Str. 2, Pecs, 7624, Hungary
| | - Peter Hardi
- Medical Skills Education and Innovation Centre, Medical School, University of Pecs, Szigeti Str. 12, Pecs, 7624, Hungary
- Department of Surgery and Vascular Surgery, Tolna County Janos Balassa Hospital, Beri Balogh Adam str. 5-7, Szekszard, 7100, Hungary
| | - Andras Matuz
- Department of Behavioural Sciences, Medical School, University of Pecs, Szigeti Str. 12, Pecs, 7624, Hungary
- Szentágothai Research Centre, University of Pecs, Pecs, Ifjusag str. 20., 7624, Hungary
| | - Adam Tibor Schlegl
- Medical Skills Education and Innovation Centre, Medical School, University of Pecs, Szigeti Str. 12, Pecs, 7624, Hungary
- Department of Orthopaedics, Medical School, University of Pecs, Akac Str. 1, Pecs, 7632, Hungary
| | - Luca Toth
- 3D Printing and Visualisation Centre, University of Pecs, Medical School, Boszorkany Str. 2, Pecs, 7624, Hungary.
- Department of Neurosurgery, Medical School, University of Pecs, 2 Ret Street, Pecs, 7624, Hungary.
| | - Ferenc Molnar
- Medical Skills Education and Innovation Centre, Medical School, University of Pecs, Szigeti Str. 12, Pecs, 7624, Hungary
| | - Balint Nagy
- Medical Skills Education and Innovation Centre, Medical School, University of Pecs, Szigeti Str. 12, Pecs, 7624, Hungary
| | - Szilard Rendeki
- Medical Skills Education and Innovation Centre, Medical School, University of Pecs, Szigeti Str. 12, Pecs, 7624, Hungary
| | - Krisztina Berner-Juhos
- Department of Surgical Research and Techniques, Heart and Vascular Centre, Semmelweis University, Nagyvarad Square 4, Budapest, 1089, Hungary
| | - Andrea Ferencz
- Department of Surgical Research and Techniques, Heart and Vascular Centre, Semmelweis University, Nagyvarad Square 4, Budapest, 1089, Hungary
| | - Krisztina Fischer
- Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, 75 Francis St, Boston, MA, 02115, USA
| | - Peter Maroti
- 3D Printing and Visualisation Centre, University of Pecs, Medical School, Boszorkany Str. 2, Pecs, 7624, Hungary.
- Medical Skills Education and Innovation Centre, Medical School, University of Pecs, Szigeti Str. 12, Pecs, 7624, Hungary.
| |
Collapse
|
2
|
Boal MWE, Anastasiou D, Tesfai F, Ghamrawi W, Mazomenos E, Curtis N, Collins JW, Sridhar A, Kelly J, Stoyanov D, Francis NK. Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review. Br J Surg 2024; 111:znad331. [PMID: 37951600 PMCID: PMC10771126 DOI: 10.1093/bjs/znad331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 09/18/2023] [Accepted: 09/19/2023] [Indexed: 11/14/2023]
Abstract
BACKGROUND There is a need to standardize training in robotic surgery, including objective assessment for accreditation. This systematic review aimed to identify objective tools for technical skills assessment, providing evaluation statuses to guide research and inform implementation into training curricula. METHODS A systematic literature search was conducted in accordance with the PRISMA guidelines. Ovid Embase/Medline, PubMed and Web of Science were searched. Inclusion criterion: robotic surgery technical skills tools. Exclusion criteria: non-technical, laparoscopy or open skills only. Manual tools and automated performance metrics (APMs) were analysed using Messick's concept of validity and the Oxford Centre of Evidence-Based Medicine (OCEBM) Levels of Evidence and Recommendation (LoR). A bespoke tool analysed artificial intelligence (AI) studies. The Modified Downs-Black checklist was used to assess risk of bias. RESULTS Two hundred and forty-seven studies were analysed, identifying: 8 global rating scales, 26 procedure-/task-specific tools, 3 main error-based methods, 10 simulators, 28 studies analysing APMs and 53 AI studies. Global Evaluative Assessment of Robotic Skills and the da Vinci Skills Simulator were the most evaluated tools at LoR 1 (OCEBM). Three procedure-specific tools, 3 error-based methods and 1 non-simulator APMs reached LoR 2. AI models estimated outcomes (skill or clinical), demonstrating superior accuracy rates in the laboratory with 60 per cent of methods reporting accuracies over 90 per cent, compared to real surgery ranging from 67 to 100 per cent. CONCLUSIONS Manual and automated assessment tools for robotic surgery are not well validated and require further evaluation before use in accreditation processes.PROSPERO: registration ID CRD42022304901.
Collapse
Affiliation(s)
- Matthew W E Boal
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
| | - Dimitrios Anastasiou
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Medical Physics and Biomedical Engineering, UCL, London, UK
| | - Freweini Tesfai
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
| | - Walaa Ghamrawi
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
| | - Evangelos Mazomenos
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Medical Physics and Biomedical Engineering, UCL, London, UK
| | - Nathan Curtis
- Department of General Surgey, Dorset County Hospital NHS Foundation Trust, Dorchester, UK
| | - Justin W Collins
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - Ashwin Sridhar
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - John Kelly
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - Danail Stoyanov
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Computer Science, UCL, London, UK
| | - Nader K Francis
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- Yeovil District Hospital, Somerset Foundation NHS Trust, Yeovil, Somerset, UK
| |
Collapse
|
3
|
Virtual reality and augmented reality in radiation oncology education – A review and expert commentary. Tech Innov Patient Support Radiat Oncol 2022; 24:25-31. [PMID: 36164438 PMCID: PMC9508152 DOI: 10.1016/j.tipsro.2022.08.007] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2022] [Revised: 08/23/2022] [Accepted: 08/24/2022] [Indexed: 12/20/2022] Open
Abstract
The field of radiation oncology is rapidly advancing through technological and biomedical innovation backed by robust research evidence. However, cancer professionals are notoriously time-poor, meaning there is a need for high quality, accessible and tailored oncologic education programs. While traditional teaching methods including lectures and other in-person delivery formats remain important, digital learning (DL) has provided additional teaching options that can be delivered flexibly and on-demand from anywhere in the world. While evidence of this digital migration has been evident for some time now, it has not always been met with the same enthusiasm by the teaching community, in part due to questions about its pedagogical effectiveness. Many of these reservations have been driven by a rudimentary utilisation of the medium and inexperience with digital best-practice. With increasing familiarity and understanding of the medium, increasingly sophisticated and pedagogically-driven learning solutions can be produced. This article will review the application of immersive digital learning tools in radiation oncology education. This includes first and second-generation Virtual Reality (VR) environments and Augmented Reality (AR). It will explore the data behind, and best-practice application of, each of these tools as well as giving practical tips for educators who are looking to implement (or refine) their use of these learning methods. It includes a discussion of how to match the digital learning methods to the content being taught and ends with a horizon scan of where the digital medium may take us in the future. This article is the second in a two-part series, with the companion piece being on Screen-Based Digital Learning Methods in Radiation Oncology. Overall, the digital space is well-placed to cater to the evolving educational needs of oncology learners. Further uptake over the next decade is likely to be driven by the desire for flexible on demand delivery, high-yield products, engaging delivery methods and programs that are tailored to individual learning needs. Educational programs that embrace these principles will have unique opportunities to thrive in this space.
Collapse
|
4
|
Eriksen JG, Boldrini L, Gershkevitsh E, Guckenberger M, van der Heide U, Heijmen B, Joiner M, Nout R, Pruschy M, Rasch C, Tan LT, Verellen D, Vozenin MC, Palmu M, La Porta L, Gasparotto C. Postgraduate education in radiation oncology during the COVID-19 pandemic - What did we learn? Tech Innov Patient Support Radiat Oncol 2022; 24:73-77. [PMID: 36247369 PMCID: PMC9554011 DOI: 10.1016/j.tipsro.2022.09.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2022] [Revised: 09/25/2022] [Accepted: 09/27/2022] [Indexed: 11/08/2022] Open
Abstract
Introduction During the COVID-19 pandemic the ESTRO School who provides international non-profit postgraduate education in Radiation Oncology and related disciplines, including Medical Physics and Radiation Technology, had to close down all live educational activities and turn online, although having only limited experience. The paper describes the experience, discusses the limitations and benefits of online education and suggests directions for the future. Materials and methods Data about format and feedback from attendees and faculty members from the course activities held in 2019, 2020 and 2021 were made available from the ESTRO School. Results In 2020, all but two out of thirty live courses that happened before the lockdown were canceled. Among the 18 courses scheduled in the second half of the year, seven went online with a short notice. Each course planned their activities quite differently, from compressed courses with consecutive full days online program to courses over several weeks with a few hours online a week. Both numbers of participants and different nationalities were higher than live courses in 2019 for the seven courses happening online, and courses were well evaluated by participants and faculties. Roughly-one-third of participants would prefer online courses in the future. Discussion Although online education was well received by the majority, pros and cons exist and especially the personal discussions and networking were missed. Online education and live education are not comparable but can complement each other. Careful balancing these activities in the future is important and strategies for online andragogy are needed.
Collapse
Affiliation(s)
- Jesper Grau Eriksen
- Corresponding author at: Dept of Experimental Clinical Oncology, Aarhus University Hospital, C108, Palle Juul-Jensens Boulevard 99, DK-8200 Aarhus, Denmark.
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
5
|
Non-technical skills in robotic surgery and impact on near-miss events: a multi-center study. Surg Endosc 2020; 35:5062-5071. [PMID: 32968920 DOI: 10.1007/s00464-020-07988-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2020] [Accepted: 09/14/2020] [Indexed: 12/27/2022]
Abstract
BACKGROUND Robotic surgery requires a set of non-technical skills (NTS), because of the complex environment. We aim to study relationship between NTS and near-miss events in robotic surgery. METHODS This is an observational study in five French centers. Three robotic procedures were observed and filmed by one of expert trainers in NTS. They established and scored a non-technical skills in robotic surgery (NTSRS) score, that included eight items, each scored from 1 to 5, to assess the whole surgical teams. The surgical teams also self-assessed their work. The number of near-miss events was recorded and classified as minor, or major but no harm incidents, independently by two surgeons. Correlations were Spearman coefficients. RESULTS Of the 26 procedures included, 15 were prostatectomy (58%), 9 nephrectomy (35%), and 2 pyeloplasty (7.7%). Half of procedures (n = 13) were performed by surgeons with extensive RS experience (more than 150 procedures). Per procedure, there was a median (quartiles) of 9 (7; 11) near-miss events. There was 1 (0; 2) major near-miss events, with no harm. The median NTSRS score was 18 (14; 21), out of 40. The number of near-miss events was strongly correlated with the NTSRS score (r = - 0.92, p < 0.001) but was not correlated with the surgeon's experience. The surgeons for fifteen (58%) procedures, and the bed-side surgeons for 11 (42%) procedures, felt that there was no need for an improvement in the quality of their NTS. None of the surgeons gave a negative self-evaluation for any procedure; in three procedures (12%), the bed-side surgeons self-assessed negatively, on ergonomics. CONCLUSION Occurrence of near-miss events was reduced in teams managing NTS. Specific NTS surgical team training is essential for robotic surgery as it may have a significant impact on risk management.
Collapse
|
6
|
Quantification of Student Radiographic Patient Positioning Using an Immersive Virtual Reality Simulation. ACTA ACUST UNITED AC 2019; 14:258-263. [DOI: 10.1097/sih.0000000000000380] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
7
|
Pathak RA, Edge CC, Thomas GM, Thiel DD, Broderick GA, La Rosa D, Lannen A, Moore MC, Frank RD, Igel TC. Vasectomy Simulation Curriculum and Trainer with Enhanced Face, Content and Construct Validity. UROLOGY PRACTICE 2019. [DOI: 10.1016/j.urpr.2018.08.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- Ram A. Pathak
- Department of Urology, Wake Forest University Baptist Medical Center, Winston-Salem, North Carolina
| | - Carl C. Edge
- Department of Urology, Wake Forest University Baptist Medical Center, Winston-Salem, North Carolina
| | - Garrett M. Thomas
- Department of Urology, Wake Forest University Baptist Medical Center, Winston-Salem, North Carolina
| | - David D. Thiel
- Department of Urology, Mayo Clinic, Jacksonville, Florida
| | | | - Delaney La Rosa
- J. Wayne and Delores Barr Weaver Simulation Center, Department of Education, Mayo Clinic, Jacksonville, Florida
| | - Amy Lannen
- J. Wayne and Delores Barr Weaver Simulation Center, Department of Education, Mayo Clinic, Jacksonville, Florida
| | - Monica C. Moore
- Department of Health Sciences Research, Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, Minnesota
| | - Ryan D. Frank
- Department of Health Sciences Research, Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, Minnesota
| | - Todd C. Igel
- Department of Urology, Mayo Clinic, Jacksonville, Florida
| |
Collapse
|
8
|
Mantica G, Pacchetti A, Aimar R, Cerasuolo M, Dotta F, Olivero A, Pini G, Passaretti G, Maffezzini M, Terrone C. Developing a five-step training model for transperineal prostate biopsies in a naïve residents' group: a prospective observational randomised study of two different techniques. World J Urol 2018; 37:1845-1850. [PMID: 30535716 DOI: 10.1007/s00345-018-2599-6] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2018] [Accepted: 12/04/2018] [Indexed: 11/28/2022] Open
Abstract
PURPOSE To evaluate a five-step training model for transperineal prostate biopsies (TPPB) and the differences in terms of the detection rate (DR) and the ease of execution when using either the "fan technique" (FT) or the use of a Free Hand technique (FH). METHODS A prospective observational randomised study was conducted from September 2015 to November 2017. Six naïve residents, who underwent the same five-steps training model, were randomly subdivided into two different groups of three residents based on the selected TPPB technique: A (FT) and B (FH). Patient characteristics (age, PSA, prostatic volume, DRE, MRI), intraoperative (operative time, number of samples) and postoperative parameters (histologic, pain) were evaluated in the 2 groups. The overall and stratified DR for PSA ranges and prostate volume (PV), operative time and complications were compared. RESULTS The overall detection rate was very high in both groups (FT 58.2% vs FH 59.6%) and not statistically different between the two techniques. There were no differences in terms of complication rates and pain. The FH showed a better detection rate in prostates smaller than 40 cc (p = 0.023) and a faster operative time (p = 0.025) compared to FT. CONCLUSIONS Within the TPPB, FH is associated with a higher detection rate in patients with prostate < 40 cc compared to an FT when performed by inexperienced trainees. Standardised training organised in consecutive steps seems to contribute to the achievement of overall high detection rates with both methods.
Collapse
Affiliation(s)
- Guglielmo Mantica
- Department of Urology, Policlinico San Martino Hospital, University of Genova, Largo Rosanna Benzi 10, 16130, Genova, Italy. .,Department of Urology, San Raffaele Turro Hospital, San Raffaele University, Milan, Italy.
| | - Andrea Pacchetti
- Department of Urology, Policlinico San Martino Hospital, University of Genova, Largo Rosanna Benzi 10, 16130, Genova, Italy
| | - Roberta Aimar
- Department of Urology, San Luigi Gonzaga Hospital, University of Turin, Turin, Italy
| | - Mattia Cerasuolo
- Department of Urology, Policlinico San Martino Hospital, University of Genova, Largo Rosanna Benzi 10, 16130, Genova, Italy
| | - Federico Dotta
- Department of Urology, Policlinico San Martino Hospital, University of Genova, Largo Rosanna Benzi 10, 16130, Genova, Italy
| | - Alberto Olivero
- Department of Urology, Policlinico San Martino Hospital, University of Genova, Largo Rosanna Benzi 10, 16130, Genova, Italy
| | - Giovannalberto Pini
- Department of Urology, San Raffaele Turro Hospital, San Raffaele University, Milan, Italy
| | - Giovanni Passaretti
- Department of Urology, San Raffaele Turro Hospital, San Raffaele University, Milan, Italy
| | - Massimo Maffezzini
- Department of Urology, Policlinico San Martino Hospital, University of Genova, Largo Rosanna Benzi 10, 16130, Genova, Italy
| | - Carlo Terrone
- Department of Urology, Policlinico San Martino Hospital, University of Genova, Largo Rosanna Benzi 10, 16130, Genova, Italy
| |
Collapse
|
9
|
Goldenberg MG, Lee JY, Kwong JCC, Grantcharov TP, Costello A. Implementing assessments of robot-assisted technical skill in urological education: a systematic review and synthesis of the validity evidence. BJU Int 2018; 122:501-519. [PMID: 29603869 DOI: 10.1111/bju.14219] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
OBJECTIVES To systematically review and synthesise the validity evidence supporting intraoperative and simulation-based assessments of technical skill in urological robot-assisted surgery (RAS), and make evidence-based recommendations for the implementation of these assessments in urological training. MATERIALS AND METHODS A literature search of the Medline, PsycINFO and Embase databases was performed. Articles using technical skill and simulation-based assessments in RAS were abstracted. Only studies involving urology trainees or faculty were included in the final analysis. RESULTS Multiple tools for the assessment of technical robotic skill have been published, with mixed sources of validity evidence to support their use. These evaluations have been used in both the ex vivo and in vivo settings. Performance evaluations range from global rating scales to psychometrics, and assessments are carried out through automation, expert analysts, and crowdsourcing. CONCLUSION There have been rapid expansions in approaches to RAS technical skills assessment, both in simulated and clinical settings. Alternative approaches to assessment in RAS, such as crowdsourcing and psychometrics, remain under investigation. Evidence to support the use of these metrics in high-stakes decisions is likely insufficient at present.
Collapse
Affiliation(s)
| | - Jason Y Lee
- Division of Urology, University of Toronto, Toronto, ON, Canada
| | | | - Teodor P Grantcharov
- Division of General Surgery, Department of Surgery, University of Toronto, Toronto, ON, Canada
| | - Anthony Costello
- Department of Surgery, Royal Melbourne Hospital, University of Melbourne, Melbourne, Vic, Australia
| |
Collapse
|
10
|
Oquendo YA, Riddle EW, Hiller D, Blinman TA, Kuchenbecker KJ. Automatically rating trainee skill at a pediatric laparoscopic suturing task. Surg Endosc 2017; 32:1840-1857. [PMID: 29071419 PMCID: PMC5845064 DOI: 10.1007/s00464-017-5873-6] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2017] [Accepted: 09/04/2017] [Indexed: 12/29/2022]
Abstract
BACKGROUND Minimally invasive surgeons must acquire complex technical skills while minimizing patient risk, a challenge that is magnified in pediatric surgery. Trainees need realistic practice with frequent detailed feedback, but human grading is tedious and subjective. We aim to validate a novel motion-tracking system and algorithms that automatically evaluate trainee performance of a pediatric laparoscopic suturing task. METHODS Subjects (n = 32) ranging from medical students to fellows performed two trials of intracorporeal suturing in a custom pediatric laparoscopic box trainer after watching a video of ideal performance. The motions of the tools and endoscope were recorded over time using a magnetic sensing system, and both tool grip angles were recorded using handle-mounted flex sensors. An expert rated the 63 trial videos on five domains from the Objective Structured Assessment of Technical Skill (OSATS), yielding summed scores from 5 to 20. Motion data from each trial were processed to calculate 280 features. We used regularized least squares regression to identify the most predictive features from different subsets of the motion data and then built six regression tree models that predict summed OSATS score. Model accuracy was evaluated via leave-one-subject-out cross-validation. RESULTS The model that used all sensor data streams performed best, achieving 71% accuracy at predicting summed scores within 2 points, 89% accuracy within 4, and a correlation of 0.85 with human ratings. 59% of the rounded average OSATS score predictions were perfect, and 100% were within 1 point. This model employed 87 features, including none based on completion time, 77 from tool tip motion, 3 from tool tip visibility, and 7 from grip angle. CONCLUSIONS Our novel hardware and software automatically rated previously unseen trials with summed OSATS scores that closely match human expert ratings. Such a system facilitates more feedback-intensive surgical training and may yield insights into the fundamental components of surgical skill.
Collapse
Affiliation(s)
- Yousi A Oquendo
- Department of Mechanical Engineering & Applied Mechanics, University of Pennsylvania, Philadelphia, USA.,Department of Computer & Information Science, University of Pennsylvania, Philadelphia, USA
| | - Elijah W Riddle
- Division of Pediatric General, Thoracic and Fetal Surgery, Children's Hospital of Philadelphia, Philadelphia, USA
| | - Dennis Hiller
- Division of Pediatric General, Thoracic and Fetal Surgery, Children's Hospital of Philadelphia, Philadelphia, USA
| | - Thane A Blinman
- Division of Pediatric General, Thoracic and Fetal Surgery, Children's Hospital of Philadelphia, Philadelphia, USA
| | - Katherine J Kuchenbecker
- Department of Mechanical Engineering & Applied Mechanics, University of Pennsylvania, Philadelphia, USA. .,Department of Computer & Information Science, University of Pennsylvania, Philadelphia, USA. .,Haptic Intelligence Department, Max Planck Institute for Intelligent Systems, Heisenbergstr. 3, 70569, Stuttgart, Germany.
| |
Collapse
|
11
|
Urology residents experience comparable workload profiles when performing live porcine nephrectomies and robotic surgery virtual reality training modules. J Robot Surg 2016; 10:49-56. [DOI: 10.1007/s11701-015-0540-1] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2015] [Accepted: 10/15/2015] [Indexed: 10/22/2022]
|
12
|
Rashid P, Gianduzzo TR. Urology technical and non-technical skills development: the emerging role of simulation. BJU Int 2015; 117 Suppl 4:9-16. [DOI: 10.1111/bju.13259] [Citation(s) in RCA: 47] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
Affiliation(s)
- Prem Rashid
- Department of Urology; Rural Clinical School; The University of New South Wales; Port Macquarie Base Hospital; Port Macquarie NSW Australia
| | - Troy R.J. Gianduzzo
- Department of Urology; School of Medicine; The University of Queensland; Royal Brisbane and Women's Hospital; Brisbane Qld Australia
| |
Collapse
|
13
|
Perez M, Xu S, Chauhan S, Tanaka A, Simpson K, Abdul-Muhsin H, Smith R. Impact of delay on telesurgical performance: study on the robotic simulator dV-Trainer. Int J Comput Assist Radiol Surg 2015; 11:581-7. [PMID: 26450105 DOI: 10.1007/s11548-015-1306-y] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2015] [Accepted: 09/21/2015] [Indexed: 12/16/2022]
Abstract
PURPOSE To determine the impact of communication latency on telesurgical performance using the robotic simulator dV-Trainer®. METHODS Surgeons were enrolled during three robotic congresses. They were randomly assigned to a delay group (ranging from 100 to 1000 ms). Each group performed three times a set of four exercises on the simulator: the first attempt without delay (Base) and the last two attempts with delay (Warm-up and Test). The impact of different levels of latency was evaluated. RESULTS Thirty-seven surgeons were involved. The different latency groups achieved similar baseline performance with a mean task completion time of 207.2 s (p > 0.05). In the Test stage, the task duration increased gradually from 156.4 to 310.7 s as latency increased from 100 to 500 ms. In separate groups, the task duration deteriorated from Base for latency stages at delays ≥300 ms, and the errors increased at 500 ms and above (p < 0.05). The subjects' performance tended to improve from the Warm-up to the Test period. Few subjects completed the tasks with a delay higher than 700 ms. CONCLUSION Gradually increasing latency has a growing impact on performances. Measurable deterioration of performance begins at 300 ms. Delays higher than 700 ms are difficult to manage especially in more complex tasks. Surgeons showed the potential to adapt to delay and may be trained to improve their telesurgical performance at lower-latency levels.
Collapse
Affiliation(s)
- Manuela Perez
- IADI Laboratory-INSERM-U947, Lorraine University, Allée du Morvan, 54500, Vandoeuvre-les-Nancy, France.
- General and Emergency Surgery Department, University Hospital of Nancy, Avenue du Maréchal de Lattre de Tassigny, 54035, Nancy, France.
- Florida Hospital Nicholson Center, 404 Celebration Place, Celebration, FL, 34747, USA.
- Hôpital Central Service de Chirurgie Générale et Urgences, Avenue du Maréchal de Lattre de Tassigny, 54035, Nancy, France.
| | - Song Xu
- IADI Laboratory-INSERM-U947, Lorraine University, Allée du Morvan, 54500, Vandoeuvre-les-Nancy, France
- Urology Department, University Hospital of Nancy, Allée du Morvan, 54511, Vandoeuvre-les-Nancy, France
| | - Sanket Chauhan
- Center for Evidence Based Simulation, Baylor University Medical Center, Dallas, TX, USA
- Associate Professor Texas A&M Health Science Center, College Station, TX, USA
| | - Alyssa Tanaka
- Florida Hospital Nicholson Center, 404 Celebration Place, Celebration, FL, 34747, USA
| | - Khara Simpson
- Florida Hospital Nicholson Center, 404 Celebration Place, Celebration, FL, 34747, USA
| | - Haidar Abdul-Muhsin
- Florida Hospital Nicholson Center, 404 Celebration Place, Celebration, FL, 34747, USA
| | - Roger Smith
- Florida Hospital Nicholson Center, 404 Celebration Place, Celebration, FL, 34747, USA
| |
Collapse
|