1
|
Boal MWE, Anastasiou D, Tesfai F, Ghamrawi W, Mazomenos E, Curtis N, Collins JW, Sridhar A, Kelly J, Stoyanov D, Francis NK. Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review. Br J Surg 2024; 111:znad331. [PMID: 37951600 PMCID: PMC10771126 DOI: 10.1093/bjs/znad331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 09/18/2023] [Accepted: 09/19/2023] [Indexed: 11/14/2023]
Abstract
BACKGROUND There is a need to standardize training in robotic surgery, including objective assessment for accreditation. This systematic review aimed to identify objective tools for technical skills assessment, providing evaluation statuses to guide research and inform implementation into training curricula. METHODS A systematic literature search was conducted in accordance with the PRISMA guidelines. Ovid Embase/Medline, PubMed and Web of Science were searched. Inclusion criterion: robotic surgery technical skills tools. Exclusion criteria: non-technical, laparoscopy or open skills only. Manual tools and automated performance metrics (APMs) were analysed using Messick's concept of validity and the Oxford Centre of Evidence-Based Medicine (OCEBM) Levels of Evidence and Recommendation (LoR). A bespoke tool analysed artificial intelligence (AI) studies. The Modified Downs-Black checklist was used to assess risk of bias. RESULTS Two hundred and forty-seven studies were analysed, identifying: 8 global rating scales, 26 procedure-/task-specific tools, 3 main error-based methods, 10 simulators, 28 studies analysing APMs and 53 AI studies. Global Evaluative Assessment of Robotic Skills and the da Vinci Skills Simulator were the most evaluated tools at LoR 1 (OCEBM). Three procedure-specific tools, 3 error-based methods and 1 non-simulator APMs reached LoR 2. AI models estimated outcomes (skill or clinical), demonstrating superior accuracy rates in the laboratory with 60 per cent of methods reporting accuracies over 90 per cent, compared to real surgery ranging from 67 to 100 per cent. CONCLUSIONS Manual and automated assessment tools for robotic surgery are not well validated and require further evaluation before use in accreditation processes.PROSPERO: registration ID CRD42022304901.
Collapse
Affiliation(s)
- Matthew W E Boal
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
| | - Dimitrios Anastasiou
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Medical Physics and Biomedical Engineering, UCL, London, UK
| | - Freweini Tesfai
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
| | - Walaa Ghamrawi
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
| | - Evangelos Mazomenos
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Medical Physics and Biomedical Engineering, UCL, London, UK
| | - Nathan Curtis
- Department of General Surgey, Dorset County Hospital NHS Foundation Trust, Dorchester, UK
| | - Justin W Collins
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - Ashwin Sridhar
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - John Kelly
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - Danail Stoyanov
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Computer Science, UCL, London, UK
| | - Nader K Francis
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- Yeovil District Hospital, Somerset Foundation NHS Trust, Yeovil, Somerset, UK
| |
Collapse
|
2
|
Feeley AA, Feeley IH, Merghani K, Sheehan E. Surgical Priming Improves Operative Performance in Surgical Trainees: A Crossover Randomized Control Trial. JOURNAL OF SURGICAL EDUCATION 2023; 80:420-427. [PMID: 36335033 DOI: 10.1016/j.jsurg.2022.10.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/04/2022] [Accepted: 10/13/2022] [Indexed: 06/16/2023]
Abstract
OBJECTIVE The aim of this study was to evaluate the impact of a surgical warm-up using a virtual reality simulator on operative performance. DESIGN This was a single-blinded cross-over randomized control trial in a single tertiary Orthopedic training center. PARTICIPANTS Orthopedic trainees were recruited, and each morning participants rostered to theatre were randomized to either undergo a simulated surgical procedure on a virtual reality simulation system prior to their first case as primary operator (priming arm), or to perform their usual preparatory routine for surgery (control arm). Consultant orthopedic trainers were recruited within the orthopedic unit to carry out subjective surgical performance assessments using a validated global rating scale tool on the first case the participant performed on the list as primary operator. RESULTS Over 3 study periods a total of 151 data points were collected, with 49 matched data points across priming status and procedural level of difficulty. Subjective assessment tools consistently demonstrated improved operative performance by participants following surgical priming (p = 0.001). CONCLUSION This study highlights that introduction of preoperative priming to improve operative preparation, and optimizes operative performance. This has not only implications for improved resident training, but also signals towards beneficial downstream effects on patient outcomes, and theatre list planning.
Collapse
Affiliation(s)
- Aoife A Feeley
- Department of Surgery Royal College of Surgeons in Ireland, Co. Dublin, Ireland; Department of Orthopaedics, Midland Regional Hospital Tullamore, Co. Offaly, Ireland.
| | - Iain H Feeley
- Department of Orthopaedics, Tallaght University Hospital, Co. Dublin, Ireland
| | - Khalid Merghani
- Department of Orthopaedics, Midland Regional Hospital Tullamore, Co. Offaly, Ireland
| | - Eoin Sheehan
- Department of Orthopaedics, Midland Regional Hospital Tullamore, Co. Offaly, Ireland
| |
Collapse
|
3
|
Rahimi AO, Ho K, Chang M, Gasper D, Ashouri Y, Dearmon-Moore D, Hsu CH, Ghaderi I. A systematic review of robotic surgery curricula using a contemporary educational framework. Surg Endosc 2022; 37:2833-2841. [PMID: 36481821 DOI: 10.1007/s00464-022-09788-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Accepted: 11/27/2022] [Indexed: 12/13/2022]
Abstract
BACKGROUND There has been a rising trend in robotic surgery. Thus, there is demand for a robotic surgery curriculum (RSC) for training surgical trainees and practicing surgeons. There are limited data available about current curricular designs and the extent to which they have incorporated educational frameworks. Our aim was to study the existing robotic surgery curricula using Kern's 6-step approach in curriculum development. METHODS A systematic review was conducted using PubMed, PubMed Central, Cochrane, Embase, and Scopus (we searched studies from 2001 to 2021). PRISMA Guidelines was used to guide the search. Curriculum designed for general surgery and its subspecialties were included. Urology and gynecology were excluded. The articles were reviewed by five reviewers. RESULTS Our review yielded 71 articles, including 39 curricula at 9 different settings. Using Kern's framework, we demonstrated that the majority of robotic surgery curricula contained all the elements of Kern's curricular design. However, there were significant deficiencies in important aspects of these curricula i.e., implementation, the quality of assessment tools for measurement of performance and evaluation of the educational value of these interventions. Most institutions used commercial virtual reality simulators (VRS) as the main component of their RSC and 23% of curricula only used VRS. CONCLUSIONS Although majority of these studies contained all the elements of Kern's framework, there are critical deficiencies in the components of existing curricula. Future curricula should be designed using established educational frameworks to improve the quality of robotic surgery training.
Collapse
|
4
|
Feeley AA, Feeley IH, Merghani K, Sheehan E. Use of procedure specific preoperative warm-up during surgical priming improves operative outcomes: A systematic review. Am J Surg 2022; 224:1126-1134. [DOI: 10.1016/j.amjsurg.2022.05.031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2022] [Revised: 05/05/2022] [Accepted: 05/24/2022] [Indexed: 11/01/2022]
|