1
|
Brentnall M, Lenihan J, Simmonds C, Malpani A, Gargiulo AR, Martino M, Levy JS. Evaluation of different approaches to define expert benchmark scores for new robotic training simulators based on the Medtronic HUGO™ RAS surgical robot experience. J Robot Surg 2024; 18:113. [PMID: 38451376 DOI: 10.1007/s11701-024-01868-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Accepted: 02/10/2024] [Indexed: 03/08/2024]
Abstract
New robot-assisted surgery platforms being developed will be required to have proficiency-based simulation training available. Scoring methodologies and performance feedback for trainees are currently not consistent across all robotic simulator platforms. Also, there are virtually no prior publications on how VR simulation passing benchmarks have been established. This paper compares methods evaluated to determine the proficiency-based scoring thresholds (a.k.a. benchmarks) for the new Medtronic Hugo™ RAS robotic simulator. Nine experienced robotic surgeons from multiple disciplines performed the 49 skills exercises 5 times each. The data were analyzed in 3 different ways: (1) include all data collected, (2) exclude first sessions, (3) exclude outliers. Eliminating the first session discounts becoming familiar with the exercise. Discounting outliers allows removal of potentially erroneous data that may be due to technical issues, unexpected distractions, etc. Outliers were identified using a common statistical technique involving the interquartile range of the data. Using each method above, mean and standard deviations were calculated, and the benchmark was set at a value of 1 standard deviation above the mean. In comparison to including all the data, when outliers are excluded, fewer data points are removed than just excluding first sessions, and the metric benchmarks are made more difficult by an average of 11%. When first sessions are excluded, the metric benchmarks are made easier by an average of about 2%. In comparison with benchmarks calculated using all data points, excluding outliers resulted in the biggest change making the benchmarks more challenging. We determined that this method provided the best representation of the data. These benchmarks should be validated with future clinical training studies.
Collapse
Affiliation(s)
| | - John Lenihan
- University of Washington School of Medicine, Seattle, WA, USA.
| | | | | | | | | | | |
Collapse
|
2
|
Leijte E, De Blaauw I, Rosman C, Botden SMBI. Transferability of the robot assisted and laparoscopic suturing learning curves. J Robot Surg 2024; 18:56. [PMID: 38280121 PMCID: PMC10821960 DOI: 10.1007/s11701-023-01753-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Accepted: 10/11/2023] [Indexed: 01/29/2024]
Abstract
Robot assisted surgery (RAS) is increasingly used, and besides conventional minimally invasive surgery (cMIS) surgeons are challenged to learn an increased array of skills. This study aimed to assess the influence of both learning curves on each other. A prospective randomized crossover study was performed. Participants without cMIS or RAS experience (Groups 1 and 2), and cMIS experienced, (Group 3) were recruited. Three suturing tasks (intracorporal suturing, tilted plane and anastomosis needle transfer) were performed on the EoSim cMIS simulator or RobotiX RAS simulator up to twenty repetitions. Subsequently, Groups 1 and 2 performed the tasks on the other modality. Outcomes were simulator parameters, validated composite and pass/fail scores. In total forty-three participants were recruited. Overall RAS suturing was better in Group 1 (cMIS followed by RAS tasks) and 3 (RAS tasks) versus Group 2 (RAS followed by cMIS tasks) for time (163 s and 157 s versus 193 s p = 0.004, p = 0.001) and composite scores (92/100 and 91/100 versus 89/100 p = 0.008, p = 0.020). The cMIS suturing was better for Group 2 versus 1 (time 287 s versus 349 s p = 0.005, composite score 96/100 versus 94/100 p = 0.002). Significant differences from the RAS suturing pass/fail were reached earlier by Group 3, followed by Groups 1 and 2 (repetition six, nine and twelve). In cMIS suturing Group 2 reached significant differences from the pass/fail earlier than Group 1 (repetition four versus six). Transferability of skills was shown for cMIS and RAS, indicating that suturing experience on cMIS or RAS is beneficial in learning either approach.
Collapse
Affiliation(s)
- E Leijte
- Department of Surgery, Radboud University Medical Centre, Geert Grooteplein 10 Route 618, 6500HB, Nijmegen, The Netherlands.
- Department of Paediatric Surgery, Radboud University Medical Centre, Nijmegen, The Netherlands.
| | - I De Blaauw
- Department of Paediatric Surgery, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - C Rosman
- Department of Surgery, Radboud University Medical Centre, Geert Grooteplein 10 Route 618, 6500HB, Nijmegen, The Netherlands
| | - S M B I Botden
- Department of Paediatric Surgery, Radboud University Medical Centre, Nijmegen, The Netherlands
| |
Collapse
|
3
|
El-Sayed C, Yiu A, Burke J, Vaughan-Shaw P, Todd J, Lin P, Kasmani Z, Munsch C, Rooshenas L, Campbell M, Bach SP. Measures of performance and proficiency in robotic assisted surgery: a systematic review. J Robot Surg 2024; 18:16. [PMID: 38217749 DOI: 10.1007/s11701-023-01756-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Accepted: 11/07/2023] [Indexed: 01/15/2024]
Abstract
Robotic assisted surgery (RAS) has seen a global rise in adoption. Despite this, there is not a standardised training curricula nor a standardised measure of performance. We performed a systematic review across the surgical specialties in RAS and evaluated tools used to assess surgeons' technical performance. Using the PRISMA 2020 guidelines, Pubmed, Embase and the Cochrane Library were searched systematically for full texts published on or after January 2020-January 2022. Observational studies and RCTs were included; review articles and systematic reviews were excluded. The papers' quality and bias score were assessed using the Newcastle Ottawa Score for the observational studies and Cochrane Risk Tool for the RCTs. The initial search yielded 1189 papers of which 72 fit the eligibility criteria. 27 unique performance metrics were identified. Global assessments were the most common tool of assessment (n = 13); the most used was GEARS (Global Evaluative Assessment of Robotic Skills). 11 metrics (42%) were objective tools of performance. Automated performance metrics (APMs) were the most widely used objective metrics whilst the remaining (n = 15, 58%) were subjective. The results demonstrate variation in tools used to assess technical performance in RAS. A large proportion of the metrics are subjective measures which increases the risk of bias amongst users. A standardised objective metric which measures all domains of technical performance from global to cognitive is required. The metric should be applicable to all RAS procedures and easily implementable. Automated performance metrics (APMs) have demonstrated promise in their wide use of accurate measures.
Collapse
Affiliation(s)
- Charlotte El-Sayed
- RCS England/HEE Robotics Research Fellow, University of Birmingham, Birmingham, United Kingdom.
| | - A Yiu
- RCS England/HEE Robotics Research Fellow, University of Birmingham, Birmingham, United Kingdom
| | - J Burke
- RCS England/HEE Robotics Research Fellow, University of Birmingham, Birmingham, United Kingdom
| | - P Vaughan-Shaw
- RCS England/HEE Robotics Research Fellow, University of Birmingham, Birmingham, United Kingdom
| | - J Todd
- RCS England/HEE Robotics Research Fellow, University of Birmingham, Birmingham, United Kingdom
| | - P Lin
- RCS England/HEE Robotics Research Fellow, University of Birmingham, Birmingham, United Kingdom
| | - Z Kasmani
- RCS England/HEE Robotics Research Fellow, University of Birmingham, Birmingham, United Kingdom
| | - C Munsch
- RCS England/HEE Robotics Research Fellow, University of Birmingham, Birmingham, United Kingdom
| | - L Rooshenas
- RCS England/HEE Robotics Research Fellow, University of Birmingham, Birmingham, United Kingdom
| | - M Campbell
- RCS England/HEE Robotics Research Fellow, University of Birmingham, Birmingham, United Kingdom
| | - S P Bach
- RCS England/HEE Robotics Research Fellow, University of Birmingham, Birmingham, United Kingdom
| |
Collapse
|
4
|
Boal MWE, Anastasiou D, Tesfai F, Ghamrawi W, Mazomenos E, Curtis N, Collins JW, Sridhar A, Kelly J, Stoyanov D, Francis NK. Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review. Br J Surg 2024; 111:znad331. [PMID: 37951600 PMCID: PMC10771126 DOI: 10.1093/bjs/znad331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 09/18/2023] [Accepted: 09/19/2023] [Indexed: 11/14/2023]
Abstract
BACKGROUND There is a need to standardize training in robotic surgery, including objective assessment for accreditation. This systematic review aimed to identify objective tools for technical skills assessment, providing evaluation statuses to guide research and inform implementation into training curricula. METHODS A systematic literature search was conducted in accordance with the PRISMA guidelines. Ovid Embase/Medline, PubMed and Web of Science were searched. Inclusion criterion: robotic surgery technical skills tools. Exclusion criteria: non-technical, laparoscopy or open skills only. Manual tools and automated performance metrics (APMs) were analysed using Messick's concept of validity and the Oxford Centre of Evidence-Based Medicine (OCEBM) Levels of Evidence and Recommendation (LoR). A bespoke tool analysed artificial intelligence (AI) studies. The Modified Downs-Black checklist was used to assess risk of bias. RESULTS Two hundred and forty-seven studies were analysed, identifying: 8 global rating scales, 26 procedure-/task-specific tools, 3 main error-based methods, 10 simulators, 28 studies analysing APMs and 53 AI studies. Global Evaluative Assessment of Robotic Skills and the da Vinci Skills Simulator were the most evaluated tools at LoR 1 (OCEBM). Three procedure-specific tools, 3 error-based methods and 1 non-simulator APMs reached LoR 2. AI models estimated outcomes (skill or clinical), demonstrating superior accuracy rates in the laboratory with 60 per cent of methods reporting accuracies over 90 per cent, compared to real surgery ranging from 67 to 100 per cent. CONCLUSIONS Manual and automated assessment tools for robotic surgery are not well validated and require further evaluation before use in accreditation processes.PROSPERO: registration ID CRD42022304901.
Collapse
Affiliation(s)
- Matthew W E Boal
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
| | - Dimitrios Anastasiou
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Medical Physics and Biomedical Engineering, UCL, London, UK
| | - Freweini Tesfai
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
| | - Walaa Ghamrawi
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
| | - Evangelos Mazomenos
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Medical Physics and Biomedical Engineering, UCL, London, UK
| | - Nathan Curtis
- Department of General Surgey, Dorset County Hospital NHS Foundation Trust, Dorchester, UK
| | - Justin W Collins
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - Ashwin Sridhar
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - John Kelly
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- University College London Hospitals NHS Foundation Trust, London, UK
| | - Danail Stoyanov
- Wellcome/ESPRC Centre for Interventional Surgical Sciences (WEISS), University College London (UCL), London, UK
- Computer Science, UCL, London, UK
| | - Nader K Francis
- The Griffin Institute, Northwick Park & St Marks’ Hospital, London, UK
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, UCL, London, UK
- Yeovil District Hospital, Somerset Foundation NHS Trust, Yeovil, Somerset, UK
| |
Collapse
|
5
|
Roll S, Dias ERM. Education in Robotic Hernia Surgery-Current Situation. JOURNAL OF ABDOMINAL WALL SURGERY : JAWS 2023; 2:12418. [PMID: 38312412 PMCID: PMC10831677 DOI: 10.3389/jaws.2023.12418] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Accepted: 11/16/2023] [Indexed: 02/06/2024]
Affiliation(s)
- Sergio Roll
- Department of Surgery, School of Medical Sciences of Santa Casa of São Paulo, São Paulo, Brazil
- Center of Abdominal Wall Surgery, Division of General Surgery, Santa Casa of São Paulo Hospital, School of Medical Sciences of Santa Casa of São Paulo, São Paulo, Brazil
- The German Hospital Oswaldo Cruz Hernia Center, São Paulo, Brazil
| | | |
Collapse
|
6
|
Rahimi AO, Ho K, Chang M, Gasper D, Ashouri Y, Dearmon-Moore D, Hsu CH, Ghaderi I. A systematic review of robotic surgery curricula using a contemporary educational framework. Surg Endosc 2022; 37:2833-2841. [PMID: 36481821 DOI: 10.1007/s00464-022-09788-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Accepted: 11/27/2022] [Indexed: 12/13/2022]
Abstract
BACKGROUND There has been a rising trend in robotic surgery. Thus, there is demand for a robotic surgery curriculum (RSC) for training surgical trainees and practicing surgeons. There are limited data available about current curricular designs and the extent to which they have incorporated educational frameworks. Our aim was to study the existing robotic surgery curricula using Kern's 6-step approach in curriculum development. METHODS A systematic review was conducted using PubMed, PubMed Central, Cochrane, Embase, and Scopus (we searched studies from 2001 to 2021). PRISMA Guidelines was used to guide the search. Curriculum designed for general surgery and its subspecialties were included. Urology and gynecology were excluded. The articles were reviewed by five reviewers. RESULTS Our review yielded 71 articles, including 39 curricula at 9 different settings. Using Kern's framework, we demonstrated that the majority of robotic surgery curricula contained all the elements of Kern's curricular design. However, there were significant deficiencies in important aspects of these curricula i.e., implementation, the quality of assessment tools for measurement of performance and evaluation of the educational value of these interventions. Most institutions used commercial virtual reality simulators (VRS) as the main component of their RSC and 23% of curricula only used VRS. CONCLUSIONS Although majority of these studies contained all the elements of Kern's framework, there are critical deficiencies in the components of existing curricula. Future curricula should be designed using established educational frameworks to improve the quality of robotic surgery training.
Collapse
|
7
|
Abstract
OBJECTIVE This systematic review aims to examine the use of standard-setting methods in the context of simulation-based training of surgical procedures. SUMMARY OF BACKGROUND Simulation-based training is increasingly used in surgical education. However, it is important to determine which level of competency trainees must reach during simulation-based training before operating on patients. Therefore, pass/fail standards must be established using systematic, transparent, and valid methods. METHODS Systematic literature search was done in four databases (Ovid MEDLINE, Embase, Web of Science, and Cochrane Library). Original studies investigating simulation-based assessment of surgical procedures with application of a standard setting were included. Quality of evidence was appraised using GRADE. RESULTS Of 24,299 studies identified by searches, 232 studies met the inclusion criteria. Publications using already established standard settings were excluded (N = 70), resulting in 162 original studies included in the final analyses. Most studies described how the standard setting was determined (N = 147, 91%) and most used the mean or median performance score of experienced surgeons (n = 65, 40%) for standard setting. We found considerable differences across most of the studies regarding study design, set-up, and expert level classification. The studies were appraised as having low and moderate evidence. CONCLUSION Surgical education is shifting towards competency-based education, and simulation-based training is increasingly used for acquiring skills and assessment. Most studies consider and describe how standard settings are established using more or less structured methods but for current and future educational programs, a critical approach is needed so that the learners receive a fair, valid and reliable assessment.
Collapse
|
8
|
Leijte E, de Blaauw I, Rosman C, Botden SMBI. Assessment of validity evidence for the RobotiX robot assisted surgery simulator on advanced suturing tasks. BMC Surg 2020; 20:183. [PMID: 32787831 PMCID: PMC7430880 DOI: 10.1186/s12893-020-00839-z] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2020] [Accepted: 07/30/2020] [Indexed: 01/24/2023] Open
Abstract
Background Robot assisted surgery has expanded considerably in the past years. Compared to conventional open or laparoscopic surgery, virtual reality (VR) training is an essential component in learning robot assisted surgery. However, for tasks to be implemented in a curriculum, the levels of validity should be studied for proficiency-based training. Therefore, this study was aimed to assess the validity evidence of advanced suturing tasks on a robot assisted VR simulator. Method Participants were voluntary recruited and divided in the robotic experienced, laparoscopic experienced or novice group, based on self-reported surgical experience. Subsequently, a questionnaire on a five-point Likert scale was completed to assess the content validity. Three component tasks of complex suturing were performed on the RobotiX simulator (Task1: tilted plane needle transfer, Task: 2 intracorporal suturing, Task 3: anastomosis needle transfer). Accordingly, the outcome of the parameters was used to assess construct validity between robotic experienced and novice participants. Composite scores (0–100) were calculated from the construct parameters and corresponding pass/fail scores with false positive (FP) and false negative (FN) percentages. Results Fifteen robotic experienced, 26 laparoscopic experienced and 29 novices were recruited. Overall content validity outcomes were scored positively on the realism (mean 3.7), didactic value (mean 4.0) and usability (mean 4.2). Robotic experienced participants significantly outperformed novices and laparoscopic experienced participants on multiple parameters on all three tasks of complex suturing. Parameters showing construct validity mainly consisted of movement parameters, needle precision and task completion time. Calculated composite pass/fail scores between robotic experienced and novice participants resulted for Task 1 in 73/100 (FP 21%, FN 5%), Task 2 in 85/100 (FP 28%, FN 4%) and Task 3 in 64/100 (FP 49%, FN 22%). Conclusion This study assessed the validity evidence on multiple levels of the three studied tasks. The participants score the RobotiX good on the content validity level. The composite pass/fail scores of Tasks 1 and 2 allow for proficiency-based training and could be implemented in a robot assisted surgery training curriculum.
Collapse
Affiliation(s)
- Erik Leijte
- Department of Surgery, Radboud University Medical Center, Geert grooteplein 10 route 618, 6500HB, Nijmegen, The Netherlands. .,Department of Pediatric Surgery, Radboud University Medical Center, Geert grooteplein 10 route 618, 6500HB, Nijmegen, The Netherlands.
| | - Ivo de Blaauw
- Department of Pediatric Surgery, Radboud University Medical Center, Geert grooteplein 10 route 618, 6500HB, Nijmegen, The Netherlands
| | - Camiel Rosman
- Department of Surgery, Radboud University Medical Center, Geert grooteplein 10 route 618, 6500HB, Nijmegen, The Netherlands
| | - Sanne M B I Botden
- Department of Pediatric Surgery, Radboud University Medical Center, Geert grooteplein 10 route 618, 6500HB, Nijmegen, The Netherlands
| |
Collapse
|
9
|
Beulens AJW, Hashish YAF, Brinkman WM, Umari P, Puliatti S, Koldewijn EL, Hendrikx AJM, van Basten JP, van Merriënboer JJG, Van der Poel HG, Bangma CH, Wagner C. Training novice robot surgeons: Proctoring provides same results as simulator-generated guidance. J Robot Surg 2020; 15:397-428. [PMID: 32651769 DOI: 10.1007/s11701-020-01118-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Accepted: 06/29/2020] [Indexed: 11/29/2022]
Abstract
To understand the influence of proctored guidance versus simulator generated guidance (SGG) on the acquisition dexterity skills in novice surgeons learning RAS (robot assisted surgery). Prospective non-blinded 3-arm randomised controlled trial (RTC). Exclusion criteria: previous experience in RAS or robotic surgery simulation. The participants were assigned to three different intervention groups and received a different form of guidance: (1) proctored guidance, (2) simulator generated guidance, (3) no guidance, during training on virtual reality (VR) simulator. All participants were asked to complete multiple questionnaires. The training was the same in all groups with the exception of the intervention part. Catharina Hospital Eindhoven, The Netherlands. A total of 70 Dutch medical students, PhD-students, and surgical residents were included in the study. The participants were randomly assigned to one of the three groups. Overall, all the participants showed a significant improvement in their dexterity skills after the training. There was no significant difference in the improvement of surgical skills between the three different intervention groups. The proctored guidance group reported a higher participant satisfaction compared to the simulator-generated guidance group, which could indicate a higher motivation to continue the training. This study showed that novice surgeons. Significantly increase their dexterity skills in RAS after a short time of practicing on simulator. The lack of difference in results between the intervention groups could indicate there is a limited impact of "human proctoring" on dexterity skills during surgical simulation training. Since there is no difference between the intervention groups the exposure alone of novice surgeons to the robotic surgery simulator could possibly be sufficient to achieve a significant improvement of dexterity skills during the initial steps of RAS learning.
Collapse
Affiliation(s)
- A J W Beulens
- Netherlands Institute for Health Services Research (NIVEL), Utrecht, The Netherlands. .,Department of Urology, Catharina Hospital, Michelangelolaan 2, 5623 EJ, Eindhoven, The Netherlands.
| | - Y A F Hashish
- Department of Urology, Catharina Hospital, Michelangelolaan 2, 5623 EJ, Eindhoven, The Netherlands
| | - W M Brinkman
- Department of Oncological Urology, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - P Umari
- Department of Translational Medicine, University of Eastern Piedmont, Novara, Italy
| | - S Puliatti
- Urology Department, University of Modena & Reggio Emilia, Modena, Italy.,Orsi Academy, Melle, Belgium.,Department of Urology, Onze Lieve Vrouw Hospital, Aalst, Belgium
| | - E L Koldewijn
- Department of Urology, Catharina Hospital, Michelangelolaan 2, 5623 EJ, Eindhoven, The Netherlands
| | - A J M Hendrikx
- Department of Urology, Catharina Hospital Eindhoven, Eindhoven, The Netherlands
| | - J P van Basten
- Department of Urology, Canisius Wilhelmina Hospital, Nijmegen, The Netherlands
| | - J J G van Merriënboer
- School of Health Professions Education, Maastricht University, Maastricht, The Netherlands
| | - H G Van der Poel
- Department of Urology, Netherlands Cancer Institute-Antoni Van Leeuwenhoek Hospital, Amsterdam, The Netherlands
| | - C H Bangma
- Department of Urology, Erasmus University Medical Centre, Rotterdam, The Netherlands
| | - C Wagner
- Netherlands Institute for Health Services Research (NIVEL), Utrecht, The Netherlands.,Amsterdam Public Health Research Institute, Amsterdam UMC, Location VUmc, Amsterdam, The Netherlands
| |
Collapse
|
10
|
DeStephano CC, Nitsche JF, Heckman MG, Banks E, Hur HC. ACOG Simulation Working Group: A Needs Assessment of Simulation Training in OB/GYN Residencies and Recommendations for Future Research. JOURNAL OF SURGICAL EDUCATION 2020; 77:661-670. [PMID: 31859227 DOI: 10.1016/j.jsurg.2019.12.002] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/06/2019] [Revised: 10/29/2019] [Accepted: 12/01/2019] [Indexed: 06/10/2023]
Abstract
OBJECTIVE To evaluate current availability and needs of simulation training among obstetrics/gynecology (OB/GYN) residency programs. DESIGN Cross-sectional survey. SETTING Accreditation Council for Graduate Medical Education accredited OB/GYN residency programs in the United States. PARTICIPANTS Residency program directors, gynecology simulation faculty, obstetrics simulation faculty, and fourth-year residents. RESULTS Of 673 invited participants, 251 (37.3%) completed the survey. Among the survey responses, OB procedures were more broadly represented compared to the GYN procedures for simulation teaching: 8 (50%) of 16 OB procedures versus 4 (18.2%) of 22 GYN procedures had simulation teaching. Among the simulated procedures, a majority of residents and faculty reported that simulation teaching was available for operative vaginal delivery, postpartum hemorrhage, shoulder dystocia, perineal laceration repair, conventional laparoscopic procedures, and robotic surgery. There were significant differences between residents and faculty perceptions regarding the availability and needs of simulated procedures with a minority of residents having knowledge of Council on Resident Education in Obstetrics and Gynecology (47.2%) and American College of Obstetrics and Gynecology (27.8%) simulation tools compared to the majority of faculty (84.7% and 72.1%, respectively). More than 80% of trainees and faculty reported they felt the average graduating resident could perform vaginal, laparoscopic, and abdominal hysterectomies independently. CONCLUSIONS Simulation is now widely available for both gynecologic and obstetric procedures, but there remains tremendous heterogeneity between programs and the perceptions of residents, program directors, and faculty. The variations in simulation training and readiness for performing different procedures following residency support the need for objective, validated assessments of actual performance to better guide resident learning and faculty teaching efforts.
Collapse
Affiliation(s)
| | - Joshua F Nitsche
- Wake Forest School of Medicine Department of OB/GYN, Winston-Salem, North Carolina
| | - Michael G Heckman
- Mayo Clinic Department of Surgical Gynecology, Jacksonville, Florida; Mayo Clinic Division of Biomedical Statistics and Informatics, Jacksonville, Florida
| | - Erika Banks
- Department of Obstetrics and Gynecology and Women's Health, Albert Einstein College of Medicine, New York, New York
| | - Hye-Chun Hur
- Division of Gynecologic Specialty Surgery, Department of Obstetrics and Gynecology, New York Presbyterian Hospital, Columbia University Medical Center, New York, New York
| |
Collapse
|
11
|
Leijte E, Claassen L, Arts E, de Blaauw I, Rosman C, Botden SMBI. Training benchmarks based on validated composite scores for the RobotiX robot-assisted surgery simulator on basic tasks. J Robot Surg 2020; 15:69-79. [PMID: 32314094 PMCID: PMC7875949 DOI: 10.1007/s11701-020-01080-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2020] [Accepted: 04/15/2020] [Indexed: 12/14/2022]
Abstract
The RobotiX robot-assisted virtual reality simulator aims to aid in the training of novice surgeons outside of the operating room. This study aimed to determine the validity evidence on multiple levels of the RobotiX simulator for basic skills. Participants were divided in either the novice, laparoscopic or robotic experienced group based on their minimally invasive surgical experience. Two basic tasks were performed: wristed manipulation (Task 1) and vessel energy dissection (Task 2). The performance scores and a questionnaire regarding the realism, didactic value, and usability were gathered (content). Composite scores (0–100), pass/fail values, and alternative benchmark scores were calculated. Twenty-seven novices, 21 laparoscopic, and 13 robotic experienced participants were recruited. Content validity evidence was scored positively overall. Statistically significant differences between novices and robotic experienced participants (construct) was found for movements left (Task 1 p = 0.009), movements right (Task 1 p = 0.009, Task 2 p = 0.021), path length left (Task 1 p = 0.020), and time (Task 1 p = 0.040, Task 2 p < 0.001). Composite scores were statistically significantly different between robotic experienced and novice participants for Task 1 (85.5 versus 77.1, p = 0.044) and Task 2 (80.6 versus 64.9, p = 0.001). The pass/fail score with false-positive/false-negative percentage resulted in a value of 75/100, 46/9.1% (Task 1) and 71/100, 39/7.0% (Task 2). Calculated benchmark scores resulted in a minority of novices passing multiple parameters. Validity evidence on multiple levels was assessed for two basic robot-assisted surgical simulation tasks. The calculated benchmark scores can be used for future surgical simulation training.
Collapse
Affiliation(s)
- Erik Leijte
- Department of Surgery, Radboud University Medical Center, Geert Grooteplein 10 route 618, 6500HB, Nijmegen, The Netherlands. .,Department of Pediatric Surgery, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Linda Claassen
- Department of Surgery, Radboud University Medical Center, Geert Grooteplein 10 route 618, 6500HB, Nijmegen, The Netherlands
| | - Elke Arts
- Department of Surgery, Radboud University Medical Center, Geert Grooteplein 10 route 618, 6500HB, Nijmegen, The Netherlands
| | - Ivo de Blaauw
- Department of Surgery, Radboud University Medical Center, Geert Grooteplein 10 route 618, 6500HB, Nijmegen, The Netherlands.,Department of Pediatric Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Camiel Rosman
- Department of Surgery, Radboud University Medical Center, Geert Grooteplein 10 route 618, 6500HB, Nijmegen, The Netherlands
| | - Sanne M B I Botden
- Department of Surgery, Radboud University Medical Center, Geert Grooteplein 10 route 618, 6500HB, Nijmegen, The Netherlands.,Department of Pediatric Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|
12
|
Alshuaibi M, Perrenot C, Hubert J, Perez M. Concurrent, face, content, and construct validity of the RobotiX Mentor simulator for robotic basic skills. Int J Med Robot 2020; 16:e2100. [PMID: 32112491 DOI: 10.1002/rcs.2100] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2019] [Revised: 02/21/2020] [Accepted: 02/23/2020] [Indexed: 01/02/2023]
Abstract
OBJECTIVES To assess several criteria, such as concurrent, face, content, and construct validity of the RobotiX Mentor (RXM) simulator for basic robotic skills and to compare virtual and actual dry lab dome. METHODS A prospective study was conducted from December 2017 to May 2018 using RXM and a da Vinci Si robot. 37 subjects, divided into three groups according to their initial surgical training (expert, intermediate, and novice), were evaluated in terms of six representative exercises of basic robotic specific skills as recommended by the fundamentals of robotic surgery. RESULTS There was a correlation between the automatic data from the RXM and the subjective evaluation with the robot. The face and content validity, which were evaluated by the experts, were generally considered high (71.5% and 62.5%, respectively). Three levels (analysis of variance [ANOVA]; P = .01) and two levels (P = .001) of experience were clearly identified by the simulator. CONCLUSION Our study proves the concurrent validity and confirms the face, content, and construct validity of the RXM.
Collapse
Affiliation(s)
- Muaath Alshuaibi
- Department of Urology, Brabois Hospital, University Hospital of Nancy, Vandœuvre-lès-Nancy, France.,Department of Urology, Hospital University, Ha'il, Saudi Arabia
| | - Cyril Perrenot
- School of Surgery, Faculty of Medicine, Lorraine University, Vandœuvre-lès-Nancy, France.,Department of Visceral, Metabolic and Cancerology Surgery, Brabois Hospital, University Hospital of Nancy, Vandœuvre-lès-Nancy, France
| | - Jacques Hubert
- Department of Urology, Brabois Hospital, University Hospital of Nancy, Vandœuvre-lès-Nancy, France.,School of Surgery, Faculty of Medicine, Lorraine University, Vandœuvre-lès-Nancy, France.,IADI Laboratory, INSERM-U1254, Nancy University, Vandœuvre-lès-Nancy, France
| | - Manuela Perez
- Department of Visceral, Metabolic and Cancerology Surgery, Brabois Hospital, University Hospital of Nancy, Vandœuvre-lès-Nancy, France.,IADI Laboratory, INSERM-U947, Nancy University, Vandœuvre-lès-Nancy, France
| |
Collapse
|
13
|
NACUL MIGUELPRESTES, MELANI ARMANDOGERALDOFRANCHINI, ZILBERSTEIN BRUNO, BENEVENUTO DYEGOSÁ, CAVAZZOLA LEANDROTOTTI, ARAUJO RAPHAELL, SALLUM RUBENSANTONIOAISSAR, AGUIAR-JR SAMUEL, TOMASICH FLÁVIO. Educational note: teaching and training in robotic surgery. An opinion of the Minimally Invasive and Robotic Surgery Committee of the Brazilian College of Surgeons. Rev Col Bras Cir 2020; 47:e20202681. [DOI: 10.1590/0100-6991e-20202681] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2020] [Accepted: 06/19/2020] [Indexed: 12/18/2022] Open
Abstract
ABSTRACT With the expansion of robotic surgical procedures, the acquisition of specific knowledge and skills for surgeons to reach proficiency seems essential before performing surgical procedures on humans. In this sense, the authors present a proposal to establish a certification based on objective and validated criteria for carrying out robotic procedures. A study was carried out by the Committee on Minimally Invasive and Robotic Surgery of the Brazilian College of Surgeons based on a reviewing strategy of the scientific literature. The study serves as a reference for the creation of a standard for the qualification and certification in robotic surgery according to a statement of the Brazilian Medical Association (AMB) announced on December 17, 2019. The standard proposes a minimum curriculum, integrating training and performance evaluation. The initial (pre-clinical) stage aims at knowledge and adaptation to a specific robotic platform and the development of psychomotor skills based on surgical simulation. Afterwards, the surgeon must accompany in person at least five surgeries in the specialty, participate as a bedside assistant in at least 10 cases and perform 10 surgeries under the supervision of a preceptor surgeon. The surgeon who completes all the steps will be considered qualified in robotic surgery in his specialty. The final certification must be issued by the specialty societies affiliated to AMB. The authors conclude that the creation of a norm for habilitation in robotic surgery should encourage Brazilian hospitals to apply objective qualification criteria for this type of procedure to qualify assistance.
Collapse
Affiliation(s)
| | | | - BRUNO ZILBERSTEIN
- Colégio Brasileiro de Cirurgiões, Brasil; Universidade Federal de São Paulo, Brazil; Faculdade de Medicina SL Mandic, Brasil
| | | | - LEANDRO TOTTI CAVAZZOLA
- Colégio Brasileiro de Cirurgiões, Brasil; Hospital de Clínicas de Porto Alegre, Brasil; Universidade Federal do Rio Grande do Sul, Brazil
| | - RAPHAEL L.C. ARAUJO
- Colégio Brasileiro de Cirurgiões, Brasil; Universidade Federal de São Paulo, Brazil; Hospital Israelita Albert Einstein, Brazil
| | | | - SAMUEL AGUIAR-JR
- Colégio Brasileiro de Cirurgiões, Brasil; Hospital AC Camargo, Brazil
| | - FLÁVIO TOMASICH
- Colégio Brasileiro de Cirurgiões, Brasil; Universidade Federal do Paraná, Brazil
| |
Collapse
|
14
|
Leijte E, de Blaauw I, Van Workum F, Rosman C, Botden S. Robot assisted versus laparoscopic suturing learning curve in a simulated setting. Surg Endosc 2019; 34:3679-3689. [PMID: 31754849 PMCID: PMC7326898 DOI: 10.1007/s00464-019-07263-2] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2019] [Accepted: 11/11/2019] [Indexed: 12/13/2022]
Abstract
Background Compared to conventional laparoscopy, robot assisted surgery is expected to have most potential in difficult areas and demanding technical skills like minimally invasive suturing. This study was performed to identify the differences in the learning curves of laparoscopic versus robot assisted suturing. Method Novice participants performed three suturing tasks on the EoSim laparoscopic augmented reality simulator or the RobotiX robot assisted virtual reality simulator. Each participant performed an intracorporeal suturing task, a tilted plane needle transfer task and an anastomosis needle transfer task. To complete the learning curve, all tasks were repeated up to twenty repetitions or until a time plateau was reached. Clinically relevant and comparable parameters regarding time, movements and safety were recorded. Intracorporeal suturing time and cumulative sum analysis was used to compare the learning curves and phases. Results Seventeen participants completed the learning curve laparoscopically and 30 robot assisted. Median first knot suturing time was 611 s (s) for laparoscopic versus 251 s for robot assisted (p < 0.001), and this was 324 s versus 165 (sixth knot, p < 0.001) and 257 s and 149 s (eleventh knot, p < 0.001) respectively on base of the found learning phases. The percentage of ‘adequate surgical knots’ was higher in the laparoscopic than in the robot assisted group. First knot: 71% versus 60%, sixth knot: 100% versus 83%, and eleventh knot: 100% versus 73%. When assessing the ‘instrument out of view’ parameter, the robot assisted group scored a median of 0% after repetition four. In the laparoscopic group, the instrument out of view increased from 3.1 to 3.9% (left) and from 3.0 to 4.1% (right) between the first and eleventh knot (p > 0.05). Conclusion The learning curve of minimally invasive suturing shows a shorter task time curve using robotic assistance compared to the laparoscopic curve. However, laparoscopic outcomes show good end results with rapid outcome improvement. Electronic supplementary material The online version of this article (10.1007/s00464-019-07263-2) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Erik Leijte
- Department of Pediatric Surgery, Radboud University Medical Centre - Amalia Children's Hospital, Geert Grooteplein 10 Route 618, 6500HB, Nijmegen, The Netherlands.
| | - Ivo de Blaauw
- Department of Pediatric Surgery, Radboud University Medical Centre - Amalia Children's Hospital, Geert Grooteplein 10 Route 618, 6500HB, Nijmegen, The Netherlands
| | - Frans Van Workum
- Department of Surgery, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Camiel Rosman
- Department of Surgery, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Sanne Botden
- Department of Pediatric Surgery, Radboud University Medical Centre - Amalia Children's Hospital, Geert Grooteplein 10 Route 618, 6500HB, Nijmegen, The Netherlands
| |
Collapse
|
15
|
Hvolbek AP, Nilsson PM, Sanguedolce F, Lund L. A prospective study of the effect of video games on robotic surgery skills using the high-fidelity virtual reality RobotiX simulator. ADVANCES IN MEDICAL EDUCATION AND PRACTICE 2019; 10:627-634. [PMID: 31616197 PMCID: PMC6699361 DOI: 10.2147/amep.s199323] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/23/2018] [Accepted: 06/15/2019] [Indexed: 05/05/2023]
Abstract
BACKGROUND Robot-assisted surgery is a growing field. Prior video game experience might give advantage to novice robotic surgeons. AIM Assessing if prior video gaming experience gives advantage in performing high-fidelity virtual reality (VR)-simulated robotic surgery. METHODS In this observational study, 30 medical students and 2 interns (17 females; 15 males) with median age 25 years (range, 24-26 years) were recruited and subsequently divided into groups according to prior gaming experience; gamers (≥6 video game hours/week) vs nongamers (<6 video game hours/week). Participants performed VR-simulated urethrovesical anastomosis on RobotiX Mentor, which measured performance parameters. Participants answered a questionnaire for demographics and gaming experience. Groups were compared using Mann-Whitney U and multiple regression. RESULTS Gamers significantly outperformed nongamers in 3 of 24 performance metrics (p<0.05), and there was a trend toward better results for 7 of the 21 remaining metrics. Males outperformed females in 5 of 24 metrics (p<0.05) but were overrepresented among gamers. CONCLUSION Prior video game experience >6 hrs/week might give advantage in simulated robotic surgery. We recommend future studies testing this hypothesis to develop simulator programs for certification of robotic surgeons.
Collapse
Affiliation(s)
| | - Philip Mørkeberg Nilsson
- Copenhagen Academy for Medical Education and Simulation, Rigshospitalet, University of Copenhagen and the Capital Region of Denmark, Copenhagen, Denmark
| | | | - Lars Lund
- Department of Urology, Odense University Hospital, Odense, Denmark
- Department of Urology, Aalborg University Hospital, Aalborg, Denmark
- Department of Clinical Research, University of Southern Denmark, Odense, Denmark
- Correspondence: Lars LundDepartment of Urology, Odense University Hospital, J.B. Winsloew Vej 4, Entrance 20, Penthouse, 2. Floor, OdenseC DK-5000, DenmarkTel +45 5 140 8982Email
| |
Collapse
|