1
|
Soenens G, Gorden L, Doyen B, Wheatcroft M, de Mestral C, Palter V, Van Herzeele I. Editor's Choice - Development and Testing of Step, Error, and Event Frameworks to Evaluate Technical Performance in Peripheral Endovascular Interventions. Eur J Vasc Endovasc Surg 2024; 68:227-235. [PMID: 38492630 DOI: 10.1016/j.ejvs.2024.03.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Revised: 12/22/2023] [Accepted: 03/11/2024] [Indexed: 03/18/2024]
Abstract
OBJECTIVE Tools for endovascular performance assessment are necessary in competency based education. This study aimed to develop and test a detailed analysis tool to assess steps, errors, and events in peripheral endovascular interventions (PVI). METHODS A modified Delphi consensus was used to identify steps, errors, and events in iliac-femoral-popliteal endovascular interventions. International experts in vascular surgery, interventional radiology, cardiology, and angiology were identified, based on their scientific track record. In an initial open ended survey round, experts volunteered a comprehensive list of steps, errors, and events. The items were then rated on a five point Likert scale until consensus was reached with a pre-defined threshold (Cronbach's alpha > 0.7) and > 70% expert agreement. An experienced endovascular surgeon applied the finalised frameworks on 10 previously videorecorded elective PVI cases. RESULTS The expert consensus panel was formed by 28 of 98 invited proceduralists, consisting of three angiologists, seven interventional radiologists, five cardiologists, and 13 vascular surgeons, with 29% from North America and 71% from Europe. The Delphi process was completed after three rounds (Cronbach's alpha; αsteps = 0.79; αerrors = 0.90; αevents = 0.90), with 15, 26, and 18 items included in the final step (73 - 100% agreement), error (73 - 100% agreement), and event (73 - 100% agreement) frameworks, respectively. The median rating time per case was 4.3 hours (interquartile range [IQR] 3.2, 5 hours). A median of 55 steps (IQR 40, 67), 27 errors (IQR 21, 49), and two events (IQR 1, 6) were identified per case. CONCLUSION An evaluation tool for the procedural steps, errors, and events in iliac-femoral-popliteal endovascular procedures was developed through a modified Delphi consensus and applied to recorded intra-operative data to identify hazardous steps, common errors, and events. Procedural mastery may be promoted by using the frameworks to provide endovascular proceduralists with detailed technical performance feedback.
Collapse
Affiliation(s)
- Gilles Soenens
- Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium. https://www.twitter.com/GillesSoenens
| | - Lauren Gorden
- Division of Vascular Surgery, University of Toronto, Canada; International Centre for Surgical Safety, Li Ka Shing Knowledge Institute of St. Michael's Hospital, Toronto, Canada; Biomedical Engineering (BME), University of Toronto, Canada
| | - Bart Doyen
- Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium
| | - Mark Wheatcroft
- Division of Vascular Surgery, University of Toronto, Canada; Department of Surgery, St. Michael's Hospital, Toronto, Canada
| | - Charles de Mestral
- Division of Vascular Surgery, University of Toronto, Canada; Department of Surgery, St. Michael's Hospital, Toronto, Canada
| | - Vanessa Palter
- International Centre for Surgical Safety, Li Ka Shing Knowledge Institute of St. Michael's Hospital, Toronto, Canada
| | - Isabelle Van Herzeele
- Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium.
| |
Collapse
|
2
|
Tesfai FM, Nagi J, Morrison I, Boal M, Olaitan A, Chandrasekaran D, Stoyanov D, Lanceley A, Francis N. Objective assessment tools in laparoscopic or robotic-assisted gynecological surgery: A systematic review. Acta Obstet Gynecol Scand 2024; 103:1480-1497. [PMID: 38610108 PMCID: PMC11266631 DOI: 10.1111/aogs.14840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 02/28/2024] [Accepted: 03/19/2024] [Indexed: 04/14/2024]
Abstract
INTRODUCTION There is a growing emphasis on proficiency-based progression within surgical training. To enable this, clearly defined metrics for those newly acquired surgical skills are needed. These can be formulated in objective assessment tools. The aim of the present study was to systematically review the literature reporting on available tools for objective assessment of minimally invasive gynecological surgery (simulated) performance and evaluate their reliability and validity. MATERIAL AND METHODS A systematic search (1989-2022) was conducted in MEDLINE, Embase, PubMed, Web of Science in accordance with PRISMA. The trial was registered with the Prospective Register of Systematic Reviews (PROSPERO) ID: CRD42022376552. Randomized controlled trials, prospective comparative studies, prospective single-group (with pre- and post-training assessment) or consensus studies that reported on the development, validation or usage of assessment tools of surgical performance in minimally invasive gynecological surgery, were included. Three independent assessors assessed study setting and validity evidence according to a contemporary framework of validity, which was adapted from Messick's validity framework. Methodological quality of included studies was assessed using the modified medical education research study quality instrument (MERSQI) checklist. Heterogeneity in data reporting on types of tools, data collection, study design, definition of expertise (novice vs. experts) and statistical values prevented a meaningful meta-analysis. RESULTS A total of 19 746 titles and abstracts were screened of which 72 articles met the inclusion criteria. A total of 37 different assessment tools were identified of which 13 represented manual global assessment tools, 13 manual procedure-specific assessment tools and 11 automated performance metrices. Only two tools showed substantive evidence of validity. Reliability and validity per tool were provided. No assessment tools showed direct correlation between tool scores and patient related outcomes. CONCLUSIONS Existing objective assessment tools lack evidence on predicting patient outcomes and suffer from limitations in transferability outside of the research environment, particularly for automated performance metrics. Future research should prioritize filling these gaps while integrating advanced technologies like kinematic data and AI for robust, objective surgical skill assessment within gynecological advanced surgical training programs.
Collapse
Affiliation(s)
- Freweini Martha Tesfai
- The Griffin InstituteNorthwick Park & St Marks' HospitalLondonUK
- EGA Institute for Women's HealthUniversity College LondonLondonUK
- Wellcome/EPSRC Center for Interventional and Surgical Sciences (WEISS)University College LondonLondonUK
| | | | - Iona Morrison
- Yeovil District HospitalSomerset Foundation NHS TrustYeovilUK
| | - Matt Boal
- The Griffin InstituteNorthwick Park & St Marks' HospitalLondonUK
- EGA Institute for Women's HealthUniversity College LondonLondonUK
- Wellcome/EPSRC Center for Interventional and Surgical Sciences (WEISS)University College LondonLondonUK
| | | | - Dhivya Chandrasekaran
- EGA Institute for Women's HealthUniversity College LondonLondonUK
- Department of Gynecological OncologyUniversity College of London HospitalsLondonUK
| | - Danail Stoyanov
- EGA Institute for Women's HealthUniversity College LondonLondonUK
- Wellcome/EPSRC Center for Interventional and Surgical Sciences (WEISS)University College LondonLondonUK
| | - Anne Lanceley
- EGA Institute for Women's HealthUniversity College LondonLondonUK
| | - Nader Francis
- The Griffin InstituteNorthwick Park & St Marks' HospitalLondonUK
- EGA Institute for Women's HealthUniversity College LondonLondonUK
- Yeovil District HospitalSomerset Foundation NHS TrustYeovilUK
| |
Collapse
|
3
|
Gorard J, Boal M, Swamynathan V, Ghamrawi W, Francis N. The application of objective clinical human reliability analysis (OCHRA) in the assessment of basic robotic surgical skills. Surg Endosc 2024; 38:116-128. [PMID: 37932602 PMCID: PMC10776495 DOI: 10.1007/s00464-023-10510-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Accepted: 10/01/2023] [Indexed: 11/08/2023]
Abstract
BACKGROUND Using a validated, objective, and standardised assessment tool to assess progression and competency is essential for basic robotic surgical training programmes. Objective clinical human reliability analysis (OCHRA) is an error-based assessment tool that provides in-depth analysis of individual technical errors. We conducted a feasibility study to assess the concurrent validity and reliability of OCHRA when applied to basic, generic robotic technical skills assessment. METHODS Selected basic robotic surgical skill tasks, in virtual reality (VR) and dry lab equivalent, were performed by novice robotic surgeons during an intensive 5-day robotic surgical skills course on da Vinci® X and Xi surgical systems. For each task, we described a hierarchical task analysis. Our developed robotic surgical-specific OCHRA methodology was applied to error events in recorded videos with a standardised definition. Statistical analysis to assess concurrent validity with existing tools and inter-rater reliability were performed. RESULTS OCHRA methodology was applied to 272 basic robotic surgical skills tasks performed by 20 novice robotic surgeons. Performance scores improved from the start of the course to the end using all three assessment tools; Global Evaluative Assessment of Robotic Skills (GEARS) [VR: t(19) = - 9.33, p < 0.001] [dry lab: t(19) = - 10.17, p < 0.001], OCHRA [VR: t(19) = 6.33, p < 0.001] [dry lab: t(19) = 10.69, p < 0.001] and automated VR [VR: t(19) = - 8.26, p < 0.001]. Correlation analysis, for OCHRA compared to GEARS and automated VR scores, shows a significant and strong inverse correlation in every VR and dry lab task; OCHRA vs GEARS [VR: mean r = - 0.78, p < 0.001] [dry lab: mean r = - 0.82, p < 0.001] and OCHRA vs automated VR [VR: mean r = - 0.77, p < 0.001]. There is very strong and significant inter-rater reliability between two independent reviewers (r = 0.926, p < 0.001). CONCLUSION OCHRA methodology provides a detailed error analysis tool in basic robotic surgical skills with high reliability and concurrent validity with existing tools. OCHRA requires further evaluation in more advanced robotic surgical procedures.
Collapse
Affiliation(s)
- Jack Gorard
- Division of Surgery & Interventional Science, Royal Free Hospital Campus, University College London, London, UK
| | - Matthew Boal
- Division of Surgery & Interventional Science, Royal Free Hospital Campus, University College London, London, UK
- The Griffin Institute, Northwick Park and St Mark's Hospital, London, UK
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, Charles Bell House, University College London, London, UK
| | - Vishaal Swamynathan
- Division of Surgery & Interventional Science, Royal Free Hospital Campus, University College London, London, UK
| | - Walaa Ghamrawi
- Division of Surgery & Interventional Science, Royal Free Hospital Campus, University College London, London, UK
- The Griffin Institute, Northwick Park and St Mark's Hospital, London, UK
| | - Nader Francis
- Division of Surgery & Interventional Science, Royal Free Hospital Campus, University College London, London, UK.
- The Griffin Institute, Northwick Park and St Mark's Hospital, London, UK.
| |
Collapse
|
4
|
De Mol L, Lievens A, De Pauw N, Vanommeslaeghe H, Van Herzeele I, Van de Voorde P, Konge L, Desender L, Willaert W. Assessing Chest Tube Insertion Skills Using a Porcine Rib Model-A Validity Study. Simul Healthc 2023:01266021-990000000-00087. [PMID: 37782127 DOI: 10.1097/sih.0000000000000750] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/03/2023]
Abstract
INTRODUCTION Assessments require sufficient validity evidence before their use. The Assessment for Competence in Chest Tube Insertion (ACTION) tool evaluates proficiency in chest tube insertion (CTI), combining a rating scale and an error checklist. The aim of this study was to collect validity evidence for the ACTION tool on a porcine rib model according to the Messick framework. METHODS A rib model, consisting of a porcine hemithorax that was placed in a wooden frame, was used as simulator. Participants were recruited from the departments of surgery, pulmonology, and emergency medicine. After familiarization with the rib model and the equipment, standardized instructions and clinical context were provided. They performed 2 CTIs while being scored with the ACTION tool. All performances were assessed live by 1 rater and by 3 blinded raters using video recordings. Generalizability-analysis was performed and mean scores and errors of both groups on the first performance were compared. A pass/fail score was established using the contrasting groups' method. RESULTS Nine novice and 8 experienced participants completed the study. Generalizability coefficients where high for the rating scale (0.92) and the error checklist (0.87). In the first CTI, novices scored lower than the experienced group (38.1/68 vs. 47.1/68, P = 0.042), but no difference was observed on the error checklist. A pass/fail score of 44/68 was established. CONCLUSION A solid validity argument for the ACTION tool's rating scale on a porcine rib model is presented, allowing formative and summative assessment of procedural skills during training before patient contact.
Collapse
Affiliation(s)
- Leander De Mol
- From the Department of Human Structure and Repair (L.D.M., A.L., N.D.P., I.V.H., L.D., W.W.), Faculty of Medicine and Health Sciences, Ghent University, Ghent, Belgium; Department of Gastrointestinal Surgery (H.V., W.W.), Ghent University Hospital, Ghent, Belgium; Department of Thoracic and Vascular Surgery (I.V.H., L.D.), Ghent University Hospital, Ghent, Belgium; Department of Basic and Applied Medical Sciences (P.V.d.V.), Faculty of Medicine and Health Sciences, Ghent University, Ghent, Belgium; Department of Emergency Medicine (P.V.d.V.), Ghent University Hospital, Ghent, Belgium; Faculty of Health and Medical Sciences, (L.K.) University of Copenhagen, Copenhagen, Denmark; and Copenhagen Academy for Medical Education and Simulation (CAMES) (L.K.), Copenhagen, Denmark
| | | | | | | | | | | | | | | | | |
Collapse
|
5
|
Grüter AAJ, Van Lieshout AS, van Oostendorp SE, Henckens SPG, Ket JCF, Gisbertz SS, Toorenvliet BR, Tanis PJ, Bonjer HJ, Tuynman JB. Video-based tools for surgical quality assessment of technical skills in laparoscopic procedures: a systematic review. Surg Endosc 2023:10.1007/s00464-023-10076-z. [PMID: 37099157 DOI: 10.1007/s00464-023-10076-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Accepted: 04/08/2023] [Indexed: 04/27/2023]
Abstract
BACKGROUND Quality of surgery has substantial impact on both short- and long-term clinical outcomes. This stresses the need for objective surgical quality assessment (SQA) for education, clinical practice and research purposes. The aim of this systematic review was to provide a comprehensive overview of all video-based objective SQA tools in laparoscopic procedures and their validity to objectively assess surgical performance. METHODS PubMed, Embase.com and Web of Science were systematically searched by two reviewers to identify all studies focusing on video-based SQA tools of technical skills in laparoscopic surgery performed in a clinical setting. Evidence on validity was evaluated using a modified validation scoring system. RESULTS Fifty-five studies with a total of 41 video-based SQA tools were identified. These tools were used in 9 different fields of laparoscopic surgery and were divided into 4 categories: the global assessment scale (GAS), the error-based assessment scale (EBAS), the procedure-specific assessment tool (PSAT) and artificial intelligence (AI). The number of studies focusing on these four categories were 21, 6, 31 and 3, respectively. Twelve studies validated the SQA tool with clinical outcomes. In 11 of those studies, a positive association between surgical quality and clinical outcomes was found. CONCLUSION This systematic review included a total of 41 unique video-based SQA tools to assess surgical technical skills in various domains of laparoscopic surgery. This study suggests that validated SQA tools enable objective assessment of surgical performance with relevance for clinical outcomes, which can be used for training, research and quality improvement programs.
Collapse
Affiliation(s)
- Alexander A J Grüter
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands.
- Cancer Center Amsterdam, Treatment and Quality of Life, Amsterdam, The Netherlands.
| | - Annabel S Van Lieshout
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Treatment and Quality of Life, Amsterdam, The Netherlands
| | - Stefan E van Oostendorp
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands
- Department of Surgery, Rode Kruis Ziekenhuis, Vondellaan 13, Beverwijk, The Netherlands
| | - Sofie P G Henckens
- Cancer Center Amsterdam, Treatment and Quality of Life, Amsterdam, The Netherlands
- Department of Surgery, Amsterdam UMC Location University of Amsterdam, Meibergdreef 9, Amsterdam, The Netherlands
| | - Johannes C F Ket
- Medical Library, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Suzanne S Gisbertz
- Department of Surgery, Amsterdam UMC Location University of Amsterdam, Meibergdreef 9, Amsterdam, The Netherlands
| | | | - Pieter J Tanis
- Department of Surgery, Amsterdam UMC Location University of Amsterdam, Meibergdreef 9, Amsterdam, The Netherlands
- Department of Surgical Oncology and Gastrointestinal Surgery, Erasmus MC, Doctor Molewaterplein 40, Rotterdam, The Netherlands
| | - Hendrik J Bonjer
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands
| | - Jurriaan B Tuynman
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands
| |
Collapse
|
6
|
Doyen B, Soenens G, Maurel B, Hertault A, Gordon L, Vlerick P, Vermassen F, Grantcharov T, van Herzeele I. Assessing endovascular team performances in a hybrid room using the Black Box system: a prospective cohort study. THE JOURNAL OF CARDIOVASCULAR SURGERY 2023; 64:82-92. [PMID: 36168949 DOI: 10.23736/s0021-9509.22.12226-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
BACKGROUND The hybrid room (HR) is a complex, high-risk environment, requiring teams (surgeons, anesthesiologists, nurses, technologists) to master various skills, including the 'As Low As Reasonably Achievable' principle of radiation safety. This prospective single center cohort reports the first use of the Operating Room Black Box (ORBB) in a HR. This medical data recording system captures procedural and audio-visual data to facilitate structured team performance analysis. METHODS Patients planned for endovascular repair of an infrarenal abdominal aortic aneurysm (EVAR) or treatment of symptomatic iliac-femoral-popliteal atherosclerotic disease (Peripheral Vascular Interventions or PVI) were included. Validated measures and established assessment tools were used to assess (non-)technical skills, radiation safety performance and environmental distractions. RESULTS Six EVAR and sixteen PVI procedures were captured. Technical performance for one EVAR was rated 19/35 on the procedure-specific scale, below the 'acceptable' score of 21. Technical skills were rated above acceptable in all PVI procedures. Shared decision making and leadership were rated highly in 12/22 cases, whereas surgical communication and nurses' task management were rated low in 14/22 cases. Team members rarely stepped back from the C-arm during digital subtraction angiography. Radiation safety behavior was scored below 'acceptable' in 14/22 cases. A median (interquartile range) number of 12 (6-23) auditory distractions was observed per procedure. CONCLUSIONS The ORBB facilitates holistic workplace-based assessment of endovascular performance in a HR by combining objective assessment parameters and rating scale-based evaluations. Strengths and weaknesses were identified in team members' (non-)technical and radiation safety practices. This technology has the potential to improve vascular surgical practice, though human input remains crucial. (NCT04854278).
Collapse
Affiliation(s)
- Bart Doyen
- Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium
| | - Gilles Soenens
- Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium
| | - Blandine Maurel
- Department of Vascular Surgery, University Hospital Centre of Nantes, Nantes, France
| | - Adrien Hertault
- Department of Vascular Surgery, Valenciennes General Hospital, Valenciennes, France
| | - Lauren Gordon
- Department of Surgery, University of Toronto, Toronto, ON, Canada
| | - Peter Vlerick
- Department of Work, Organization and Society, Ghent University, Ghent, Belgium
| | - Frank Vermassen
- Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium
| | - Teodor Grantcharov
- Department of Surgery, Stanford University, Stanford, CA, USA.,Clinical Excellence Research Center, Stanford University, Stanford, CA, USA
| | - Isabelle van Herzeele
- Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium -
| |
Collapse
|
7
|
Koskinen J, He W, Elomaa AP, Kaipainen A, Hussein A, Zheng B, Huotarinen A, Bednarik R. Utilizing Grasp Monitoring to Predict Microsurgical Expertise. J Surg Res 2023; 282:101-108. [PMID: 36265429 DOI: 10.1016/j.jss.2022.09.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Revised: 07/22/2022] [Accepted: 09/18/2022] [Indexed: 11/07/2022]
Abstract
INTRODUCTION Most microsurgical procedures require the surgeon to use tools to grasp and hold fragile objects in the surgical site. Prior research on grasping in surgery has mostly either been in other surgical techniques or used grasping as an auxiliary metric. We focus on microsurgery and investigate what grasping can tell about microsurgical skill and suturing performance. This study lays groundwork for using automatic detection of grasps to evaluate surgical skill. METHODS Five expert surgeons and six novices completed sutures on a microsurgical training board. Video recordings of the performance were annotated for the number of grasps, while an eye tracker recorded the participants' pupil dilations for cognitive workload assessment. Performance was measured with suturing duration and the University of Western Ontario Microsurgical Skills Assessment instrument (UWOMSA). Differences in skill, suturing performance and cognitive workload were compared with grasping behavior. RESULTS Novices needed significantly more grasps to complete sutures and failed to grasp more often than the experts. The number of grasps affected the suturing duration more in novices. Decreasing suturing efficiency as measured by UWOMSA instrument was associated with increase in grasps, even when we controlled for overall skill differences. Novices displayed larger pupil dilations when averaged over a sufficiently large sample, and the difference increased after the grasp. CONCLUSIONS Grasping action during microsurgical procedures can be used as a conceptually simple yet objective proxy in microsurgical performance assessment. If the grasps could be detected automatically, they could be used to aid in computational evaluation of surgical trainees' performance.
Collapse
Affiliation(s)
- Jani Koskinen
- School of Computing, University of Eastern Finland, Joensuu, Finland.
| | - Wenjing He
- Department of Surgery, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Manitoba, Canada
| | - Antti-Pekka Elomaa
- Department of Neurosurgery, Institute of Clinical Medicine, Kuopio University Hospital, Kuopio, Finland; Microsurgery Center, Kuopio University Hospital, Kuopio, Finland; School of Medicine, University of Eastern Finland, Kuopio, Finland
| | - Aku Kaipainen
- Department of Neurosurgery, Institute of Clinical Medicine, Kuopio University Hospital, Kuopio, Finland; Microsurgery Center, Kuopio University Hospital, Kuopio, Finland
| | - Ahmed Hussein
- Department of Neurosurgery, Institute of Clinical Medicine, Kuopio University Hospital, Kuopio, Finland; Microsurgery Center, Kuopio University Hospital, Kuopio, Finland
| | - Bin Zheng
- Surgical Simulation Research Lab, Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | - Antti Huotarinen
- Department of Neurosurgery, Institute of Clinical Medicine, Kuopio University Hospital, Kuopio, Finland; Microsurgery Center, Kuopio University Hospital, Kuopio, Finland
| | - Roman Bednarik
- School of Computing, University of Eastern Finland, Joensuu, Finland
| |
Collapse
|
8
|
Daniel R, McKechnie T, Kruse CC, Levin M, Lee Y, Doumouras AG, Hong D, Eskicioglu C. Video-based coaching for surgical residents: a systematic review and meta-analysis. Surg Endosc 2023; 37:1429-1439. [PMID: 35739431 PMCID: PMC9225812 DOI: 10.1007/s00464-022-09379-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Accepted: 06/03/2022] [Indexed: 11/28/2022]
Abstract
BACKGROUND Video-based coaching (VBC) is used to supplement current teaching methods in surgical education and may be useful in competency-based frameworks. Whether VBC can effectively improve surgical skill in surgical residents has yet to be fully elucidated. The objective of this study is to compare surgical residents receiving and not receiving VBC in terms of technical surgical skill. METHODS The following databases were searched from database inception to October 2021: Medline, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL), and PubMed. Articles were included if they were randomized controlled trials (RCTs) comparing surgical residents receiving and not receiving VBC. The primary outcome, as defined prior to data collection, was change in objective measures of technical surgical skill following implementation of either VBC or control. A pairwise meta-analyses using inverse variance random effects was performed. Standardized mean differences (SMD) were used as the primary outcome measure to account for differences in objective surgical skill evaluation tools. RESULTS From 2734 citations, 11 RCTs with 157 residents receiving VBC and 141 residents receiving standard surgical teaching without VBC were included. There was no significant difference in post-coaching scores on objective surgical skill evaluation tools between groups (SMD 0.53, 95% CI 0.00 to 1.01, p = 0.05, I2 = 74%). The improvement in scores pre- and post-intervention was significantly greater in residents receiving VBC compared to those not receiving VBC (SMD 1.62, 95% CI 0.62 to 2.63, p = 0.002, I2 = 85%). These results were unchanged with leave-one-out sensitivity analysis and subgroup analysis according to operative setting. CONCLUSION VBC can improve objective surgical skills in surgical residents of various levels. The benefit may be most substantial for trainees with lower baseline levels of objective skill. Further studies are required to determine the impact of VBC on competency-based frameworks.
Collapse
Affiliation(s)
- Ryan Daniel
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON Canada
| | - Tyler McKechnie
- Division of General Surgery, Department of Surgery, St. Joseph’s Healthcare Hamilton, McMaster University, 50 Charlton Avenue East, Hamilton, ON L8N 4A6 Canada
| | - Colin C. Kruse
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON Canada
| | - Marc Levin
- Division of Head and Neck, Otolaryngology Surgery, Department of Surgery, University of Toronto, Toronto, ON Canada
| | - Yung Lee
- Division of General Surgery, Department of Surgery, St. Joseph’s Healthcare Hamilton, McMaster University, 50 Charlton Avenue East, Hamilton, ON L8N 4A6 Canada
| | - Aristithes G. Doumouras
- Division of General Surgery, Department of Surgery, St. Joseph’s Healthcare Hamilton, McMaster University, 50 Charlton Avenue East, Hamilton, ON L8N 4A6 Canada ,Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON Canada ,Division of General Surgery, Department of Surgery, St. Joseph’s Healthcare Hamilton, Hamilton, ON Canada
| | - Dennis Hong
- Division of General Surgery, Department of Surgery, St. Joseph’s Healthcare Hamilton, McMaster University, 50 Charlton Avenue East, Hamilton, ON L8N 4A6 Canada ,Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON Canada ,Division of General Surgery, Department of Surgery, St. Joseph’s Healthcare Hamilton, Hamilton, ON Canada
| | - Cagla Eskicioglu
- Division of General Surgery, Department of Surgery, St. Joseph's Healthcare Hamilton, McMaster University, 50 Charlton Avenue East, Hamilton, ON, L8N 4A6, Canada. .,Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada. .,Division of General Surgery, Department of Surgery, St. Joseph's Healthcare Hamilton, Hamilton, ON, Canada.
| |
Collapse
|
9
|
Hybrid Spatiotemporal Contrastive Representation Learning for Content-Based Surgical Video Retrieval. ELECTRONICS 2022. [DOI: 10.3390/electronics11091353] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
In the medical field, due to their economic and clinical benefits, there is a growing interest in minimally invasive surgeries and microscopic surgeries. These types of surgeries are often recorded during operations, and these recordings have become a key resource for education, patient disease analysis, surgical error analysis, and surgical skill assessment. However, manual searching in this collection of long-term surgical videos is an extremely labor-intensive and long-term task, requiring an effective content-based video analysis system. In this regard, previous methods for surgical video retrieval are based on handcrafted features which do not represent the video effectively. On the other hand, deep learning-based solutions were found to be effective in both surgical image and video analysis, where CNN-, LSTM- and CNN-LSTM-based methods were proposed in most surgical video analysis tasks. In this paper, we propose a hybrid spatiotemporal embedding method to enhance spatiotemporal representations using an adaptive fusion layer on top of the LSTM and temporal causal convolutional modules. To learn surgical video representations, we propose exploring the supervised contrastive learning approach to leverage label information in addition to augmented versions. By validating our approach to a video retrieval task on two datasets, Surgical Actions 160 and Cataract-101, we significantly improve on previous results in terms of mean average precision, 30.012 ± 1.778 vs. 22.54 ± 1.557 for Surgical Actions 160 and 81.134 ± 1.28 vs. 33.18 ± 1.311 for Cataract-101. We also validate the proposed method’s suitability for surgical phase recognition task using the benchmark Cholec80 surgical dataset, where our approach outperforms (with 90.2% accuracy) the state of the art.
Collapse
|
10
|
Video-Based Coaching: Current Status and Role in Surgical Practice (Part 1) From the Society for Surgery of the Alimentary Tract, Health Care Quality and Outcomes Committee. J Gastrointest Surg 2021; 25:2439-2446. [PMID: 34355331 DOI: 10.1007/s11605-021-05102-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/14/2021] [Accepted: 07/21/2021] [Indexed: 01/31/2023]
Abstract
Patient safety and outcomes are directly related to surgical performance. Surgical training emphasizes the importance of the surgeon in determining these outcomes. After training is complete, there is a lack of structured programs for surgeons to audit their skills and continue their individual development. There is a significant linear relationship between surgeon technical skill and surgical outcomes; however, measuring technical performance is difficult. Video-based coaching matches an individual surgeon in practice with a surgical colleague who has been trained in the core principles of coaching for individualizing instruction. It can provide objective assessment for teaching higher-level concepts, such as technical skills, cognitive skills, and decision-making. There are many benefits to video-based coaching. While the concept is gaining acceptance as a method of surgical education, it is still novel in clinical practice. As more surgeons look towards video-based coaching for quality improvement, a consistent definition of the program, goals, and metrics for assessment will be critical. This paper is a review on the status of the video-based coaching as it applies to practicing surgeons.
Collapse
|
11
|
Hennings LI, Sørensen JL, Hybscmann J, Strandbygaard J. Tools for measuring technical skills during gynaecologic surgery: a scoping review. BMC MEDICAL EDUCATION 2021; 21:402. [PMID: 34311735 PMCID: PMC8314568 DOI: 10.1186/s12909-021-02790-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/01/2020] [Accepted: 06/09/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND Standardised assessment is key to structured surgical training. Currently, there is no consensus on which surgical assessment tool to use in live gynaecologic surgery. The purpose of this review is to identify assessment tools measuring technical skills in gynaecologic surgery and evaluate the measurement characteristics of each tool. METHOD We utilized the scoping review methodology and searched PubMed, Medline, Embase and Cochrane. Inclusion criteria were studies that analysed assessment tools in live gynaecologic surgery. Kane's validity argument was applied to evaluate the assessment tools in the included studies. RESULTS Eight studies out of the 544 identified fulfilled the inclusion criteria. The assessment tools were categorised as global rating scales, global and procedure rating scales combined, procedure-specific rating scales or as a non-procedure-specific error assessment tool. CONCLUSION This scoping review presents the current different tools for observational assessment of technical skills in intraoperative, gynaecologic surgery. This scoping review can serve as a guide for surgical educators who want to apply a scale or a specific tool in surgical assessment.
Collapse
Affiliation(s)
| | - Jette Led Sørensen
- Juliane Marie Centre for children, women and reproduction, Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - Jane Hybscmann
- Juliane Marie Centre for children, women and reproduction, Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | | |
Collapse
|
12
|
Rastegari E, Orn D, Zahiri M, Nelson C, Ali H, Siu KC. Assessing Laparoscopic Surgical Skills Using Similarity Network Models: A Pilot Study. Surg Innov 2021; 28:600-610. [PMID: 33745371 DOI: 10.1177/15533506211002753] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Background: Medical devices are becoming more complex, and doctors need to learn quickly how to use new medical tools. However, it is challenging to objectively assess the fundamental laparoscopic surgical skill level and determine skill readiness for advancement. There is a lack of objective models to compare performance between medical trainees and experienced doctors. Methods: This article discusses the use of similarity network models for individual tasks and a combination of tasks to show the level of similarity between residents and medical students while performing each task and their overall laparoscopic surgical skill level using a medical device (eg laparoscopic instruments). When a medical student is connected to most residents, that student is competent to the next training level. Performance of sixteen participants (5 residents and 11 students) while performing 3 tasks in 3 different training schedules is used in this study. Results: The promising result shows the general positive progression of students over 4 training sessions. Our results also indicate that students with different training schedules have different performance levels. Students' progress in performing a task is quicker if the training sessions are held more closely compared to when the training sessions are far apart in time. Conclusions: This study provides a graph-based framework for evaluating new learners' performance on medical devices and their readiness for advancement. This similarity network method could be used to classify students' performance using similarity thresholds, facilitating decision-making related to training and progression through curricula.
Collapse
Affiliation(s)
- Elham Rastegari
- Department of Business Intelligence and Analytics, 6216Creighton University, Omaha, NE, USA
| | - Donovan Orn
- College of Information Science and Technology, 14720University of Nebraska at Omaha, Omaha, NE, USA
| | - Mohsen Zahiri
- Senior Research Scientist, BioSensics LLC, Watertown, MA, USA
| | - Carl Nelson
- Department of Mechanical and Materials Engineering, 14719University of Nebraska-Lincoln, Lincoln, NE, USA
| | - Hesham Ali
- College of Information Science and Technology, 14720University of Nebraska at Omaha, Omaha, NE, USA
| | - Ka-Chun Siu
- College of Allied Health Professions, University of Nebraska Medical Center, Omaha, NE, USA
| |
Collapse
|
13
|
Systematic Review of Intraoperative Assessment Tools in Minimally Invasive Gynecologic Surgery. J Minim Invasive Gynecol 2020; 28:692-697. [PMID: 33086146 PMCID: PMC7568765 DOI: 10.1016/j.jmig.2020.10.007] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2020] [Revised: 10/07/2020] [Accepted: 10/15/2020] [Indexed: 11/24/2022]
Abstract
OBJECTIVE To collect, summarize, and evaluate the currently available intraoperative rating tools used in abdominal minimally invasive gynecologic surgery (MIGS). DATA SOURCES Medline, Embase, and Scopus databases from January 1, 2000, to May 12, 2020. METHODS OF STUDY SELECTION A systematic search strategy was designed and executed. Published studies evaluating an assessment tool in abdominal MIGS cases were included. Studies focused on simulation, reviews, and abstracts without a published manuscript were excluded. Risk of bias and methodological quality were assessed for each study. TABULATION, INTEGRATION, AND RESULTS Disparate study methods prevented quantitative synthesis of the data. Ten studies were included in the analysis. The tools were grouped into global (n = 4) and procedure-specific assessments (n = 6). Most studies evaluated small numbers of surgeons and lacked a comparison group to evaluate the effectiveness of the tool. All studies demonstrated content validity and at least 1 dimension of reliability, and 2 have external validity. The intraoperative procedure-specific tools have been more thoroughly evaluated than the global scales. CONCLUSION Procedure-specific intraoperative assessment tools for MIGS cases are more thoroughly evaluated than global tools; however, poor-quality studies and borderline reliability limit their use. Well-designed, controlled studies evaluating the effectiveness of intraoperative assessment tools in MIGS are needed.
Collapse
|
14
|
Beulens AJW, Brinkman WM, Koldewijn EL, Hendrikx AJM, van Basten JPA, van Merriënboer JJG, Van der Poel HG, Bangma CH, Wagner C. A Prospective, Observational, Multicentre Study Concerning Nontechnical Skills in Robot-assisted Radical Cystectomy Versus Open Radical Cystectomy. EUR UROL SUPPL 2020; 19:37-44. [PMID: 34337453 PMCID: PMC8317860 DOI: 10.1016/j.euros.2020.05.003] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2020] [Revised: 04/20/2020] [Accepted: 05/16/2020] [Indexed: 11/28/2022] Open
Abstract
Introduction and hypotheses valuation of surgical skills, both technical and nontechnical, is possible through observations and video analysis. Besides technical failures, adverse outcomes in surgery can also be related to hampered communication, moderate teamwork, lack of leadership, and loss of situational awareness. Even though some surgeons are convinced about nontechnical skills being an important part of their professionalisation, there is paucity of data about a possible relationship between nontechnical skills and surgical outcome. In robot-assisted surgery, the surgeon sits behind the console and is at a remote position from the surgical field and team, making communication more important than in open surgery and conventional laparoscopy. A lack of structured research makes it difficult to assess the value of the different analysis methods for nontechnical skills, particularly in robot-assisted surgery. Our hypothesis includes the following: (1) introduction of robot-assisted surgery leads to an initial decay in nontechnical skills behaviour during the learning curve of the team, (2) nontechnical skills behaviour is more explicitly expressed in experienced robot-assisted surgery teams than in experienced open surgery teams, and (3) introduction of robot-assisted surgery leads to the development of different forms of nontechnical skills behaviour compared with open surgery. Design This study is a prospective, observational, multicentre, nonrandomised, case-control study including bladder cancer patients undergoing either an open radical cystectomy or a robot-assisted radical cystectomy at the Catharina Hospital Eindhoven, the Netherlands, or at the Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital Amsterdam. All patients are eligible for inclusion; there are no exclusion criteria. The Catharina Hospital Eindhoven, the Netherlands, performs on average 35 radical cystectomies a year. The Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital Amsterdam, performs on average 100 radical cystectomies a year. Protocol overview The choice of treatment is at the discretion of the patient and the surgeon. Patient results will be obtained prospectively. Pathology results as well as complications occurring within 90 d following surgery will be registered. Surgical complications will be registered according to the Clavien-Dindo system. Measurements Nontechnical skills will be observed using five different methods: (1) NOTSS: Nontechnical Skills for Surgeons; (2) Oxford NOTECHS II: a modified theatre team nontechnical skills scoring system; (3) OTAS: Observational Teamwork Assessment for Surgery; (4) Interpersonal and Cognitive Assessment for Robotic Surgery (ICARS): evaluation of nontechnical skills in robotic surgery; and (5) analysis of human factors. Technical skills in robot-assisted radical cystectomy will be analysed using two different methods: (1) GEARS: Global Evaluative Assessment of Robotic Skill and (2) GERT: Generic Error Rating Tool. Safety criteria and reporting Formal ethical approval has been provided by Medical research Ethics Committees United (MEC-U), The Netherlands (reference number W19.048). We hope to present the results of this study to the scientific community at conferences and in peer-reviewed journals. Statistical analysis Frequency statistics will be calculated for patient demographical data, and a Shapiro-Wilk test with p > 0.05 will be used to define normal distribution. Univariate analysis will be conducted to test for statistically significant differences in observation scores between open radical cystectomy and robot-assisted radical cystectomy cohorts across all variables, using independent sample t tests and Mann-Whitney U testing, as appropriate. A variable-selection strategy will be used to create multivariate models. Binary logistic regression will be conducted to calculate odds ratios and 95% confidence intervals for significant predictors on univariate analysis and clinically relevant covariates. Statistical significance is set at p < 0.05 based on a two-tailed comparison. Summary This study uses a structured approach to the analysis of nontechnical skills using extracorporeal videos of both open radical cystectomy and robot-assisted radical cystectomy surgeries, in order to obtain detailed data on nontechnical skills during open and minimally invasive surgeries. The results of this study could possibly be used to develop team-training programmes, specifically for the introduction of the surgical robot in relation to changes in nontechnical skills. Additional analysis of technical skills using the intracorporeal footage of the surgical robot will be used to elucidate the role of surgical skills and surgical events in nontechnical skills.
Collapse
Affiliation(s)
- Alexander J W Beulens
- Netherlands Institute for Health Services Research (NIVEL), Utrecht, The Netherlands.,Department of Urology, Catharina Hospital, Eindhoven, The Netherlands
| | - Willem M Brinkman
- Department of Oncological Urology, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Evert L Koldewijn
- Department of Urology, Catharina Hospital, Eindhoven, The Netherlands
| | - Ad J M Hendrikx
- Department of Urology, Catharina Hospital, Eindhoven, The Netherlands
| | | | | | - Henk G Van der Poel
- Department of Urology, Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
| | - Chris H Bangma
- Department of Urology, Erasmus University Medical Centre, Rotterdam, The Netherlands
| | - Cordula Wagner
- Netherlands Institute for Health Services Research (NIVEL), Utrecht, The Netherlands.,Amsterdam Public Health Research Institute, Amsterdam UMC, Location VUmc, Amsterdam, The Netherlands
| |
Collapse
|
15
|
Beulens AJW, Namba HF, Brinkman WM, Meijer RP, Koldewijn EL, Hendrikx AJM, van Basten JP, van Merriënboer JJG, Van der Poel HG, Bangma C, Wagner C. Analysis of the video motion tracking system "Kinovea" to assess surgical movements during robot-assisted radical prostatectomy. Int J Med Robot 2020; 16:e2090. [PMID: 32034977 DOI: 10.1002/rcs.2090] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Revised: 01/16/2020] [Accepted: 02/03/2020] [Indexed: 11/11/2022]
Abstract
BACKGROUNDS Robot-assisted surgery facilitated the possibility to evaluate the surgeon's skills by recording and evaluating the robot surgical images. The aim of this study was to investigate the possibility of using a computer programme (Kinovea) for objective assessment of surgical movements in previously recorded in existing robot-assisted radical prostatectomy (RARP) videos. METHODS Twelve entire RARP procedures were analysed by a trained researcher using the computer programme "Kinovea" to perform semi-automated assessment of surgical movements. RESULTS Data analysis showed Kinovea was on average able to automatically assess only 22% of the total surgical duration per video of the robot-assisted surgery. On average, it lasted 4 hours of continued monitoring by the researcher to assess one RARP using Kinovea. CONCLUSION Although we proved it is technically possible to use the Kinovea system in retrospective analysis of surgical movement in robot-assisted surgery, the acquired data do not give a comprehensive enough analysis of the video to be used in skills assessment.
Collapse
Affiliation(s)
- Alexander J W Beulens
- Netherlands Institute for Health Services Research (NIVEL), Utrecht, The Netherlands.,Department of Urology, Catharina Hospital, Eindhoven, The Netherlands
| | - Hanae F Namba
- Department of Urology, Catharina Hospital, Eindhoven, The Netherlands.,Faculty of Medicine, Utrecht University, Utrecht, The Netherlands
| | - Willem M Brinkman
- Department of Oncological Urology, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Richard P Meijer
- Department of Oncological Urology, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Evert L Koldewijn
- Department of Urology, Catharina Hospital, Eindhoven, The Netherlands
| | | | | | | | - Henk G Van der Poel
- Department of Urology, Dutch Cancer Institute-Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
| | - Chris Bangma
- Department of Urology, Erasmus Medical Centre, Rotterdam, The Netherlands
| | - Cordula Wagner
- Netherlands Institute for Health Services Research (NIVEL), Utrecht, The Netherlands.,Amsterdam Public Health Research Institute, Amsterdam UMC, Amsterdam, The Netherlands
| |
Collapse
|
16
|
Vaidya A, Aydin A, Ridgley J, Raison N, Dasgupta P, Ahmed K. Current Status of Technical Skills Assessment Tools in Surgery: A Systematic Review. J Surg Res 2020; 246:342-378. [DOI: 10.1016/j.jss.2019.09.006] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2019] [Revised: 08/29/2019] [Accepted: 09/11/2019] [Indexed: 12/18/2022]
|
17
|
Technical Performance as a Predictor of Clinical Outcomes in Laparoscopic Gastric Cancer Surgery. Ann Surg 2020; 270:115-120. [PMID: 29578907 DOI: 10.1097/sla.0000000000002741] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
OBJECTIVE The purpose of this study was to evaluate the relationship between technical performance and patient outcomes in laparoscopic gastric cancer surgery. BACKGROUND Laparoscopic gastrectomy for cancer is an advanced procedure with high rate of postoperative morbidity and mortality. Many variables including patient, disease, and perioperative management factors have been shown to impact postoperative outcomes; however, the role of surgical performance is insufficiently investigated. METHODS A retrospective review was performed for all patients who had undergone laparoscopic gastrectomy for cancer at 3 teaching institutions between 2009 and 2015. Patients with available, unedited video-recording of their procedure were included in the study. Video files were rated for technical performance, using Objective Structured Assessments of Technical Skills (OSATS) and Generic Error Rating Tool instruments. The main outcome variable was major short-term complications. The effect of technical performance on patient outcomes was assessed using logistic regression analysis with backward selection strategy. RESULTS Sixty-one patients with available video recordings were included in the study. The overall complication rate was 29.5%. The mean Charlson comorbidity index, type of procedure, and the global OSATS score were included in the final predictive model. Lower performance score (OSATS ≤29) remained an independent predictor for major short-term outcomes (odds ratio 6.49), while adjusting for comorbidities and type of procedure. CONCLUSIONS Intraoperative technical performance predicts major short-term outcomes in laparoscopic gastrectomy for cancer. Ongoing assessment and enhancement of surgical skills using modern, evidence-based strategies might improve short-term patient outcomes. Future work should focus on developing and studying the effectiveness of such interventions in laparoscopic gastric cancer surgery.
Collapse
|
18
|
Augestad KM, Butt K, Ignjatovic D, Keller DS, Kiran R. Video-based coaching in surgical education: a systematic review and meta-analysis. Surg Endosc 2019; 34:521-535. [PMID: 31748927 DOI: 10.1007/s00464-019-07265-0] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2019] [Accepted: 11/12/2019] [Indexed: 01/22/2023]
Abstract
BACKGROUND In the era of competency-based surgical education, VBC has gained increased attention and may enhance the efficacy of surgical education. The objective of this systematic review was to summarize the existing evidence of video-based coaching (VBC) and compare VBC to traditional master-apprentice-based surgical education. METHODS We performed a systematic review and meta-analysis of randomized controlled trials (RCT) assessing VBC according to the PRISMA and Cochrane guidelines. The MEDLINE, EMBASE, and COCHRANE and Researchgate databases were searched for eligible manuscripts. Standard mean difference (SMD) of performance scoring scales was used to assess the effect of VBC versus traditional training without VBC (control). RESULTS Of 627 studies identified, 24 RCTs were eligible and evaluated. The studies included 778 surgical trainees (n = 386 VBC vs. n = 392 control). 13 performance scoring scales were used to assess technical competence; OSATS-GRS was the most common (n = 15). VBC was provided preoperative (n = 11), intraoperative (n = 1), postoperative (n = 10), and perioperative (n = 2). The majority of studies were unstructured, where identified coaching frameworks were PRACTICE (n = 1), GROW (n = 2) and Wisconsin Coaching Framework (n = 1). There was an effect on performance scoring scales in favor of VBC coaching (SMD 0.87, p < 0.001). In subgroup analyses, the residents had a larger relative effect (SMD 1.13; 0.61-1.65, p < 0.001) of VBC compared to medical students (SMD 0.43, 0.06-0.81, p < 0.001). The greatest source of potential bias was absence of blinding of the participants and personnel (n = 20). CONCLUSION Video-based coaching increases technical performance of medical students and surgical residents. There exist significant study and intervention heterogeneity that warrants further exploration, showing the need to structure and standardize video-based coaching tools.
Collapse
Affiliation(s)
- Knut Magne Augestad
- Department of Postgraduate Surgical Education, University Hospital North Norway, Tromsö, Norway. .,Department of GI Surgery, Sandnessjøen Regional Hospital, Sandnessjøen, Norway. .,Division of Colorectal Surgery, Columbia University Medical Center, New York, NY, USA.
| | - Khayam Butt
- Department of GI Surgery, Nordlandssykehuset, Bodø, Norway
| | - Dejan Ignjatovic
- Department of GI Surgery, Akershus University Hospital, Oslo, Norway.,Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Deborah S Keller
- Division of Colorectal Surgery, Columbia University Medical Center, New York, NY, USA
| | - Ravi Kiran
- Division of Colorectal Surgery, Columbia University Medical Center, New York, NY, USA
| |
Collapse
|
19
|
Linking surgical skills to postoperative outcomes: a Delphi study on the robot-assisted radical prostatectomy. J Robot Surg 2019; 13:675-687. [PMID: 30610535 DOI: 10.1007/s11701-018-00916-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2018] [Accepted: 12/18/2018] [Indexed: 10/27/2022]
Abstract
OBJECTIVE To develop an assessment instrument for the evaluation of surgical videos to elucidate the association between surgical skills and postoperative outcomes after a robot-assisted radical prostatectomy (RARP). DESIGN A Delphi study consisting of two consecutive online surveys and a consensus group meeting. SETTING Urology departments of general, teaching and university hospitals in the Netherlands. PARTICIPANTS All Dutch urologists with a specialization in RARP. RESULTS Of 18 invited experts, 12 (67%) participated in the first online survey. In the second round, 9 of the 18 invited experts participated (50%). The Delphi meeting was attended by 5 of the 18 (27%) invited experts. The panel identified seven surgical steps with a possible association to postoperative outcomes. The experts also expected an association between adverse postoperative outcomes and the frequency of camera removals, the number of stitches placed, the amount of bleeding, and the extent of coagulation. These factors were incorporated into an assessment instrument. CONCLUSIONS Experts in the field of RARP achieved consensus on 7 surgical steps and 4 aspects of the RARP procedure that may be related to adverse postoperative outcomes. The resulting assessment instrument will be tested in future research to determine its validity.
Collapse
|
20
|
A scoping review of assessment tools for laparoscopic suturing. Surg Endosc 2018; 32:3009-3023. [PMID: 29721749 DOI: 10.1007/s00464-018-6199-8] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2018] [Accepted: 04/21/2018] [Indexed: 10/17/2022]
Abstract
BACKGROUND A needs assessment identified a gap in teaching and assessment of laparoscopic suturing (LS) skills. The purpose of this review is to identify assessment tools that were used to assess LS skills, to evaluate validity evidence available, and to provide guidance for selecting the right assessment tool for specific assessment conditions. METHODS Bibliographic databases were searched till April 2017. Full-text articles were included if they reported on assessment tools used in the operating room/simulation to (1) assess procedures that require LS or (2) specifically assess LS skills. RESULTS Forty-two tools were identified, of which 26 were used for assessing LS skills specifically and 26 for procedures that require LS. Tools had the most evidence in internal structure and relationship to other variables, and least in consequences. CONCLUSION Through identification and evaluation of assessment tools, the results of this review could be used as a guideline when implementing assessment tools into training programs.
Collapse
|
21
|
Yılmaz EM, Soyder A, Aksu M, Bozdağ AD, Boylu Ş, Edizsoy A, Ballı Ş, Tekindal MA. Contribution of an educational video to surgical education in laparoscopic appendectomy. Turk J Surg 2017; 33:237-242. [PMID: 29260126 DOI: 10.5152/turkjsurg.2017.3610] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2016] [Accepted: 06/17/2016] [Indexed: 11/22/2022]
Abstract
Objective With recent widespread advances in laparoscopy, providing education on this subject has become a significant challenge. The aim of this study was to determine the degree of contribution made by surgeons to laparoscopic education through an educational video. Material and Methods A total of 40 volunteer general surgery residents and specialists participated in our study. Before watching the approximately six-minute educational video prepared for laparoscopic appendectomy, the participants were asked to fill out participant information forms and information measurement questionnaire forms. After the video, the participants were asked to fill out the information measurement questionnaire forms a second time; additionally, attitude evaluation forms and education evaluation questionnaire forms were presented to the participants for completion, and statistical analysis was performed. Furthermore, the total watching duration and the number of times the video was paused were recorded. Results A total of 40 surgeons participated in the study (75% residents and 25% specialists). When the results of the information determination questionnaire forms filled out by the resident and specialist groups before and after the video were compared, it was observed that the scores of both groups significantly increased after watching the video (p=0.001). A statistical significance was identified between the length of time the video was watched and the education evaluation form scores of the participants (p<0.01). It was observed that the longer the video was watched, the greater the increase in the education evaluation scores. The results of the attitude evaluation forms implied that the video could be produced more professionally. Conclusion Although education is an inevitable requirement of laparoscopic surgery, many teaching methods are available. Awareness-enhancing videos prepared on this topic can be efficient in providing laparoscopic education.
Collapse
Affiliation(s)
- Eyüp Murat Yılmaz
- Department of General Surgery, Adnan Menderes University School of Medicine, Aydın, Turkey
| | - Aykut Soyder
- Department of General Surgery, Adnan Menderes University School of Medicine, Aydın, Turkey
| | - Murat Aksu
- Department of Ethics, İzmir University School of Medicine, İzmir, Turkey
| | - Ali Doğan Bozdağ
- Department of General Surgery, Adnan Menderes University School of Medicine, Aydın, Turkey
| | - Şükrü Boylu
- Department of General Surgery, Adnan Menderes University School of Medicine, Aydın, Turkey
| | - Akay Edizsoy
- Department of General Surgery, Adnan Menderes University School of Medicine, Aydın, Turkey
| | - Şadi Ballı
- Department of General Surgery, Adnan Menderes University School of Medicine, Aydın, Turkey
| | | |
Collapse
|
22
|
Goldenberg MG, Grantcharov TP. Video-analysis for the assessment of practical skill. ACTA ACUST UNITED AC 2016. [DOI: 10.1007/s13629-016-0156-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
23
|
Avoiding complications by a hands-on mentor programme. Best Pract Res Clin Obstet Gynaecol 2016; 35:3-12. [DOI: 10.1016/j.bpobgyn.2015.11.003] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2015] [Accepted: 11/05/2015] [Indexed: 11/19/2022]
|
24
|
Polterauer S, Husslein H, Kranawetter M, Schwameis R, Reinthaller A, Heinze G, Grimm C. Effect of Preoperative Warm-up Exercise Before Laparoscopic Gynecological Surgery: A Randomized Trial. JOURNAL OF SURGICAL EDUCATION 2016; 73:429-432. [PMID: 26778744 DOI: 10.1016/j.jsurg.2015.11.017] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/06/2015] [Revised: 11/28/2015] [Accepted: 11/30/2015] [Indexed: 06/05/2023]
Abstract
BACKGROUND Laparoscopic surgical procedures require a high level of cognitive and psychomotoric skills. Thus, effective training methods to acquire an adequate level of expertise are crucial. The aim of this study was to investigate the effect of preoperative warm up training on surgeon׳s performance during gynecologic laparoscopic surgery. MATERIALS AND METHODS In this randomized controlled trial, surgeons performed a preoperative warm up training using a virtual reality simulator before laparoscopic unilateral salpingo-oophorectomy. Serving as their own controls, each subject performed 2 pairs of laparoscopic cases, each pair consisting of 1 case with and 1 without warm up before surgery. Surgeries were videotaped and psychomotoric skills were rated using objective structured assessment of technical skills (OSATS) and the generic error rating tool by a masked observer. Perioperative complications were assessed. Statistical analysis was performed using a mixed model, and mean OSATS scores were compared between both the groups. RESULTS In total, data of 10 surgeons and 17 surgeries were available for analysis. No differences between educational level and surgical experiences were observed between the groups. Mean standard error psychomotoric and task-specific OSATS scores of 19.8 (1.7) and 3.7 (0.2) were observed in the warm up group compared with 18.6 (1.7) and 3.8 (0.2) in the no warm up group, respectively (p = 0.51 and p = 0.29). Using generic error rating tool, the total number of errors was 8.75 (2.15) in the warm up group compared with 10.8 (2.18) in the no warm-up group (p = 0.53). Perioperative complications and operating time did not differ between both the groups. DISCUSSION The present study suggests that warm-up before laparoscopic salpingo-oophorectomy does not increase psychomotoric skills during surgery. Moreover, it does not influence operating time and complication rates. (Medical University of Vienna-IRB approval number, 1072/2011, ClinicalTrials.gov number, NCT01712607).
Collapse
Affiliation(s)
- Stephan Polterauer
- Department of General Gynecology and Gynecologic Oncology, Medical University of Vienna, Vienna, Austria; Karl Landsteiner Institute for General Gynecology and Experimental Gynecologic Oncology, Vienna, Austria.
| | - Heinrich Husslein
- Department of General Gynecology and Gynecologic Oncology, Medical University of Vienna, Vienna, Austria
| | - Marlene Kranawetter
- Department of General Gynecology and Gynecologic Oncology, Medical University of Vienna, Vienna, Austria
| | - Richard Schwameis
- Department of General Gynecology and Gynecologic Oncology, Medical University of Vienna, Vienna, Austria
| | - Alexander Reinthaller
- Department of General Gynecology and Gynecologic Oncology, Medical University of Vienna, Vienna, Austria; Karl Landsteiner Institute for General Gynecology and Experimental Gynecologic Oncology, Vienna, Austria
| | - Georg Heinze
- Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, Austria
| | - Christoph Grimm
- Department of General Gynecology and Gynecologic Oncology, Medical University of Vienna, Vienna, Austria
| |
Collapse
|