The Generic Error Rating Tool: A Novel Approach to Assessment of Performance and Surgical Education in Gynecologic Laparoscopy

Editor's Choice - Development and Testing of Step, Error, and Event Frameworks to Evaluate Technical Performance in Peripheral Endovascular Interventions

OBJECTIVE

Tools for endovascular performance assessment are necessary in competency based education. This study aimed to develop and test a detailed analysis tool to assess steps, errors, and events in peripheral endovascular interventions (PVI).

METHODS

A modified Delphi consensus was used to identify steps, errors, and events in iliac-femoral-popliteal endovascular interventions. International experts in vascular surgery, interventional radiology, cardiology, and angiology were identified, based on their scientific track record. In an initial open ended survey round, experts volunteered a comprehensive list of steps, errors, and events. The items were then rated on a five point Likert scale until consensus was reached with a pre-defined threshold (Cronbach's alpha > 0.7) and > 70% expert agreement. An experienced endovascular surgeon applied the finalised frameworks on 10 previously videorecorded elective PVI cases.

RESULTS

The expert consensus panel was formed by 28 of 98 invited proceduralists, consisting of three angiologists, seven interventional radiologists, five cardiologists, and 13 vascular surgeons, with 29% from North America and 71% from Europe. The Delphi process was completed after three rounds (Cronbach's alpha; αsteps = 0.79; αerrors = 0.90; αevents = 0.90), with 15, 26, and 18 items included in the final step (73 - 100% agreement), error (73 - 100% agreement), and event (73 - 100% agreement) frameworks, respectively. The median rating time per case was 4.3 hours (interquartile range [IQR] 3.2, 5 hours). A median of 55 steps (IQR 40, 67), 27 errors (IQR 21, 49), and two events (IQR 1, 6) were identified per case.

CONCLUSION

An evaluation tool for the procedural steps, errors, and events in iliac-femoral-popliteal endovascular procedures was developed through a modified Delphi consensus and applied to recorded intra-operative data to identify hazardous steps, common errors, and events. Procedural mastery may be promoted by using the frameworks to provide endovascular proceduralists with detailed technical performance feedback.

Objective assessment tools in laparoscopic or robotic-assisted gynecological surgery: A systematic review

INTRODUCTION

There is a growing emphasis on proficiency-based progression within surgical training. To enable this, clearly defined metrics for those newly acquired surgical skills are needed. These can be formulated in objective assessment tools. The aim of the present study was to systematically review the literature reporting on available tools for objective assessment of minimally invasive gynecological surgery (simulated) performance and evaluate their reliability and validity.

MATERIAL AND METHODS

A systematic search (1989-2022) was conducted in MEDLINE, Embase, PubMed, Web of Science in accordance with PRISMA. The trial was registered with the Prospective Register of Systematic Reviews (PROSPERO) ID: CRD42022376552. Randomized controlled trials, prospective comparative studies, prospective single-group (with pre- and post-training assessment) or consensus studies that reported on the development, validation or usage of assessment tools of surgical performance in minimally invasive gynecological surgery, were included. Three independent assessors assessed study setting and validity evidence according to a contemporary framework of validity, which was adapted from Messick's validity framework. Methodological quality of included studies was assessed using the modified medical education research study quality instrument (MERSQI) checklist. Heterogeneity in data reporting on types of tools, data collection, study design, definition of expertise (novice vs. experts) and statistical values prevented a meaningful meta-analysis.

RESULTS

A total of 19 746 titles and abstracts were screened of which 72 articles met the inclusion criteria. A total of 37 different assessment tools were identified of which 13 represented manual global assessment tools, 13 manual procedure-specific assessment tools and 11 automated performance metrices. Only two tools showed substantive evidence of validity. Reliability and validity per tool were provided. No assessment tools showed direct correlation between tool scores and patient related outcomes.

CONCLUSIONS

Existing objective assessment tools lack evidence on predicting patient outcomes and suffer from limitations in transferability outside of the research environment, particularly for automated performance metrics. Future research should prioritize filling these gaps while integrating advanced technologies like kinematic data and AI for robust, objective surgical skill assessment within gynecological advanced surgical training programs.

The application of objective clinical human reliability analysis (OCHRA) in the assessment of basic robotic surgical skills

BACKGROUND

Using a validated, objective, and standardised assessment tool to assess progression and competency is essential for basic robotic surgical training programmes. Objective clinical human reliability analysis (OCHRA) is an error-based assessment tool that provides in-depth analysis of individual technical errors. We conducted a feasibility study to assess the concurrent validity and reliability of OCHRA when applied to basic, generic robotic technical skills assessment.

METHODS

Selected basic robotic surgical skill tasks, in virtual reality (VR) and dry lab equivalent, were performed by novice robotic surgeons during an intensive 5-day robotic surgical skills course on da Vinci® X and Xi surgical systems. For each task, we described a hierarchical task analysis. Our developed robotic surgical-specific OCHRA methodology was applied to error events in recorded videos with a standardised definition. Statistical analysis to assess concurrent validity with existing tools and inter-rater reliability were performed.

RESULTS

OCHRA methodology was applied to 272 basic robotic surgical skills tasks performed by 20 novice robotic surgeons. Performance scores improved from the start of the course to the end using all three assessment tools; Global Evaluative Assessment of Robotic Skills (GEARS) [VR: t(19) = - 9.33, p < 0.001] [dry lab: t(19) = - 10.17, p < 0.001], OCHRA [VR: t(19) = 6.33, p < 0.001] [dry lab: t(19) = 10.69, p < 0.001] and automated VR [VR: t(19) = - 8.26, p < 0.001]. Correlation analysis, for OCHRA compared to GEARS and automated VR scores, shows a significant and strong inverse correlation in every VR and dry lab task; OCHRA vs GEARS [VR: mean r = - 0.78, p < 0.001] [dry lab: mean r = - 0.82, p < 0.001] and OCHRA vs automated VR [VR: mean r = - 0.77, p < 0.001]. There is very strong and significant inter-rater reliability between two independent reviewers (r = 0.926, p < 0.001).

CONCLUSION

OCHRA methodology provides a detailed error analysis tool in basic robotic surgical skills with high reliability and concurrent validity with existing tools. OCHRA requires further evaluation in more advanced robotic surgical procedures.

Assessing Chest Tube Insertion Skills Using a Porcine Rib Model-A Validity Study

INTRODUCTION

Assessments require sufficient validity evidence before their use. The Assessment for Competence in Chest Tube Insertion (ACTION) tool evaluates proficiency in chest tube insertion (CTI), combining a rating scale and an error checklist. The aim of this study was to collect validity evidence for the ACTION tool on a porcine rib model according to the Messick framework.

METHODS

A rib model, consisting of a porcine hemithorax that was placed in a wooden frame, was used as simulator. Participants were recruited from the departments of surgery, pulmonology, and emergency medicine. After familiarization with the rib model and the equipment, standardized instructions and clinical context were provided. They performed 2 CTIs while being scored with the ACTION tool. All performances were assessed live by 1 rater and by 3 blinded raters using video recordings. Generalizability-analysis was performed and mean scores and errors of both groups on the first performance were compared. A pass/fail score was established using the contrasting groups' method.

RESULTS

Nine novice and 8 experienced participants completed the study. Generalizability coefficients where high for the rating scale (0.92) and the error checklist (0.87). In the first CTI, novices scored lower than the experienced group (38.1/68 vs. 47.1/68, P = 0.042), but no difference was observed on the error checklist. A pass/fail score of 44/68 was established.

CONCLUSION

A solid validity argument for the ACTION tool's rating scale on a porcine rib model is presented, allowing formative and summative assessment of procedural skills during training before patient contact.

Video-based tools for surgical quality assessment of technical skills in laparoscopic procedures: a systematic review

BACKGROUND

Quality of surgery has substantial impact on both short- and long-term clinical outcomes. This stresses the need for objective surgical quality assessment (SQA) for education, clinical practice and research purposes. The aim of this systematic review was to provide a comprehensive overview of all video-based objective SQA tools in laparoscopic procedures and their validity to objectively assess surgical performance.

METHODS

PubMed, Embase.com and Web of Science were systematically searched by two reviewers to identify all studies focusing on video-based SQA tools of technical skills in laparoscopic surgery performed in a clinical setting. Evidence on validity was evaluated using a modified validation scoring system.

RESULTS

Fifty-five studies with a total of 41 video-based SQA tools were identified. These tools were used in 9 different fields of laparoscopic surgery and were divided into 4 categories: the global assessment scale (GAS), the error-based assessment scale (EBAS), the procedure-specific assessment tool (PSAT) and artificial intelligence (AI). The number of studies focusing on these four categories were 21, 6, 31 and 3, respectively. Twelve studies validated the SQA tool with clinical outcomes. In 11 of those studies, a positive association between surgical quality and clinical outcomes was found.

CONCLUSION

This systematic review included a total of 41 unique video-based SQA tools to assess surgical technical skills in various domains of laparoscopic surgery. This study suggests that validated SQA tools enable objective assessment of surgical performance with relevance for clinical outcomes, which can be used for training, research and quality improvement programs.

Assessing endovascular team performances in a hybrid room using the Black Box system: a prospective cohort study

BACKGROUND

The hybrid room (HR) is a complex, high-risk environment, requiring teams (surgeons, anesthesiologists, nurses, technologists) to master various skills, including the 'As Low As Reasonably Achievable' principle of radiation safety. This prospective single center cohort reports the first use of the Operating Room Black Box (ORBB) in a HR. This medical data recording system captures procedural and audio-visual data to facilitate structured team performance analysis.

METHODS

Patients planned for endovascular repair of an infrarenal abdominal aortic aneurysm (EVAR) or treatment of symptomatic iliac-femoral-popliteal atherosclerotic disease (Peripheral Vascular Interventions or PVI) were included. Validated measures and established assessment tools were used to assess (non-)technical skills, radiation safety performance and environmental distractions.

RESULTS

Six EVAR and sixteen PVI procedures were captured. Technical performance for one EVAR was rated 19/35 on the procedure-specific scale, below the 'acceptable' score of 21. Technical skills were rated above acceptable in all PVI procedures. Shared decision making and leadership were rated highly in 12/22 cases, whereas surgical communication and nurses' task management were rated low in 14/22 cases. Team members rarely stepped back from the C-arm during digital subtraction angiography. Radiation safety behavior was scored below 'acceptable' in 14/22 cases. A median (interquartile range) number of 12 (6-23) auditory distractions was observed per procedure.

CONCLUSIONS

The ORBB facilitates holistic workplace-based assessment of endovascular performance in a HR by combining objective assessment parameters and rating scale-based evaluations. Strengths and weaknesses were identified in team members' (non-)technical and radiation safety practices. This technology has the potential to improve vascular surgical practice, though human input remains crucial. (NCT04854278).

Utilizing Grasp Monitoring to Predict Microsurgical Expertise

INTRODUCTION

Most microsurgical procedures require the surgeon to use tools to grasp and hold fragile objects in the surgical site. Prior research on grasping in surgery has mostly either been in other surgical techniques or used grasping as an auxiliary metric. We focus on microsurgery and investigate what grasping can tell about microsurgical skill and suturing performance. This study lays groundwork for using automatic detection of grasps to evaluate surgical skill.

METHODS

Five expert surgeons and six novices completed sutures on a microsurgical training board. Video recordings of the performance were annotated for the number of grasps, while an eye tracker recorded the participants' pupil dilations for cognitive workload assessment. Performance was measured with suturing duration and the University of Western Ontario Microsurgical Skills Assessment instrument (UWOMSA). Differences in skill, suturing performance and cognitive workload were compared with grasping behavior.

RESULTS

Novices needed significantly more grasps to complete sutures and failed to grasp more often than the experts. The number of grasps affected the suturing duration more in novices. Decreasing suturing efficiency as measured by UWOMSA instrument was associated with increase in grasps, even when we controlled for overall skill differences. Novices displayed larger pupil dilations when averaged over a sufficiently large sample, and the difference increased after the grasp.

CONCLUSIONS

Grasping action during microsurgical procedures can be used as a conceptually simple yet objective proxy in microsurgical performance assessment. If the grasps could be detected automatically, they could be used to aid in computational evaluation of surgical trainees' performance.

Video-based coaching for surgical residents: a systematic review and meta-analysis

BACKGROUND

Video-based coaching (VBC) is used to supplement current teaching methods in surgical education and may be useful in competency-based frameworks. Whether VBC can effectively improve surgical skill in surgical residents has yet to be fully elucidated. The objective of this study is to compare surgical residents receiving and not receiving VBC in terms of technical surgical skill.

METHODS

The following databases were searched from database inception to October 2021: Medline, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL), and PubMed. Articles were included if they were randomized controlled trials (RCTs) comparing surgical residents receiving and not receiving VBC. The primary outcome, as defined prior to data collection, was change in objective measures of technical surgical skill following implementation of either VBC or control. A pairwise meta-analyses using inverse variance random effects was performed. Standardized mean differences (SMD) were used as the primary outcome measure to account for differences in objective surgical skill evaluation tools.

RESULTS

From 2734 citations, 11 RCTs with 157 residents receiving VBC and 141 residents receiving standard surgical teaching without VBC were included. There was no significant difference in post-coaching scores on objective surgical skill evaluation tools between groups (SMD 0.53, 95% CI 0.00 to 1.01, p = 0.05, I² = 74%). The improvement in scores pre- and post-intervention was significantly greater in residents receiving VBC compared to those not receiving VBC (SMD 1.62, 95% CI 0.62 to 2.63, p = 0.002, I² = 85%). These results were unchanged with leave-one-out sensitivity analysis and subgroup analysis according to operative setting.

CONCLUSION

VBC can improve objective surgical skills in surgical residents of various levels. The benefit may be most substantial for trainees with lower baseline levels of objective skill. Further studies are required to determine the impact of VBC on competency-based frameworks.

Hybrid Spatiotemporal Contrastive Representation Learning for Content-Based Surgical Video Retrieval

In the medical field, due to their economic and clinical benefits, there is a growing interest in minimally invasive surgeries and microscopic surgeries. These types of surgeries are often recorded during operations, and these recordings have become a key resource for education, patient disease analysis, surgical error analysis, and surgical skill assessment. However, manual searching in this collection of long-term surgical videos is an extremely labor-intensive and long-term task, requiring an effective content-based video analysis system. In this regard, previous methods for surgical video retrieval are based on handcrafted features which do not represent the video effectively. On the other hand, deep learning-based solutions were found to be effective in both surgical image and video analysis, where CNN-, LSTM- and CNN-LSTM-based methods were proposed in most surgical video analysis tasks. In this paper, we propose a hybrid spatiotemporal embedding method to enhance spatiotemporal representations using an adaptive fusion layer on top of the LSTM and temporal causal convolutional modules. To learn surgical video representations, we propose exploring the supervised contrastive learning approach to leverage label information in addition to augmented versions. By validating our approach to a video retrieval task on two datasets, Surgical Actions 160 and Cataract-101, we significantly improve on previous results in terms of mean average precision, 30.012 ± 1.778 vs. 22.54 ± 1.557 for Surgical Actions 160 and 81.134 ± 1.28 vs. 33.18 ± 1.311 for Cataract-101. We also validate the proposed method’s suitability for surgical phase recognition task using the benchmark Cholec80 surgical dataset, where our approach outperforms (with 90.2% accuracy) the state of the art.

Video-Based Coaching: Current Status and Role in Surgical Practice (Part 1) From the Society for Surgery of the Alimentary Tract, Health Care Quality and Outcomes Committee

Patient safety and outcomes are directly related to surgical performance. Surgical training emphasizes the importance of the surgeon in determining these outcomes. After training is complete, there is a lack of structured programs for surgeons to audit their skills and continue their individual development. There is a significant linear relationship between surgeon technical skill and surgical outcomes; however, measuring technical performance is difficult. Video-based coaching matches an individual surgeon in practice with a surgical colleague who has been trained in the core principles of coaching for individualizing instruction. It can provide objective assessment for teaching higher-level concepts, such as technical skills, cognitive skills, and decision-making. There are many benefits to video-based coaching. While the concept is gaining acceptance as a method of surgical education, it is still novel in clinical practice. As more surgeons look towards video-based coaching for quality improvement, a consistent definition of the program, goals, and metrics for assessment will be critical. This paper is a review on the status of the video-based coaching as it applies to practicing surgeons.

Tools for measuring technical skills during gynaecologic surgery: a scoping review

BACKGROUND

Standardised assessment is key to structured surgical training. Currently, there is no consensus on which surgical assessment tool to use in live gynaecologic surgery. The purpose of this review is to identify assessment tools measuring technical skills in gynaecologic surgery and evaluate the measurement characteristics of each tool.

METHOD

We utilized the scoping review methodology and searched PubMed, Medline, Embase and Cochrane. Inclusion criteria were studies that analysed assessment tools in live gynaecologic surgery. Kane's validity argument was applied to evaluate the assessment tools in the included studies.

RESULTS

Eight studies out of the 544 identified fulfilled the inclusion criteria. The assessment tools were categorised as global rating scales, global and procedure rating scales combined, procedure-specific rating scales or as a non-procedure-specific error assessment tool.

CONCLUSION

This scoping review presents the current different tools for observational assessment of technical skills in intraoperative, gynaecologic surgery. This scoping review can serve as a guide for surgical educators who want to apply a scale or a specific tool in surgical assessment.

Assessing Laparoscopic Surgical Skills Using Similarity Network Models: A Pilot Study

Background: Medical devices are becoming more complex, and doctors need to learn quickly how to use new medical tools. However, it is challenging to objectively assess the fundamental laparoscopic surgical skill level and determine skill readiness for advancement. There is a lack of objective models to compare performance between medical trainees and experienced doctors. Methods: This article discusses the use of similarity network models for individual tasks and a combination of tasks to show the level of similarity between residents and medical students while performing each task and their overall laparoscopic surgical skill level using a medical device (eg laparoscopic instruments). When a medical student is connected to most residents, that student is competent to the next training level. Performance of sixteen participants (5 residents and 11 students) while performing 3 tasks in 3 different training schedules is used in this study. Results: The promising result shows the general positive progression of students over 4 training sessions. Our results also indicate that students with different training schedules have different performance levels. Students' progress in performing a task is quicker if the training sessions are held more closely compared to when the training sessions are far apart in time. Conclusions: This study provides a graph-based framework for evaluating new learners' performance on medical devices and their readiness for advancement. This similarity network method could be used to classify students' performance using similarity thresholds, facilitating decision-making related to training and progression through curricula.

Systematic Review of Intraoperative Assessment Tools in Minimally Invasive Gynecologic Surgery

OBJECTIVE

To collect, summarize, and evaluate the currently available intraoperative rating tools used in abdominal minimally invasive gynecologic surgery (MIGS).

DATA SOURCES

Medline, Embase, and Scopus databases from January 1, 2000, to May 12, 2020.

METHODS OF STUDY SELECTION

A systematic search strategy was designed and executed. Published studies evaluating an assessment tool in abdominal MIGS cases were included. Studies focused on simulation, reviews, and abstracts without a published manuscript were excluded. Risk of bias and methodological quality were assessed for each study.

TABULATION, INTEGRATION, AND RESULTS

Disparate study methods prevented quantitative synthesis of the data. Ten studies were included in the analysis. The tools were grouped into global (n = 4) and procedure-specific assessments (n = 6). Most studies evaluated small numbers of surgeons and lacked a comparison group to evaluate the effectiveness of the tool. All studies demonstrated content validity and at least 1 dimension of reliability, and 2 have external validity. The intraoperative procedure-specific tools have been more thoroughly evaluated than the global scales.

CONCLUSION

Procedure-specific intraoperative assessment tools for MIGS cases are more thoroughly evaluated than global tools; however, poor-quality studies and borderline reliability limit their use. Well-designed, controlled studies evaluating the effectiveness of intraoperative assessment tools in MIGS are needed.

A Prospective, Observational, Multicentre Study Concerning Nontechnical Skills in Robot-assisted Radical Cystectomy Versus Open Radical Cystectomy

Introduction and hypotheses

valuation of surgical skills, both technical and nontechnical, is possible through observations and video analysis. Besides technical failures, adverse outcomes in surgery can also be related to hampered communication, moderate teamwork, lack of leadership, and loss of situational awareness. Even though some surgeons are convinced about nontechnical skills being an important part of their professionalisation, there is paucity of data about a possible relationship between nontechnical skills and surgical outcome. In robot-assisted surgery, the surgeon sits behind the console and is at a remote position from the surgical field and team, making communication more important than in open surgery and conventional laparoscopy. A lack of structured research makes it difficult to assess the value of the different analysis methods for nontechnical skills, particularly in robot-assisted surgery. Our hypothesis includes the following: (1) introduction of robot-assisted surgery leads to an initial decay in nontechnical skills behaviour during the learning curve of the team, (2) nontechnical skills behaviour is more explicitly expressed in experienced robot-assisted surgery teams than in experienced open surgery teams, and (3) introduction of robot-assisted surgery leads to the development of different forms of nontechnical skills behaviour compared with open surgery.

Design

This study is a prospective, observational, multicentre, nonrandomised, case-control study including bladder cancer patients undergoing either an open radical cystectomy or a robot-assisted radical cystectomy at the Catharina Hospital Eindhoven, the Netherlands, or at the Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital Amsterdam. All patients are eligible for inclusion; there are no exclusion criteria. The Catharina Hospital Eindhoven, the Netherlands, performs on average 35 radical cystectomies a year. The Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital Amsterdam, performs on average 100 radical cystectomies a year.

Protocol overview

The choice of treatment is at the discretion of the patient and the surgeon. Patient results will be obtained prospectively. Pathology results as well as complications occurring within 90 d following surgery will be registered. Surgical complications will be registered according to the Clavien-Dindo system.

Measurements

Nontechnical skills will be observed using five different methods: (1) NOTSS: Nontechnical Skills for Surgeons; (2) Oxford NOTECHS II: a modified theatre team nontechnical skills scoring system; (3) OTAS: Observational Teamwork Assessment for Surgery; (4) Interpersonal and Cognitive Assessment for Robotic Surgery (ICARS): evaluation of nontechnical skills in robotic surgery; and (5) analysis of human factors. Technical skills in robot-assisted radical cystectomy will be analysed using two different methods: (1) GEARS: Global Evaluative Assessment of Robotic Skill and (2) GERT: Generic Error Rating Tool.

Safety criteria and reporting

Formal ethical approval has been provided by Medical research Ethics Committees United (MEC-U), The Netherlands (reference number W19.048). We hope to present the results of this study to the scientific community at conferences and in peer-reviewed journals.

Statistical analysis

Frequency statistics will be calculated for patient demographical data, and a Shapiro-Wilk test with p > 0.05 will be used to define normal distribution. Univariate analysis will be conducted to test for statistically significant differences in observation scores between open radical cystectomy and robot-assisted radical cystectomy cohorts across all variables, using independent sample t tests and Mann-Whitney U testing, as appropriate. A variable-selection strategy will be used to create multivariate models. Binary logistic regression will be conducted to calculate odds ratios and 95% confidence intervals for significant predictors on univariate analysis and clinically relevant covariates. Statistical significance is set at p < 0.05 based on a two-tailed comparison.

Summary

This study uses a structured approach to the analysis of nontechnical skills using extracorporeal videos of both open radical cystectomy and robot-assisted radical cystectomy surgeries, in order to obtain detailed data on nontechnical skills during open and minimally invasive surgeries. The results of this study could possibly be used to develop team-training programmes, specifically for the introduction of the surgical robot in relation to changes in nontechnical skills. Additional analysis of technical skills using the intracorporeal footage of the surgical robot will be used to elucidate the role of surgical skills and surgical events in nontechnical skills.

Analysis of the video motion tracking system "Kinovea" to assess surgical movements during robot-assisted radical prostatectomy

BACKGROUNDS

Robot-assisted surgery facilitated the possibility to evaluate the surgeon's skills by recording and evaluating the robot surgical images. The aim of this study was to investigate the possibility of using a computer programme (Kinovea) for objective assessment of surgical movements in previously recorded in existing robot-assisted radical prostatectomy (RARP) videos.

METHODS

Twelve entire RARP procedures were analysed by a trained researcher using the computer programme "Kinovea" to perform semi-automated assessment of surgical movements.

RESULTS

Data analysis showed Kinovea was on average able to automatically assess only 22% of the total surgical duration per video of the robot-assisted surgery. On average, it lasted 4 hours of continued monitoring by the researcher to assess one RARP using Kinovea.

CONCLUSION

Although we proved it is technically possible to use the Kinovea system in retrospective analysis of surgical movement in robot-assisted surgery, the acquired data do not give a comprehensive enough analysis of the video to be used in skills assessment.

Technical Performance as a Predictor of Clinical Outcomes in Laparoscopic Gastric Cancer Surgery

OBJECTIVE

The purpose of this study was to evaluate the relationship between technical performance and patient outcomes in laparoscopic gastric cancer surgery.

BACKGROUND

Laparoscopic gastrectomy for cancer is an advanced procedure with high rate of postoperative morbidity and mortality. Many variables including patient, disease, and perioperative management factors have been shown to impact postoperative outcomes; however, the role of surgical performance is insufficiently investigated.

METHODS

A retrospective review was performed for all patients who had undergone laparoscopic gastrectomy for cancer at 3 teaching institutions between 2009 and 2015. Patients with available, unedited video-recording of their procedure were included in the study. Video files were rated for technical performance, using Objective Structured Assessments of Technical Skills (OSATS) and Generic Error Rating Tool instruments. The main outcome variable was major short-term complications. The effect of technical performance on patient outcomes was assessed using logistic regression analysis with backward selection strategy.

RESULTS

Sixty-one patients with available video recordings were included in the study. The overall complication rate was 29.5%. The mean Charlson comorbidity index, type of procedure, and the global OSATS score were included in the final predictive model. Lower performance score (OSATS ≤29) remained an independent predictor for major short-term outcomes (odds ratio 6.49), while adjusting for comorbidities and type of procedure.

CONCLUSIONS

Intraoperative technical performance predicts major short-term outcomes in laparoscopic gastrectomy for cancer. Ongoing assessment and enhancement of surgical skills using modern, evidence-based strategies might improve short-term patient outcomes. Future work should focus on developing and studying the effectiveness of such interventions in laparoscopic gastric cancer surgery.

Video-based coaching in surgical education: a systematic review and meta-analysis

BACKGROUND

In the era of competency-based surgical education, VBC has gained increased attention and may enhance the efficacy of surgical education. The objective of this systematic review was to summarize the existing evidence of video-based coaching (VBC) and compare VBC to traditional master-apprentice-based surgical education.

METHODS

We performed a systematic review and meta-analysis of randomized controlled trials (RCT) assessing VBC according to the PRISMA and Cochrane guidelines. The MEDLINE, EMBASE, and COCHRANE and Researchgate databases were searched for eligible manuscripts. Standard mean difference (SMD) of performance scoring scales was used to assess the effect of VBC versus traditional training without VBC (control).

RESULTS

Of 627 studies identified, 24 RCTs were eligible and evaluated. The studies included 778 surgical trainees (n = 386 VBC vs. n = 392 control). 13 performance scoring scales were used to assess technical competence; OSATS-GRS was the most common (n = 15). VBC was provided preoperative (n = 11), intraoperative (n = 1), postoperative (n = 10), and perioperative (n = 2). The majority of studies were unstructured, where identified coaching frameworks were PRACTICE (n = 1), GROW (n = 2) and Wisconsin Coaching Framework (n = 1). There was an effect on performance scoring scales in favor of VBC coaching (SMD 0.87, p < 0.001). In subgroup analyses, the residents had a larger relative effect (SMD 1.13; 0.61-1.65, p < 0.001) of VBC compared to medical students (SMD 0.43, 0.06-0.81, p < 0.001). The greatest source of potential bias was absence of blinding of the participants and personnel (n = 20).

CONCLUSION

Video-based coaching increases technical performance of medical students and surgical residents. There exist significant study and intervention heterogeneity that warrants further exploration, showing the need to structure and standardize video-based coaching tools.

Linking surgical skills to postoperative outcomes: a Delphi study on the robot-assisted radical prostatectomy

OBJECTIVE

To develop an assessment instrument for the evaluation of surgical videos to elucidate the association between surgical skills and postoperative outcomes after a robot-assisted radical prostatectomy (RARP).

DESIGN

A Delphi study consisting of two consecutive online surveys and a consensus group meeting.

SETTING

Urology departments of general, teaching and university hospitals in the Netherlands.

PARTICIPANTS

All Dutch urologists with a specialization in RARP.

RESULTS

Of 18 invited experts, 12 (67%) participated in the first online survey. In the second round, 9 of the 18 invited experts participated (50%). The Delphi meeting was attended by 5 of the 18 (27%) invited experts. The panel identified seven surgical steps with a possible association to postoperative outcomes. The experts also expected an association between adverse postoperative outcomes and the frequency of camera removals, the number of stitches placed, the amount of bleeding, and the extent of coagulation. These factors were incorporated into an assessment instrument.

CONCLUSIONS

Experts in the field of RARP achieved consensus on 7 surgical steps and 4 aspects of the RARP procedure that may be related to adverse postoperative outcomes. The resulting assessment instrument will be tested in future research to determine its validity.

A scoping review of assessment tools for laparoscopic suturing

BACKGROUND

A needs assessment identified a gap in teaching and assessment of laparoscopic suturing (LS) skills. The purpose of this review is to identify assessment tools that were used to assess LS skills, to evaluate validity evidence available, and to provide guidance for selecting the right assessment tool for specific assessment conditions.

METHODS

Bibliographic databases were searched till April 2017. Full-text articles were included if they reported on assessment tools used in the operating room/simulation to (1) assess procedures that require LS or (2) specifically assess LS skills.

RESULTS

Forty-two tools were identified, of which 26 were used for assessing LS skills specifically and 26 for procedures that require LS. Tools had the most evidence in internal structure and relationship to other variables, and least in consequences.

CONCLUSION

Through identification and evaluation of assessment tools, the results of this review could be used as a guideline when implementing assessment tools into training programs.

Contribution of an educational video to surgical education in laparoscopic appendectomy

Objective

With recent widespread advances in laparoscopy, providing education on this subject has become a significant challenge. The aim of this study was to determine the degree of contribution made by surgeons to laparoscopic education through an educational video.

Material and Methods

A total of 40 volunteer general surgery residents and specialists participated in our study. Before watching the approximately six-minute educational video prepared for laparoscopic appendectomy, the participants were asked to fill out participant information forms and information measurement questionnaire forms. After the video, the participants were asked to fill out the information measurement questionnaire forms a second time; additionally, attitude evaluation forms and education evaluation questionnaire forms were presented to the participants for completion, and statistical analysis was performed. Furthermore, the total watching duration and the number of times the video was paused were recorded.

Results

A total of 40 surgeons participated in the study (75% residents and 25% specialists). When the results of the information determination questionnaire forms filled out by the resident and specialist groups before and after the video were compared, it was observed that the scores of both groups significantly increased after watching the video (p=0.001). A statistical significance was identified between the length of time the video was watched and the education evaluation form scores of the participants (p<0.01). It was observed that the longer the video was watched, the greater the increase in the education evaluation scores. The results of the attitude evaluation forms implied that the video could be produced more professionally.

Conclusion

Although education is an inevitable requirement of laparoscopic surgery, many teaching methods are available. Awareness-enhancing videos prepared on this topic can be efficient in providing laparoscopic education.

Effect of Preoperative Warm-up Exercise Before Laparoscopic Gynecological Surgery: A Randomized Trial

BACKGROUND

Laparoscopic surgical procedures require a high level of cognitive and psychomotoric skills. Thus, effective training methods to acquire an adequate level of expertise are crucial. The aim of this study was to investigate the effect of preoperative warm up training on surgeon׳s performance during gynecologic laparoscopic surgery.

MATERIALS AND METHODS

In this randomized controlled trial, surgeons performed a preoperative warm up training using a virtual reality simulator before laparoscopic unilateral salpingo-oophorectomy. Serving as their own controls, each subject performed 2 pairs of laparoscopic cases, each pair consisting of 1 case with and 1 without warm up before surgery. Surgeries were videotaped and psychomotoric skills were rated using objective structured assessment of technical skills (OSATS) and the generic error rating tool by a masked observer. Perioperative complications were assessed. Statistical analysis was performed using a mixed model, and mean OSATS scores were compared between both the groups.

RESULTS

In total, data of 10 surgeons and 17 surgeries were available for analysis. No differences between educational level and surgical experiences were observed between the groups. Mean standard error psychomotoric and task-specific OSATS scores of 19.8 (1.7) and 3.7 (0.2) were observed in the warm up group compared with 18.6 (1.7) and 3.8 (0.2) in the no warm up group, respectively (p = 0.51 and p = 0.29). Using generic error rating tool, the total number of errors was 8.75 (2.15) in the warm up group compared with 10.8 (2.18) in the no warm-up group (p = 0.53). Perioperative complications and operating time did not differ between both the groups.

DISCUSSION

The present study suggests that warm-up before laparoscopic salpingo-oophorectomy does not increase psychomotoric skills during surgery. Moreover, it does not influence operating time and complication rates. (Medical University of Vienna-IRB approval number, 1072/2011, ClinicalTrials.gov number, NCT01712607).