Validation study of a skill assessment tool for education and outcome prediction of laparoscopic distal gastrectomy

Surgical quality assessment of critical view of safety in 283 laparoscopic cholecystectomy videos by surgical residents and surgeons

INTRODUCTION

Surgical quality assessment has improved the efficacy and efficiency of surgical training and has the potential to optimize the surgical learning curve. In laparoscopic cholecystectomy (LC), the critical view of safety (CVS) can be assessed with a 6-point competency assessment tool (CAT), a task commonly performed by experienced surgeons. The aim of this study is to determine the capability of surgical residents to perform this assessment.

METHODS

Both surgeons and surgical residents assessed unedited LC videos using a 6-point CVS, a CAT, using an online video assessment platform. The CAT consists of the following three criteria: 1. clearance of hepatocystic triangle, 2. cystic plate, and 3. two structures connect to the gallbladder, with a maximum of 2 points available for each criterion. A higher score indicates superior surgical performance. The intraclass correlation coefficient (ICC) was employed to assess the inter-rater reliability between surgeons and surgical residents.

RESULTS

In total, 283 LC videos were assessed by 19 surgeons and 31 surgical residents. The overall ICC for all criteria was 0.628. Specifically, the ICC scores were 0.504 for criterion 1, 0.639 for criterion 2, and 0.719 for the criterion involving the two structures connected to the gallbladder. Consequently, only the criterion regarding clearance of the hepatocystic triangle exhibited fair agreement, whereas the other two criteria, as well as the overall scores, demonstrated good agreement. In 71% of cases, both surgeons and surgical residents scored a total score either ranging from 0 to 4 or from 5 to 6.

CONCLUSION

Compared to the gold standard, i.e., the surgeons' assessments, surgical residents are equally skilled at assessing critical view of safety (CVS) in laparoscopic cholecystectomy (LC) videos. By incorporating video-based assessments of surgical procedures into their training, residents could potentially enhance their learning pace, which may result in better clinical outcomes.

Nationwide standardization of minimally invasive right hemicolectomy for colon cancer and development and validation of a video-based competency assessment tool (the Right study)

BACKGROUND

Substantial variation exists when performing a minimally invasive right hemicolectomy (MIRH) due to disparities in training, expertise and differences in implementation of innovations. This study aimed to achieve national consensus on an optimal and standardized MIRH technique for colon cancer and to develop and validate a video-based competency assessment tool (CAT) for MIRH.

METHOD

Statements covering all elements of MIRH were formulated. Subsequently, the Delphi technique was used to reach consensus on a standardized MIRH among 76 colorectal surgeons from 43 different centres. A CAT was developed based on the Delphi results. Nine surgeons assessed the same 12 unedited full-length videos using the CAT, allowing evaluation of the intraclass correlation coefficient (ICC).

RESULTS

After three Delphi rounds, consensus (≥80% agreement) was achieved on 23 of the 24 statements. Consensus statements included the use of low intra-abdominal pressure, detailed anatomical outline how to perform complete mesocolic excision with central vascular ligation, the creation of an intracorporeal anastomosis, and specimen extraction through a Pfannenstiel incision using a wound protector. The CAT included seven consecutive steps to measure competency of the MIRH and showed high consistency among surgeons with an overall ICC of 0.923.

CONCLUSION

Nationwide consensus on a standardized and optimized technique of MIRH was reached. The CAT developed showed excellent interrater reliability. These achievements are crucial steps to an ongoing nationwide quality improvement project (the Right study).

National implementation of an optimal standardised technique for right-sided colon cancer: protocol of an interventional sequential cohort study (Right study)

PURPOSE

Minimally invasive right hemicolectomy (MIRH) is the cornerstone of treatment for patients with right-sided colon cancer. This operation has evolved during recent decades, with many innovations and improvements but this has also resulted in high variability of uptake with subsequent substantial variableness. The aim of this ongoing study is to identify current surgical variations, determine the most optimal and standardised MIRH and nationally train and implement that technique to improve short-term clinical and long-term oncological outcomes.

METHODS

The Right study is a national multicentre prospective interventional sequential cohort study. Firstly, current local practice was evaluated. Subsequently, a standardised surgical technique for right-sided colon cancer was determined using the Delphi consensus method, and this procedure was trained during hands-on courses. The standardised MIRH will be implemented with proctoring (implementation cohort), after which the performance will be monitored (consolidation cohort). Patients who will receive a minimally invasive (extended) right hemicolectomy for cT1-3N0-2M0 colon cancer will be included. The primary outcome is patient safety reflected in the 90-day overall complication rate according to the Clavien-Dindo classification. Secondary outcomes will include intraoperative complications, 90-day mortality rate, number of resected tumour-positive lymph nodes, completeness of mesocolic excision, surgical quality score, locoregional and distant recurrence and 5-year overall survival. A total number of 1095 patients (365 per cohort) will be included.

DISCUSSION

The Right study is designed to safely implement the best surgical practice concerning patients with right-sided colon cancer aiming to standardise and improve the surgical quality of MIRH at a national level.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT04889456, May 2021.

Surgical skills assessment of pancreaticojejunostomy using a simulator may predict patient outcomes: A multicenter prospective observational study

BACKGROUND

Pancreatoduodenectomy, an advanced surgical procedure with a high complication rate, requires surgical skill in performing pancreaticojejunostomy, which correlates with operative outcomes. We aimed to analyze the correlation between pancreaticojejunostomy assessment conducted in a simulator environment and the operating room and patient clinical outcomes.

METHODS

We recruited 30 surgeons (with different experience levels in pancreatoduodenectomy) from 11 institutes. Three trained blinded raters assessed the videos of the pancreaticojejunostomy procedure performed in the operating room using a simulator according to an objective structured assessment of technical skill and a newly developed pancreaticojejunostomy assessment scale. The correlations between the assessment score of the pancreaticojejunostomy performed in the operating room and using the simulator and between each assessment score and patient outcomes were calculated. The participants were also surveyed regarding various aspects of the simulator as a training tool.

RESULTS

There was no correlation between the average score of the pancreaticojejunostomy performed in the operating room and that in the simulator environment (r = 0.047). Pancreaticojejunostomy scores using the simulator were significantly lower in patients with postoperative pancreatic fistula than in those without postoperative pancreatic fistula (P = .05). Multivariate analysis showed that pancreaticojejunostomy assessment scores were independent factors in postoperative pancreatic fistula (P = .09). The participants highly rated the simulator and considered that it had the potential to be used for training.

CONCLUSION

There was no correlation between pancreaticojejunostomy surgical performance in the operating room and the simulation environment. Surgical skills evaluated in the simulation setting could predict patient surgical outcomes.

Video-based tools for surgical quality assessment of technical skills in laparoscopic procedures: a systematic review

BACKGROUND

Quality of surgery has substantial impact on both short- and long-term clinical outcomes. This stresses the need for objective surgical quality assessment (SQA) for education, clinical practice and research purposes. The aim of this systematic review was to provide a comprehensive overview of all video-based objective SQA tools in laparoscopic procedures and their validity to objectively assess surgical performance.

METHODS

PubMed, Embase.com and Web of Science were systematically searched by two reviewers to identify all studies focusing on video-based SQA tools of technical skills in laparoscopic surgery performed in a clinical setting. Evidence on validity was evaluated using a modified validation scoring system.

RESULTS

Fifty-five studies with a total of 41 video-based SQA tools were identified. These tools were used in 9 different fields of laparoscopic surgery and were divided into 4 categories: the global assessment scale (GAS), the error-based assessment scale (EBAS), the procedure-specific assessment tool (PSAT) and artificial intelligence (AI). The number of studies focusing on these four categories were 21, 6, 31 and 3, respectively. Twelve studies validated the SQA tool with clinical outcomes. In 11 of those studies, a positive association between surgical quality and clinical outcomes was found.

CONCLUSION

This systematic review included a total of 41 unique video-based SQA tools to assess surgical technical skills in various domains of laparoscopic surgery. This study suggests that validated SQA tools enable objective assessment of surgical performance with relevance for clinical outcomes, which can be used for training, research and quality improvement programs.