Crowdsourced Assessment of Inanimate Biotissue Drills: A Valid and Cost-Effective Way to Evaluate Surgical Trainees

Pancreatic anastomosis training models: Current status and future directions

Postoperative pancreatic fistula (POPF) is a major cause of morbidity and mortality after pancreatoduodenectomy (PD), and previous research has focused on patient-related risk factors and comparisons between anastomotic techniques. However, it is recognized that surgeon experience is an important factor in POPF outcomes, and that there is a significant learning curve for the pancreatic anastomosis. The aim of this study was to review the current literature on training models for the pancreatic anastomosis, and to explore areas for future research. It is concluded that research is needed to understand the mechanical properties of the human pancreas in an effort to develop a synthetic model that closely mimics its mechanical properties. Virtual reality (VR) is an attractive alternative to synthetic models for surgical training, and further work is needed to develop a VR pancreatic anastomosis training module that provides both high fidelity and haptic feedback.

The development of a deep learning model for automated segmentation of the robotic pancreaticojejunostomy

BACKGROUND

Minimally invasive surgery provides an unprecedented opportunity to review video for assessing surgical performance. Surgical video analysis is time-consuming and expensive. Deep learning provides an alternative for analysis. Robotic pancreaticoduodenectomy (RPD) is a complex and morbid operation. Surgeon technical performance of pancreaticojejunostomy (PJ) has been associated with postoperative pancreatic fistula. In this work, we aimed to utilize deep learning to automatically segment PJ RPD videos.

METHODS

This was a retrospective review of prospectively collected videos from 2011 to 2022 that were in libraries at tertiary referral centers, including 111 PJ videos. Each frame of a robotic PJ video was categorized based on 6 tasks. A 3D convolutional neural network was trained for frame-level visual feature extraction and classification. All the videos were manually annotated for the start and end of each task.

RESULTS

Of the 100 videos assessed, 60 videos were used for the training the model, 10 for hyperparameter optimization, and 30 for the testing of performance. All the frames were extracted (6 frames/second) and annotated. The accuracy and mean per-class F1 scores were 88.01% and 85.34% for tasks.

CONCLUSION

The deep learning model performed well for automated segmentation of PJ videos. Future work will focus on skills assessment and outcome prediction.

Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review

BACKGROUND

There is a need to standardize training in robotic surgery, including objective assessment for accreditation. This systematic review aimed to identify objective tools for technical skills assessment, providing evaluation statuses to guide research and inform implementation into training curricula.

METHODS

A systematic literature search was conducted in accordance with the PRISMA guidelines. Ovid Embase/Medline, PubMed and Web of Science were searched. Inclusion criterion: robotic surgery technical skills tools. Exclusion criteria: non-technical, laparoscopy or open skills only. Manual tools and automated performance metrics (APMs) were analysed using Messick's concept of validity and the Oxford Centre of Evidence-Based Medicine (OCEBM) Levels of Evidence and Recommendation (LoR). A bespoke tool analysed artificial intelligence (AI) studies. The Modified Downs-Black checklist was used to assess risk of bias.

RESULTS

Two hundred and forty-seven studies were analysed, identifying: 8 global rating scales, 26 procedure-/task-specific tools, 3 main error-based methods, 10 simulators, 28 studies analysing APMs and 53 AI studies. Global Evaluative Assessment of Robotic Skills and the da Vinci Skills Simulator were the most evaluated tools at LoR 1 (OCEBM). Three procedure-specific tools, 3 error-based methods and 1 non-simulator APMs reached LoR 2. AI models estimated outcomes (skill or clinical), demonstrating superior accuracy rates in the laboratory with 60 per cent of methods reporting accuracies over 90 per cent, compared to real surgery ranging from 67 to 100 per cent.

CONCLUSIONS

Manual and automated assessment tools for robotic surgery are not well validated and require further evaluation before use in accreditation processes.PROSPERO: registration ID CRD42022304901.

Research Hotspots and Trend Exploration on the Clinical Translational Outcome of Simulation-Based Medical Education: A 10-Year Scientific Bibliometric Analysis From 2011 to 2021

BACKGROUND

In recent decades, an increasing number of studies have focused on the clinical translational effect of simulation-based medical education (SBME). However, few scientific bibliometric studies have analyzed the research hotspots and publication trends. This study aimed to investigate research hotspots and future direction in the clinical translational outcome of SBME via bibliometrics.

METHOD

Relevant publications on the clinical translational outcomes of SBME from 2011 to 2021 were identified and retrieved from the Web of Science Core Collection (WOSCC). Software including VOSviewer (1.6.17) and CiteSpace (5.8R3) and a platform (bibliometric.com) were employed to conduct bibliographic and visualized analysis on the literature.

RESULTS

A total of 1,178 publications were enrolled. An increasing number of publications were observed in the past decades from 48 in 2011 to 175 in 2021. The United States accounted for the largest number of publications (488, 41.4%) and citations (10,432); the University of Toronto and Northwestern University were the leading institutions. Academic Medicine was the most productive journal concerning this field. McGaghie W C and Konge L were the most influential authors in this area. The hot topic of the translational outcome of SBME was divided into 3 stages, laboratory phase, individual skill improvement, and patient outcome involving both technical skills and non-technical skills. Translational research of comprehensive impact and collateral outcomes could be obtained in the future.

CONCLUSION

From the overall trend of 10 years of research, we can see that the research is roughly divided into three phases, from laboratory stage, individual skill improvement to the patient outcomes, and comprehensive impacts such as skill retention and collateral effect as cost-effectiveness is a major trend of future research. More objective evaluation measurement should be designed to assess the diverse impact and further meta-analysis and randomized controlled trials are needed to provide more clinical evidence of SBME as translational science.

dV-Trainer vs. da Vinci Simulator: Comparison of Virtual Reality Platforms for Robotic Surgery

BACKGROUND

A virtual reality (VR) curriculum performed on the da Vinci Simulation System (DVSS) was previously shown to be effective in training fellows. The dV-Trainer is a separate platform with similar features to the da Vinci console, but its efficacy and utility versus the DVSS simulator are not well known.

MATERIALS AND METHODS

A mastery-based VR curriculum was completed by surgical fellows on the DVSS (2014-2016) and on the dV-Trainer (2016-2018) at a large academic center. Pre-test/post-test scores were used to evaluate performance between the two groups. Data was collected prospectively.

RESULTS

Forty-six fellows enrolled in the curriculum: surgical oncology (n=31), hepatobiliary (n=5), head/neck (n=4), endocrine (n=2), cardiothoracic (n=2), gynecology (n=1) and transplant surgery (n=1). Twenty-four used the DVSS and twenty-two used the dV-Trainer. Compared to the DVSS, the dV-Trainer was associated with lower scores on 2 of 3 VR modules in the pre-test (P=0.027, P<0.001, respectively) and post-test (P=0.021, P<0.001, respectively). Fellows in the dV-Trainer era scored lower on inanimate drills as well. Average VR curriculum score was lower on the dV-Trainer (71.3% vs 83.34%, P<0.001). dV-Trainer users spent more time completing the pre-test and post-test; however, overall simulator time to complete the curriculum was not significantly different (297 vs 231 minutes, P=0.142). Both groups showed improvement in scores after completion of the VR curriculum.

CONCLUSIONS

The dV-Trainer simulator allows for more usability outside the operating room to complete VR modules; however, the DVSS simulator group outperformed the dV-Trainer group on the post-test.

Robotic gastrointestinal surgery: learning curve, educational programs and outcomes

The use of the robotic platform for gastrointestinal surgery was introduced nearly 20 years ago. However, significant growth and advancement has occurred primarily in the last decade. This is due to several advantages over traditional laparoscopic surgery allowing for more complex dissections and reconstructions. Several randomized controlled trials and retrospective reviews have demonstrated equivalent oncologic outcomes compared to open surgery with improved short-term outcomes. Unfortunately, there are currently no universally accepted or implemented training programs for robotic surgery and robotic surgery experience varies greatly. Additionally, several limitations to the robotic platform exist resulting in a distinct learning curve associated with various procedures. Therefore, implementation of robotic surgery requires a multidisciplinary team approach with commitment and investment from clinical faculty, operating room staff and hospital administrators. Additionally, there is a need for wider distribution of educational modules to train more surgeons and reduce the associated learning curve. This article will focus on the implementation of the robotic platform for surgery of the pancreas, stomach, liver, colon and rectum with an emphasis on the associated learning curve, educational platforms to develop proficiency and perioperative outcomes.

State of the art robotic distal pancreatectomy: a review of the literature

Minimally invasive distal pancreatectomy has become increasingly used in practice. While laparoscopic approach is the most commonly used technique, robotic distal pancreatectomy (RDP) has emerged as a safe, feasible and effective approach for distal pancreatectomy. Most studies have shown that RDP improved perioperative surgical outcomes and has equivalent oncologic outcomes to open technique. Widespread adoption is limited by a steep learning curve, higher costs and the need for institutional training protocols in place for safe integration of the platform into practice.

500 Minimally Invasive Robotic Pancreatoduodenectomies: One Decade of Optimizing Performance

OBJECTIVES

This study aims to present the outcomes of our decade-long experience of robotic pancreatoduodenectomy and provide insights into successful program implementation.

BACKGROUND

Despite significant improvement in mortality over the past 30 years, morbidity following open pancreatoduodenectomy remains high. We implemented a minimally invasive pancreatic surgery program based on the robotic platform as one potential method of improving outcomes for this operation.

METHODS

A retrospective review of a prospectively maintained institutional database was performed to identify patients who underwent robotic pancreatoduodenectomy (RPD) between 2008 and 2017 at the University of Pittsburgh.

RESULTS

In total, 500 consecutive RPDs were included. Operative time, conversion to open, blood loss, and clinically relevant postoperative pancreatic fistula improved early in the experience and have remained low despite increasing complexity of case selection as reflected by increasing number of patients with pancreatic cancer, vascular resections, and higher Charlson Comorbidity scores (all P<0.05). Operating room time plateaued after 240 cases at a median time of 391 minutes (interquartile rang 340-477). Major complications (Clavien >2) occurred in less than 24%, clinically relevant postoperative pancreatic fistula in 7.8%, 30- and 90-day mortality were 1.4% and 3.1% respectively, and median length of stay was 8 days. Outcomes were not impacted by integration of trainees or expansion of selection criteria.

CONCLUSIONS

Structured implementation of robotic pancreatoduodenectomy can be associated with excellent outcomes. In the largest series of RPD, we establish benchmarks for the surgical community to consider when adopting this approach.

Will It Play in Peoria? A Pilot Study of a Robotic Skills Curriculum for Surgical Oncology Fellows

INTRODUCTION

To implement a mastery-based robotic surgery curriculum using virtual reality (VR) and inanimate reality (IR) drills at multiple Complex General Surgical Oncology (CGSO) fellowships.

PATIENTS AND METHODS

A prospective study of curriculum feasibility and efficacy was conducted at four CGSO fellowship sites. All sites had simulators, and kits were provided to perform 19 biotissue drills. Fellows from three non-UPMC sites (n = 15) in 2016-2018 were compared with fellows from University of Pittsburgh (UPMC; n = 15) where the curriculum was validated in 2014-2018.

RESULTS

All fellows completed the pre- and post-test. There was no difference in pre-test scores between UPMC and non-UPMC sites. Only 7 of 15 non-UPMC fellows completed the VR curriculum (47% compliance) compared with all 15 UPMC fellows completing the VR curriculum (100% compliance). UPMC had higher curriculum times (217 versus 93 mins) and % mastery (86% versus 55%). Time spent on curriculum was associated with % mastery (p = 0.01). Both groups showed improvement between pre- and post-test. Post-test VR scores trended higher for UPMC (221 versus 180). Between the non-UPMC sites, there was a difference in compliance (p = 0.03) and % mastery (p = 0.03). Zero non-UPMC fellows performed the biotissue drills, while five contemporary UPMC fellows completed 253 biotissue drills. Approximately 140 UPMC faculty and 300 staff hours were spent on the pilot.

CONCLUSIONS

A proficiency curriculum can result in improved robotic console skills. However, multiple barriers to implementation potentially exist, including availability of simulators, availability of a training robot, on-site support staff, and universal buy-in from fellows, faculty, and leadership.

Association of Mentorship and a Formal Robotic Proficiency Skills Curriculum With Subsequent Generations' Learning Curve and Safety for Robotic Pancreaticoduodenectomy

Importance

Learning curves are unavoidable for practicing surgeons when adopting new technologies. However, patient outcomes are worse in the early stages of a learning curve vs after mastery. Therefore, it is critical to find a way to decrease these learning curves without compromising patient safety.

Objective

To evaluate the association of mentorship and a formal proficiency-based skills curriculum with the learning curves of 3 generations of surgeons and to determine the association with increased patient safety.

Design, Setting, and Participants

All consecutive robotic pancreaticoduodenectomies (RPDs) performed at the University of Pittsburgh Medical Center between 2008 and 2017 were included in this study. Surgeons were split into generations based on their access to mentorship and a proficiency-based skills curriculum. The generations are (1) no mentorship or curriculum, (2) mentorship but no curriculum, and (3) mentorship and curriculum. Univariable and multivariable analyses were used to create risk-adjusted learning curves by surgical generation and to analyze factors associated with operating room time, complications, and fellows completing the full resection. The participants include surgical oncology attending surgeons and fellows who participated in an RPD at University of Pittsburgh Medical Center between 2008 and 2017.

Main Outcomes and Measures

The primary outcome was operating room time (ORT). Secondary outcomes were postoperative pancreatic fistula and Clavien-Dindo classification higher than grade 2.

Results

We identified 514 RPDs completed between 2008 and 2017, of which 258 (50.2%) were completed by first-generation surgeons, 151 (29.3%) were completed by the second generation, and 82 (15.9%) were completed by the third generation. There was no statistically significant difference between groups with respect to age (66.3-67.3 years; P = .52) or female sex (n = 34 [41.5%] vs n = 121 [46.9%]; P = .60). There was a significant decrease in ORT (P < .001), from 450.8 minutes for the first-generation surgeons to 348.6 minutes for the third generation. Additionally, across generations, Clavien-Dindo classification higher than grade 2 (n = 74 [28.7%] vs n = 30 [9.9%] vs n = 12 [14.6%]; P = .01), conversion rates (n = 18 [7.0%] vs n = 7 [4.6%] vs n = 0; P = .006), and estimated blood loss (426 mL vs 288.6 mL vs 254.7 mL; P < .001) decreased significantly with subsequent generations. There were no significant differences in postoperative pancreatic fistula.

Conclusions and Relevance

In this study, ORT, conversion rates, and estimated blood loss decreased across generations without a concomitant rise in adverse patient outcomes. These findings suggest that a proficiency-based curriculum coupled with mentorship allows for the safe introduction of less experienced surgeons to RPD without compromising patient safety.

How to train and evaluate minimally invasive pancreas surgery

Training for minimally invasive pancreas surgery is critical as an evolving body of literature supports its use with acceptable outcomes during training and improved short term outcomes following completion. Although case volume needed to achieve mastery remains unclear, improved outcomes for both laparoscopic and robotic pancreatectomy are demonstrated following a learning curve and inflection point. Therefore, dedicated training curricula for both laparoscopic and robotic pancreatectomy have been developed to mitigate this learning curve and improve outcomes.

Oncologic Outcomes After Robotic Pancreatic Resections Are Not Inferior to Open Surgery

OBJECTIVE

Compare oncologic outcomes after open and robotic pancreatic resections for pancreatic adenocarcinoma (PDAC).

SUMMARY OF BACKGROUND DATA

Receipt of adjuvant chemotherapy improves survival after resected PDAC. Complications after pancreatectomy have been shown to prohibit the administration of adjuvant chemotherapy and survival. We examined the effect of surgical approach on receipt of adjuvant chemotherapy, complications, and overall survival after pancreatectomy.

METHODS

A single-institution retrospective review of all patients with PDAC who underwent robotic or open pancreatectomy from 2011 to 2016 with 24-month follow-up.

RESULTS

Four hundred fifty-six patients underwent resection: 226 robotic and 230 open. No significant difference was identified in major complications or receipt of adjuvant chemotherapy between robotic and open pancreatectomy, nor was approach an independent predictor of these outcomes. Robotic pancreatectomy patients had a shorter length of stay than patients who underwent open pancreatectomy (7 days vs 9 days; P < 0.001). Additionally, wound infection rate (32.3% vs 12.4%, P < 0.0001) and transfusion (39.6% vs 12.4%, P < 0.0001) was improved in robotic pancreatectomy group with no differences in perioperative mortality. Improved median overall survival approached statistical significance for the robotic cohort (25.6 months vs 23.9 months; P = 0.055); however, on multivariable analysis the robotic approach predicted overall survival, (hazard ratio 0.77, P = 0.041). Robotic approach was an independent predictor of decreased blood loss and less transfusions than the open approach.

CONCLUSIONS

Robotic pancreatectomy was not inferior compared to open pancreatectomy in a high-volume experienced center for oncologic outcomes and due to decreased blood loss and transfusion may have improved survival.