Proficiency-based Progression Training: A Scientific Approach to Learning Surgical Skills

Objective assessment tools in laparoscopic or robotic-assisted gynecological surgery: A systematic review

INTRODUCTION

There is a growing emphasis on proficiency-based progression within surgical training. To enable this, clearly defined metrics for those newly acquired surgical skills are needed. These can be formulated in objective assessment tools. The aim of the present study was to systematically review the literature reporting on available tools for objective assessment of minimally invasive gynecological surgery (simulated) performance and evaluate their reliability and validity.

MATERIAL AND METHODS

A systematic search (1989-2022) was conducted in MEDLINE, Embase, PubMed, Web of Science in accordance with PRISMA. The trial was registered with the Prospective Register of Systematic Reviews (PROSPERO) ID: CRD42022376552. Randomized controlled trials, prospective comparative studies, prospective single-group (with pre- and post-training assessment) or consensus studies that reported on the development, validation or usage of assessment tools of surgical performance in minimally invasive gynecological surgery, were included. Three independent assessors assessed study setting and validity evidence according to a contemporary framework of validity, which was adapted from Messick's validity framework. Methodological quality of included studies was assessed using the modified medical education research study quality instrument (MERSQI) checklist. Heterogeneity in data reporting on types of tools, data collection, study design, definition of expertise (novice vs. experts) and statistical values prevented a meaningful meta-analysis.

RESULTS

A total of 19 746 titles and abstracts were screened of which 72 articles met the inclusion criteria. A total of 37 different assessment tools were identified of which 13 represented manual global assessment tools, 13 manual procedure-specific assessment tools and 11 automated performance metrices. Only two tools showed substantive evidence of validity. Reliability and validity per tool were provided. No assessment tools showed direct correlation between tool scores and patient related outcomes.

CONCLUSIONS

Existing objective assessment tools lack evidence on predicting patient outcomes and suffer from limitations in transferability outside of the research environment, particularly for automated performance metrics. Future research should prioritize filling these gaps while integrating advanced technologies like kinematic data and AI for robust, objective surgical skill assessment within gynecological advanced surgical training programs.

Neurocognitive Concepts of Arthroscopic Surgical Training

» Arthroscopy is used to treat a broad variety of orthopaedic conditions.» The technical aspects of arthroscopic surgery are distinct from traditional open surgery and require different approaches to education and training.» There are neurocognitive concepts related to learning, memory, and performance that are recognized and understood in many fields and relevant but not commonly and specifically considered in orthopaedic surgery.» The purpose of this review was to introduce and discuss neurocognitive principles and concepts of visuospatial motor skill acquisition and proficiency to provide a background to support the development of arthroscopy educational curricula and training.

International consensus panel for transurethral resection of bladder tumours metrics: assessment of face and content validity

OBJECTIVE

To develop performance metrics that objectively define a reference approach to a transurethral resection of bladder tumours (TURBT) procedure, seek consensus on the performance metrics from a group of international experts.

METHODS

The characterisation of a reference approach to a TURBT procedure was performed by identifying phases and explicitly defined procedure events (i.e., steps, errors, and critical errors). An international panel of experienced urologists (i.e., Delphi panel) was then assembled to scrutinise the metrics using a modified Delphi process. Based on the panel's feedback, the proposed metrics could be edited, supplemented, or deleted. A voting process was conducted to establish the consensus level on the metrics. Consensus was defined as the panel majority (i.e., >80%) agreeing that the metric definitions were accurate and acceptable. The number of metric units before and after the Delphi meeting were presented.

RESULTS

A core metrics group (i.e., characterisation group) deconstructed the TURBT procedure. The reference case was identified as an elective TURBT on a male patient, diagnosed after full diagnostic evaluation with three or fewer bladder tumours of ≤3 cm. The characterisation group identified six procedure phases, 60 procedure steps, 43 errors, and 40 critical errors. The metrics were presented to the Delphi panel which included 15 experts from six countries. After the Delphi, six procedure phases, 63 procedure steps, 47 errors, and 41 critical errors were identified. The Delphi panel achieved a 100% consensus.

CONCLUSION

Performance metrics to characterise a reference approach to TURBT were developed and an international panel of experts reached 100% consensus on them. This consensus supports their face and content validity. The metrics can now be used for a proficiency-based progression training curriculum for TURBT.

Comparative analysis of robot-assisted simple prostatectomy: the HUGO™ RAS system versus the DaVinci® Xi system

BACKGROUND

Robot-Assisted Simple Prostatectomy (RASP) has emerged as a promising alternative in the treatment of benign prostatic obstruction (BPO). However, there is currently a lack of comparative studies evaluating different robotic platforms for performing RASP. Therefore, we aimed to compare perioperative and functional outcomes of RASP performed using the HUGO™ RAS System versus the DaVinci® Xi System.

METHODS

Forty consecutive cases of RASP performed between May 2021 and March 2023 with the HUGO™ RAS and the DaVinci® Xi at OLV Hospital (Aalst, Belgium) were included in this retrospective study. All surgeries were performed by three experienced surgeons using the same approach. Baseline characteristics, peri-operative and functional outcomes were collected and compared between the two groups.

RESULTS

The population was equally divided between the two groups with 20 patients in each group. There were no significant differences in preoperative patient characteristics between the two groups, except for the presence of bladder stones prior to the surgery (p = 0.03). No significant differences in total operative time and console time between the two groups were reported (p = 0.3). No cases required conversion to open surgery or additional port placement. During one case performed with the HUGO™ RAS, a malfunctioning monopolar curved shear had to be replaced. However, there was no statistically significant differences in terms of technical robotic problems between the groups (p = 0.3). There was no significant difference between the two groups in perioperative and functional outcomes (all p ≥ 0.2).

CONCLUSIONS

We did not observe any statistically significant difference in perioperative and functional outcomes in case of RASP performed with the HUGO™ RAS System and with the DaVinci® Xi System. These findings provide compelling support for considering the HUGO™ RAS as a promising tool for robot-assisted procedures, thereby expanding the utilization of robotics for benign conditions.

There is a need for a paradigm shift in laparoscopic surgical training: results of a nationwide survey among teaching hospitals in Switzerland

BACKGROUND

Surgical training curricula have changed little over the past decades. Current advances in surgical techniques, especially in minimally invasive surgery, as well as the rapidly changing socioeconomic environment pose a major challenge for the training of young surgeons. The aim of this survey was to provide a representative overview of the surgical training landscape in Switzerland focusing on laparoscopic surgical training: How do department chairs of teaching hospitals deal with the above challenges, and what should a future training curriculum look like?

METHODS

This is a prospective, questionnaire-based, cross-sectional study among the heads of departments of all certified surgical teaching hospitals in Switzerland.

RESULTS

The overall response rate was 56% (48/86) and 86% (19/22) for tertiary centers. Two-thirds of the centers (32) organize themselves in training networks. Laparoscopic training courses are offered in 25 (52%) hospitals, mainly in tertiary centers. Self-training opportunities exist in 40 (83%) hospitals. In addition to commercial (27) and self-built (7) box trainers, high-fidelity trainers are available in 16 (33%) hospitals. A mandatory training curriculum exists in 7 (15%) facilities, and a training assessment is performed in 15 (31%) institutions. Thirty-two (65%) heads of departments indicated that residents have sufficient practical exposure in the operating room, but the ability to work independently with obtaining the specialist title is seen critically (71%). They state that the surgical catalog does not adequately reflect the manual skills of the resident (64%). The desire is for training to be restructured from a numbers-based to a performance-based curriculum (53%) and for tools to assess residents' manual skills (56%) to be introduced.

CONCLUSIONS

Department chairs stated that the existing curriculum in Switzerland does not meet the requirements of a modern training curriculum. This study highlights the need to create an improved, competency-based curriculum that ensures the training of a new generation of surgeons, taking into account the growing evidence of the effectiveness of state-of-the-art training modalities such as simulation or proficiency-based training.

Comparison of Global Evaluative Assessment of Robotic Surgery with objective performance indicators for the assessment of skill during robotic-assisted thoracic surgery

BACKGROUND

The Global Evaluative Assessment of Robotic Skills is a popular but ultimately subjective assessment tool in robotic-assisted surgery. An alternative approach is to record system or console events or calculate instrument kinematics to derive objective performance indicators. The aim of this study was to compare these 2 approaches and correlate the Global Evaluative Assessment of Robotic Skills with different types of objective performance indicators during robotic-assisted lobectomy.

METHODS

Video, system event, and kinematic data were recorded from the robotic surgical system during left upper lobectomy on a standardized perfused and pulsatile ex vivo porcine heart-lung model. Videos were segmented into steps, and the superior vein dissection was graded independently by 2 blinded expert surgeons with Global Evaluative Assessment of Robotic Skills. Objective performance indicators representing categories for energy use, event data, movement, smoothness, time, and wrist articulation were calculated for the same task and compared to Global Evaluative Assessment of Robotic Skills scores.

RESULTS

Video and data from 51 cases were analyzed (44 fellows, 7 attendings). Global Evaluative Assessment of Robotic Skills scores were significantly higher for attendings (P < .05), but there was a significant difference in raters' scores of 31.4% (defined as >20% difference in total score). The interclass correlation was 0.44 for 1 rater and 0.61 for 2 raters. Objective performance indicators correlated with Global Evaluative Assessment of Robotic Skills to varying degrees. The most highly correlated Global Evaluative Assessment of Robotic Skills domain was efficiency. Instrument movement and smoothness were highly correlated among objective performance indicator categories. Of individual objective performance indicators, right-hand median jerk, an objective performance indicator of change of acceleration, had the highest correlation coefficient (0.55).

CONCLUSION

There was a relatively poor overall correlation between the Global Evaluative Assessment of Robotic Skills and objective performance indicators. However, both appear strongly correlated for certain metrics such as efficiency and smoothness. Objective performance indicators may be a potentially more quantitative and granular approach to assessing skill, given that they can be calculated mathematically and automatically without subjective interpretation.

A spaced retraining schedule with 2-day interval improves the acquisition and retention of simulation-based basic arthroscopic skills

PURPOSE

To compare the effect of three differently spaced retraining schedules (1-day, 2-day, and 1-week intervals) on the acquisition of basic arthroscopic skills and skill retention after 3 months.

METHODS

Thirty orthopaedic residents without arthroscopic experience were enrolled in a double-blind, randomised, parallel-controlled trial. Spaced retaining schedules were divided into massed training and retraining phases. Participants were required to obtain perfect scores in all tasks on the simulator in the massed training phase, followed by a pretest to evaluate the training effect. During the retraining phase, participants were randomly assigned to Groups A (1-day interval), B (2-day interval) or C (1-week interval). A posttest was used to evaluate the effect of different retraining patterns. Follow-up evaluations were conducted at 1 week, 1 month and 3 months after the completion of spaced retraining schedules to measure skill retention. One-way ANOVA and paired-sample t tests were used for statistical analysis.

RESULTS

Significant between-group differences in diagnostic arthroscopy (137.0 ± 24.8 vs. 140.1 ± 21.3 vs. 175.3 ± 27.4 s, P(A-C) = 0.005, P(B-C) = 0.010) and loose body removal (193.1 ± 33.9 vs. 182.0 ± 32.1 vs. 228.7 ± 42.9 s, P(B-C) = 0.025) completion times were observed. No significant differences were found in other posttest metrics. An assessment of skill retention after the 3-month follow-up (Evaluation 3) showed significant differences in diagnostic arthroscopy completion time (202.5 ± 53.3 vs. 172.0 ± 27.2 vs. 225.5 ± 42.1 s, P(B-C) = 0.026). No significant differences were found in other Evaluation 3 metrics.

CONCLUSION

The 2-day retraining schedule was the most effective for the acquisition and retention of basic arthroscopic skills and could be integrated into arthroscopic skills curricula. After a 3-month follow-up, residents who followed this schedule showed better skill retention than those who followed the 1-week interval schedule.

LEVEL OF EVIDENCE

Level I.

Discrimination, Reliability, Sensitivity, and Specificity of Robotic Surgical Proficiency Assessment With Global Evaluative Assessment of Robotic Skills and Binary Scoring Metrics: Results From a Randomized Controlled Trial

Objective

To compare binary metrics and Global Evaluative Assessment of Robotic Skills (GEARS) evaluations of training outcome assessments for reliability, sensitivity, and specificity.

Background

GEARS-Likert-scale skills assessment are a widely accepted tool for robotic surgical training outcome evaluations. Proficiency-based progression (PBP) training is another methodology but uses binary performance metrics for evaluations.

Methods

In a prospective, randomized, and blinded study, we compared conventional with PBP training for a robotic suturing, knot-tying anastomosis task. Thirty-six surgical residents from 16 Belgium residency programs were randomized. In the skills laboratory, the PBP group trained until they demonstrated a quantitatively defined proficiency benchmark. The conventional group were yoked to the same training time but without the proficiency requirement. The final trial was video recorded and assessed with binary metrics and GEARS by robotic surgeons blinded to individual, group, and residency program. Sensitivity and specificity of the two assessment methods were evaluated with area under the curve (AUC) and receiver operating characteristics (ROC) curves.

Results

The PBP group made 42% fewer objectively assessed performance errors than the conventional group (P < 0.001) and scored 15% better on the GEARS assessment (P = 0.033). The mean interrater reliability for binary metrics and GEARS was 0.87 and 0.38, respectively. Binary total error metrics AUC was 97% and for GEARS 85%. With a sensitivity threshold of 0.8, false positives rates were 3% and 25% for, respectively, the binary and GEARS assessments.

Conclusions

Binary metrics for scoring a robotic VUA task demonstrated better psychometric properties than the GEARS assessment.

The learning curve for open and minimally-invasive kidney transplantation: a systematic review

INTRODUCTION

There is lack of evidence on the impact of surgeons' learning curve on postoperative outcomes after open (OKT) or minimally-invasive (robot-assisted) kidney transplantation (RAKT). The aim of the review was to assess the learning curve (LC) for OKT and RAKT, focusing on intra-, perioperative and functional outcomes.

EVIDENCE ACQUISITION

A systematic review of the English-language literature published between 01/01/2000 - 10/12/2021 was conducted using the MEDLINE (Via PubMed), Web of Science and the Cochrane Library databases according to the principles highlighted by the EAU Guidelines Office and the PRISMA statement recommendations. The review protocol was registered on PROSPERO (CRD42022301132). The overall quality of evidence was assessed according to GRADE recommendations.

EVIDENCE SYNTHESIS

Twelve studies were included in the qualitative analysis. Surgical competence in terms of operative and re-warming times was defined after 30 cases in OKT and after 11-35 cases in RAKT. Decreased complications rates were observed after 20-33 cases in OKT and 10-30 cases in RAKT. Optimal functional outcomes were achieved after 33 cases in OKT and 15-25 cases in RAKT. However, while a poor OKT experience did not influence the LC for RAKT, lack of robotic surgery exposure could lead to a longer LC for the robotic approach.

CONCLUSIONS

OKT and RAKT appear to have similar LCs and might require about 30 cases to achieve optimal surgical and functional outcomes. Previous expertise in OKT is warranted to shorten the LC for RAKT. Further research is needed to validate these thresholds using standardized reporting metrics.

Transferability of robotic console skills by early robotic surgeons: a multi-platform crossover trial of simulation training

Robotic surgical training is undergoing a period of transition now that new robotic operating platforms are entering clinical practice. As this occurs, training will need to be adapted to include strategies to train across various consoles. These new consoles differ in multiple ways, with some new vendors using flat screen open source 3D enhanced vision with glasses and differences in design will require surgeons to learn new skills. This process has parallels with aviation credentialling across different aircraft described as type rating. This study was designed to test the hypothesis that technical robotic console operating skills are transferrable across different robotic operating platforms. Ten participants sequentially completed four Mimic®(Surgical Science) simulation exercises on two different robotic operating platforms (DaVinci®, Intuitive Surgical and HUGO™ RAS, Medtronic). Ethical approval and informed consent were obtained for this study. Groups were balanced for key demographics including previous robotic simulator experience. Data for simulation metrics and time to proficiency were collected for each attempt at the simulated exercise and analysed. Qualitative feedback on multi-platform learning was sought via unstructured interviews and a questionnaire. Participants were divided into two groups of 5. Group 1 completed the simulation exercises on console A first then repeated these exercises on console B. Group 2 completed the simulated exercises on console B first then repeated these exercises on console A. Group 1 candidates adapted quicker to the second console and Group 2 candidates reached proficiency faster on the first console. Participants were slower on the second attempt of the final exercise regardless of their allocated group. Quality and efficiency metrics and risk and safety metrics were equivalent across consoles. The data from this investigation suggests that console operating skills are transferrable across different platforms. Overall risk and safety metrics are within acceptable limits regardless of the order of progression of console indicating that training can safely occur across multiple consoles contemporaneously. This data has implications for the design of training and certification as new platforms progress to market and supports a proficiency-based approach.

Assessing the efficacy of dissection gestures in robotic surgery

Our group previously defined a dissection gesture classification system that deconstructs robotic tissue dissection into its most elemental yet meaningful movements. The purpose of this study was to expand upon this framework by adding an assessment of gesture efficacy (ineffective, effective, or erroneous) and analyze dissection patterns between groups of surgeons of varying experience. We defined three possible gesture efficacies as ineffective (no meaningful effect on the tissue), effective (intended effect on the tissue), and erroneous (unintended disruption of the tissue). Novices (0 prior robotic cases), intermediates (1-99 cases), and experts (≥ 100 cases) completed a robotic dissection task in a dry-lab training environment. Video recordings were reviewed to classify each gesture and determine its efficacy, then dissection patterns between groups were analyzed. 23 participants completed the task, with 9 novices, 8 intermediates with median caseload 60 (IQR 41-80), and 6 experts with median caseload 525 (IQR 413-900). For gesture selection, we found increasing experience associated with increasing proportion of overall dissection gestures (p = 0.009) and decreasing proportion of retraction gestures (p = 0.009). For gesture efficacy, novices performed the greatest proportion of ineffective gestures (9.8%, p < 0.001), intermediates commit the greatest proportion of erroneous gestures (26.8%, p < 0.001), and the three groups performed similar proportions of overall effective gestures, though experts performed the greatest proportion of effective retraction gestures (85.6%, p < 0.001). Between groups of experience, we found significant differences in gesture selection and gesture efficacy. These relationships may provide insight into further improving surgical training.