A Systematic Review and Meta-analysis on the Impact of Proficiency-based Progression Simulation Training on Performance Outcomes

Efficacy of a perfused cadaver model for simulated trauma resuscitation in advanced surgical skills training

Background

To develop a perfused cadaveric model for trauma surgery simulation, and to evaluate its efficacy in trauma resuscitation advanced surgical skills training.

Methods

Fourteen fourth-year general surgery residents attended this workshop at Siriraj Hospital (Bangkok, Thailand). Inflow and outflow cannulae and a cardiopulmonary bypass pump were used to create the perfusion circuit. Inflow was achieved by cannulating the right common carotid artery, and outflow by cannulation of both the right common femoral artery and the internal jugular vein. Arterial line monitoring was used to monitor resuscitation response and to control perfusion pressure. The perfusion solution comprised saline solution mixed 1:1 with glycerol (50%) and water with red food dye added. Advanced surgical skills during life-threatening injuries and damage control resuscitation operations were practiced starting from the airway to the neck, chest, peripheral vessels, abdomen, and pelvis. Resuscitative endovascular balloon occlusion of the aorta (REBOA) was also practiced. Post-workshop survey questions were grouped into three categories, including comparison with previous training methods; the realism of anatomical correlation and procedures; and, satisfaction, safety, and confidence. All questions and tasks were discussed among all members of the development team, and were agreed upon by at least 90% of experts from each participating medical specialty/subspecialty.

Results

The results of the three main groups of post-workshop survey questions are, as follows: (1) How the training compared with previous surgical training methods—mean score: 4.26/5.00, high score: 4.73/5.00; (2) Realism of anatomical correlation and procedures—mean score: 4.03/5.00, high score: 4.60/5.00; and, (3) Satisfaction, safety, and confidence—mean score: 4.24/5.00, high score: 4.47/5.00.

Conclusion

The developed perfused cadaveric model demonstrated potential advantages over previously employed conventional surgical training techniques for teaching vascular surgery at our center as evidenced by the improvement in the satisfaction scores from students attending perfused cadaveric training compared to the scores reported by students who attended earlier training sessions that employed other training techniques. Areas of improvement included ‘a more realistic training experience’ and ‘improved facilitation of decision-making and damage control practice during trauma surgery’.

New imaging technologies for robotic kidney cancer surgery

Objective

Kidney cancers account for approximately 2% of all newly diagnosed cancer in 2020. Among the primary treatment options for kidney cancer, urologist may choose between radical or partial nephrectomy, or ablative therapies. Nowadays, robotic-assisted partial nephrectomy (RAPN) for the management of renal cancers has gained popularity, up to being considered the gold standard. However, RAPN is a challenging procedure with a steep learning curve.

Methods

In this narrative review, different imaging technologies used to guide and aid RAPN are discussed.

Results

Three-dimensional visualization technology has been extensively discussed in RAPN, showing its value in enhancing robotic-surgery training, patient counseling, surgical planning, and intraoperative guidance. Intraoperative imaging technologies such as intracorporeal ultrasound, near-infrared fluorescent imaging, and intraoperative pathological examination can also be used to improve the outcomes following RAPN. Finally, artificial intelligence may play a role in the field of RAPN soon.

Conclusion

RAPN is a complex surgery; however, many imaging technologies may play an important role in facilitating it.

Learning Curve Analysis for Intracorporeal Robot-assisted Radical Cystectomy: Results from the EAU Robotic Urology Section Scientific Working Group

Background

The utilisation of robot-assisted radical cystectomy with intracorporeal reconstruction (iRARC) has increased in recent years. Little is known about the length of the learning curve (LC) for this procedure.

Objective

To study the length of the LC for iRARC in terms of 90-d major complications (MC90; Clavien-Dindo grade ≥3), 90-d overall complications (OC90, Clavien-Dindo grades 1–5), operating time (OT), estimated blood loss (EBL), and length of hospital stay (LOS).

Design, setting, and participants

This was a retrospective analysis of all consecutive iRARC cases from nine European high-volume hospitals with ≥100 cases. All patients had bladder cancer for which iRARC was performed, with an ileal conduit or neobladder as the urinary diversion.

Outcome measurements and statistical analysis

Outcome parameters used as a proxy for LC length were the number of consecutive cases needed to reach a plateau level in two-piece mixed-effects models for MC90, OC90, OT, EBL, and LOS.

Results and limitations

A total of 2186 patients undergoing iRARC between 2003 and 2018were included. The plateau levels for MC90 and OC90 were reached after 137 cases (95% confidence interval [CI] 80–193) and 97 cases (95% CI 41–154), respectively. The mean MC90 rate at the plateau was 14% (95% CI 7–21%). The plateau level was reached after 75 cases (95% CI 65–86) for OT, 88 cases (95% CI 70–106) for EBL, and 198 cases (95% CI 130–266) for LOS. A major limitation of the study is the difference in the balance of urinary diversion types between centres.

Conclusions

This multicentre retrospective analysis for the iRARC LC among nine European centres showed that 137 consecutive cases were needed to reach a stable MC90 rate.

Patient summary

We carried out a multicentre analysis of the surgical learning curve for robot-assisted removal of the bladder and bladder reconstruction in patients with bladder cancer. We found that 137 consecutive cases were needed to reach a stable rate of serious complications.

Using an application to measure trainees' procedural knowledge before chest tube insertion

PURPOSE OF THE STUDY

To collect validity evidence for the chest tube insertion (CTI) test mode on the medical simulation application Touch Surgery. This was done by using Messick's contemporary framework.

METHODS

Novice, intermediate and experienced participants provided informed consent and demographic information. After familiarisation with the application, they completed the CTI test mode. Validity evidence was collected from four sources: content, response process, relation to other variables and consequences. A post-study questionnaire with 5-point Likert scales assessed the perceived realism, relevance and utility of the assessment. Mean scores of the three groups were compared.

RESULTS

A total of 25 novices, 11 intermediates and 19 experienced participants were recruited. Content evidence was collected by an expert in CTI and was based on published literature and guidelines. All users were familiarised with the application, and received standardised instructions throughout the test. Most users rated the simulation as realistic and suitable to assess cognitive skills. Novices received significantly lower (55.9±7.5) test mode scores than intermediates (80.6±4.4) (p<0.001) and experienced participants (82.3±5.3) (p<0.001). There was no significant difference in score between intermediate and experienced participants (p=0.75). Consequences evidence was provided by establishing a pass/fail score of 71% using the contrasting groups method, which resulted in one observed false positive and no false negatives.

CONCLUSION

A robust validity argument was constructed for the CTI test mode, which can be implemented in surgical curricula to assess learners' cognitive skills prior to hands-on simulation practice.

Training to proficiency in surgery using simulation: is there a moral obligation?

A deontological approach to surgical ethics advocates that patients have the right to receive the best care that can be provided. The 'learning curve' in surgical skill is an observable and measurable phenomenon. Surgical training may therefore carry risk to patients. This can occur directly, through inadvertent harm, or indirectly through theatre inefficiency and associated costs. Trainee surgeon operating, however, is necessary from a utilitarian perspective, with potential risk balanced by the greater societal need to train future independent surgeons.New technology means that the surgical learning curve could take place, at least in part, outside of the operating theatre. Simulation-based deliberate practice could be used to obtain a predetermined level of proficiency in a safe environment, followed by simulation-based assessment of operative competence. Such an approach would require an overhaul of the current training paradigm and significant investment in simulator technology. This may increasingly be viewed as necessary in light of well-discussed pressures on surgical trainees and trainers.This article discusses the obligations to trainees, trainers and training bodies raised by simulation technology, and outlines the current arguments both against and in favour of a simulation-based training-to-proficiency model in surgery. The significant changes to the current training paradigm that would be required to implement such a model are also discussed.

OUP accepted manuscript

Background

This study aimed to evaluate the use of binary metric-based (proficiency-based progression; PBP) performance assessments and global evaluative assessment of robotic skills (GEARS) of a robotic-assisted low anterior rectal resection (RA-LAR) procedure.

Method

A prospective study of video analysis of RA-LAR procedures was carried out using the PBP metrics with binary parameters previously developed, and GEARS. Recordings were collected from five novice surgeons (≤30 RA-LAR previously performed) and seven experienced surgeons (>30 RA-LAR previously performed). Two consultant colorectal surgeons were trained to be assessors in the use of PBP binary parameters to evaluate the procedure phases, surgical steps, errors, and critical errors in male and female patients and GEARS scores. Novice and experienced surgeons were categorized and assessed using PBP metrics and GEARS; mean scores obtained were compared for statistical purpose. Also, the inter-rater reliability (IRR) of these assessment tools was evaluated.

Results

Twenty unedited recordings of RA-LAR procedures were blindly assessed. Overall, using PBP metric-based assessment, a subgroup of experienced surgeons made more errors (20 versus 16, P = 0.158) and critical errors (9.2 versus 7.8, P = 0.417) than the novice group, although not significantly. However, during the critical phase of RA-LAR, experienced surgeons made significantly fewer errors than the novice group (95% CI of the difference, Lower = 0.104 – Upper = 5.155, df = 11.9, t = 2.23, p = 0.042), and a similar pattern was observed for critical errors. The PBP metric and GEARS assessment tools distinguished between the objectively assessed performance of experienced and novice colorectal surgeons performing RA-LAR (total error scores with PBP metrics, P = 0.019–0.008; GEARS scores, P = 0.029–0.025). GEARS demonstrated poor IRR (mean IRR 0.49) and weaker discrimination between groups (15–41 per cent difference). PBP binary metrics demonstrated good IRR (mean 0.94) and robust discrimination particularly for total error scores (58–64 per cent).

Conclusions

PBP binary metrics seem to be useful for metric-based training for surgeons learning RA-LAR procedures.

Proficiency Based Progression (PBP) training- the future model for dental operative skills training?: A systematic review and meta-analysis of existing literature

OBJECTIVE

To evaluate the effectiveness of Proficiency-Based Progression (PBP) operative training using validated performance metrics, by comparing this to standard, conventional training methods.

DATA

This systematic review was conducted in accordance with the guidelines of the Transparent Reporting of Systematic Reviews and Meta-Analyses (PRISMA). Study quality was assessed using the MERSQI tool and the Cochrane Risk of Bias tool. Results were pooled using biased corrected standardized mean difference and ratio-of-means (ROM). Summary effects were evaluated using a series of fixed and random effects models. The primary outcome was the number of procedural errors performed comparing PBP and non-PBP-based training pathways. In quantitative synthesis testing for procedural errors, a pooled meta-analysis on 87 trainees was conducted using random-effects models. In a ROM analysis, PBP was estimated to reduce the mean rate of errors by 62%, when compared to standard training (ROM 0.38, 95% CI: 0.25; 0.58; p < 0.001) Sources: The electronic databases of PubMed, Embase, Web of Science, MEDLINE and Cochrane library's CENTRAL were searched from inception to 8/11/2021. Filters activated were Randomized Controlled trials, clinical trial.

STUDY SELECTION

13 studies were included for review with 11 included in the quantitative synthesis from 174 potentially relevant publications identified by the search strategy. Main inclusion criteria were studies comparing standard surgical/operative training with proficiency-based simulation training using validated metrics based on expert performance.

CONCLUSIONS

Our meta-analysis found that PBP training improved trainees' performances, by decreasing procedural errors. There is sufficient evidence to explore PBP training for use in dental skills training. PBP training was estimated to reduce the mean rate of operative errors by 62%, when compared to standard training. Given that there is a direct correlation between operative skill and patient outcomes, these data suggest that there is sufficient evidence to explore PBP training for use in dental skills training.

Potential Contenders for the Leadership in Robotic Surgery

Purpose: To summarize the scientific published literature on new robotic surgical platforms with potential use in the urological field, reviewing their evolution from presentation until the present day. Our goal is to describe the current characteristics and possible prospects for these platforms. Materials and Methods: A nonsystematic search of the PubMed, Cochrane library's Central, EMBASE, MEDLINE, and Scopus databases was conducted to identify scientific literature about new robotic platforms other than the Da Vinci^® system, reviewing their evolution from inception until December 2020. Only English language publications were included. The following keywords were used: "new robotic platforms," "Revo-I robot," "Versius robot," and "Senhance robot." All relevant English-language original studies were analyzed by one author (R.F.) and summarized after discussion with an independent third party (E.M., S.Y., S.P., and M.A.). Results: Since 1995, Intuitive Surgical, Inc., with the Da Vinci surgical system, is the leading company in the robotic surgical market. However, Revo-I^®, Versius^®, and Senhance^® are the other three platforms that recently appeared on the market with available articles published in peer-reviewed journals. Among these three new surgical systems, the Senhance robot has the most substantial scientific proof of its capacity to perform minimally invasive urological surgery and as such, it might become a contender of the Da Vinci robot. Conclusions: The Da Vinci surgical platform has allowed the diffusion of robotic surgery worldwide and showed the different advantages of this type of technique. However, its use has some drawbacks, especially its price. New robotic platforms characterized by unique features are under development. Of note, they might be less expensive compared with the Da Vinci robotic system. We found that these new platforms are still at the beginning of their technical and scientific validation. However, the Senhance robot is in a more advanced stage, with clinical studies supporting its full implementation.

Development and validation of the metric-based assessment of a robotic vessel dissection, vessel loop positioning, clip applying and bipolar coagulation task on an avian model

The evolution of robotic technology and its diffusion does not seem to have been adequately accompanied by the development and implementation of surgeon training programs that ensure skilled and safe device use at the start of the learning curve. The objective of the study is to develop and validate performance metrics for vessel dissection, vessel loop positioning, clip applying and bipolar coagulation using an avian model. Three robotic surgeons and a behavioral scientist characterized the performance metrics of the task according to the proficiency-based progression methodology. Fourteen experienced robotic surgeons from different European countries participated in a modified online Delphi consensus. Eight experienced surgeons and eight novices performed the robotic task twice. In the Delphi meeting, 100% consensus was reached on the performance metrics. Novice surgeons took 26 min to complete the entire task on trial 1 and 20 min on trial 2. Experts took 10.1 min and 9.5 min. On average the Expert Group completed the task 137% faster than the Novice Group. The amount of time to reach the vessel part of the task was also calculated. Novice surgeons took 26 min on trial 1 and 20 min on trial 2. Experts took 5.5 min and 4.8 min. On average the experts reached the vessel 200% faster than the novices. The Expert Group made 155% fewer performance errors than the Novice Group. The mean IRR of video-recorded performance assessments for all metrics was 0.96 (95% confidence intervals (CI) lower = 0.94-upper = 0.98). We report the development and validation for a standard and replicable basic robotic vessel dissection, vessel loop positioning, clip applying and bipolar coagulation task on an avian model. The development of objective performance metrics, based on a transparent and fair methodology (i.e., PBP), is the first fundamental step toward quality assured training. This task developed on the avian model proved to have good results in the validation study.