Surgical Artificial Intelligence in Urology: Educational Applications

Surgical education has seen immense change recently. Increased demand for iterative evaluation of trainees from medical school to independent practice has led to the generation of an overwhelming amount of data related to an individual's competency. Artificial intelligence has been proposed as a solution to automate and standardize the ability of stakeholders to assess the technical and nontechnical abilities of a surgical trainee. In both the simulation and clinical environments, evidence supports the use of machine learning algorithms to both evaluate trainee skill and provide real-time and automated feedback, enabling a shortened learning curve for many key procedural skills and ensuring patient safety.

Severity Grading Systems for Intraoperative Adverse Events. A Systematic Review of the Literature and Citation Analysis

INTRODUCTION

The accurate assessment and grading of adverse events (AE) is essential to ensure comparisons between surgical procedures and outcomes. The current lack of a standardized severity grading system may limit our understanding of the true morbidity attributed to AEs in surgery. The aim of this study is to review the prevalence in which intraoperative adverse event (iAE) severity grading systems are used in the literature, evaluate the strengths and limitations of these systems, and appraise their applicability in clinical studies.

METHODS

A systematic review was conducted in accordance with Preferred Reporting Items for Systematic Reviews and Meta-analysis guidelines. PubMed, Web of Science, and Scopus were queried to yield all clinical studies reporting the proposal and/or the validation of iAE severity grading systems. Google Scholar, Web of Science, and Scopus were searched separately to identify the articles citing the systems to grade iAEs identified in the first search.

RESULTS

Our search yielded 2957 studies, with 7 studies considered for the qualitative synthesis. Five studies considered only surgical/interventional iAEs, while 2 considered both surgical/interventional and anesthesiologic iAEs. Two included studies validated the iAE severity grading system prospectively. A total of 357 citations were retrieved, with an overall self/nonself-citation ratio of 0.17 (53/304). The majority of citing articles were clinical studies (44.1%). The average number of citations per year was 6.7 citations for each classification/severity system, with only 2.05 citations/year for clinical studies. Of the 158 clinical studies citing the severity grading systems, only 90 (56.9%) used them to grade the iAEs. The appraisal of applicability (mean%/median%) was below the 70% threshold in 3 domains: stakeholder involvement (46/47), clarity of presentation (65/67), and applicability (57/56).

CONCLUSION

Seven severity grading systems for iAEs have been published in the last decade. Despite the importance of collecting and grading the iAEs, these systems are poorly adopted, with only a few studies per year using them. A uniform globally implemented severity grading system is needed to produce comparable data across studies and develop strategies to decrease iAEs, further improving patient safety.

A Multi-institution Study on the Association of Virtual Reality Skills with Continence Recovery after Robot-assisted Radical Prostatectomy

BACKGROUND

Virtual reality (VR) simulators are increasingly being used for surgical skills training. It is unclear what skills are best improved via VR, translate to live surgical skills, and influence patient outcomes.

OBJECTIVE

To assess surgeons in VR and live surgery using a suturing assessment tool and evaluate the association between technical skills and a clinical outcome.

DESIGN, SETTING, AND PARTICIPANTS

This prospective five-center study enrolled participants who completed VR suturing exercises and provided live surgical video. Graders provided skill assessments using the validated End-To-End Assessment of Suturing Expertise (EASE) suturing evaluation tool.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

A hierarchical Poisson model was used to compare skill scores among cohorts and evaluate the association of scores with clinical outcomes. Spearman's method was used to assess correlation between VR and live skills.

RESULTS AND LIMITATIONS

Ten novices, ten surgeons with intermediate expertise (median 64 cases, interquartile range [IQR] 6-80), and 26 expert surgeons (median 850 cases, IQR 375-3000) participated in this study. Intermediate and expert surgeons were significantly more likely to have ideal scores in comparison to novices for the subskills needle hold angle, wrist rotation, and wrist rotation needle withdrawal (p < 0.01). For both intermediate and expert surgeons, there was positive correlation between VR and live skills for needle hold angle (p < 0.05). For expert surgeons, there was a positive association between ideal scores for VR needle hold angle and driving smoothness subskills and 3-mo continence recovery (p < 0.05). Limitations include the size of the intermediate surgeon sample and clinical data limited to expert surgeons.

CONCLUSIONS

EASE can be used in VR to identify skills to improve for trainee surgeons. Technical skills that influence postoperative outcomes may be assessable in VR.

PATIENT SUMMARY

This study provides insights into surgical skills that translate from virtual simulation to live surgery and that have an impact on urinary continence after robot-assisted removal of the prostate. We also highlight the usefulness of virtual reality in surgical education.

Development of a Classification System for Live Surgical Feedback

Importance

Live feedback in the operating room is essential in surgical training. Despite the role this feedback plays in developing surgical skills, an accepted methodology to characterize the salient features of feedback has not been defined.

Objective

To quantify the intraoperative feedback provided to trainees during live surgical cases and propose a standardized deconstruction for feedback.

Design, Setting, and Participants

In this qualitative study using a mixed methods analysis, surgeons at a single academic tertiary care hospital were audio and video recorded in the operating room from April to October 2022. Urological residents, fellows, and faculty attending surgeons involved in robotic teaching cases during which trainees had active control of the robotic console for at least some portion of a surgery were eligible to voluntarily participate. Feedback was time stamped and transcribed verbatim. An iterative coding process was performed using recordings and transcript data until recurring themes emerged.

Exposure

Feedback in audiovisual recorded surgery.

Main Outcomes and Measures

The primary outcomes were the reliability and generalizability of a feedback classification system in characterizing surgical feedback. Secondary outcomes included assessing the utility of our system.

Results

In 29 surgical procedures that were recorded and analyzed, 4 attending surgeons, 6 minimally invasive surgery fellows, and 5 residents (postgraduate years, 3-5) were involved. For the reliability of the system, 3 trained raters achieved moderate to substantial interrater reliability in coding cases using 5 types of triggers, 6 types of feedback, and 9 types of responses (prevalence-adjusted and bias-adjusted κ range: a 0.56 [95% CI, 0.45-0.68] minimum for triggers to a 0.99 [95% CI, 0.97-1.00] maximum for feedback and responses). For the generalizability of the system, 6 types of surgical procedures and 3711 instances of feedback were analyzed and coded with types of triggers, feedback, and responses. Significant differences in triggers, feedback, and responses reflected surgeon experience level and surgical task being performed. For example, as a response, attending surgeons took over for safety concerns more often for fellows than residents (prevalence rate ratio [RR], 3.97 [95% CI, 3.12-4.82]; P = .002), and suturing involved more errors that triggered feedback than dissection (RR, 1.65 [95% CI, 1.03-3.33]; P = .007). For the utility of the system, different combinations of trainer feedback had associations with rates of different trainee responses. For example, technical feedback with a visual component was associated with an increased rate of trainee behavioral change or verbal acknowledgment responses (RR, 1.11 [95% CI, 1.03-1.20]; P = .02).

Conclusions and Relevance

These findings suggest that identifying different types of triggers, feedback, and responses may be a feasible and reliable method for classifying surgical feedback across several robotic procedures. Outcomes suggest that a system that can be generalized across surgical specialties and for trainees of different experience levels may help galvanize novel surgical education strategies.

Urinary complications after penile inversion vaginoplasty in transgender women: Systematic review and meta-analysis

Introduction: Penile inversion vaginoplasty (PIV) remains the gold standard technique for vaginoplasty, a gender-affirming feminizing surgery, but has been associated with urinary complications; however, there is little literature synthesizing urinary complications after PIV surgery, and there is a need to compile these complications to counsel patients pre- and postoperatively on managing surgical expectations. In this systematic review, we summarize the prevalence of urinary complications following PIV. Methods: We searched the MEDLINE, EMBASE, CINAHL, and Scopus databases in July 2020. The primary outcome was the prevalence of urinary and surgical complications in patients after penile inversion vaginoplasty. Pooled prevalence was determined from extrapolated data. ROBINS-I tool was used to assess study quality. The study was prospectively registered on PROSPERO (CRD42020204139). Results: Of 843 unique records, 27 articles were pooled for synthesis, with 3388 patients in total. Overall patient satisfaction ranged from 80–100%. The most common urological complications included poor/splayed stream (11.7%, 95% confidence interval [CI] 5.7–19.3), meatal stenosis (6.9%, 95% CI 2.7–12.7), and irritative symptoms (frequency, urgency, nocturia) (11.5%, 95% CI 2.6–25.1). Other urinary complications included retention requiring catheterization (5.1%, 95% CI 0.3–13.8), incontinence (8.7%, 95% CI 3.4–15.6), urethral stricture (4.6%, 95% CI 1.2–9.8), and urinary tract infection (5.6%, 95% CI 2.7–9.4). Most pooled studies had moderate risk of bias. Conclusions: The available evidence suggests that there is a low prevalence of urinary complications following PIV. Overall, there is a need for standardization of data in transgender surgical care to better understand surgical outcomes and improve postoperative management.

Multi-Modal Deep Learning for Assessing Surgeon Technical Skill

This paper introduces a new dataset of a surgical knot-tying task, and a multi-modal deep learning model that achieves comparable performance to expert human raters on this skill assessment task. Seventy-two surgical trainees and faculty were recruited for the knot-tying task, and were recorded using video, kinematic, and image data. Three expert human raters conducted the skills assessment using the Objective Structured Assessment of Technical Skill (OSATS) Global Rating Scale (GRS). We also designed and developed three deep learning models: a ResNet-based image model, a ResNet-LSTM kinematic model, and a multi-modal model leveraging the image and time-series kinematic data. All three models demonstrate performance comparable to the expert human raters on most GRS domains. The multi-modal model demonstrates the best overall performance, as measured using the mean squared error (MSE) and intraclass correlation coefficient (ICC). This work is significant since it demonstrates that multi-modal deep learning has the potential to replicate human raters on a challenging human-performed knot-tying task. The study demonstrates an algorithm with state-of-the-art performance in surgical skill assessment. As objective assessment of technical skill continues to be a growing, but resource-heavy, element of surgical education, this study is an important step towards automated surgical skill assessment, ultimately leading to reduced burden on training faculty and institutes.

Standardized Reporting of Machine Learning Applications in Urology: The STREAM-URO Framework

The Standardized Reporting of Machine Learning Applications in Urology (STREAM-URO) framework was developed to provide a set of recommendations to help standardize how machine learning studies in urology are reported. This framework serves three purposes: (1) to promote high-quality studies and streamline the peer review process; (2) to enhance reproducibility, comparability, and interpretability of results; and (3) to improve engagement and literacy of machine learning within the urological community.

Can video games enhance surgical skills acquisition for medical students? A systematic review

OBJECTIVE

To systematically assess literature examining the impact of being a gamer or participating in video-game-based training on surgical skills acquisition amongst medical students.

BACKGROUND

Video games and surgical procedures share similar skills such as visuospatial abilities and hand-eye coordination; therefore, video games can be a valuable tool for surgical training amongst medical students. However, comprehensive, up-to-date systematic reviews are necessary to confirm.

METHODS

A systematic literature search of PubMed, MEDLINE, and EMBASE was performed in April 2020 with no limits set on the date of publication. Observational and randomized controlled studies were included. Quality and bias were assessed using the Newcastle-Ottawa Scale for nonrandomized studies and the Grading of Recommendations Assessment, Development and Evaluation system for randomized studies.

RESULTS

A total of 575 participants from 16 studies were included. The most common surgical skills tested were laparoscopy (n = 283, from 8 studies) and robotic surgery (n = 199, from 5 studies). A history of gaming and video-game-based training were associated with improved metrics in robotic surgery and laparoscopy, respectively. Neither was beneficial in arthroscopy or bronchoscopy. Studies using the Wii U and Underground reported significant improvement in overall laparoscopic performance.

CONCLUSION

Video games demonstrate potential as adjunctive training in surgical skill education, with a history of gaming and video-game-based training being beneficial in robotic surgery and laparoscopy, respectively. Methodological heterogeneity amongst included studies limit the ability to make conclusive decisions; thus, future studies with long-term follow-up, larger sample sizes, outcomes stratified by video-game characteristics, and up-to-date technology are necessary.

Quantifying the "Assistant Effect" in Robotic-Assisted Radical Prostatectomy (RARP): Measures of Technical Performance

PURPOSE

Surgeons are reliant on the bedside assistant during robotic surgeries. Using a modified global rating scale (GRS), we aim to assess the association between an assistant's technical skill on surgeon performance in Robotic-Assisted Radical Prostatectomy (RARP).

METHODS

Prospective, intraoperative video from RARP cases at three centers were collected. Baseline demographic and RARP-experience data were collected from participating surgeons and trainees. The dissection of the prostatic pedicle and neurovascular bundle step (NVB) was analyzed. Expert analysts scored the console surgeon performance using the Global Evaluative Assessment of Robotic Skills (GEARS), and the bedside assistant performance using a modified Objective Structured Assessment of Technical Skills (aOSATS). The primary outcome is the association between console surgeon performance, as measured by GEARS, and assistant skill, as measured by aOSATS. Spearman's rho correlations were used to test the relationship between assistant and surgeon technical performance, and a multivariable linear regression model was created to test this association while controlling for patient factors.

RESULTS

92 RARP cases were available for the analysis, comprising 14 console surgeons and 22 different bedside assistants. In only 5 (5.4%) cases, the neurovascular bundle step was completed by a trainee, and in 13 (14.1%) of cases, a staff-level surgeon acted as the bedside assistant. aOSATS score was significantly associated with robotic console experience (P = 0.011), and prior laparoscopic experience (P < 0.001). Assistant aOSATS score showed a weak but significant correlation with surgeon GEARS score during the neurovascular bundle step (spearman's rho = 0.248, P = 0.028). On linear regression, aOSATS remained a significant predictor of console surgeon performance (P = 0.016), after controlling for patient age and BMI, prostate volume, tumor stage, and presence of nerve-sparing.

CONCLUSIONS

This is the first study to assess the association between assistant technical skill and surgeon performance in RARP. Additionally, we have provided validity evidence for a modified OSATS global rating scale for training and assessing bedside assistant performance.

Current Educational Interventions for Improving Technical Skills of Urology Trainees in Endourological Procedures: A Systematic Review

Objective: Endourology continues to grow with the introduction of new technologies into clinical practice. Simulators and training models have been developed to improve comfort and proficiency in endoscopic procedures. The purpose of this systematic review was to examine the current educational interventions utilized to improve the performance of endourology trainees and to critically appraise the strengths and limitations of each. Methods: A search of the Ovid MEDLINE, EMBASE, PsycINFO, and the Cochrane Library databases was performed to identify literature focused on current educational interventions for improving technical skills of trainees in endourologic procedures. The Medical Education Research Study Quality Instrument (MERSQI) was used to evaluate the methodological quality of the abstracted articles. Results: Of the 2236 articles identified, 22 met the inclusion criteria. The types of educational interventions included: bench/wet lab models, virtual reality simulators, and instructional courses. Metrics used to quantify the impact of these interventions include global rating scales, Objective Structured Assessment of Technical Skills (OSATS) scores, and task-specific checklists. The setting of these evaluations comprises both virtual reality simulators and live surgery. Conclusions: In the surgical education literature, simulation-based training and assessment continues to play a prominent role in urologic training. The educational interventions highlighted in this review address various aspects of endourology, from stone management to transurethral resection. Additional work is needed to correlate technical performance in clinical and nonclinical settings with patient outcomes and develop a focused approach to nontechnical skill training.

From box ticking to the black box: the evolution of operating room safety

PURPOSE

Efforts to improve the safety of patients in the operating room have focused on mitigating harm through the standardization of system, team, and human level factors. This article highlights existing and future methods for enhancing safety in the perioperative setting, and the theory and principles that underpin them.

METHODS

Evidence surrounding the development and implementation of select surgical safety interventions is discussed.

RESULTS

Work in human factors and engineering that has inspired safety interventions such as the WHO Safety Checklist, and more recently operating room recorders, represents a movement away from traditional, retrospective or reactive methods of studying surgical safety, to prospective and proactive ones.

CONCLUSIONS

Future work will examine the effectiveness of these interventions for improving patient outcomes and minimizing iatrogenic harm.

Psychology and learning: The role of the clinical learning environment

The clinical learning environment for the postgraduate education of physicians significantly influences the learning process and the outcomes of learning. Two critical aspects of the learning environment, when viewed through a psychology lens are (1) constructs from psychology relevant to learning, such as cognitive load theory and learner self-efficacy; and (2) psychological attributes of the context in which learning occurs such as psychological safety and "Just Culture". In this paper, we address selected psychological aspects of the clinical learning environment, with a particular focus on the establishment and sustainment of psychological safety in the clinical learning environment for physicians. Psychological safety is defined as individuals' perceptions that they can speak out in the learning or working context without consequences for their professional standing or risks to their status on work teams or groups. We close with seven critical strategies for use by educators, learners, health systems leaders, and other stakeholders to contribute to a clinical environment that optimizes learning. These dimensions can also provide avenues for future research to enhance the community's understanding of psychological constructs operating in the clinical learning environment.

Understanding and Assessing Nontechnical Skills in Robotic Urological Surgery: A Systematic Review and Synthesis of the Validity Evidence

OBJECTIVE

Robotic urological surgery (RUS) has seen widespread adoption across institutions in the last decade. To match this rapid growth, it is imperative to develop a structured RUS curriculum that addresses both technical and nontechnical competencies. Emerging evidence has shown that nontechnical skills form a critical component of RUS training. The purpose of this review is to examine the validity evidence of available nontechnical skills assessment tools in RUS.

METHODS

A literature search of MEDLINE, EMBASE, and PsycINFO was conducted to identify primary articles using nontechnical skills assessment tools in RUS. Messick's validity framework and the Medical Education Research Study Quality Instrument were utilized to evaluate the quality of the validity evidence of the abstracted articles.

RESULTS

Of the 566 articles identified, 12 used nontechnical skills assessment tools in RUS. The metrics used ranged from self-assessment using global rating scales, to objective measures such as electroencephalography. The setting of these evaluations ranged from immersive and virtual reality-based simulators to live surgery.

CONCLUSIONS

Limited effort has been made to develop nontechnical skills assessment tools in RUS. Recently, there has been a shift from subjective to objective measures of nontechnical performance, as well as the development of assessments specific to RUS. However, the validity evidence supporting these nontechnical assessments is limited at this time, including their relationship to technical skills, and their impact on surgical outcomes.

Simulation-Based Laparoscopic Surgery Crisis Resource Management Training-Predicting Technical and Nontechnical Skills

OBJECTIVES

To develop a unique simulation-based assessment using a laparoscopic inferior vena cava (IVC) injury scenario that allows for the safe assessment of urology resident's technical and nontechnical skills, and investigate the effect of personality traits performance in a surgical crisis.

METHODS

Urology residents from our institution were recruited to participate in a simulation-based training laparoscopic nephrectomy exercise. Residents completed demographic and multidimensional personality questionnaires and were instructed to play the role of staff urologist. A vasovagal response to pneumoperitoneum and an IVC injury event were scripted into the scenario. Technical and nontechnical skills were assessed by expert laparoscopic surgeons using validated tools (task checklist, GOALS, and NOTSS).

RESULTS

Ten junior and five senior urology residents participated. Five residents were unable to complete the exercise safely. Senior residents outperformed juniors on technical (checklist score 15.1 vs 9.9, p < 0.01, GOALS score 18.0 vs 13.3, p < 0.01) and nontechnical performance (NOTSS score 13.8 vs 10.1, p = 0.03). Technical performance scores correlated with NOTSS scores (p < 0.01) and pass/fail rating correlated with technical performance (p < 0.01 for both checklist and GOALS), NOTSS score (p = 0.02), and blood loss (p < 0.01). Only the conscientiousness dimension of the big five inventory correlated with technical score (p = 0.03) and pass/fail rating (p = 0.04).

CONCLUSIONS

Resident level of training and laparoscopic experience correlated with technical performance during a simulation-based laparoscopic IVC injury crisis management scenario, as well as multiple domains of nontechnical performance. Personality traits of our surgical residents are similar and did not predict technical skill.

Implementing assessments of robot-assisted technical skill in urological education: a systematic review and synthesis of the validity evidence

OBJECTIVES

To systematically review and synthesise the validity evidence supporting intraoperative and simulation-based assessments of technical skill in urological robot-assisted surgery (RAS), and make evidence-based recommendations for the implementation of these assessments in urological training.

MATERIALS AND METHODS

A literature search of the Medline, PsycINFO and Embase databases was performed. Articles using technical skill and simulation-based assessments in RAS were abstracted. Only studies involving urology trainees or faculty were included in the final analysis.

RESULTS

Multiple tools for the assessment of technical robotic skill have been published, with mixed sources of validity evidence to support their use. These evaluations have been used in both the ex vivo and in vivo settings. Performance evaluations range from global rating scales to psychometrics, and assessments are carried out through automation, expert analysts, and crowdsourcing.

CONCLUSION

There have been rapid expansions in approaches to RAS technical skills assessment, both in simulated and clinical settings. Alternative approaches to assessment in RAS, such as crowdsourcing and psychometrics, remain under investigation. Evidence to support the use of these metrics in high-stakes decisions is likely insufficient at present.

Feasibility of expert and crowd-sourced review of intraoperative video for quality improvement of intracorporeal urinary diversion during robotic radical cystectomy

INTRODUCTION

Development of uretero-ileal stricture (UIS) after robotic-assisted radical cystectomy (RARC) may be dependent on surgical technique. Video review of intraoperative technique is an emerging paradigm for surgical quality improvement. We examined whether surgeon-perceived risk of UIS or crowd-sourced assessment of robotic skill are associated with the development of UIS.

METHODS

We conducted a case-control study comparing the operative technique of uretero-ileal anastomoses resulting in clinically significant UIS with the contralateral anastomosis for the same patient. De-identified videos were analyzed by 1) five high-volume surgeons; and 2) crowd workers (Crowd-Sourced Assessment of Technical Skill, C-SATS) to determine Global Evaluative Assessment of Robotic Skill (GEARS) score. Mantel-Haenszel common odds ratio (OR) estimates were calculated to assess the association between surgeon performance and the development of UIS. Logistic regression models were used to examine the association between GEARS scores and the development of UIS.

RESULTS

A total of 10 UIS videos were compared to eight control videos by five surgeons and 2142 crowd workers. Expert surgeons systematically evaluated intraoperative footage, however, no association between the expert mode response and UIS (OR 0.42; 95% confidence interval [CI] 0.05-3.45; p=0.91) was identified. Crowd-sourced assessment was not predictive of UIS (p=0.62).

CONCLUSIONS

We used video review to systematically analyze procedure-specific content and technique. The inability of surgeons to predict UIS may reflect the questionnaire, uncontrolled patient factors, or a lack of power. Crowd-sourced GEARS score was unsuccessful in predicting UIS after RARC.

Surgeon Performance Predicts Early Continence After Robot-Assisted Radical Prostatectomy

INTRODUCTION

There is limited, yet compelling evidence supporting the role of surgeon technical performance in influencing patient outcomes. To date, this concept has been underexplored in endourologic procedures. We hypothesized that a surgeon's technical performance plays a role in predicting an early return to continence after robot-assisted radical prostatectomy (RARP).

MATERIALS AND METHODS

We conducted a retrospective, matched case-control analysis of prospectively collected unedited RARP endoscopic videos performed by a single surgeon. A blinded observer with expertise in intraoperative video analysis evaluated clinically relevant steps of RARP using the global evaluative assessment of robotic skill (GEARS) and the generic error rating tool (GERT). The primary outcome was continence status at 3 months postoperatively, defined as patient use of less than or equal to a single precautionary pad. Mann-Whitney U tests examined differences in predictor variables between cases and controls, and multivariate analysis was conducted using binary logistic regression models.

RESULTS

Twenty-four incontinent patients were matched for age, body mass index, preoperative International Prostate Symptoms Score, use of posterior/anterior hitch, prostate weight, and learning curve position. No statistically significant difference in errors between groups was observed using the GERT. On multivariate analysis, overall case GEARS score was independently predictive of 3-month continence status (odds ratios [OR] = 0.55, 95% confidence interval [CI] 0.33-0.91), as were urethrovesical anastomosis (OR = 0.70, 95% CI 0.50-0.97) and bladder neck GEARS scores (OR = 0.69, 95% CI 0.51-0.94).

CONCLUSIONS

Our study generates the hypothesis that there may be a link between surgeon technical performance and functional outcomes in RARP. This relationship may have implications for the accreditation and training of future urologists and warrants further investigation.

Systematic review to establish absolute standards for technical performance in surgery

BACKGROUND

Standard setting allows educators to create benchmarks that distinguish between those who pass and those who fail an assessment. It can also be used to create standards in clinical and simulated procedural skill. The objective of this review was to perform a systematic review of the literature using absolute standard-setting methodology to create benchmarks in technical performance.

METHODS

A systematic review was conducted by searching MEDLINE, Embase, PsycINFO and the Cochrane Database of Systematic Reviews. Abstracts of retrieved studies were reviewed and those meeting the inclusion criteria were selected for full-text review. The quality of evidence presented in the included studies was assessed using the Medical Education Research Study Quality Instrument (MERSQI), where a score of 14 or more of 18 indicates high-quality evidence.

RESULTS

Of 1809 studies identified, 37 used standard-setting methodology for assessment of procedural skill. Of these, 24 used participant-centred and 13 employed item-centred methods. Thirty studies took place in a simulated environment, and seven in a clinical setting. The included studies assessed residents (26 of 37), fellows (6 of 37) and staff physicians (17 of 37). Seventeen articles achieved a MERSQI score of 14 or more of 18, whereas 20 did not meet this mark.

CONCLUSION

Absolute standard-setting methodologies can be used to establish cut-offs for procedural skill assessments.

Baseline Laparoscopic Skill May Predict Baseline Robotic Skill and Early Robotic Surgery Learning Curve

INTRODUCTION

Robotic surgery is associated with a learning curve unique to each trainee. Knowledge about a trainee's baseline skill and learning curve would facilitate the development of a more individualized training curriculum. The aim of our study was to determine whether baseline laparoscopic skill is predictive of one's baseline robotic skill and short-term learning curve.

METHODS

Trainees from four different surgical specialties were included in the study. Each trainee participated in a 4-week, simulation-based robotic surgery basic skills training course. Precourse, baseline laparoscopic and robotic skills were assessed using validated test tasks; a basic peg transfer (PT) and an advanced intracorporeal suturing and knot tying (ISKT) task. Trainee robotic skill was assessed again 1 week postcourse. Each task performance was video recorded and scored by two blinded expert surgeons.

RESULTS

A total of 32 trainees were included; 14 urology, 7 gynecology, 8 thoracic Sx, 3 general Sx. Most (91%) were senior residents or clinical fellows and 50% had no prior robotic experience. There were no differences in baseline laparoscopic and robotic skill related to reported prior robotic experience. Between specialties, no differences were seen on baseline laparoscopic skill and only small differences were seen on baseline robotic skill. Both baseline Lap PT (p = 0.01) and Lap ISKT (p = 0.01) performances correlated with baseline robotic ISKT performance, but not robotic PT scores. Only baseline Lap ISKT performance correlated with postcourse robotic PT (p = 0.01) and ISKT (p < 0.01) performance. Baseline robotic ISKT scores, but not PT scores, correlated with postcourse robotic performance (p = 0.02 for PT, p < 0.01 for ISKT).

CONCLUSIONS

In this study, a trainee's baseline laparoscopic skill correlated with certain baseline robotic skills. Better baseline performance on an advanced, but not basic, laparoscopic and robotic skill task may correlate with a shorter learning curve for basic robotic skills. Further exploration of this finding may yield better training curricula.