Utilizing Machine Learning and Automated Performance Metrics to Evaluate Robot-Assisted Radical Prostatectomy Performance and Predict Outcomes

The use and applicability of machine learning algorithms in predicting the surgical outcome for patients with benign prostatic enlargement. Which model to use?

OBJECTIVES

Artificial intelligence (AI) is increasingly used in medicine, but data on benign prostatic enlargement (BPE) management are lacking. This study aims to test the performance of several machine learning algorithms, in predicting clinical outcomes during BPE surgical management.

METHODS

Clinical data were extracted from a prospectively collected database for 153 men with BPE, treated with transurethral resection (monopolar or bipolar) or vaporization of the prostate. Due to small sample size, we applied a method for increasing our dataset, Synthetic Minority Oversampling Technique (SMOTE). The new dataset created with SMOTE has been expanded by 453 synthetic instances, in addition to the original 153. The WEKA Data Mining Software was used for constructing predictive models, while several appropriate statistical measures, like Correlation coefficient (R), Mean Absolute Error (MAE), Root Mean-Squared Error (RMSE), were calculated with several supervised regression algorithms - techniques (Linear Regression, Multilayer Perceptron, SMOreg, k-Nearest Neighbors, Bagging, M5Rules, M5P - Pruned Model Tree, and Random forest).

RESULTS

The baseline characteristics of patients were extracted, with age, prostate volume, method of operation, baseline Qmax and baseline IPSS being used as independent variables. Using the Random Forest algorithm resulted in values of R, MAE, RMSE that indicate the ability of these models to better predict % Qmax increase. The Random Forest model also demonstrated the best results in R, MAE, RMSE for predicting % IPSS reduction.

CONCLUSIONS

Machine Learning techniques can be used for making predictions regarding clinical outcomes of surgical BPRE management. Wider-scale validation studies are necessary to strengthen our results in choosing the best model.

Application of Artificial Intelligence to Overcome Clinical Information Overload in Urologic Cancer

OBJECTIVE

To describe the use of artificial intelligence (AI) in medical literature and trial data extraction, and its applications in uro-oncology. This bridging review, which consolidates information from the diverse applications of AI, highlights how AI users can investigate more sophisticated queries than with traditional methods, leading to synthesis of raw data and complex outputs into more actionable and personalized results, particularly in the field of uro-oncology.

METHODS

Literature and clinical trial searches were performed in PubMed, Dimensions, Embase and Google (1999-2020). The searches focused on the use of AI and its various forms to facilitate literature searches, clinical guidelines development, and clinical trial data extraction in uro-oncology. To illustrate how AI can be applied toaddress questions about optimizing therapeutic decision making and individualizing treatment regimens, the Dimensions-linked information platform was searched for "prostate cancer" keywords (76 publications were identified; 48 were included).

RESULTS

AI offers the promise of transforming raw data and complex outputs into actionable insights. Literature and clinical trial searches can be automated, enabling clinicians to develop and analyze publications expeditiously on complex issues such as therapeutic sequencing and to obtain updates on documents that evolve at the pace and scope of the landscape. An AI-based platform inclusive of 12 trial databases and >100 scientific literature sources enabled the creation of an interactive visualization.

CONCLUSION

As the literature and clinical trial landscape continues to grow in complexity and with increasing speed, the ability to pull the right information at the right time from different search engines and resources while excluding social media bias becomes more challenging. This review demonstrates that by applying natural language processing and machine learning algorithms, validated and optimized AI leads to a speedier, more personalized, efficient and focused search compared with traditional methods.

Review of automated performance metrics to assess surgical technical skills in robot-assisted laparoscopy

INTRODUCTION

Robot-assisted laparoscopy is a safe surgical approach with several studies suggesting correlations between complication rates and the surgeon's technical skills. Surgical skills are usually assessed by questionnaires completed by an expert observer. With the advent of surgical robots, automated surgical performance metrics (APMs)-objective measures related to instrument movements-can be computed. The aim of this systematic review was thus to assess APMs use in robot-assisted laparoscopic procedures. The primary outcome was the assessment of surgical skills by APMs and the secondary outcomes were the association between APM and surgeon parameters and the prediction of clinical outcomes.

METHODS

A systematic review following the PRISMA guidelines was conducted. PubMed and Scopus electronic databases were screened with the query "robot-assisted surgery OR robotic surgery AND performance metrics" between January 2010 and January 2021. The quality of the studies was assessed by the medical education research study quality instrument. The study settings, metrics, and applications were analysed.

RESULTS

The initial search yielded 341 citations of which 16 studies were finally included. The study settings were either simulated virtual reality (VR) (4 studies) or real clinical environment (12 studies). Data to compute APMs were kinematics (motion tracking), and system and specific events data (actions from the robot console). APMs were used to differentiate expertise levels, and thus validate VR modules, predict outcomes, and integrate datasets for automatic recognition models. APMs were correlated with clinical outcomes for some studies.

CONCLUSIONS

APMs constitute an objective approach for assessing technical skills. Evidence of associations between APMs and clinical outcomes remain to be confirmed by further studies, particularly, for non-urological procedures. Concurrent validation is also required.

Machine learning algorithms can more efficiently predict biochemical recurrence after robot-assisted radical prostatectomy

OBJECTIVES

To develop a model for predicting biochemical recurrence (BCR) in patients with long follow-up periods using clinical parameters and the machine learning (ML) methods.

MATERIALS METHOD

Patients who underwent robot-assisted radical prostatectomy between January 2014 and December 2019 were retrospectively reviewed. Patients who did not have BCR were assigned to Group 1, while those diagnosed with BCR were assigned to Group 2. The patient's demographic data, preoperative and postoperative parameters were all recorded in the database. Three different ML algorithms were employed: random forest, K-nearest neighbour, and logistic regression.

RESULTS

Three hundred and sixty-eight patients were included in this study. Among these patients, 295 (80.1%) did not have BCR (Group 1), while 73 (19.8%) had BCR (Group 2). The mean duration of follow-up and duration until the diagnosis of BCR was calculated as 35.2 ± 16.7 and 11.5 ± 11.3 months, respectively. The multivariate analysis revealed that NLR, PSAd, risk classification, PIRADS score, T stage, presence or absence of positive surgical margin, and seminal vesicle invasion were predictive for BCR. Classic Cox regression analysis had an area under the curve (AUC) of 0.915 with a sensitivity and specificity of 90.6% and 79.8%. The AUCs for receiver-operating characteristic curves for random forest, K nearest neighbour, and logistic regression were 0.95, 0.93, and 0.93, respectively. All ML models outperformed the conventional statistical regression model in the prediction of BCR after prostatectomy.

CONCLUSION

The construction of more reliable and potent models will provide the clinicians and patients with advantages such as more accurate risk classification, prognosis estimation, early intervention, avoidance of unnecessary treatments, relatively lower morbidity and mortality. The ML methods are cheap, and their powers increase with increasing data input; we believe that their clinical use will increase over time.

Evolving robotic surgery training and improving patient safety, with the integration of novel technologies

INTRODUCTION

Robot-assisted surgery is becoming increasingly adopted by multiple surgical specialties. There is evidence of inherent risks of utilising new technologies that are unfamiliar early in the learning curve. The development of standardised and validated training programmes is crucial to deliver safe introduction. In this review, we aim to evaluate the current evidence and opportunities to integrate novel technologies into modern digitalised robotic training curricula.

METHODS

A systematic literature review of the current evidence for novel technologies in surgical training was conducted online and relevant publications and information were identified. Evaluation was made on how these technologies could further enable digitalisation of training.

RESULTS

Overall, the quality of available studies was found to be low with current available evidence consisting largely of expert opinion, consensus statements and small qualitative studies. The review identified that there are several novel technologies already being utilised in robotic surgery training. There is also a trend towards standardised validated robotic training curricula. Currently, the majority of the validated curricula do not incorporate novel technologies and training is delivered with more traditional methods that includes centralisation of training services with wet laboratories that have access to cadavers and dedicated training robots.

CONCLUSIONS

Improvements to training standards and understanding performance data have good potential to significantly lower complications in patients. Digitalisation automates data collection and brings data together for analysis. Machine learning has potential to develop automated performance feedback for trainees. Digitalised training aims to build on the current gold standards and to further improve the 'continuum of training' by integrating PBP training, 3D-printed models, telementoring, telemetry and machine learning.

Use of Artificial Intelligence in Non-Oncologic Interventional Radiology: Current State and Future Directions

The future of radiology is disproportionately linked to the applications of artificial intelligence (AI). Recent exponential advancements in AI are already beginning to augment the clinical practice of radiology. Driven by a paucity of review articles in the area, this article aims to discuss applications of AI in non-oncologic IR across procedural planning, execution, and follow-up along with a discussion on the future directions of the field. Applications in vascular imaging, radiomics, touchless software interactions, robotics, natural language processing, post-procedural outcome prediction, device navigation, and image acquisition are included. Familiarity with AI study analysis will help open the current 'black box' of AI research and help bridge the gap between the research laboratory and clinical practice.

Virtual Reality vs. Dry-Lab Models: Comparing automated performance metrics and cognitive workload during robotic simulation training

Background This study compares surgical performance during analogous vesico-urethral anastomosis (VUA) tasks in two robotic training environments, virtual reality (VR) and dry-lab (DL), in order to investigate transferability of skills assessment across the two platforms. Utilizing computer-generated performance metrics and pupillary data we evaluated the two environments' ability to distinguish surgical expertise and ultimately whether performance in the VR simulation correlates to performance on the live robot in the dry-lab. Materials and Methods Experts (≥ 300 cases) and trainees (<300) performed analogous VUAs during VR and dry-lab sessions on a da Vinci robotic console. 22 metrics were generated in each environment (kinematic metrics, tissue metrics, biometrics). The dry-lab included 18 previously validated automated performance metrics (APMs) (kinematics, events metrics) and were captured by an Intuitive systems data recorder. In both settings, Tobii Pro Glasses 2 recorded task-evoked pupillary response (reported as Index of Cognitive Activity [ICA]) to indicate cognitive workload, analyzed by EyeTracking Cognitive Workload Software. Pearson Correlation, Mann-Whitney and Independent t-tests were used for the comparative analyses. Results Our study included 6 experts (median caseload 1300 [interquartile range 400-3000]) and 11 trainees (25 [0-250]). 8/9 metrics directly comparable between VR and DL showed significant positive correlation (r≥0.554, p≤0.032). 5/22 VR metrics distinguished expertise including: task time (p=0.031), clutch usage (p=0.040), unnecessary needle piercings (p=0.026) and suspected injury to endopelvic fascia (p=0.040). This contrasts with 14/22 APMs in dry-lab (p≤0.038) including: linear velocities of all three instruments (p≤0.038) and dominant-hand instrument wrist articulation (p=0.013). Trainees experienced higher cognitive workload (ICA) in both environments when compared to experts (p<0.036). Conclusions A majority of performance metrics between VR and dry-lab exhibited moderate to strong correlations, showing transferability of skills across the platforms. Comparing training environments, APMs during dry-lab tasks are better able to distinguish expertise than VR-generated metrics.

How the use of the artificial intelligence could improve surgical skills in urology: state of the art and future perspectives

PURPOSE OF REVIEW

As technology advances, surgical training has evolved in parallel over the previous decade. Training is commonly seen as a way to prepare surgeons for their day-to-day work; however, more importantly, it allows for certification of skills to ensure maximum patient safety. This article reviews advances in the use of machine learning and artificial intelligence for improvements of surgical skills in urology.

RECENT FINDINGS

Six studies have been published, which met the inclusion criteria. All articles assessed the application of artificial intelligence in improving surgical training. Different approaches were taken, such as using machine learning to identify and classify suturing gestures, creating automated objective evaluation reports, and determining surgical technical skill levels to predict clinical outcomes. The articles illustrated the continuously growing role of artificial intelligence to address the difficulties currently present in evaluating urological surgical skills.

SUMMARY

Artificial intelligence allows us to efficiently analyze the surmounting data related to surgical training and use it to come to conclusions that normally would require human intelligence. Although these metrics have been shown to predict surgeon expertise and surgical outcomes, evidence is still scarce regarding their ability to directly improve patient outcomes. Considering this, current active research is growing on the topic of deep learning-based computer vision to provide automated metrics needed for real-time surgeon feedback.

Machine learning applications to enhance patient specific care for urologic surgery

PURPOSE

As computational power has improved over the past 20 years, the daily application of machine learning methods has become more prevalent in daily life. Additionally, there is increasing interest in the clinical application of machine learning techniques. We sought to review the current literature regarding machine learning applications for patient-specific urologic surgical care.

METHODS

We performed a broad search of the current literature via the PubMed-Medline and Google Scholar databases up to Dec 2020. The search terms "urologic surgery" as well as "artificial intelligence", "machine learning", "neural network", and "automation" were used.

RESULTS

The focus of machine learning applications for patient counseling is disease-specific. For stone disease, multiple studies focused on the prediction of stone-free rate based on preoperative characteristics of clinical and imaging data. For kidney cancer, many studies focused on advanced imaging analysis to predict renal mass pathology preoperatively. Machine learning applications in prostate cancer could provide for treatment counseling as well as prediction of disease-specific outcomes. Furthermore, for bladder cancer, the reviewed studies focus on staging via imaging, to better counsel patients towards neoadjuvant chemotherapy. Additionally, there have been many efforts on automatically segmenting and matching preoperative imaging with intraoperative anatomy.

CONCLUSION

Machine learning techniques can be implemented to assist patient-centered surgical care and increase patient engagement within their decision-making processes. As data sets improve and expand, especially with the transition to large-scale EHR usage, these tools will improve in efficacy and be utilized more frequently.

Artificial intelligence in urological oncology: An update and future applications

There continues to be rapid developments and research in the field of Artificial Intelligence (AI) in Urological Oncology worldwide. In this review we discuss the basics of AI, application of AI per tumour group (Renal, Prostate and Bladder Cancer) and application of AI in Robotic Urological Surgery. We also discuss future applications of AI being developed with the benefits to patients with Urological Oncology.

Development and head-to-head comparison of machine-learning models to identify patients requiring prostate biopsy

Background

Machine learning has many attractive theoretic properties, specifically, the ability to handle non predefined relations. Additionally, studies have validated the clinical utility of mpMRI for the detection and localization of CSPCa (Gleason score ≥ 3 + 4). In this study, we sought to develop and compare machine-learning models incorporating mpMRI parameters with traditional logistic regression analysis for prediction of PCa (Gleason score ≥ 3 + 3) and CSPCa on initial biopsy.

Methods

A total of 688 patients with no prior prostate cancer diagnosis and tPSA ≤ 50 ng/ml, who underwent mpMRI and prostate biopsy were included between 2016 and 2020. We used four supervised machine-learning algorithms in a hypothesis-free manner to build models to predict PCa and CSPCa. The machine-learning models were compared to the logistic regression analysis using AUC, calibration plot, and decision curve analysis.

Results

The artificial neural network (ANN), support vector machine (SVM), and random forest (RF) yielded similar diagnostic accuracy with logistic regression, while classification and regression tree (CART, AUC = 0.834 and 0.867) had significantly lower diagnostic accuracy than logistic regression (AUC = 0.894 and 0.917) in prediction of PCa and CSPCa (all P < 0.05). However, the CART illustrated best calibration for PCa (SSR = 0.027) and CSPCa (SSR = 0.033). The ANN, SVM, RF, and LR for PCa had higher net benefit than CART across the threshold probabilities above 5%, and the five models for CSPCa displayed similar net benefit across the threshold probabilities below 40%. The RF (53% and 57%, respectively) and SVM (52% and 55%, respectively) for PCa and CSPCa spared more unnecessary biopsies than logistic regression (35% and 47%, respectively) at 95% sensitivity for detection of CSPCa.

Conclusion

Machine-learning models (SVM and RF) yielded similar diagnostic accuracy and net benefit, while spared more biopsies at 95% sensitivity for detection of CSPCa, compared with logistic regression. However, no method achieved desired performance. All methods should continue to be explored and used in complementary ways.

Applications of neural networks in urology: a systematic review

PURPOSE OF REVIEW

Over the last decade, major advancements in artificial intelligence technology have emerged and revolutionized the extent to which physicians are able to personalize treatment modalities and care for their patients. Artificial intelligence technology aimed at mimicking/simulating human mental processes, such as deep learning artificial neural networks (ANNs), are composed of a collection of individual units known as 'artificial neurons'. These 'neurons', when arranged and interconnected in complex architectural layers, are capable of analyzing the most complex patterns. The aim of this systematic review is to give a comprehensive summary of the contemporary applications of deep learning ANNs in urological medicine.

RECENT FINDINGS

Fifty-five articles were included in this systematic review and each article was assigned an 'intermediate' score based on its overall quality. Of these 55 articles, nine studies were prospective, but no nonrandomized control trials were identified.

SUMMARY

In urological medicine, the application of novel artificial intelligence technologies, particularly ANNs, have been considered to be a promising step in improving physicians' diagnostic capabilities, especially with regards to predicting the aggressiveness and recurrence of various disorders. For benign urological disorders, for example, the use of highly predictive and reliable algorithms could be helpful for the improving diagnoses of male infertility, urinary tract infections, and pediatric malformations. In addition, articles with anecdotal experiences shed light on the potential of artificial intelligence-assisted surgeries, such as with the aid of virtual reality or augmented reality.

Current methods for assessing technical skill in cataract surgery

Surgery is a major source of errors in patient care. Preventing complications from surgical errors in the operating room is estimated to lead to reduction of up to 41 846 readmissions and save $620.3 million per year. It is now established that poor technical skill is associated with an increased risk of severe adverse events postoperatively and traditional models to train surgeons are being challenged by rapid advances in technology, an intensified patient-safety culture, and a need for value-driven health systems. This review discusses the current methods available for evaluating technical skills in cataract surgery and the recent technological advancements that have enabled capture and analysis of large amounts of complex surgical data for more automated objective skills assessment.

Contemporary application of artificial intelligence in prostate cancer: an i-TRUE study

Artificial intelligence (AI) involves technology that is able to emulate tasks previously carried out by humans. The growing incidence, novel diagnostic strategies and newer available therapeutic options have had resource and economic impacts on the healthcare organizations providing prostate cancer care. AI has the potential to be an adjunct to and, in certain cases, a replacement for human input in prostate cancer care delivery. Automation can also address issues such as inter- and intra-observer variability and has the ability to deliver analysis of large volume datasets quickly and accurately. The continuous training and testing of AI algorithms will facilitate development of futuristic AI models that will have integral roles to play in diagnostics, enhanced training and surgical outcomes and developments of prostate cancer predictive tools. These AI related innovations will enable clinicians to provide individualized care. Despite its potential benefits, it is vital that governance with AI related care is maintained and responsible adoption is achieved.

Adoption of Machine Learning Algorithm-Based Intelligent Basketball Training Robot in Athlete Injury Prevention

In order to effectively prevent sports injuries caused by collisions in basketball training, realize efficient shooting, and reduce collisions, the machine learning algorithm was applied to intelligent robot for path planning in this study. First of all, combined with the basketball motion trajectory model, the sport recognition in basketball training was analyzed. Second, the mathematical model of the basketball motion trajectory of the shooting motion was established, and the factors affecting the shooting were analyzed. Thirdly, on this basis, the machine learning-based improved Q-Learning algorithm was proposed, the path planning of the moving robot was realized, and the obstacle avoidance behavior was accomplished effectively. In the path planning, the principle of fuzzy controller was applied, and the obstacle ultrasonic signals acquired around the robot were taken as input to effectively avoid obstacles. Finally, the robot was able to approach the target point while avoiding obstacles. The results of simulation experiment show that the obstacle avoidance path obtained by the improved Q-Learning algorithm is flatter, indicating that the algorithm is more suitable for the obstacle avoidance of the robot. Besides, it only takes about 250 s for the robot to find the obstacle avoidance path to the target state for the first time, which is far lower than the 700 s of the previous original algorithm. As a result, the fuzzy controller applied to the basketball robot can effectively avoid the obstacles in the robot movement process, and the motion trajectory curve obtained is relatively smooth. Therefore, the proposed machine learning algorithm has favorable obstacle avoidance effect when it is applied to path planning in basketball training, and can effectively prevent sports injuries in basketball activities.

Dr. Answer AI for Prostate Cancer: Predicting Biochemical Recurrence Following Radical Prostatectomy

OBJECTIVES

To develop a model to predict biochemical recurrence (BCR) after radical prostatectomy (RP), using artificial intelligence (AI) techniques.

PATIENTS AND METHODS

This study collected data from 7,128 patients with prostate cancer (PCa) who received RP at 3 tertiary hospitals. After preprocessing, we used the data of 6,755 cases to generate the BCR prediction model. There were 16 input variables with BCR as the outcome variable. We used a random forest to develop the model. Several sampling techniques were used to address class imbalances.

RESULTS

We achieved good performance using a random forest with synthetic minority oversampling technique (SMOTE) using Tomek links, edited nearest neighbors (ENN), and random oversampling: accuracy = 96.59%, recall = 95.49%, precision = 97.66%, F1 score = 96.59%, and ROC AUC = 98.83%.

CONCLUSION

We developed a BCR prediction model for RP. The Dr. Answer AI project, which was developed based on our BCR prediction model, helps physicians and patients to make treatment decisions in the clinical follow-up process as a clinical decision support system.

Robotic Surgery: The Impact of Simulation and Other Innovative Platforms on Performance and Training

OBJECTIVE

To review the current status of robotic training and the impact of various training platforms on the performance of robotic surgical trainees.

DATA SOURCES

Literature review of Google Scholar and PubMed. The search terms included a combination of the following: "robotic training," "simulation," "robotic curriculum," "obgyn residency robotic training," "virtual reality robotic training," "DaVinci training," "surgical simulation," "gyn surgical training." The sources considered for inclusion included peer-reviewed articles, literature reviews, textbook chapters, and statements from various institutions involved in resident training.

METHODS OF STUDY SELECTION

A literature search of Google Scholar and PubMed using terms related to robotic surgery and robotics training, as mentioned in the "Data Sources" section.

RESULTS

Multiple novel platforms that use machine learning and real-time video feedback to teach and evaluate robotic surgical skills have been developed over recent years. Various training curricula, virtual reality simulators, and other robotic training tools have been shown to enhance robotic surgical education and improve surgical skills. The integration of didactic learning, simulation, and intraoperative teaching into more comprehensive training curricula shows positive effects on robotic skills proficiency. Few robotic surgery training curricula have been validated through peer-reviewed study, and there is more work to be completed in this area. In addition, there is a lack of information about how the skills obtained through robotics curricula and simulation translate into operating room performance and patient outcomes.

CONCLUSION

Data collected to date show promising advances in the training of robotic surgeons. A diverse array of curricula for training robotic surgeons continue to emerge, and existing teaching modalities are evolving to keep up with the rapidly growing demand for proficient robotic surgeons. Futures areas of growth include establishing competency benchmarks for existing training tools, validating existing curricula, and determining how to translate the acquired skills in simulation into performance in the operating room and patient outcomes. Many surgical training platforms are beginning to expand beyond discrete robotic skills training to procedure-specific and team training. There is still a wealth of research to be done to understand how to create an effective training experience for gynecologic surgical trainees and robotics teams.

Expert Consensus Recommendations for Robotic Surgery Credentialing

OBJECTIVE

To define criteria for robotic credentialing using expert consensus.

BACKGROUND

A recent review of institutional robotic credentialing policies identified significant variability and determined current policies are largely inadequate to ensure surgeon proficiency and may threaten patient safety.

METHODS

28 national robotic surgery experts were invited to participate in a consensus conference. After review of available institutional policies and discussion, the group developed a 91 proposed criteria. Using a modified Delphi process the experts were asked to indicate their agreement with the proposed criteria in three electronic survey rounds after the conference. Criteria that achieved 80% or more in agreement (consensus) in all rounds were included in the final list.

RESULTS

All experts agreed that there is a need for standardized robotic surgery credentialing criteria across institutions that promote surgeon proficiency. 49 items reached consensus in the first round, 19 in the second, and 8 in the third for a total of 76 final items. Experts agreed that privileges should be granted based on video review of surgical performance and attainment of clearly defined objective proficiency benchmarks. Parameters for ongoing outcome monitoring were determined and recommendations for technical skills training, proctoring, and performance assessment were defined.

CONCLUSIONS

Using a systematic approach, detailed credentialing criteria for robotic surgery were defined. Implementation of these criteria uniformly across institutions will promote proficiency of robotic surgeons and has the potential to positively impact patient outcomes.

Current Trends in Artificial Intelligence Application for Endourology and Robotic Surgery

With the advent of electronic medical records and digitalization of health care over the past 2 decades, artificial intelligence (AI) has emerged as an enabling tool to manage complex datasets and deliver streamlined data-driven patient care. AI algorithms have the ability to extract meaningful signal from complex datasets through an iterative process akin to human learning. Through advancements over the past decade in deep learning, AI-driven innovations have accelerated applications in health care. Herein, the authors explore the development of these emerging AI technologies, focusing on the application of AI to endourology and robotic surgery.

Development and validation of non-guided bladder-neck and neurovascular-bundle dissection modules of the RobotiX-Mentor® full-procedure robotic-assisted radical prostatectomy virtual reality simulation

Background

Full‐procedure virtual reality (VR) simulator training in robotic‐assisted radical prostatectomy (RARP) is a new tool in surgical education.

Methods

Description of the development of a VR RARP simulation model, (RobotiX‐Mentor®) including non‐guided bladder neck (ngBND) and neurovascular bundle dissection (ngNVBD) modules, and assessment of face, content, and construct validation of the ngBND and ngNVBD modules by robotic surgeons with different experience levels.

Results

Simulator and ngBND/ngNVBD modules were rated highly by all surgeons for realism and usability as training tool. In the ngBND‐task construct, validation was not achieved in task‐specific performance metrics. In the ngNVBD, task‐specific performance of the expert/intermediately experienced surgeons was significantly better than that of novices.

Conclusions

We proved face and content validity of simulator and both modules, and construct validity for generic metrics of the ngBND module and for generic and task‐specific metrics of the ngNVBD module.

Bidirectional long short-term memory for surgical skill classification of temporally segmented tasks

PURPOSE

The majority of historical surgical skill research typically analyzes holistic summary task-level metrics to create a skill classification for a performance. Recent advances in machine learning allow time series classification at the sub-task level, allowing predictions on segments of tasks, which could improve task-level technical skill assessment.

METHODS

A bidirectional long short-term memory (LSTM) network was used with 8-s windows of multidimensional time-series data from the Basic Laparoscopic Urologic Skills dataset. The network was trained on experts and novices from four common surgical tasks. Stratified cross-validation with regularization was used to avoid overfitting. The misclassified cases were re-submitted for surgical technical skill assessment to crowds using Amazon Mechanical Turk to re-evaluate and to analyze the level of agreement with previous scores.

RESULTS

Performance was best for the suturing task, with 96.88% accuracy at predicting whether a performance was an expert or novice, with 1 misclassification, when compared to previously obtained crowd evaluations. When compared with expert surgeon ratings, the LSTM predictions resulted in a Spearman coefficient of 0.89 for suturing tasks. When crowds re-evaluated misclassified performances, it was found that for all 5 misclassified cases from peg transfer and suturing tasks, the crowds agreed more with our LSTM model than with the previously obtained crowd scores.

CONCLUSION

The technique presented shows results not incomparable with labels which would be obtained from crowd-sourced labels of surgical tasks. However, these results bring about questions of the reliability of crowd sourced labels in videos of surgical tasks. We, as a research community, should take a closer look at crowd labeling with higher scrutiny, systematically look at biases, and quantify label noise.

The Application of Artificial Intelligence in Prostate Cancer Management—What Improvements Can Be Expected? A Systematic Review

Artificial Intelligence (AI) is progressively remodeling our daily life. A large amount of information from “big data” now enables machines to perform predictions and improve our healthcare system. AI has the potential to reshape prostate cancer (PCa) management thanks to growing applications in the field. The purpose of this review is to provide a global overview of AI in PCa for urologists, pathologists, radiotherapists, and oncologists to consider future changes in their daily practice. A systematic review was performed, based on PubMed MEDLINE, Google Scholar, and DBLP databases for original studies published in English from January 2009 to January 2019 relevant to PCa, AI, Machine Learning, Artificial Neural Networks, Convolutional Neural Networks, and Natural-Language Processing. Only articles with full text accessible were considered. A total of 1008 articles were reviewed, and 48 articles were included. AI has potential applications in all fields of PCa management: analysis of genetic predispositions, diagnosis in imaging, and pathology to detect PCa or to differentiate between significant and non-significant PCa. AI also applies to PCa treatment, whether surgical intervention or radiotherapy, skills training, or assessment, to improve treatment modalities and outcome prediction. AI in PCa management has the potential to provide a useful role by predicting PCa more accurately, using a multiomic approach and risk-stratifying patients to provide personalized medicine.

Temporal variability of surgical technical skill perception in real robotic surgery

PURPOSE

Summary score metrics, either from crowds of non-experts, faculty surgeons or from automated performance metrics, have been trusted as the prevailing method of reporting surgeon technical skill. The aim of this paper is to learn whether there exist significant fluctuations in the technical skill assessments of a surgeon throughout long durations of surgical footage.

METHODS

A set of 12 videos of robotic surgery cases from common human patient robotic surgeries were used to evaluate the perceived technical skill at each individual minute of the surgical videos, which were originally 12-15 min in length. A linear mixed-effects model for each video was used to compare the ratings of each minute to those from every other minute in order to learn whether a change in scores over time can be detected and reliably measured apart from inter- and intrarater variation.

RESULTS

Modeling the change over time of the global evaluative assessment of robotic skills scores significantly contributed to the prediction models for 11 of the 12 surgeons. This demonstrates that measurable changes in technical skill occur over time during robotic surgery.

CONCLUSION

The findings from this research raise questions about the optimal duration of footage needed to be evaluated to arrive at an accurate rating of surgical technical skill for longer procedures. This may imply non-negligible label noise for supervised machine learning approaches. In the future, it may be necessary to report a surgeon's skill variability in addition to their mean score to have proper knowledge of a surgeon's overall skill level.

The what? How? And Who? Of video based assessment

BACKGROUND

Currently, there is significant variability in the development, implementation and overarching goals of video review for assessment of surgical performance.

METHODS

This paper evaluates the current methods in which video review is used for evaluation of surgical performance and identifies which processes are critical for successful, widespread implementation of video-based assessment.

RESULTS

Despite the advances in video capture technology and growing interest in video-based assessment, there is a notable gap in the implementation and longitudinal use of formative and summative assessment using video.

CONCLUSION

Validity, scalability and discoverability are current but removable barriers to video-based assessment.

The effect of video playback speed on surgeon technical skill perception

PURPOSE

 Finding effective methods of discriminating surgeon technical skill has proved a complex problem to solve computationally. Previous research has shown that obtaining non-expert crowd evaluations of surgical performances is as accurate as the gold standard, expert surgeon review. The aim of this research is: (1) to learn whether crowdsourced evaluators give higher ratings of technical skill to video of performances with increased playback speed, (2) its effect in discriminating skill levels, and (3) whether this increase is related to the evaluator consciously being aware that the video is manually manipulated.

METHODS

 A set of ten peg transfer videos (five novices, five experts) were used to evaluate the perceived technical skill of the performers at each video playback speed used ([Formula: see text]). Objective metrics used for measuring technical skill were also computed for comparison by manipulating the corresponding kinematic data of each performance. Two videos of an expert and novice performing dry laboratory laparoscopic trials of peg transfer tasks were used to obtain evaluations at each playback speed ([Formula: see text]) of perception of whether a video is played at real-time playback speed or not.

RESULTS

 We found that while both novices and experts had increased perceived technical skill as the video playback was increased, the amount of increase was significantly greater for experts. Each increase in the playback speed by [Formula: see text] was associated with, on average, a 0.72-point increase in the GOALS score (95% CI 0.60-0.84 point increase; [Formula: see text]) for expert videos and only a 0.24-point increase in the GOALS score (95% CI 0.13-0.36 point increase; [Formula: see text]) for novice videos.

CONCLUSION

 Due to the differential increase in perceived technical skill due to increased playback speed for experts, the difference between novice and expert skill levels of surgical performances may be more easily discerned by manually increasing the video playback speed.

Evaluation of Deep Learning Models for Identifying Surgical Actions and Measuring Performance

IMPORTANCE

When evaluating surgeons in the operating room, experienced physicians must rely on live or recorded video to assess the surgeon's technical performance, an approach prone to subjectivity and error. Owing to the large number of surgical procedures performed daily, it is infeasible to review every procedure; therefore, there is a tremendous loss of invaluable performance data that would otherwise be useful for improving surgical safety.

OBJECTIVE

To evaluate a framework for assessing surgical video clips by categorizing them based on the surgical step being performed and the level of the surgeon's competence.

DESIGN, SETTING, AND PARTICIPANTS

This quality improvement study assessed 103 video clips of 8 surgeons of various levels performing knot tying, suturing, and needle passing from the Johns Hopkins University-Intuitive Surgical Gesture and Skill Assessment Working Set. Data were collected before 2015, and data analysis took place from March to July 2019.

MAIN OUTCOMES AND MEASURES

Deep learning models were trained to estimate categorical outputs such as performance level (ie, novice, intermediate, and expert) and surgical actions (ie, knot tying, suturing, and needle passing). The efficacy of these models was measured using precision, recall, and model accuracy.

RESULTS

The provided architectures achieved accuracy in surgical action and performance calculation tasks using only video input. The embedding representation had a mean (root mean square error [RMSE]) precision of 1.00 (0) for suturing, 0.99 (0.01) for knot tying, and 0.91 (0.11) for needle passing, resulting in a mean (RMSE) precision of 0.97 (0.01). Its mean (RMSE) recall was 0.94 (0.08) for suturing, 1.00 (0) for knot tying, and 0.99 (0.01) for needle passing, resulting in a mean (RMSE) recall of 0.98 (0.01). It also estimated scores on the Objected Structured Assessment of Technical Skill Global Rating Scale categories, with a mean (RMSE) precision of 0.85 (0.09) for novice level, 0.67 (0.07) for intermediate level, and 0.79 (0.12) for expert level, resulting in a mean (RMSE) precision of 0.77 (0.04). Its mean (RMSE) recall was 0.85 (0.05) for novice level, 0.69 (0.14) for intermediate level, and 0.80 (0.13) for expert level, resulting in a mean (RMSE) recall of 0.78 (0.03).

CONCLUSIONS AND RELEVANCE

The proposed models and the accompanying results illustrate that deep machine learning can identify associations in surgical video clips. These are the first steps to creating a feedback mechanism for surgeons that would allow them to learn from their experiences and refine their skills.

Analysis of the video motion tracking system "Kinovea" to assess surgical movements during robot-assisted radical prostatectomy

BACKGROUNDS

Robot-assisted surgery facilitated the possibility to evaluate the surgeon's skills by recording and evaluating the robot surgical images. The aim of this study was to investigate the possibility of using a computer programme (Kinovea) for objective assessment of surgical movements in previously recorded in existing robot-assisted radical prostatectomy (RARP) videos.

METHODS

Twelve entire RARP procedures were analysed by a trained researcher using the computer programme "Kinovea" to perform semi-automated assessment of surgical movements.

RESULTS

Data analysis showed Kinovea was on average able to automatically assess only 22% of the total surgical duration per video of the robot-assisted surgery. On average, it lasted 4 hours of continued monitoring by the researcher to assess one RARP using Kinovea.

CONCLUSION

Although we proved it is technically possible to use the Kinovea system in retrospective analysis of surgical movement in robot-assisted surgery, the acquired data do not give a comprehensive enough analysis of the video to be used in skills assessment.

Video review for measuring and improving skill in urological surgery

Interest is growing within the urological surgery community for objective assessments of technical skill. Surgical video review relies on the use of objective assessment tools to evaluate both global and procedure-specific skill. These evaluations provide structured feedback to surgeons with the aim of improving technique, which has been associated with patient outcomes. Currently, skill assessments can be performed by using expert peer-review, crowdsourcing or computer-based methods. Given the relationship between skill and patient outcomes, surgeons might be required in the future to provide empirical evidence of their technical skill for certification, employment, credentialing and quality improvement. Interventions such as coaching and skills workshops incorporating video review might help surgeons improve their skill, with the ultimate goal of improving patient outcomes.

Artificial intelligence and neural networks in urology: current clinical applications

INTRODUCTION

As we enter the era of "big data," an increasing amount of complex health-care data will become available. These data are often redundant, "noisy," and characterized by wide variability. In order to offer a precise and transversal view of a clinical scenario the artificial intelligence (AI) with machine learning (ML) algorithms and Artificial neuron networks (ANNs) process were adopted, with a promising wide diffusion in the near future. The present work aims to provide a comprehensive and critical overview of the current and potential applications of AI and ANNs in urology.

EVIDENCE ACQUISITION

A non-systematic review of the literature was performed by screening Medline, PubMed, the Cochrane Database, and Embase to detect pertinent studies regarding the application of AI and ANN in Urology.

EVIDENCE SYNTHESIS

The main application of AI in urology is the field of genitourinary cancers. Focusing on prostate cancer, AI was applied for the prediction of prostate biopsy results. For bladder cancer, the prediction of recurrence-free probability and diagnostic evaluation were analysed with ML algorithms. For kidney and testis cancer, anecdotal experiences were reported for staging and prediction of diseases recurrence. More recently, AI has been applied in non-oncological diseases like stones and functional urology.

CONCLUSIONS

AI technologies are growing their role in health care; but, up to now, their "real-life" implementation remains limited. However, in the near future, the potential of AI-driven era could change the clinical practice in Urology, improving overall patient outcomes.

Artificial intelligence and robotics: a combination that is changing the operating room

PURPOSE

The aim of the current narrative review was to summarize the available evidence in the literature on artificial intelligence (AI) methods that have been applied during robotic surgery.

METHODS

A narrative review of the literature was performed on MEDLINE/Pubmed and Scopus database on the topics of artificial intelligence, autonomous surgery, machine learning, robotic surgery, and surgical navigation, focusing on articles published between January 2015 and June 2019. All available evidences were analyzed and summarized herein after an interactive peer-review process of the panel.

LITERATURE REVIEW

The preliminary results of the implementation of AI in clinical setting are encouraging. By providing a readout of the full telemetry and a sophisticated viewing console, robot-assisted surgery can be used to study and refine the application of AI in surgical practice. Machine learning approaches strengthen the feedback regarding surgical skills acquisition, efficiency of the surgical process, surgical guidance and prediction of postoperative outcomes. Tension-sensors on the robotic arms and the integration of augmented reality methods can help enhance the surgical experience and monitor organ movements.

CONCLUSIONS

The use of AI in robotic surgery is expected to have a significant impact on future surgical training as well as enhance the surgical experience during a procedure. Both aim to realize precision surgery and thus to increase the quality of the surgical care. Implementation of AI in master-slave robotic surgery may allow for the careful, step-by-step consideration of autonomous robotic surgery.

Current and future applications of machine and deep learning in urology: a review of the literature on urolithiasis, renal cell carcinoma, and bladder and prostate cancer

PURPOSE

The purpose of the study was to provide a comprehensive review of recent machine learning (ML) and deep learning (DL) applications in urological practice. Numerous studies have reported their use in the medical care of various urological disorders; however, no critical analysis has been made to date.

METHODS

A detailed search of original articles was performed using the PubMed MEDLINE database to identify recent English literature relevant to ML and DL applications in the fields of urolithiasis, renal cell carcinoma (RCC), bladder cancer (BCa), and prostate cancer (PCa).

RESULTS

In total, 43 articles were included addressing these four subfields. The most common ML and DL application in urolithiasis is in the prediction of endourologic surgical outcomes. The main area of research involving ML and DL in RCC concerns the differentiation between benign and malignant small renal masses, Fuhrman nuclear grade prediction, and gene expression-based molecular signatures. BCa studies employ radiomics and texture feature analysis for the distinction between low- and high-grade tumors, address accurate image-based cytology, and use algorithms to predict treatment response, tumor recurrence, and patient survival. PCa studies aim at developing algorithms for Gleason score prediction, MRI computer-aided diagnosis, and surgical outcomes and biochemical recurrence prediction. Studies consistently found the superiority of these methods over traditional statistical methods.

CONCLUSIONS

The continuous incorporation of clinical data, further ML and DL algorithm retraining, and generalizability of models will augment the prediction accuracy and enhance individualized medicine.

Current status of artificial intelligence applications in urology and their potential to influence clinical practice

OBJECTIVE

To investigate the applications of artificial intelligence (AI) in diagnosis, treatment and outcome predictionin urologic diseases and evaluate its advantages over traditional models and methods.

MATERIALS AND METHODS

A literature search was performed after PROSPERO registration (CRD42018103701) and in compliance with Preferred Reported Items for Systematic Reviews and Meta-Analyses (PRISMA) methods. Articles between 1994 and 2018 using the search terms "urology", "artificial intelligence", "machine learning" were included and categorized by the application of AI in urology. Review articles, editorial comments, articles with no full-text access, and nonurologic studies were excluded.

RESULTS

Initial search yielded 231 articles, but after excluding duplicates and following full-text review and examination of article references, only 111 articles were included in the final analysis. AI applications in urology include: utilizing radiomic imaging or ultrasonic echo data to improve or automate cancer detection or outcome prediction, utilizing digitized tissue specimen images to automate detection of cancer on pathology slides, and combining patient clinical data, biomarkers, or gene expression to assist disease diagnosis or outcome prediction. Some studies employed AI to plan brachytherapy and radiation treatments while others used video based or robotic automated performance metrics to objectively evaluate surgical skill. Compared to conventional statistical analysis, 71.8% of studies concluded that AI is superior in diagnosis and outcome prediction.

CONCLUSION

AI has been widely adopted in urology. Compared to conventional statistics AI approaches are more accurate in prediction and more explorative for analyzing large data cohorts. With an increasing library of patient data accessible to clinicians, AI may help facilitate evidence-based and individualized patient care.

A deep-learning model using automated performance metrics and clinical features to predict urinary continence recovery after robot-assisted radical prostatectomy

OBJECTIVES

To predict urinary continence recovery after robot-assisted radical prostatectomy (RARP) using a deep learning (DL) model, which was then used to evaluate surgeon's historical patient outcomes.

SUBJECTS AND METHODS

Robotic surgical automated performance metrics (APMs) during RARP, and patient clinicopathological and continence data were captured prospectively from 100 contemporary RARPs. We used a DL model (DeepSurv) to predict postoperative urinary continence. Model features were ranked based on their importance in prediction. We stratified eight surgeons based on the five top-ranked features. The top four surgeons were categorized in 'Group 1/APMs', while the remaining four were categorized in 'Group 2/APMs'. A separate historical cohort of RARPs (January 2015 to August 2016) performed by these two surgeon groups was then used for comparison. Concordance index (C-index) and mean absolute error (MAE) were used to measure the model's prediction performance. Outcomes of historical cases were compared using the Kruskal-Wallis, chi-squared and Fisher's exact tests.

RESULTS

Continence was attained in 79 patients (79%) after a median of 126 days. The DL model achieved a C-index of 0.6 and an MAE of 85.9 in predicting continence. APMs were ranked higher by the model than clinicopathological features. In the historical cohort, patients in Group 1/APMs had superior rates of urinary continence at 3 and 6 months postoperatively (47.5 vs 36.7%, P = 0.034, and 68.3 vs 59.2%, P = 0.047, respectively).

CONCLUSION

Using APMs and clinicopathological data, the DeepSurv DL model was able to predict continence after RARP. In this feasibility study, surgeons with more efficient APMs achieved higher continence rates at 3 and 6 months after RARP.

Effect of surgeon experience and bony pelvic dimensions on surgical performance and patient outcomes in robot-assisted radical prostatectomy

OBJECTIVES

To evaluate the effects of surgeon experience, body habitus, and bony pelvic dimensions on surgeon performance and patient outcomes after robot-assisted radical prostatectomy (RARP).

PATIENTS, SUBJECTS AND METHODS

The pelvic dimensions of 78 RARP patients were measured on preoperative magnetic resonance imaging and computed tomography by three radiologists. Surgeon automated performance metrics (APMs [instrument motion tracking and system events data, i.e., camera movement, third-arm swap, energy use]) were obtained by a systems data recorder (Intuitive Surgical, Sunnyvale, CA, USA) during RARP. Two analyses were performed: Analysis 1, examined effects of patient characteristics, pelvic dimensions and prior surgeon RARP caseload on APMs using linear regression; Analysis 2, the effects of patient body habitus, bony pelvic measurement, and surgeon experience on short- and long-term outcomes were analysed by multivariable regression.

RESULTS

Analysis 1 showed that while surgeon experience affected the greatest number of APMs (P < 0.044), the patient's body mass index, bony pelvic dimensions, and prostate size also affected APMs during each surgical step (P < 0.043, P < 0.046, P < 0.034, respectively). Analysis 2 showed that RARP duration was significantly affected by pelvic depth (β = 13.7, P = 0.039) and prostate volume (β = 0.5, P = 0.024). A wider and shallower pelvis was less likely to result in a positive margin (odds ratio 0.25, 95% confidence interval [CI] 0.09-0.72). On multivariate analysis, urinary continence recovery was associated with surgeon's prior RARP experience (hazard ratio [HR] 2.38, 95% CI 1.18-4.81; P = 0.015), but not on pelvic dimensions (HR 1.44, 95% CI 0.95-2.17).

CONCLUSION

Limited surgical workspace, due to a narrower and deeper pelvis, does affect surgeon performance and patient outcomes, most notably in longer surgery time and an increased positive margin rate.

Novel evaluation of surgical activity recognition models using task-based efficiency metrics

PURPOSE

Surgical task-based metrics (rather than entire procedure metrics) can be used to improve surgeon training and, ultimately, patient care through focused training interventions. Machine learning models to automatically recognize individual tasks or activities are needed to overcome the otherwise manual effort of video review. Traditionally, these models have been evaluated using frame-level accuracy. Here, we propose evaluating surgical activity recognition models by their effect on task-based efficiency metrics. In this way, we can determine when models have achieved adequate performance for providing surgeon feedback via metrics from individual tasks.

METHODS

We propose a new CNN-LSTM model, RP-Net-V2, to recognize the 12 steps of robotic-assisted radical prostatectomies (RARP). We evaluated our model both in terms of conventional methods (e.g., Jaccard Index, task boundary accuracy) as well as novel ways, such as the accuracy of efficiency metrics computed from instrument movements and system events.

RESULTS

Our proposed model achieves a Jaccard Index of 0.85 thereby outperforming previous models on RARP. Additionally, we show that metrics computed from tasks automatically identified using RP-Net-V2 correlate well with metrics from tasks labeled by clinical experts.

CONCLUSION

We demonstrate that metrics-based evaluation of surgical activity recognition models is a viable approach to determine when models can be used to quantify surgical efficiencies. We believe this approach and our results illustrate the potential for fully automated, postoperative efficiency reports.

Evidence that surgical performance predicts clinical outcomes

PURPOSE

Assessment of surgeon performance in the operating room has been identified as a direct method of measuring surgical quality. Studies published in urology and other surgical disciplines have investigated this link directly by measuring surgeon and team performance using methodology supported by validity evidence. This article highlights the key findings of these studies and associated underlying concepts.

METHODS

​Seminal literature from urology and related areas of research was used to inform this review of the performance-outcome relationship in surgery. Current efforts to further our understanding of this concept are discussed, including relevant quality improvement and educational interventions that utilize this relationship.

RESULTS

Evidence from multiple surgical specialties and procedures has established the association between surgeon skill and clinically significant patient outcomes. Novel methods of measuring performance utilize surgeon kinematics and artificial intelligence techniques to more reliably and objectively quantify surgical performance.

CONCLUSIONS

Future directions include the use of this data to create interventions for quality improvement, as well as innovate the credentialing and recertification process for practicing surgeons.

Automatic and near real-time stylistic behavior assessment in robotic surgery

PURPOSE

Automatic skill evaluation is of great importance in surgical robotic training. Extensive research has been done to evaluate surgical skill, and a variety of quantitative metrics have been proposed. However, these methods primarily use expert selected features which may not capture latent information in movement data. In addition, these features are calculated over the entire task time and are provided to the user after the completion of the task. Thus, these quantitative metrics do not provide users with information on how to modify their movements to improve performance in real time. This study focuses on automatic stylistic behavior recognition that has the potential to be implemented in near real time.

METHODS

We propose a sparse coding framework for automatic stylistic behavior recognition in short time intervals using only position data from the hands, wrist, elbow, and shoulder. A codebook is built for each stylistic adjective using the positive and negative labels provided for each trial through crowd sourcing. Sparse code coefficients are obtained for short time intervals (0.25 s) in a trial using this codebook. A support vector machine classifier is trained and validated through tenfold cross-validation using the sparse codes from the training set.

RESULTS

The results indicate that the proposed dictionary learning method is able to assess stylistic behavior performance in near real time using user joint position data with improved accuracy compared to using PCA features or raw data.

CONCLUSION

The possibility to automatically evaluate a trainee's style of movement in short time intervals could provide the user with online customized feedback and thus improve performance during surgical tasks.

Experts vs super-experts: differences in automated performance metrics and clinical outcomes for robot-assisted radical prostatectomy

OBJECTIVES

To evaluate automated performance metrics (APMs) and clinical data of experts and super-experts for four cardinal steps of robot-assisted radical prostatectomy (RARP): bladder neck dissection; pedicle dissection; prostate apex dissection; and vesico-urethral anastomosis.

SUBJECTS AND METHODS

We captured APMs (motion tracking and system events data) and synchronized surgical video during RARP. APMs were compared between two experience levels: experts (100-750 cases) and super-experts (2100-3500 cases). Clinical outcomes (peri-operative, oncological and functional) were then compared between the two groups. APMs and outcomes were analysed for 125 RARPs using multi-level mixed-effect modelling.

RESULTS

For the four cardinal steps selected, super-experts showed differences in select APMs compared with experts (P < 0.05). Despite similar PSA and Gleason scores, super-experts outperformed experts clinically with regard to peri-operative outcomes, with a greater lymph node yield of 22.6 vs 14.9 nodes, respectively (P < 0.01), less blood loss (125 vs 130 mL, respectively; P < 0.01), and fewer readmissions at 30 days (1% vs 13%, respectively; P = 0.02). A similar but nonsignificant trend was seen for oncological and functional outcomes, with super-experts having a lower rate of biochemical recurrence compared with experts (5% vs 15%, respectively; P = 0.13) and a higher continence rate at 3 months (36% vs 18%, respectively; P = 0.14).

CONCLUSION

We found that experts and super-experts differed significantly in select APMs for the four cardinal steps of RARP, indicating that surgeons do continue to improve in performance even after achieving expertise. We hope ultimately to identify associations between APMs and clinical outcomes to tailor interventions to surgeons and optimize patient outcomes.