An automatic skill evaluation framework for robotic surgery training

Measures of performance and proficiency in robotic assisted surgery: a systematic review

Robotic assisted surgery (RAS) has seen a global rise in adoption. Despite this, there is not a standardised training curricula nor a standardised measure of performance. We performed a systematic review across the surgical specialties in RAS and evaluated tools used to assess surgeons' technical performance. Using the PRISMA 2020 guidelines, Pubmed, Embase and the Cochrane Library were searched systematically for full texts published on or after January 2020-January 2022. Observational studies and RCTs were included; review articles and systematic reviews were excluded. The papers' quality and bias score were assessed using the Newcastle Ottawa Score for the observational studies and Cochrane Risk Tool for the RCTs. The initial search yielded 1189 papers of which 72 fit the eligibility criteria. 27 unique performance metrics were identified. Global assessments were the most common tool of assessment (n = 13); the most used was GEARS (Global Evaluative Assessment of Robotic Skills). 11 metrics (42%) were objective tools of performance. Automated performance metrics (APMs) were the most widely used objective metrics whilst the remaining (n = 15, 58%) were subjective. The results demonstrate variation in tools used to assess technical performance in RAS. A large proportion of the metrics are subjective measures which increases the risk of bias amongst users. A standardised objective metric which measures all domains of technical performance from global to cognitive is required. The metric should be applicable to all RAS procedures and easily implementable. Automated performance metrics (APMs) have demonstrated promise in their wide use of accurate measures.

Implementation of a Virtual Reality Based Digital-Twin Robotic Minimally Invasive Surgery Simulator

The rapid development of computers and robots has seen robotic minimally invasive surgery (RMIS) gradually enter the public's vision. RMIS can effectively eliminate the hand vibrations of surgeons and further reduce wounds and bleeding. However, suitable RMIS and virtual reality-based digital-twin surgery trainers are still in the early stages of development. Extensive training is required for surgeons to adapt to different operating modes compared to traditional MIS. A virtual-reality-based digital-twin robotic minimally invasive surgery (VRDT-RMIS) simulator was developed in this study, and its effectiveness was introduced. Twenty-five volunteers were divided into two groups for the experiment, the Expert Group and the Novice Group. The use of the VRDT-RMIS simulator for face, content, and structural validation training, including the peg transfer module and the soft tissue cutting module, was evaluated. Through subjective and objective evaluations, the potential roles of vision and haptics in robot surgery training were explored. The simulator can effectively distinguish surgical skill proficiency between experts and novices.

Artificial Intelligence Technologies in the Microsurgical Operating Room (Review)

Surgery performed by a novice neurosurgeon under constant supervision of a senior surgeon with the experience of thousands of operations, able to handle any intraoperative complications and predict them in advance, and never getting tired, is currently an elusive dream, but can become a reality with the development of artificial intelligence methods. This paper has presented a review of the literature on the use of artificial intelligence technologies in the microsurgical operating room. Searching for sources was carried out in the PubMed text database of medical and biological publications. The key words used were "surgical procedures", "dexterity", "microsurgery" AND "artificial intelligence" OR "machine learning" OR "neural networks". Articles in English and Russian were considered with no limitation to publication date. The main directions of research on the use of artificial intelligence technologies in the microsurgical operating room have been highlighted. Despite the fact that in recent years machine learning has been increasingly introduced into the medical field, a small number of studies related to the problem of interest have been published, and their results have not proved to be of practical use yet. However, the social significance of this direction is an important argument for its development.

Machine learning for technical skill assessment in surgery: a systematic review

Accurate and objective performance assessment is essential for both trainees and certified surgeons. However, existing methods can be time consuming, labor intensive, and subject to bias. Machine learning (ML) has the potential to provide rapid, automated, and reproducible feedback without the need for expert reviewers. We aimed to systematically review the literature and determine the ML techniques used for technical surgical skill assessment and identify challenges and barriers in the field. A systematic literature search, in accordance with the PRISMA statement, was performed to identify studies detailing the use of ML for technical skill assessment in surgery. Of the 1896 studies that were retrieved, 66 studies were included. The most common ML methods used were Hidden Markov Models (HMM, 14/66), Support Vector Machines (SVM, 17/66), and Artificial Neural Networks (ANN, 17/66). 40/66 studies used kinematic data, 19/66 used video or image data, and 7/66 used both. Studies assessed the performance of benchtop tasks (48/66), simulator tasks (10/66), and real-life surgery (8/66). Accuracy rates of over 80% were achieved, although tasks and participants varied between studies. Barriers to progress in the field included a focus on basic tasks, lack of standardization between studies, and lack of datasets. ML has the potential to produce accurate and objective surgical skill assessment through the use of methods including HMM, SVM, and ANN. Future ML-based assessment tools should move beyond the assessment of basic tasks and towards real-life surgery and provide interpretable feedback with clinical value for the surgeon.

PROSPERO: CRD42020226071

Exploring the roles of artificial intelligence in surgical education: A scoping review

BACKGROUND

Technology-enhanced teaching and learning, including Artificial Intelligence (AI) applications, has started to evolve in surgical education. Hence, the purpose of this scoping review is to explore the current and future roles of AI in surgical education.

METHODS

Nine bibliographic databases were searched from January 2010 to January 2021. Full-text articles were included if they focused on AI in surgical education.

RESULTS

Out of 14,008 unique sources of evidence, 93 were included. Out of 93, 84 were conducted in the simulation setting, and 89 targeted technical skills. Fifty-six studies focused on skills assessment/classification, and 36 used multiple AI techniques. Also, increasing sample size, having balanced data, and using AI to provide feedback were major future directions mentioned by authors.

CONCLUSIONS

AI can help optimize the education of trainees and our results can help educators and researchers identify areas that need further investigation.

Automated Assessment of Neonatal Endotracheal Intubation Measured by a Virtual Reality Simulation System

Manual assessment from experts in neonatal endotracheal intubation (ETI) training is a time-consuming and tedious process. Such subjective, highly variable, and resource-intensive assessment method may not only introduce inter-rater/intra-rater variability, but also represent a serious limitation in many large-scale training programs. Moreover, poor visualization during the procedure prevents instructors from observing the events occurring within the manikin or the patient, which introduces an additional source of error into the assessment. In this paper, we propose a physics-based virtual reality (VR) ETI simulation system that captures the entire motions of the laryngoscope and the endotracheal tube (ETT) in relation to the internal anatomy of the virtual patient. Our system provides a complete visualization of the procedure, offering instructors with comprehensive information for accurate assessment. More importantly, an interpretable machine learning algorithm was developed to automatically assess the ETI performance by training on the performance parameters extracted from the motions and the scores rated by experts. Our results show that the leave-one-out-cross-validation (LOOCV) classification accuracy of the automated assessment algorithm is 80%, which indicates that our system can reliably conduct a consistent and standardized assessment for ETI training.