A systematic review on artificial intelligence in robot-assisted surgery

Classification of subtask types and skill levels in robot-assisted surgery using EEG, eye-tracking, and machine learning

BACKGROUND

Objective and standardized evaluation of surgical skills in robot-assisted surgery (RAS) holds critical importance for both surgical education and patient safety. This study introduces machine learning (ML) techniques using features derived from electroencephalogram (EEG) and eye-tracking data to identify surgical subtasks and classify skill levels.

METHOD

The efficacy of this approach was assessed using a comprehensive dataset encompassing nine distinct classes, each representing a unique combination of three surgical subtasks executed by surgeons while performing operations on pigs. Four ML models, logistic regression, random forest, gradient boosting, and extreme gradient boosting (XGB) were used for multi-class classification. To develop the models, 20% of data samples were randomly allocated to a test set, with the remaining 80% used for training and validation. Hyperparameters were optimized through grid search, using fivefold stratified cross-validation repeated five times. Model reliability was ensured by performing train-test split over 30 iterations, with average measurements reported.

RESULTS

The findings revealed that the proposed approach outperformed existing methods for classifying RAS subtasks and skills; the XGB and random forest models yielded high accuracy rates (88.49% and 88.56%, respectively) that were not significantly different (two-sample t-test; P-value = 0.9).

CONCLUSION

These results underscore the potential of ML models to augment the objectivity and precision of RAS subtask and skill evaluation. Future research should consider exploring ways to optimize these models, particularly focusing on the classes identified as challenging in this study. Ultimately, this study marks a significant step towards a more refined, objective, and standardized approach to RAS training and competency assessment.

Development and accuracy assessment of a crown lengthening surgery robot for use in the esthetic zone: An in vitro study

STATEMENT OF PROBLEM

Crown lengthening surgery has been widely used to enhance the health and esthetics of anterior teeth, and its accuracy significantly influences surgical outcomes. However, the feasibility and accuracy of a robot system for crown lengthening surgery remains unknown.

PURPOSE

The purpose of this in vitro study was to develop a crown lengthening surgery robot and evaluate its accuracy.

MATERIAL AND METHODS

A robotic crown lengthening surgery system consisting of a robotic arm, a robotic software system, and an optical tracking device was designed. Intraoral scanning and cone beam computed tomography (CBCT) were performed on 18 artificial dentition models. The data were imported into the planning software program to synthesize a surgical path for gingivectomy and alveolectomy. Subsequently, a robotic arm equipped with a high-speed handpiece was used to perform these surgical procedures. Postoperatively, the models were rescanned for evaluation, with the accuracy (trueness ±precision) of the surgical outcomes of gingivectomy and alveolectomy being assessed from the trajectories in the highest, lowest, and overall regions. Differences between groups were analyzed by using the independent sample t test and the Levene test (α=.05).

RESULTS

Crown lengthening surgery was feasible in vitro using the robot developed in this study. The overall robot-assisted crown lengthening surgery accuracy (trueness ±precision) of gingivectomy (0.23 ±0.08 mm) was significantly higher than that of alveolectomy (0.33 ±0.11 mm) (P<.05).

CONCLUSIONS

Robot-assisted crown lengthening surgery had acceptable accuracy generally and can be considered a feasible treatment option.

Near-infrared II fluorescence-guided glioblastoma surgery targeting monocarboxylate transporter 4 combined with photothermal therapy

BACKGROUND

Surgery is crucial for glioma treatment, but achieving complete tumour removal remains challenging. We evaluated the effectiveness of a probe targeting monocarboxylate transporter 4 (MCT4) in recognising gliomas, and of near-infrared window II (NIR-II) fluorescent molecular imaging and photothermal therapy as treatment strategies.

METHODS

We combined an MCT4-specific monoclonal antibody with indocyanine green to create the probe. An orthotopic mouse model and a transwell model were used to evaluate its ability to guide tumour resection using NIR-II fluorescence and to penetrate the blood-brain barrier (BBB), respectively. A subcutaneous tumour model was established to confirm photothermal therapy efficacy. Probe specificity was assessed in brain tissue from mice and humans. Finally, probe effectiveness in photothermal therapy was investigated.

FINDINGS

MCT4 was differentially expressed in tumour and normal brain tissue. The designed probe exhibited precise tumour targeting. Tumour imaging was precise, with a signal-to-background (SBR) ratio of 2.8. Residual tumour cells were absent from brain tissue postoperatively (SBR: 6.3). The probe exhibited robust penetration of the BBB. Moreover, the probe increased the tumour temperature to 50 °C within 5 min of laser excitation. Photothermal therapy significantly reduced tumour volume and extended survival time in mice without damage to vital organs.

INTERPRETATION

These findings highlight the potential efficacy of our probe for fluorescence-guided surgery and therapeutic interventions.

FUNDING

Jilin Province Department of Science and Technology (20200403079SF), Department of Finance (2021SCZ06) and Development and Reform Commission (20200601002JC); National Natural Science Foundation of China (92059207, 92359301, 62027901, 81930053, 81227901, U21A20386); and CAS Youth Interdisciplinary Team (JCTD-2021-08).

The usefulness of artificial intelligence in breast reconstruction: a systematic review

BACKGROUND

Artificial Intelligence (AI) offers an approach to predictive modeling. The model learns to determine specific patterns of undesirable outcomes in a dataset. Therefore, a decision-making algorithm can be built based on these patterns to prevent negative results. This systematic review aimed to evaluate the usefulness of AI in breast reconstruction.

METHODS

A systematic review was conducted in August 2022 following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. MEDLINE, EMBASE, SCOPUS, and Google Scholar online databases were queried to capture all publications studying the use of artificial intelligence in breast reconstruction.

RESULTS

A total of 23 studies were full text-screened after removing duplicates, and twelve articles fulfilled our inclusion criteria. The Machine Learning algorithms applied for neuropathic pain, lymphedema diagnosis, microvascular abdominal flap failure, donor site complications associated to muscle sparing Transverse Rectus Abdominis flap, surgical complications, financial toxicity, and patient-reported outcomes after breast surgery demonstrated that AI is a helpful tool to accurately predict patient results. In addition, one study used Computer Vision technology to assist in Deep Inferior Epigastric Perforator Artery detection for flap design, considerably reducing the preoperative time compared to manual identification.

CONCLUSIONS

In breast reconstruction, AI can help the surgeon by optimizing the perioperative patients' counseling to predict negative outcomes, allowing execution of timely interventions and reducing the postoperative burden, which leads to obtaining the most successful results and improving patient satisfaction.

The Future of Artificial Intelligence in Surgery

Until recently, innovations in surgery were largely represented by extensions or augmentations of the surgeon's perception. This includes advancements such as the operating microscope, tumor fluorescence, intraoperative ultrasound, and minimally invasive surgical instrumentation. However, introducing artificial intelligence (AI) into the surgical disciplines represents a transformational event. Not only does AI contribute substantively to enhancing a surgeon's perception with such methodologies as three-dimensional anatomic overlays with augmented reality, AI-improved visualization for tumor resection, and AI-formatted endoscopic and robotic surgery guidance. What truly makes AI so different is that it also provides ways to augment the surgeon's cognition. By analyzing enormous databases, AI can offer new insights that can transform the operative environment in several ways. It can enable preoperative risk assessment and allow a better selection of candidates for procedures such as organ transplantation. AI can also increase the efficiency and throughput of operating rooms and staff and coordinate the utilization of critical resources such as intensive care unit beds and ventilators. Furthermore, AI is revolutionizing intraoperative guidance, improving the detection of cancers, permitting endovascular navigation, and ensuring the reduction in collateral damage to adjacent tissues during surgery (e.g., identification of parathyroid glands during thyroidectomy). AI is also transforming how we evaluate and assess surgical proficiency and trainees in postgraduate programs. It offers the potential for multiple, serial evaluations, using various scoring systems while remaining free from the biases that can plague human supervisors. The future of AI-driven surgery holds promising trends, including the globalization of surgical education, the miniaturization of instrumentation, and the increasing success of autonomous surgical robots. These advancements raise the prospect of deploying fully autonomous surgical robots in the near future into challenging environments such as the battlefield, disaster areas, and even extraplanetary exploration. In light of these transformative developments, it is clear that the future of surgery will belong to those who can most readily embrace and harness the power of AI.

The Millennia-Long Development of Drugs Associated with the 80-Year-Old Artificial Intelligence Story: The Therapeutic Big Bang?

The journey of drug discovery (DD) has evolved from ancient practices to modern technology-driven approaches, with Artificial Intelligence (AI) emerging as a pivotal force in streamlining and accelerating the process. Despite the vital importance of DD, it faces challenges such as high costs and lengthy timelines. This review examines the historical progression and current market of DD alongside the development and integration of AI technologies. We analyse the challenges encountered in applying AI to DD, focusing on drug design and protein-protein interactions. The discussion is enriched by presenting models that put forward the application of AI in DD. Three case studies are highlighted to demonstrate the successful application of AI in DD, including the discovery of a novel class of antibiotics and a small-molecule inhibitor that has progressed to phase II clinical trials. These cases underscore the potential of AI to identify new drug candidates and optimise the development process. The convergence of DD and AI embodies a transformative shift in the field, offering a path to overcome traditional obstacles. By leveraging AI, the future of DD promises enhanced efficiency and novel breakthroughs, heralding a new era of medical innovation even though there is still a long way to go.

A new artificial intelligence system for both stomach and small-bowel capsule endoscopy

BACKGROUND AND AIMS

Despite the benefits of artificial intelligence in small-bowel (SB) capsule endoscopy (CE) image reading, information on its application in the stomach and SB CE is lacking.

METHODS

In this multicenter, retrospective diagnostic study, gastric imaging data were added to the deep learning-based SmartScan (SS), which has been described previously. A total of 1069 magnetically controlled GI CE examinations (comprising 2,672,542 gastric images) were used in the training phase for recognizing gastric pathologies, producing a new artificial intelligence algorithm named SS Plus. A total of 342 fully automated, magnetically controlled CE examinations were included in the validation phase. The performance of both senior and junior endoscopists with both the SS Plus-assisted reading (SSP-AR) and conventional reading (CR) modes was assessed.

RESULTS

SS Plus was designed to recognize 5 types of gastric lesions and 17 types of SB lesions. SS Plus reduced the number of CE images required for review to 873.90 (median, 1000; interquartile range [IQR], 814.50-1000) versus 44,322.73 (median, 42,393; IQR, 31,722.75-54,971.25) for CR. Furthermore, with SSP-AR, endoscopists took 9.54 minutes (median, 8.51; IQR, 6.05-13.13) to complete the CE video reading. In the 342 CE videos, SS Plus identified 411 gastric and 422 SB lesions, whereas 400 gastric and 368 intestinal lesions were detected with CR. Moreover, junior endoscopists remarkably improved their CE image reading ability with SSP-AR.

CONCLUSIONS

Our study shows that the newly upgraded deep learning-based algorithm SS Plus can detect GI lesions and help improve the diagnostic performance of junior endoscopists in interpreting CE videos.

RARPKB: a knowledge-guide decision support platform for personalized robot-assisted surgery in prostate cancer

BACKGROUND

Robot-assisted radical prostatectomy (RARP) has emerged as a pivotal surgical intervention for the treatment of prostate cancer (PCa). However, the complexity of clinical cases, heterogeneity of PCa, and limitations in physician expertise pose challenges to rational decision-making in RARP. To address these challenges, the authors aimed to organize the knowledge of previously complex cohorts and establish an online platform named the RARP knowledge base (RARPKB) to provide reference evidence for personalized treatment plans.

MATERIALS AND METHODS

PubMed searches over the past two decades were conducted to identify publications describing RARP. The authors collected, classified, and structured surgical details, patient information, surgical data, and various statistical results from the literature. A knowledge-guided decision-support tool was established using MySQL, DataTable, ECharts, and JavaScript. ChatGPT-4 and two assessment scales were used to validate and compare the platform.

RESULTS

The platform comprised 583 studies, 1589 cohorts, 1 911 968 patients, and 11 986 records, resulting in 54 834 data entries. The knowledge-guided decision support tool provide personalized surgical plan recommendations and potential complications on the basis of patients' baseline and surgical information. Compared with ChatGPT-4, RARPKB outperformed in authenticity (100% vs. 73%), matching (100% vs. 53%), personalized recommendations (100% vs. 20%), matching of patients (100% vs. 0%), and personalized recommendations for complications (100% vs. 20%). Postuse, the average System Usability Scale score was 88.88±15.03, and the Net Promoter Score of RARPKB was 85. The knowledge base is available at: http://rarpkb.bioinf.org.cn .

CONCLUSIONS

The authors introduced the pioneering RARPKB, the first knowledge base for robot-assisted surgery, with an emphasis on PCa. RARPKB can assist in personalized and complex surgical planning for PCa to improve its efficacy. RARPKB provides a reference for the future applications of artificial intelligence in clinical practice.

Role of Robotic-Assisted Surgery in Public Health: Its Advantages and Challenges

The modern hospital setting is closely related to engineering and technology. In a hospital, modern equipment is abundant in every department, including the operating room, intensive care unit, and laboratories. Thus, the quality of treatment provided in hospitals and technology advancements are closely tied. Robotic systems are used to support and improve the accuracy and agility of human surgeons during medical procedures. This surgical approach is commonly referred to as robotic surgery or robotic-assisted surgery (RAS). These systems are not entirely autonomous; they are managed by skilled surgeons who carry out procedures with improved accuracy and minimized invasiveness using a console and specialized instruments. Because RAS offers increased surgical precision, less discomfort after surgery, shorter hospital stays, and faster recovery time, all of which improve patient outcomes and lessen the strain on healthcare resources, it plays a critical role in public health. Its minimally invasive technique benefits patients and the healthcare system by lowering problems, reducing the requirement for blood transfusions, and reducing the danger of infections related to medical care. Furthermore, the possibility of remote surgery via robotic systems can increase access to specialized care, reducing regional differences and advancing fairness in public health. In this review article, we will be covering how RAS has its role in public health.

An Automated Video Analysis System for Retrospective Assessment and Real-Time Monitoring of Endoscopic Procedures (with Video)

BACKGROUND AND AIMS

Accurate recognition of endoscopic instruments facilitates quantitative evaluation and quality control of endoscopic procedures. However, no relevant research has been reported. In this study, we aimed to develop a computer-assisted system, EndoAdd, for automated endoscopic surgical video analysis based on our dataset of endoscopic instrument images.

METHODS

Large training and validation datasets containing 45,143 images of 10 different endoscopic instruments and a test dataset of 18,375 images collected from several medical centers were used in this research. Annotated image frames were used to train the state-of-the-art object detection model, YOLO-v5, to identify the instruments. Based on the frame-level prediction results, we further developed a hidden Markov model to perform video analysis and generate heatmaps to summarize the videos.

RESULTS

EndoAdd achieved high accuracy (>97%) on the test dataset for all 10 endoscopic instrument types. The mean average accuracy, precision, recall, and F1-score were 99.1%, 92.0%, 88.8%, and 89.3%, respectively. The area under the curve values exceeded 0.94 for all instrument types. Heatmaps of endoscopic procedures were generated for both retrospective and real-time analyses.

CONCLUSIONS

We successfully developed an automated endoscopic video analysis system, EndoAdd, which supports retrospective assessment and real-time monitoring. It can be used for data analysis and quality control of endoscopic procedures in clinical practice.

What More Can We Ask of Robotics?

The future of robotics relies heavily on the ongoing synergy between robotic surgery and artificial intelligence. To unlock their full potential, we should address issues such as accessibility, education, data privacy, and ethics.

Expert consensus on Prospective Precision Diagnosis and Treatment Strategies for Osteoporotic Fractures

Osteoporotic fractures are the most severe complications of osteoporosis, characterized by poor bone quality, difficult realignment and fixation, slow fracture healing, and a high risk of recurrence. Clinically managing these fractures is relatively challenging, and in the context of rapid aging, they pose significant social hazards. The rapid advancement of disciplines such as biophysics and biochemistry brings new opportunities for future medical diagnosis and treatment. However, there has been limited attention to precision diagnosis and treatment strategies for osteoporotic fractures both domestically and internationally. In response to this, the Chinese Medical Association Orthopaedic Branch Youth Osteoporosis Group, Chinese Geriatrics Society Geriatric Orthopaedics Committee, Chinese Medical Doctor Association Orthopaedic Physicians Branch Youth Committee Osteoporosis Group, and Shanghai Association of Integrated Traditional Chinese and Western Medicine Osteoporosis Professional Committee have collaborated to develop this consensus. It aims to elucidate emerging technologies that may play a pivotal role in both diagnosis and treatment, advocating for clinicians to embrace interdisciplinary approaches and incorporate these new technologies into their practice. Ultimately, the goal is to improve the prognosis and quality of life for elderly patients with osteoporotic fractures.

Systematic reviews of machine learning in healthcare: a literature review

INTRODUCTION

The increasing availability of data and computing power has made machine learning (ML) a viable approach to faster, more efficient healthcare delivery.

METHODS

A systematic literature review (SLR) of published SLRs evaluating ML applications in healthcare settings published between1 January 2010 and 27 March 2023 was conducted.

RESULTS

In total 220 SLRs covering 10,462 ML algorithms were reviewed. The main application of AI in medicine related to the clinical prediction and disease prognosis in oncology and neurology with the use of imaging data. Accuracy, specificity, and sensitivity were provided in 56%, 28%, and 25% SLRs respectively. Internal and external validation was reported in 53% and less than 1% of the cases respectively. The most common modeling approach was neural networks (2,454 ML algorithms), followed by support vector machine and random forest/decision trees (1,578 and 1,522 ML algorithms, respectively).

EXPERT OPINION

The review indicated considerable reporting gaps in terms of the ML's performance, both internal and external validation. Greater accessibility to healthcare data for developers can ensure the faster adoption of ML algorithms into clinical practice.

[Status Quo of Surgical Navigation]

Surgical navigation, also referred to as computer-assisted or image-guided surgery, is a technique that employs a variety of methods - such as 3D imaging, tracking systems, specialised software, and robotics to support surgeons during surgical interventions. These emerging technologies aim not only to enhance the accuracy and precision of surgical procedures, but also to enable less invasive approaches, with the objective of reducing complications and improving operative outcomes for patients. By harnessing the integration of emerging digital technologies, surgical navigation holds the promise of assisting complex procedures across various medical disciplines. In recent years, the field of surgical navigation has witnessed significant advances. Abdominal surgical navigation, particularly endoscopy, laparoscopic, and robot-assisted surgery, is currently undergoing a phase of rapid evolution. Emphases include image-guided navigation, instrument tracking, and the potential integration of augmented and mixed reality (AR, MR). This article will comprehensively delve into the latest developments in surgical navigation, spanning state-of-the-art intraoperative technologies like hyperspectral and fluorescent imaging, to the integration of preoperative radiological imaging within the intraoperative setting.

Three-dimensional automatic artificial intelligence driven augmented-reality selective biopsy during nerve-sparing robot-assisted radical prostatectomy: A feasibility and accuracy study

Objective

To evaluate the accuracy of our new three-dimensional (3D) automatic augmented reality (AAR) system guided by artificial intelligence in the identification of tumour's location at the level of the preserved neurovascular bundle (NVB) at the end of the extirpative phase of nerve-sparing robot-assisted radical prostatectomy.

Methods

In this prospective study, we enrolled patients with prostate cancer (clinical stages cT1c-3, cN0, and cM0) with a positive index lesion at target biopsy, suspicious for capsular contact or extracapsular extension at preoperative multiparametric magnetic resonance imaging. Patients underwent robot-assisted radical prostatectomy at San Luigi Gonzaga Hospital (Orbassano, Turin, Italy), from December 2020 to December 2021. At the end of extirpative phase, thanks to our new AAR artificial intelligence driven system, the virtual prostate 3D model allowed to identify the tumour's location at the level of the preserved NVB and to perform a selective excisional biopsy, sparing the remaining portion of the bundle. Perioperative and postoperative data were evaluated, especially focusing on the positive surgical margin (PSM) rates, potency, continence recovery, and biochemical recurrence.

Results

Thirty-four patients were enrolled. In 15 (44.1%) cases, the target lesion was in contact with the prostatic capsule at multiparametric magnetic resonance imaging (Wheeler grade L2) while in 19 (55.9%) cases extracapsular extension was detected (Wheeler grade L3). 3D AAR guided biopsies were negative in all pathological tumour stage 2 (pT2) patients while they revealed the presence of cancer in 14 cases in the pT3 cohort (14/16; 87.5%). PSM rates were 0% and 7.1% in the pathological stages pT2 and pT3 (<3 mm, Gleason score 3), respectively.

Conclusion

With the proposed 3D AAR system, it is possible to correctly identify the lesion's location on the NVB in 87.5% of pT3 patients and perform a 3D-guided tailored nerve-sparing even in locally advanced diseases, without compromising the oncological safety in terms of PSM rates.

Artificial Intelligence in Surgical Training for Kidney Cancer: A Systematic Review of the Literature

The prevalence of renal cell carcinoma (RCC) is increasing due to advanced imaging techniques. Surgical resection is the standard treatment, involving complex radical and partial nephrectomy procedures that demand extensive training and planning. Furthermore, artificial intelligence (AI) can potentially aid the training process in the field of kidney cancer. This review explores how artificial intelligence (AI) can create a framework for kidney cancer surgery to address training difficulties. Following PRISMA 2020 criteria, an exhaustive search of PubMed and SCOPUS databases was conducted without any filters or restrictions. Inclusion criteria encompassed original English articles focusing on AI's role in kidney cancer surgical training. On the other hand, all non-original articles and articles published in any language other than English were excluded. Two independent reviewers assessed the articles, with a third party settling any disagreement. Study specifics, AI tools, methodologies, endpoints, and outcomes were extracted by the same authors. The Oxford Center for Evidence-Based Medicine's evidence levels were employed to assess the studies. Out of 468 identified records, 14 eligible studies were selected. Potential AI applications in kidney cancer surgical training include analyzing surgical workflow, annotating instruments, identifying tissues, and 3D reconstruction. AI is capable of appraising surgical skills, including the identification of procedural steps and instrument tracking. While AI and augmented reality (AR) enhance training, challenges persist in real-time tracking and registration. The utilization of AI-driven 3D reconstruction proves beneficial for intraoperative guidance and preoperative preparation. Artificial intelligence (AI) shows potential for advancing surgical training by providing unbiased evaluations, personalized feedback, and enhanced learning processes. Yet challenges such as consistent metric measurement, ethical concerns, and data privacy must be addressed. The integration of AI into kidney cancer surgical training offers solutions to training difficulties and a boost to surgical education. However, to fully harness its potential, additional studies are imperative.

Intelligence brings responsibility - Even smart AI assistants are held responsible

People will not hold cars responsible for traffic accidents, yet they do when artificial intelligence (AI) is involved. AI systems are held responsible when they act or merely advise a human agent. Does this mean that as soon as AI is involved responsibility follows? To find out, we examined whether purely instrumental AI systems stay clear of responsibility. We compared AI-powered with non-AI-powered car warning systems and measured their responsibility rating alongside their human users. Our findings show that responsibility is shared when the warning system is powered by AI but not by a purely mechanical system, even though people consider both systems as mere tools. Surprisingly, whether the warning prevents the accident introduces an outcome bias: the AI takes higher credit than blame depending on what the human manages or fails to do.

In the Hands of a Robot, From the Operating Room to the Courtroom: The Medicolegal Considerations of Robotic Surgery

Robotic surgery has rapidly evolved as a groundbreaking field in medicine, revolutionizing surgical practices across various specialties. Despite its numerous benefits, the adoption of robotic surgery faces significant medicolegal challenges. This article delves into the underexplored legal implications of robotic surgery and identifies three distinct medicolegal problems. First, the lack of standardized training and credentialing for robotic surgery poses potential risks to patient safety and surgeon competence. Second, informed consent processes require additional considerations to ensure patients are fully aware of the technology's capabilities and potential risks. Finally, the issue of legal liability becomes complex due to the involvement of multiple stakeholders in the functioning of robotic systems. The article highlights the need for comprehensive guidelines, regulations, and training programs to navigate the medicolegal aspects of robotic surgery effectively, thereby unlocking its full potential for the future..

Robotic Surgery: A Comprehensive Review of the Literature and Current Trends

Robotic surgery (RS) is an evolution of minimally invasive surgery that combines medical science, robotics, and engineering. The first robots approved by the Food and Drug Administration (FDA) were the Da Vinci Surgical System and the ZEUS Robotic Surgical System, which have been improving over time. Through the decades, the equipment applied to RS had undergone a wide transformation as a response to the development of new techniques and facilities for the assembly and implementation of the own. RS has revolutionized the field of urology, enabling surgeons to perform complex procedures with greater precision and accuracy, and many other surgical specialties such as gynecology, general surgery, otolaryngology, cardiothoracic surgery, and neurosurgery. Several benefits, such as a better approach to the surgical site, a three-dimensional image that improves depth perception, and smaller scars, enhance range of motion, allowing the surgeon to conduct more complicated surgical operations, and reduced postoperative complications have made robotic-assisted surgery an increasingly popular approach. However, some points like the cost of surgical procedures, equipment-instrument, and maintenance are important aspects to consider. Machine learning will likely have a role to play in surgical training shortly through "automated performance metrics," where algorithms observe and "learn" individual surgeons' techniques, assess performance, and anticipate surgical outcomes with the potential to individualize surgical training and aid decision-making in real time.

Evolution of a surgical system using deep learning in minimally invasive surgery (Review)

Recently, artificial intelligence (AI) has been applied in various fields due to the development of new learning methods, such as deep learning, and the marked progress in computational processing speed. AI is also being applied in the medical field for medical image recognition and omics analysis of genomes and other data. Recently, AI applications for videos of minimally invasive surgeries have also advanced, and studies on such applications are increasing. In the present review, studies that focused on the following topics were selected: i) Organ and anatomy identification, ii) instrument identification, iii) procedure and surgical phase recognition, iv) surgery-time prediction, v) identification of an appropriate incision line, and vi) surgical education. The development of autonomous surgical robots is also progressing, with the Smart Tissue Autonomous Robot (STAR) and RAVEN systems being the most reported developments. STAR, in particular, is currently being used in laparoscopic imaging to recognize the surgical site from laparoscopic images and is in the process of establishing an automated suturing system, albeit in animal experiments. The present review examined the possibility of fully autonomous surgical robots in the future.

Acquisition and usage of robotic surgical data for machine learning analysis

BACKGROUND

The increasing use of robot-assisted surgery (RAS) has led to the need for new methods of assessing whether new surgeons are qualified to perform RAS, without the resource-demanding process of having expert surgeons do the assessment. Computer-based automation and artificial intelligence (AI) are seen as promising alternatives to expert-based surgical assessment. However, no standard protocols or methods for preparing data and implementing AI are available for clinicians. This may be among the reasons for the impediment to the use of AI in the clinical setting.

METHOD

We tested our method on porcine models with both the da Vinci Si and the da Vinci Xi. We sought to capture raw video data from the surgical robots and 3D movement data from the surgeons and prepared the data for the use in AI by a structured guide to acquire and prepare video data using the following steps: 'Capturing image data from the surgical robot', 'Extracting event data', 'Capturing movement data of the surgeon', 'Annotation of image data'.

RESULTS

15 participant (11 novices and 4 experienced) performed 10 different intraabdominal RAS procedures. Using this method we captured 188 videos (94 from the surgical robot, and 94 corresponding movement videos of the surgeons' arms and hands). Event data, movement data, and labels were extracted from the raw material and prepared for use in AI.

CONCLUSION

With our described methods, we could collect, prepare, and annotate images, events, and motion data from surgical robotic systems in preparation for its use in AI.

A systematic review of annotation for surgical process model analysis in minimally invasive surgery based on video

BACKGROUND

Annotated data are foundational to applications of supervised machine learning. However, there seems to be a lack of common language used in the field of surgical data science. The aim of this study is to review the process of annotation and semantics used in the creation of SPM for minimally invasive surgery videos.

METHODS

For this systematic review, we reviewed articles indexed in the MEDLINE database from January 2000 until March 2022. We selected articles using surgical video annotations to describe a surgical process model in the field of minimally invasive surgery. We excluded studies focusing on instrument detection or recognition of anatomical areas only. The risk of bias was evaluated with the Newcastle Ottawa Quality assessment tool. Data from the studies were visually presented in table using the SPIDER tool.

RESULTS

Of the 2806 articles identified, 34 were selected for review. Twenty-two were in the field of digestive surgery, six in ophthalmologic surgery only, one in neurosurgery, three in gynecologic surgery, and two in mixed fields. Thirty-one studies (88.2%) were dedicated to phase, step, or action recognition and mainly relied on a very simple formalization (29, 85.2%). Clinical information in the datasets was lacking for studies using available public datasets. The process of annotation for surgical process model was lacking and poorly described, and description of the surgical procedures was highly variable between studies.

CONCLUSION

Surgical video annotation lacks a rigorous and reproducible framework. This leads to difficulties in sharing videos between institutions and hospitals because of the different languages used. There is a need to develop and use common ontology to improve libraries of annotated surgical videos.

Artificial Intelligence Splint in Orthognathic Surgery for Skeletal Class III Malocclusion: Design and Application

BACKGROUND

Digital splints are indispensable in orthognathic surgery. However, the present design process of splints is time-consuming and has low reproducibility. To solve these problems, an algorithm for artificial intelligent splints has been developed in this study, making the automatic design of splints accessible.

METHODS

Firstly, the algorithm and program of the artificial intelligence splint were created. Then a total of 54 patients with skeletal class III malocclusion were included in this study from 2018 to 2020. Pre and postoperative radiographic examinations were performed. The cephalometric measurements were recorded and the difference between virtual simulation and postoperative images was measured. The time cost and differences between artificial intelligent splints and digital splints were analyzed through both model surgery and radiographic images.

RESULTS

The results showed that the efficiency of designing splints is significantly improved. And the mean difference between artificial intelligent splints and digital splints was <0.15 mm in model surgery. Meanwhile, there was no significant difference between the artificial intelligent splints and digital splints in radiological image analysis.

CONCLUSIONS

In conclusion, compared with digital splints, artificial intelligent splints could save time for preoperative design while ensuring accuracy. The authors believed that it is conducive to the presurgical design of orthognathic surgery.

Artificial Intelligence in Surgical Learning

(1) Background: Artificial Intelligence (AI) is transforming healthcare on all levels. While AI shows immense potential, the clinical implementation is lagging. We present a concise review of AI in surgical learning; (2) Methods: A non-systematic review of AI in surgical learning of the literature in English is provided; (3) Results: AI shows utility for all components of surgical competence within surgical learning. AI presents with great potential within robotic surgery specifically (4) Conclusions: Technology will evolve in ways currently unimaginable, presenting us with novel applications of AI and derivatives thereof. Surgeons must be open to new modes of learning to be able to implement all evidence-based applications of AI in the future. Systematic analyses of AI in surgical learning are needed.

A bibliometric and visual analysis of publications on artificial intelligence in colorectal cancer (2002-2022)

Background

Colorectal cancer (CRC) has the third-highest incidence and second-highest mortality rate of all cancers worldwide. Early diagnosis and screening of CRC have been the focus of research in this field. With the continuous development of artificial intelligence (AI) technology, AI has advantages in many aspects of CRC, such as adenoma screening, genetic testing, and prediction of tumor metastasis.

Objective

This study uses bibliometrics to analyze research in AI in CRC, summarize the field's history and current status of research, and predict future research directions.

Method

We searched the SCIE database for all literature on CRC and AI. The documents span the period 2002-2022. we used bibliometrics to analyze the data of these papers, such as authors, countries, institutions, and references. Co-authorship, co-citation, and co-occurrence analysis were the main methods of analysis. Citespace, VOSviewer, and SCImago Graphica were used to visualize the results.

Result

This study selected 1,531 articles on AI in CRC. China has published a maximum number of 580 such articles in this field. The U.S. had the most quality publications, boasting an average citation per article of 46.13. Mori Y and Ding K were the two authors with the highest number of articles. Scientific Reports, Cancers, and Frontiers in Oncology are this field's most widely published journals. Institutions from China occupy the top 9 positions among the most published institutions. We found that research on AI in this field mainly focuses on colonoscopy-assisted diagnosis, imaging histology, and pathology examination.

Conclusion

AI in CRC is currently in the development stage with good prospects. AI is currently widely used in colonoscopy, imageomics, and pathology. However, the scope of AI applications is still limited, and there is a lack of inter-institutional collaboration. The pervasiveness of AI technology is the main direction of future housing development in this field.

Surgical Gesture Recognition in Laparoscopic Tasks Based on the Transformer Network and Self-Supervised Learning

In this study, we propose a deep learning framework and a self-supervision scheme for video-based surgical gesture recognition. The proposed framework is modular. First, a 3D convolutional network extracts feature vectors from video clips for encoding spatial and short-term temporal features. Second, the feature vectors are fed into a transformer network for capturing long-term temporal dependencies. Two main models are proposed, based on the backbone framework: C3DTrans (supervised) and SSC3DTrans (self-supervised). The dataset consisted of 80 videos from two basic laparoscopic tasks: peg transfer (PT) and knot tying (KT). To examine the potential of self-supervision, the models were trained on 60% and 100% of the annotated dataset. In addition, the best-performing model was evaluated on the JIGSAWS robotic surgery dataset. The best model (C3DTrans) achieves an accuracy of 88.0%, a 95.2% clip level, and 97.5% and 97.9% (gesture level), for PT and KT, respectively. The SSC3DTrans performed similar to C3DTrans when training on 60% of the annotated dataset (about 84% and 93% clip-level accuracies for PT and KT, respectively). The performance of C3DTrans on JIGSAWS was close to 76% accuracy, which was similar to or higher than prior techniques based on a single video stream, no additional video training, and online processing.

Breaking down the silos of artificial intelligence in surgery: glossary of terms

BACKGROUND

The literature on artificial intelligence (AI) in surgery has advanced rapidly during the past few years. However, the published studies on AI are mostly reported by computer scientists using their own jargon which is unfamiliar to surgeons.

METHODS

A literature search was conducted in using PubMed following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement. The primary outcome of this review is to provide a glossary with definitions of the commonly used AI terms in surgery to improve their understanding by surgeons.

RESULTS

One hundred ninety-five studies were included in this review, and 38 AI terms related to surgery were retrieved. Convolutional neural networks were the most frequently culled term by the search, accounting for 74 studies on AI in surgery, followed by classification task (n = 62), artificial neural networks (n = 53), and regression (n = 49). Then, the most frequent expressions were supervised learning (reported in 24 articles), support vector machine (SVM) in 21, and logistic regression in 16. The rest of the 38 terms was seldom mentioned.

CONCLUSIONS

The proposed glossary can be used by several stakeholders. First and foremost, by residents and attending consultant surgeons, both having to understand the fundamentals of AI when reading such articles. Secondly, junior researchers at the start of their career in Surgical Data Science and thirdly experts working in the regulatory sections of companies involved in the AI Business Software as a Medical Device (SaMD) preparing documents for submission to the Food and Drug Administration (FDA) or other agencies for approval.

Ensemble deep learning for the prediction of proficiency at a virtual simulator for robot-assisted surgery

BACKGROUND

Artificial intelligence (AI) has the potential to enhance patient safety in surgery, and all its aspects, including education and training, will derive considerable benefit from AI. In the present study, deep-learning models were used to predict the rates of proficiency acquisition in robot-assisted surgery (RAS), thereby providing surgical programs directors information on the levels of the innate ability of trainees to facilitate the implementation of flexible personalized training.

METHODS

176 medical students, without prior experience with surgical simulators, were trained to reach proficiency in five tasks on a virtual simulator for RAS. Ensemble deep neural networks (DNN) models were developed and compared with other ensemble AI algorithms, i.e., random forests and gradient boosted regression trees (GBRT).

RESULTS

DNN models achieved a higher accuracy than random forests and GBRT in predicting time to proficiency, 0.84 vs. 0.70 and 0.77, respectively (Peg board 2), 0.83 vs. 0.79 and 0.78 (Ring walk 2), 0.81 vs 0.81 and 0.80 (Match board 1), 0.79 vs. 0.75 and 0.71 (Ring and rail 2), and 0.87 vs. 0.86 and 0.84 (Thread the rings 2). Ensemble DNN models outperformed random forests and GBRT in predicting number of attempts to proficiency, with an accuracy of 0.87 vs. 0.86 and 0.83, respectively (Peg board 2), 0.89 vs. 0.88 and 0.89 (Ring walk 2), 0.91 vs. 0.89 and 0.89 (Match board 1), 0.89 vs. 0.87 and 0.83 (Ring and rail 2), and 0.96 vs. 0.94 and 0.94 (Thread the rings 2).

CONCLUSIONS

Ensemble DNN models can identify at an early stage the acquisition rates of surgical technical proficiency of trainees and identify those struggling to reach the required expected proficiency level.

Artificial intelligence using advanced imaging techniques and cholangiocarcinoma: Recent advances and future direction

While cholangiocarcinoma represents only about 3% of all gastrointestinal tumors, it has a dismal survival rate, usually because it is diagnosed at a late stage. The utilization of Artificial Intelligence (AI) in medicine in general, and in gastroenterology has made gigantic steps. However, the application of AI for biliary disease, in particular for cholangiocarcinoma, has been sub-optimal. The use of AI in combination with clinical data, cross-sectional imaging (computed tomography, magnetic resonance imaging) and endoscopy (endoscopic ultrasound and cholangioscopy) has the potential to significantly improve early diagnosis and the choice of optimal therapeutic options, leading to a transformation in the prognosis of this feared disease. In this review we summarize the current knowledge on the use of AI for the diagnosis and management of cholangiocarcinoma and point to future directions in the field.

Three-dimensional imaging reconstruction of the kidney's anatomy for a tailored minimal invasive partial nephrectomy: A pilot study

Objective

Methods

Results

Conclusion

New imaging technologies for robotic kidney cancer surgery

Objective

Kidney cancers account for approximately 2% of all newly diagnosed cancer in 2020. Among the primary treatment options for kidney cancer, urologist may choose between radical or partial nephrectomy, or ablative therapies. Nowadays, robotic-assisted partial nephrectomy (RAPN) for the management of renal cancers has gained popularity, up to being considered the gold standard. However, RAPN is a challenging procedure with a steep learning curve.

Methods

In this narrative review, different imaging technologies used to guide and aid RAPN are discussed.

Results

Three-dimensional visualization technology has been extensively discussed in RAPN, showing its value in enhancing robotic-surgery training, patient counseling, surgical planning, and intraoperative guidance. Intraoperative imaging technologies such as intracorporeal ultrasound, near-infrared fluorescent imaging, and intraoperative pathological examination can also be used to improve the outcomes following RAPN. Finally, artificial intelligence may play a role in the field of RAPN soon.

Conclusion

RAPN is a complex surgery; however, many imaging technologies may play an important role in facilitating it.