Towards automatic verification of the critical view of the myopectineal orifice with artificial intelligence

Anatomical recognition of dissection layers, nerves, vas deferens, and microvessels using artificial intelligence during transabdominal preperitoneal inguinal hernia repair

PURPOSE

In laparoscopic inguinal hernia surgery, proper recognition of loose connective tissue, nerves, vas deferens, and microvessels is important to prevent postoperative complications, such as recurrence, pain, sexual dysfunction, and bleeding. EUREKA (Anaut Inc., Tokyo, Japan) is a system that uses artificial intelligence (AI) for anatomical recognition. This system can intraoperatively confirm the aforementioned anatomical landmarks. In this study, we validated the accuracy of EUREKA in recognizing dissection layers, nerves, vas deferens, and microvessels during transabdominal preperitoneal inguinal hernia repair (TAPP).

METHODS

We used TAPP videos to compare EUREKA's recognition of loose connective tissue, nerves, vas deferens, and microvessels with the original surgical video and examined whether EUREKA accurately identified these structures. Intersection over Union (IoU) and F1/Dice scores were calculated to quantitively evaluate AI predictive images.

RESULTS

The mean IoU and F1/Dice scores were 0.33 and 0.50 for connective tissue, 0.24 and 0.38 for nerves, 0.50 and 0.66 for the vas deferens, and 0.30 and 0.45 for microvessels, respectively. Compared with the images without EUREKA visualization, dissection layers were very clearly recognized and displayed when appropriate tension was applied.

Usefulness of an Artificial Intelligence Model in Recognizing Recurrent Laryngeal Nerves During Robot-Assisted Minimally Invasive Esophagectomy

BACKGROUND

Recurrent laryngeal nerve (RLN) palsy is a common complication in esophagectomy and its main risk factor is reportedly intraoperative procedure associated with surgeons' experience. We aimed to improve surgeons' recognition of the RLN during robot-assisted minimally invasive esophagectomy (RAMIE) by developing an artificial intelligence (AI) model.

METHODS

We used 120 RAMIE videos from four institutions to develop an AI model and eight other surgical videos from another institution for AI model evaluation. AI performance was measured using the Intersection over Union (IoU). Furthermore, to verify the AI's clinical validity, we conducted the two experiments on the early identification of RLN and recognition of its location by eight trainee surgeons with or without AI.

RESULTS

The IoUs for AI recognition of the right and left RLNs were 0.40 ± 0.26 and 0.34 ± 0.27, respectively. The recognition of the right RLN presence in the beginning of right RLN lymph node dissection (LND) by surgeons with AI (81.3%) was significantly more accurate (p = 0.004) than that by surgeons without AI (46.9%). The IoU of right RLN during right RLN LND recognized by surgeons with AI (0.59 ± 0.18) was significantly higher (p = 0.010) than that by surgeons without AI (0.40 ± 0.29).

CONCLUSIONS

Surgeons' recognition of anatomical structures in RAMIE was improved by our AI system with high accuracy. Especially in right RLN LND, surgeons could recognize the RLN more quickly and accurately by using the AI model.

Evaluating surgical expertise with AI-based automated instrument recognition for robotic distal gastrectomy

Introduction

Complexities of robotic distal gastrectomy (RDG) give reason to assess physician's surgical skill. Varying levels in surgical skill affect patient outcomes. We aim to investigate how a novel artificial intelligence (AI) model can be used to evaluate surgical skill in RDG by recognizing surgical instruments.

Methods

Fifty-five consecutive robotic surgical videos of RDG for gastric cancer were analyzed. We used Deeplab, a multi-stage temporal convolutional network, and it trained on 1234 manually annotated images. The model was then tested on 149 annotated images for accuracy. Deep learning metrics such as Intersection over Union (IoU) and accuracy were assessed, and the comparison between experienced and non-experienced surgeons based on usage of instruments during infrapyloric lymph node dissection was performed.

Results

We annotated 540 Cadiere forceps, 898 Fenestrated bipolars, 359 Suction tubes, 307 Maryland bipolars, 688 Harmonic scalpels, 400 Staplers, and 59 Large clips. The average IoU and accuracy were 0.82 ± 0.12 and 87.2 ± 11.9% respectively. Moreover, the percentage of each instrument's usage to overall infrapyloric lymphadenectomy duration predicted by AI were compared. The use of Stapler and Large clip were significantly shorter in the experienced group compared to the non-experienced group.

Conclusions

This study is the first to report that surgical skill can be successfully and accurately determined by an AI model for RDG. Our AI gives us a way to recognize and automatically generate instance segmentation of the surgical instruments present in this procedure. Use of this technology allows unbiased, more accessible RDG surgical skill.

Real-time detection of active bleeding in laparoscopic colectomy using artificial intelligence

BACKGROUND

Most intraoperative adverse events (iAEs) result from surgeons' errors, and bleeding is the majority of iAEs. Recognizing active bleeding timely is important to ensure safe surgery, and artificial intelligence (AI) has great potential for detecting active bleeding and providing real-time surgical support. This study aimed to develop a real-time AI model to detect active intraoperative bleeding.

METHODS

We extracted 27 surgical videos from a nationwide multi-institutional surgical video database in Japan and divided them at the patient level into three sets: training (n = 21), validation (n = 3), and testing (n = 3). We subsequently extracted the bleeding scenes and labeled distinctively active bleeding and blood pooling frame by frame. We used pre-trained YOLOv7_6w and developed a model to learn both active bleeding and blood pooling. The Average Precision at an Intersection over Union threshold of 0.5 (AP.50) for active bleeding and frames per second (FPS) were quantified. In addition, we conducted two 5-point Likert scales (5 = Excellent, 4 = Good, 3 = Fair, 2 = Poor, and 1 = Fail) questionnaires about sensitivity (the sensitivity score) and number of overdetection areas (the overdetection score) to investigate the surgeons' assessment.

RESULTS

We annotated 34,117 images of 254 bleeding events. The AP.50 for active bleeding in the developed model was 0.574 and the FPS was 48.5. Twenty surgeons answered two questionnaires, indicating a sensitivity score of 4.92 and an overdetection score of 4.62 for the model.

CONCLUSIONS

We developed an AI model to detect active bleeding, achieving real-time processing speed. Our AI model can be used to provide real-time surgical support.

Automated artificial intelligence-based phase-recognition system for esophageal endoscopic submucosal dissection (with video)

BACKGROUND AND AIMS

Endoscopic submucosal dissection (ESD) for superficial esophageal cancer is a multistep treatment involving several endoscopic processes. Although analyzing each phase separately is worthwhile, it is not realistic in practice owing to the need for considerable manpower. To solve this problem, we aimed to establish a state-of-the-art artificial intelligence (AI)-based system, specifically, an automated phase-recognition system that can automatically identify each endoscopic phase based on video images.

METHODS

Ninety-four videos of ESD procedures for superficial esophageal cancer were evaluated in this single-center study. A deep neural network-based phase-recognition system was developed in an automated manner to recognize each of the endoscopic phases. The system was trained with the use of videos that were annotated and verified by 2 GI endoscopists.

RESULTS

The overall accuracy of the AI model for automated phase recognition was 90%, and the average precision, recall, and F value rates were 91%, 90%, and 90%, respectively. Two representative ESD videos predicted by the model indicated the usability of AI in clinical practice.

CONCLUSIONS

We demonstrated that an AI-based automated phase-recognition system for esophageal ESD can be established with high accuracy. To the best of our knowledge, this is the first report on automated recognition of ESD treatment phases. Because this system enabled a detailed analysis of phases, collecting large volumes of data in the future may help to identify quality indicators for treatment techniques and uncover unmet medical needs that necessitate the creation of new treatment methods and devices.

A novel high accuracy model for automatic surgical workflow recognition using artificial intelligence in laparoscopic totally extraperitoneal inguinal hernia repair (TEP)

INTRODUCTION

Artificial intelligence and computer vision are revolutionizing the way we perceive video analysis in minimally invasive surgery. This emerging technology has increasingly been leveraged successfully for video segmentation, documentation, education, and formative assessment. New, sophisticated platforms allow pre-determined segments chosen by surgeons to be automatically presented without the need to review entire videos. This study aimed to validate and demonstrate the accuracy of the first reported AI-based computer vision algorithm that automatically recognizes surgical steps in videos of totally extraperitoneal (TEP) inguinal hernia repair.

METHODS

Videos of TEP procedures were manually labeled by a team of annotators trained to identify and label surgical workflow according to six major steps. For bilateral hernias, an additional change of focus step was also included. The videos were then used to train a computer vision AI algorithm. Performance accuracy was assessed in comparison to the manual annotations.

RESULTS

A total of 619 full-length TEP videos were analyzed: 371 were used to train the model, 93 for internal validation, and the remaining 155 as a test set to evaluate algorithm accuracy. The overall accuracy for the complete procedure was 88.8%. Per-step accuracy reached the highest value for the hernia sac reduction step (94.3%) and the lowest for the preperitoneal dissection step (72.2%).

CONCLUSIONS

These results indicate that the novel AI model was able to provide fully automated video analysis with a high accuracy level. High-accuracy models leveraging AI to enable automation of surgical video analysis allow us to identify and monitor surgical performance, providing mathematical metrics that can be stored, evaluated, and compared. As such, the proposed model is capable of enabling data-driven insights to improve surgical quality and demonstrate best practices in TEP procedures.