Phase Segmentation Methods for an Automatic Surgical Workflow Analysis

A novel high accuracy model for automatic surgical workflow recognition using artificial intelligence in laparoscopic totally extraperitoneal inguinal hernia repair (TEP)

INTRODUCTION

Artificial intelligence and computer vision are revolutionizing the way we perceive video analysis in minimally invasive surgery. This emerging technology has increasingly been leveraged successfully for video segmentation, documentation, education, and formative assessment. New, sophisticated platforms allow pre-determined segments chosen by surgeons to be automatically presented without the need to review entire videos. This study aimed to validate and demonstrate the accuracy of the first reported AI-based computer vision algorithm that automatically recognizes surgical steps in videos of totally extraperitoneal (TEP) inguinal hernia repair.

METHODS

Videos of TEP procedures were manually labeled by a team of annotators trained to identify and label surgical workflow according to six major steps. For bilateral hernias, an additional change of focus step was also included. The videos were then used to train a computer vision AI algorithm. Performance accuracy was assessed in comparison to the manual annotations.

RESULTS

A total of 619 full-length TEP videos were analyzed: 371 were used to train the model, 93 for internal validation, and the remaining 155 as a test set to evaluate algorithm accuracy. The overall accuracy for the complete procedure was 88.8%. Per-step accuracy reached the highest value for the hernia sac reduction step (94.3%) and the lowest for the preperitoneal dissection step (72.2%).

CONCLUSIONS

These results indicate that the novel AI model was able to provide fully automated video analysis with a high accuracy level. High-accuracy models leveraging AI to enable automation of surgical video analysis allow us to identify and monitor surgical performance, providing mathematical metrics that can be stored, evaluated, and compared. As such, the proposed model is capable of enabling data-driven insights to improve surgical quality and demonstrate best practices in TEP procedures.

Value of applying interactive micro-courses on occupational protection in operating room for nursing trainees

This study aimed to investigate the application value of an interactive micro class on occupational protection in the operating room for nursing interns. Using a cluster sampling approach, 200 junior college nursing interns were selected as participants at our hospital, who were practicing from June 2020 to April 2021. They were randomly assigned to either the observation group or the control group, with 100 participants in each. Data on teaching-related evaluation indicators, including clarity of teaching objectives, learning atmosphere, rational use of learning resources, effectiveness of teaching process regulation, and degree of students' participation in activities, were collected for both groups. Additionally, the scores of occupational protection assessment in the operating room, which encompassed physical, chemical, biological, environmental, and physiological and psychological factors, were also recorded. The comparative analysis of teaching-related evaluation indicators between the 2 groups revealed statistically significant differences. Specifically, significant differences were found between the 2 groups in the clarity of teaching objectives (P = .007), learning atmosphere (P < .001), rational use of learning resources (P < .001), effectiveness of teaching process regulation (P = .01), and degree of students' participation in activities (P < .001). Furthermore, the scores of all items in the observation group were higher than those in the control group. Additionally, the comparison results of the scores of the occupational protection assessment in the operating room between the 2 groups showed that while there was no statistically significant difference between the 2 groups before the intervention (P > .05). However, after the intervention, statistically significant differences were found between the 2 groups in physical (P < .001), chemical (P = .001), biological (P < .001), environmental (P < .001), and physiological and psychological factors (P < .001). Additionally, the scores of all items in the observation group were higher than those in the control group. The implementation of the interactive micro class enhanced the quality of teaching related to occupational protection in the operating room for interning nurses, thus verifying its benefits in clinical teaching practices.

Evolution of the digital operating room: the place of video technology in surgery

PURPOSE

The aim of this review was to collate current evidence wherein digitalisation, through the incorporation of video technology and artificial intelligence (AI), is being applied to the practice of surgery. Applications are vast, and the literature investigating the utility of surgical video and its synergy with AI has steadily increased over the last 2 decades. This type of technology is widespread in other industries, such as autonomy in transportation and manufacturing.

METHODS

Articles were identified primarily using the PubMed and MEDLINE databases. The MeSH terms used were "surgical education", "surgical video", "video labelling", "surgery", "surgical workflow", "telementoring", "telemedicine", "machine learning", "deep learning" and "operating room". Given the breadth of the subject and the scarcity of high-level data in certain areas, a narrative synthesis was selected over a meta-analysis or systematic review to allow for a focussed discussion of the topic.

RESULTS

Three main themes were identified and analysed throughout this review, (1) the multifaceted utility of surgical video recording, (2) teleconferencing/telemedicine and (3) artificial intelligence in the operating room.

CONCLUSIONS

Evidence suggests the routine collection of intraoperative data will be beneficial in the advancement of surgery, by driving standardised, evidence-based surgical care and personalised training of future surgeons. However, many barriers stand in the way of widespread implementation, necessitating close collaboration between surgeons, data scientists, medicolegal personnel and hospital policy makers.

Tracking Clinical Staff Behaviors in an Operating Room

Inadequate staff behaviors in an operating room (OR) may lead to environmental contamination and increase the risk of surgical site infection. In order to assess this statement objectively, we have developed an approach to analyze OR staff behaviors using a motion tracking system. The present article introduces a solution for the assessment of individual displacements in the OR by: (1) detecting human presence and quantifying movements using a motion capture (MOCAP) system and (2) observing doors’ movements by means of a wireless network of inertial sensors fixed on the doors and synchronized with the MOCAP system. The system was used in eight health care facilities sites during 30 cardiac and orthopedic surgery interventions. A total of 119 h of data were recorded and analyzed. Three hundred thirty four individual displacements were reconstructed. On average, only 10.6% individual positions could not be reconstructed and were considered undetermined, i.e., the presence in the room of the corresponding staff member could not be determined. The article presents the hardware and software developed together with the obtained reconstruction performances.

Monitoring tool usage in surgery videos using boosted convolutional and recurrent neural networks

This paper investigates the automatic monitoring of tool usage during a surgery, with potential applications in report generation, surgical training and real-time decision support. Two surgeries are considered: cataract surgery, the most common surgical procedure, and cholecystectomy, one of the most common digestive surgeries. Tool usage is monitored in videos recorded either through a microscope (cataract surgery) or an endoscope (cholecystectomy). Following state-of-the-art video analysis solutions, each frame of the video is analyzed by convolutional neural networks (CNNs) whose outputs are fed to recurrent neural networks (RNNs) in order to take temporal relationships between events into account. Novelty lies in the way those CNNs and RNNs are trained. Computational complexity prevents the end-to-end training of "CNN+RNN" systems. Therefore, CNNs are usually trained first, independently from the RNNs. This approach is clearly suboptimal for surgical tool analysis: many tools are very similar to one another, but they can generally be differentiated based on past events. CNNs should be trained to extract the most useful visual features in combination with the temporal context. A novel boosting strategy is proposed to achieve this goal: the CNN and RNN parts of the system are simultaneously enriched by progressively adding weak classifiers (either CNNs or RNNs) trained to improve the overall classification accuracy. Experiments were performed in a dataset of 50 cataract surgery videos, where the usage of 21 surgical tools was manually annotated, and a dataset of 80 cholecystectomy videos, where the usage of 7 tools was manually annotated. Very good classification performance are achieved in both datasets: tool usage could be labeled with an average area under the ROC curve of A_z=0.9961 and A_z=0.9939, respectively, in offline mode (using past, present and future information), and A_z=0.9957 and A_z=0.9936, respectively, in online mode (using past and present information only).