Performance and Capability Assessment in Surgical Subtask Automation

Assessment of Surgeons' Stress Levels with Digital Sensors during Robot-Assisted Surgery: An Experimental Study

Robot-Assisted Minimally Invasive Surgery (RAMIS) marks a paradigm shift in surgical procedures, enhancing precision and ergonomics. Concurrently it introduces complex stress dynamics and ergonomic challenges regarding the human-robot interface and interaction. This study explores the stress-related aspects of RAMIS, using the da Vinci XI Surgical System and the Sea Spikes model as a standard skill training phantom to establish a link between technological advancement and human factors in RAMIS environments. By employing different physiological and kinematic sensors for heart rate variability, hand movement tracking, and posture analysis, this research aims to develop a framework for quantifying the stress and ergonomic loads applied to surgeons. Preliminary findings reveal significant correlations between stress levels and several of the skill-related metrics measured by external sensors or the SURG-TLX questionnaire. Furthermore, early analysis of this preliminary dataset suggests the potential benefits of applying machine learning for surgeon skill classification and stress analysis. This paper presents the initial findings, identified correlations, and the lessons learned from the clinical setup, aiming to lay down the cornerstones for wider studies in the fields of clinical situation awareness and attention computing.

Levels of autonomy in FDA-cleared surgical robots: a systematic review

The integration of robotics in surgery has increased over the past decade, and advances in the autonomous capabilities of surgical robots have paralleled that of assistive and industrial robots. However, classification and regulatory frameworks have not kept pace with the increasing autonomy of surgical robots. There is a need to modernize our classification to understand technological trends and prepare to regulate and streamline surgical practice around these robotic systems. We present a systematic review of all surgical robots cleared by the United States Food and Drug Administration (FDA) from 2015 to 2023, utilizing a classification system that we call Levels of Autonomy in Surgical Robotics (LASR) to categorize each robot's decision-making and action-taking abilities from Level 1 (Robot Assistance) to Level 5 (Full Autonomy). We searched the 510(k), De Novo, and AccessGUDID databases in December 2023 and included all medical devices fitting our definition of a surgical robot. 37,981 records were screened to identify 49 surgical robots. Most surgical robots were at Level 1 (86%) and some reached Level 3 (Conditional Autonomy) (6%). 2 surgical robots were recognized by the FDA to have machine learning-enabled capabilities, while more were reported to have these capabilities in their marketing materials. Most surgical robots were introduced via the 510(k) pathway, but a growing number via the De Novo pathway. This review highlights trends toward greater autonomy in surgical robotics. Implementing regulatory frameworks that acknowledge varying levels of autonomy in surgical robots may help ensure their safe and effective integration into surgical practice.

Sensor-Based Measurement Method to Support the Assessment of Robot-Assisted Radiofrequency Ablation

Digital surgery technologies, such as interventional robotics and sensor systems, not only improve patient care but also aid in the development and optimization of traditional invasive treatments and methods. Atrial Fibrillation (AF) is the most common cardiac arrhythmia with critical clinical relevance today. Delayed intervention can lead to heart failure, stroke, or sudden cardiac death. Although many advances have been made in the field of radiofrequency (RF) catheter ablation (CA), it can be further developed by incorporating sensor technology to improve its efficacy and safety. Automation can be utilized to shorten the duration of RF ablation, provided that the interactions between the tissue and the RF tools are well understood and adequately modeled. Further research is needed to develop the optimal catheter design. This paper describes the systematic methodology developed to support robot-assisted RF CA characterization measurements. The article describes the custom instruments developed for the experiments, particularly the contact force limiter, the measurement procedure, and the evaluation of the results, as enablers for new results. The aim was to establish an objective, repeatable, robust measurement method and adjacent procedure.

A Novel Vascular Anastomotic Coupling Device for End-to-End Anastomosis of Arteries and Veins

OBJECTIVE

Hand-sutured (HS) techniques remain the gold standard for most microvascular anastomoses in microsurgery. HS techniques can result in endothelial lacerations and back wall suturing, leading to complications such as thrombosis and free tissue loss. A novel force-interference-fit vascular coupling device (FIF-VCD) system can potentially reduce the need for HS and improve end-to-end anastomosis. This study aims to describe the development and testing of a novel FIF-VCD system for 1.5 to 4.0 mm outside diameter arteries and veins.

METHODS

Benchtop anastomoses were performed using porcine cadaver arteries and veins. Decoupling force and anastomotic leakage were tested under simulated worst-case intravital physiological conditions. The 1.5 mm FIF-VCD system was used to perform cadaver rat abdominal aorta anastomoses.

RESULTS

Benchtop testing showed that the vessels coupled with the FIF-VCD system could withstand simulated worst-case intravital physiological conditions with a 95% confidence interval for the average decoupling force safety factor of 8.2 ± 1.0 (5.2 ± 1.0 N) and a 95% confidence interval for the average leakage rate safety factor of 26 ± 3.6 (8.4 ± 0.14 and 95 ± 1.4 μL/s at 150 and 360 mmHg, respectively) when compared to HS anastomotic leakage rates (310 ± 14 and 2,100 ± 72 μL/s at 150 and 360 mmHg, respectively). The FIF-VCD system was successful in performing cadaver rat abdominal aorta anastomoses.

CONCLUSION

The FIF-VCD system can potentially replace HS in microsurgery, allowing the safe and effective connection of arteries and veins. Further studies are needed to confirm the clinical viability and effectiveness of the FIF-VCD system.

Clean visual field reconstruction in robot-assisted laparoscopic surgery based on dynamic prediction

Robot-assisted minimally invasive surgery has been broadly employed in complicated operations. However, the multiple surgical instruments may occupy a large amount of visual space in complex operations performed in narrow spaces, which affects the surgeon's judgment on the shape and position of the lesion as well as the course of its adjacent vessels/lacunae. In this paper, a surgical scene reconstruction method is proposed, which involves the tracking and removal of surgical instruments and the dynamic prediction of the obscured region. For tracking and segmentation of instruments, the image sequences are preprocessed by a modified U-Net architecture composed of a pre-trained ResNet101 encoder and a redesigned decoder. Also, the segmentation boundaries of the instrument shafts are extended using image filtering and a real-time index mask algorithm to achieve precise localization of the obscured elements. For predicting the deformation of soft tissues, a soft tissue deformation prediction algorithm is proposed based on dense optical flow gravitational field and entropy increase, which can achieve local dynamic visualization of the surgical scene by integrating image morphological operations. Finally, the preliminary experiments and the pre-clinical evaluation were presented to demonstrate the performance of the proposed method. The results show that the proposed method can provide the surgeon with a clean and comprehensive surgical scene, reconstruct the course of important vessels/lacunae, and avoid inadvertent injuries.

Robotic ultrasound imaging: State-of-the-art and future perspectives

Ultrasound (US) is one of the most widely used modalities for clinical intervention and diagnosis due to the merits of providing non-invasive, radiation-free, and real-time images. However, free-hand US examinations are highly operator-dependent. Robotic US System (RUSS) aims at overcoming this shortcoming by offering reproducibility, while also aiming at improving dexterity, and intelligent anatomy and disease-aware imaging. In addition to enhancing diagnostic outcomes, RUSS also holds the potential to provide medical interventions for populations suffering from the shortage of experienced sonographers. In this paper, we categorize RUSS as teleoperated or autonomous. Regarding teleoperated RUSS, we summarize their technical developments, and clinical evaluations, respectively. This survey then focuses on the review of recent work on autonomous robotic US imaging. We demonstrate that machine learning and artificial intelligence present the key techniques, which enable intelligent patient and process-specific, motion and deformation-aware robotic image acquisition. We also show that the research on artificial intelligence for autonomous RUSS has directed the research community toward understanding and modeling expert sonographers' semantic reasoning and action. Here, we call this process, the recovery of the "language of sonography". This side result of research on autonomous robotic US acquisitions could be considered as valuable and essential as the progress made in the robotic US examination itself. This article will provide both engineers and clinicians with a comprehensive understanding of RUSS by surveying underlying techniques. Additionally, we present the challenges that the scientific community needs to face in the coming years in order to achieve its ultimate goal of developing intelligent robotic sonographer colleagues. These colleagues are expected to be capable of collaborating with human sonographers in dynamic environments to enhance both diagnostic and intraoperative imaging.

Preoperative Planning Framework for Robot-Assisted Dental Implant Surgery: Finite-Parameter Surrogate Model and Optimization of Instrument Placement

For robot-assisted dental implant surgery, it is necessary to feed the instrument into a specified position to perform surgery. To improve safety and efficiency, a preoperative planning framework, including a finite-parameter surrogate model (FPSM) and an automatic instrument-placement method, is proposed in this paper. This framework is implemented via two-stage optimization. In the first stage, a group of closed curves in polar coordinates is used to represent the oral cavity. By optimizing a finite number of parameters for these curves, the oral structure is simplified to form the FPSM. In the second stage, the FPSM serves as a fast safety estimator with which the target position/orientation of the instrument for the feeding motion is automatically determined through particle swarm optimization (PSO). The optimized feeding target can be used to generate a virtual fixture (VF) to avoid undesired operations and to lower the risk of collision. This proposed framework has the advantages of being safe, fast, and accurate, overcoming the computational burden and insufficient real-time performance of complex 3D models. The framework has been developed and tested, preliminarily verifying its feasibility, efficiency, and effectiveness.

Evolution of a surgical system using deep learning in minimally invasive surgery (Review)

Recently, artificial intelligence (AI) has been applied in various fields due to the development of new learning methods, such as deep learning, and the marked progress in computational processing speed. AI is also being applied in the medical field for medical image recognition and omics analysis of genomes and other data. Recently, AI applications for videos of minimally invasive surgeries have also advanced, and studies on such applications are increasing. In the present review, studies that focused on the following topics were selected: i) Organ and anatomy identification, ii) instrument identification, iii) procedure and surgical phase recognition, iv) surgery-time prediction, v) identification of an appropriate incision line, and vi) surgical education. The development of autonomous surgical robots is also progressing, with the Smart Tissue Autonomous Robot (STAR) and RAVEN systems being the most reported developments. STAR, in particular, is currently being used in laparoscopic imaging to recognize the surgical site from laparoscopic images and is in the process of establishing an automated suturing system, albeit in animal experiments. The present review examined the possibility of fully autonomous surgical robots in the future.

HUMANISE: Human-Inspired Smart Management, towards a Healthy and Safe Industrial Collaborative Robotics

The workplace is evolving towards scenarios where humans are acquiring a more active and dynamic role alongside increasingly intelligent machines. Moreover, the active population is ageing and consequently emerging risks could appear due to health disorders of workers, which requires intelligent intervention both for production management and workers' support. In this sense, the innovative and smart systems oriented towards monitoring and regulating workers' well-being will become essential. This work presents HUMANISE, a novel proposal of an intelligent system for risk management, oriented to workers suffering from disease conditions. The developed support system is based on Computer Vision, Machine Learning and Intelligent Agents. Results: The system was applied to a two-arm Cobot scenario during a Learning from Demonstration task for collaborative parts transportation, where risk management is critical. In this environment with a worker suffering from a mental disorder, safety is successfully controlled by means of human/robot coordination, and risk levels are managed through the integration of human/robot behaviour models and worker's models based on the workplace model of the World Health Organization. The results show a promising real-time support tool to coordinate and monitoring these scenarios by integrating workers' health information towards a successful risk management strategy for safe industrial Cobot environments.

Interactive 3D Force/Torque Parameter Acquisition and Correlation Identification during Primary Trocar Insertion in Laparoscopic Abdominal Surgery: 5 Cases

Laparoscopic procedures have become indispensable in gastrointestinal surgery. As a minimally invasive process, it begins with primary trocar insertion. However, this step poses the threat of injuries to the gastrointestinal tract and blood vessels. As such, the comprehension of the insertion process is crucial to the development of robotic-assisted/automated surgeries. To sustain robotic development, this research aims to study the interactive force/torque (F/T) behavior between the trocar and the abdomen during the trocar insertion process. For force/torque (F/T) data acquisition, a trocar interfaced with a six-axis F/T sensor was used by surgeons for the insertion. The study was conducted during five abdominal hernia surgical cases in the Department of Surgery, Faculty of Medicine, Ramathibodi Hospital, Mahidol University. The real-time F/T data were further processed and analyzed. The fluctuation in the force/torque (F/T) parameter was significant, with peak force ranging from 16.83 N to 61.86 N and peak torque ranging from 0.552 Nm to 1.76 Nm. The force parameter was observed to positively correlate with procedural time, while torque was found to be negatively correlated. Although during the process a surgeon applied force and torque in multiple axes, for a robotic system, the push and turn motion in a single axis was observed to be sufficient. For minimal tissue damage in less procedural time, a system with low push force and high torque was observed to be advantageous. These understandings will eventually benefit the development of computer-assisted or robotics technology to improve the outcome of the primary trocar insertion procedure.

Sensor-Based Automated Detection of Electrosurgical Cautery States

In computer-assisted surgery, it is typically required to detect when the tool comes into contact with the patient. In activated electrosurgery, this is known as the energy event. By continuously tracking the electrosurgical tools' location using a navigation system, energy events can help determine locations of sensor-classified tissues. Our objective was to detect the energy event and determine the settings of electrosurgical cautery-robustly and automatically based on sensor data. This study aims to demonstrate the feasibility of using the cautery state to detect surgical incisions, without disrupting the surgical workflow. We detected current changes in the wires of the cautery device and grounding pad using non-invasive current sensors and an oscilloscope. An open-source software was implemented to apply machine learning on sensor data to detect energy events and cautery settings. Our methods classified each cautery state at an average accuracy of 95.56% across different tissue types and energy level parameters altered by surgeons during an operation. Our results demonstrate the feasibility of automatically identifying energy events during surgical incisions, which could be an important safety feature in robotic and computer-integrated surgery. This study provides a key step towards locating tissue classifications during breast cancer operations and reducing the rate of positive margins.