New real-time MR image-guided surgical robotic system for minimally invasive precision surgery

State of the Art and Future Opportunities in MRI-Guided Robot-Assisted Surgery and Interventions

Magnetic resonance imaging (MRI) can provide high-quality 3-D visualization of target anatomy, surrounding tissue, and instrumentation, but there are significant challenges in harnessing it for effectively guiding interventional procedures. Challenges include the strong static magnetic field, rapidly switching magnetic field gradients, high-power radio frequency pulses, sensitivity to electrical noise, and constrained space to operate within the bore of the scanner. MRI has a number of advantages over other medical imaging modalities, including no ionizing radiation, excellent soft-tissue contrast that allows for visualization of tumors and other features that are not readily visible by other modalities, true 3-D imaging capabilities, including the ability to image arbitrary scan plane geometry or perform volumetric imaging, and capability for multimodality sensing, including diffusion, dynamic contrast, blood flow, blood oxygenation, temperature, and tracking of biomarkers. The use of robotic assistants within the MRI bore, alongside the patient during imaging, enables intraoperative MR imaging (iMRI) to guide a surgical intervention in a closed-loop fashion that can include tracking of tissue deformation and target motion, localization of instrumentation, and monitoring of therapy delivery. With the ever-expanding clinical use of MRI, MRI-compatible robotic systems have been heralded as a new approach to assist interventional procedures to allow physicians to treat patients more accurately and effectively. Deploying robotic systems inside the bore synergizes the visual capability of MRI and the manipulation capability of robotic assistance, resulting in a closed-loop surgery architecture. This article details the challenges and history of robotic systems intended to operate in an MRI environment and outlines promising clinical applications and associated state-of-the-art MRI-compatible robotic systems and technology for making this possible.

Analysis of Current Deep Learning Networks for Semantic Segmentation of Anatomical Structures in Laparoscopic Surgery

Semantic segmentation of anatomical structures in laparoscopic videos is a crucial task to enable the development of new computer-assisted systems that can assist surgeons during surgery. However, this is a difficult task due to artifacts and similar visual characteristics of anatomical structures on the laparoscopic videos. Recently, deep learning algorithms have been showed promising results on the segmentation of laparoscopic instruments. However, due to the lack of large public datasets for semantic segmentation of anatomical structures, there are only a few studies on this task. In this work, we evaluate the performance of five networks, namely U-Net, U-Net++, DynUNet, UNETR and DeepLabV3+, for segmentation of laparoscopic cholecystectomy images from the recently released CholecSeg8k dataset. To the best of our knowledge, this is the first benchmark performed on this dataset. Training was performed with dice loss. The networks were evaluated on segmentation of 8 anatomical structures and instruments, performance was quantified through the dice coefficient, intersection over union, recall, and precision. Apart from the U-Net, all networks obtained scores similar to each other, with the U-Net++ being the network with the best overall score with a mean Dice value of 0.62. Overall, the results show that there is still room for improvement in the segmentation of anatomical structures from laparoscopic videos. Clinical Relevance- The results of this study show the potential of deep neural networks for the segmentation of anatomical structures in laparoscopic images which can later be incorporated into computer-aided systems for surgeons.

NeuroPlan: A Surgical Planning Toolkit for an MRI-Compatible Stereotactic Neurosurgery Robot

The adoption of robotic image-guided surgeries has enabled physicians to perform therapeutic and diagnostic procedures with less invasiveness and higher accuracy. One example is the MRI-guided stereotactic robotic-assisted surgery for conformal brain tumor ablation, where the robot is used to position and orient a thin probe to target a desired region within the brain. Requirements such as the remote center of motion and precise manipulation, impose the use of complex kinematic structures, which result in non-trivial workspaces in these robots. The lack of workspace visualization poses a challenge in selecting valid entry and target points during the surgical planning and navigation stage. In this paper, we present a surgical planning toolkit called the "NeuroPlan" for our MRI-compatible stereotactic neurosurgery robot developed as a module for 3D Slicer software. This toolkit streamlines the current surgical workflow by rendering and overlaying the robot's reachable workspace on the MRI image. It also assists with identifying the optimal entry point by segmenting the cranial burr hole volume and locating its center. We demonstrate the accuracy of the workspace rendering and burr hole parameter detection through both phantom and MR-images acquired from previously conducted animal studies.

Deep learning for semantic segmentation of organs and tissues in laparoscopic surgery

Semantic segmentation of organs and tissue types is an important sub-problem in image based scene understanding for laparoscopic surgery and is a prerequisite for context-aware assistance and cognitive robotics. Deep Learning (DL) approaches are prominently applied to segmentation and tracking of laparoscopic instruments. This work compares different combinations of neural networks, loss functions, and training strategies in their application to semantic segmentation of different organs and tissue types in human laparoscopic images in order to investigate their applicability as components in cognitive systems. TernausNet-11 trained on Soft-Jaccard loss with a pretrained, trainable encoder performs best in regard to segmentation quality (78.31% mean Intersection over Union [IoU]) and inference time (28.07 ms) on a single GTX 1070 GPU.

Development of a robotic system with six-degrees-of-freedom robotic tool manipulators for single-port surgery

BACKGROUND

Current robotic systems have limitations for single-port surgery (SPS) because the instruments are large, the arms collide and the field of vision requires manual readjustment. We have developed an SPS robotic system that manipulates the vision field.

METHODS

The master-slave system included a six degrees of freedom (DOFs) tool manipulator, an easy set-up mechanism, a flexible shaft drive with sufficient DOFs and a vertical arrangement for improved vision. The robot manipulates an endoscope within the body. After assembly, the system was tested in vitro and in vivo.

RESULTS

In vitro testing showed that the tool manipulators resected tissue precisely, with the range of motion required. In vivo testing indicated that an abdominal organ is accurately approached and diseased tissue removed by combined endoscopy and robotics.

CONCLUSIONS

Our robotics allowed the operator to adjust the vision field intuitively. The tool manipulator approached and resected diseased tissue precisely.

Intraoperative magnetic resonance imaging–conditional robotic devices for therapy and diagnosis

Magnetic resonance imaging presents high-resolution preoperative scans of target tissue and allows for the availability of intraoperative real-time images without the exposure of patients to ionizing radiation. This has motivated scientists and engineers to integrate medical robotics with the magnetic resonance imaging modality to allow robot-assisted, image-guided diagnosis and therapy. This article provides a review of the state-of-the-art medical robotic systems available for use in conjunction with intraoperative magnetic resonance imaging. The robot functionalities and mechanical designs for a wide range of magnetic resonance imaging interventions are presented, including their magnetic resonance imaging compatibility, actuation, kinematics and the mechanical and electrical designs of the robots. Classification and comparative study of various intraoperative magnetic resonance image guided robotic systems are provided. The robotic systems reviewed are summarized in a table in detail. Current technologies for magnetic resonance imaging–conditional robotics are reviewed and their potential future directions are sketched.

Emerging robotic platforms for minimally invasive surgery

Recent technological advances in surgery have resulted in the development of a range of new techniques that have reduced patient trauma, shortened hospitalization, and improved diagnostic accuracy and therapeutic outcome. Despite the many appreciated benefits of minimally invasive surgery (MIS) compared to traditional approaches, there are still significant drawbacks associated with conventional MIS including poor instrument control and ergonomics caused by rigid instrumentation and its associated fulcrum effect. The use of robot assistance has helped to realize the full potential of MIS with improved consistency, safety and accuracy. The development of articulated, precision tools to enhance the surgeon's dexterity has evolved in parallel with advances in imaging and human-robot interaction. This has improved hand-eye coordination and manual precision down to micron scales, with the capability of navigating through complex anatomical pathways. In this review paper, clinical requirements and technical challenges related to the design of robotic platforms for flexible access surgery are discussed. Allied technical approaches and engineering challenges related to instrument design, intraoperative guidance, and intelligent human-robot interaction are reviewed. We also highlight emerging designs and research opportunities in the field by assessing the current limitations and open technical challenges for the wider clinical uptake of robotic platforms in MIS.

In vivo experiments of a surgical robot with vision field control for Single Port Endoscopic Surgery

Recently, robotics systems are focused to assist in Single Port Endoscopic Surgery (SPS). However, the existing system required a manual operation of vision and viewpoint, hindering the surgical task. We proposed a surgical endoscopic robot for SPS with dynamic vision control, the endoscopic view being manipulated by a master controller. The prototype robot consists of a manipulator for vision control, and dual tool tissue manipulators (gripping: 5DOFs, cautery: 3DOFs) can be attached at the tip of sheath manipulator. In particular, this paper focuses on an in vivo experiment. We showed that vision control in the stomach and a cautery task by a cautery tool could be effectively achieved.

Pilot study of design method for surgical robot using workspace reproduction system

Recent development methods for surgical robots have an inherent problem. The user-friendliness of operating robot cannot be revealed until completion of the robot. To assist the design of a surgical robot that is user-friendly in terms of surgeon's operation, we propose a system that considers the operation manner of surgeon during the design phase of the robot. This system includes the following functionality: 1) a master manipulator that measures the operation manner of the surgeon (operator), and 2) a slave simulator in which the mechanical parameters can be configured freely. The operator can use the master manipulator to operate the slave simulator. Using this system, we investigate the necessity of considering the operator's manner when developing a surgical robot. In the experiment, we used three instruments with mechanisms that differed with respect to the length between bending joints and measured the trajectory of each instrument tip position during the surgical task. The results show that there are differences in the trajectories of each mechanism. Based on the results, changes in the mechanism of the surgical robot influenced the operator's manner. Therefore, when designing the mechanism for a surgical robot, there is a need to consider how this influences the operator's manner.

Development of a Tool Manipulator Driven by a Flexible Shaft for Single-Port Endoscopic Surgery

Recently, a robotic system was developed to assist in Single-Port Endoscopic Surgery (SPS). However, the existing system required a manual operation of vision and viewpoint, hindering the surgical task. We proposed a surgical endoscopic robot for SPS with a dynamic vision control, the endoscopic view being manipulated by a master controller. The prototype robot consists of a manipulator for vision control, and dual tool tissue manipulators (gripping: five DOFs; cautery: three DOFs) can be attached at the tip of the sheath manipulator. In particular, this paper focuses on the details of the mechanism and control scheme of the tool manipulator. The experimental results show that our manipulator exhibits a response with a precision of less than 0.15 mm and a time delay of less than 31 ms, when the input frequency is 1.0 Hz.

Detection of Curved Robots using 3D Ultrasound

Three-dimensional ultrasound can be an effective imaging modality for image-guided interventions since it enables visualization of both the instruments and the tissue. For robotic applications, its realtime frame rates create the potential for image-based instrument tracking and servoing. These capabilities can enable improved instrument visualization, compensation for tissue motion as well as surgical task automation. Continuum robots, whose shape comprises a smooth curve along their length, are well suited for minimally invasive procedures. Existing techniques for ultrasound tracking, however, are limited to straight, laparoscopic-type instruments and thus are not applicable to continuum robot tracking. Toward the goal of developing tracking algorithms for continuum robots, this paper presents a method for detecting a robot comprised of a single constant curvature in a 3D ultrasound volume. Computational efficiency is achieved by decomposing the six-dimensional circle estimation problem into two sequential three-dimensional estimation problems. Simulation and experiment are used to evaluate the proposed method.