Tracking multiple surgical instruments in a near-infrared optical system

Surgical tool segmentation and localization using spatio-temporal deep network

Laparoscopic cholecystectomy surgery is a minimally invasive surgery to remove the gallbladder, where surgical instruments are inserted through small incisions in the abdomen with the help of a laparoscope. Identification of tool presence and precise segmentation of tools from the video is very important in understanding the quality of the surgery and training budding surgeons. Precise segmentation of tools is required to track the tools during real-time surgeries. In this paper, a new pixel-wise instance segmentation algorithm is proposed, which segments and localizes the surgical tool using spatio-temporal deep network. The performance of the proposed has been compared with the state-of-the-art image-based instance segmentation method using the Cholec80 dataset. It is also compared with methods in the literature using frame-level presence detection and spatial detection with good results.

Optical and Electromagnetic Tracking Systems for Biomedical Applications: A Critical Review on Potentialities and Limitations

Optical and electromagnetic tracking systems represent the two main technologies integrated into commercially-available surgical navigators for computer-assisted image-guided surgery so far. Optical Tracking Systems (OTSs) work within the optical spectrum to track the position and orientation, i.e., pose of target surgical instruments. OTSs are characterized by high accuracy and robustness to environmental conditions. The main limitation of OTSs is the need of a direct line-of-sight between the optical markers and the camera sensor, rigidly fixed into the operating theatre. Electromagnetic Tracking Systems (EMTSs) use electromagnetic field generator to detect the pose of electromagnetic sensors. EMTSs do not require such a direct line-of-sight, however the presence of metal or ferromagnetic sources in the operating workspace can significantly affect the measurement accuracy. The aim of the proposed review is to provide a complete and detailed overview of optical and electromagnetic tracking systems, including working principles, source of error and validation protocols. Moreover, commercial and research-oriented solutions, as well as clinical applications, are described for both technologies. Finally, a critical comparative analysis of the state of the art which highlights the potentialities and the limitations of each tracking system for a medical use is provided.

Robust Real-Time Detection of Laparoscopic Instruments in Robot Surgery Using Convolutional Neural Networks with Motion Vector Prediction

More than half of post-operative complications can be prevented, and operation performances can be improved based on the feedback gathered from operations or notifications of the risks during operations in real time. However, existing surgical analysis methods are limited, because they involve time-consuming processes and subjective opinions. Therefore, the detection of surgical instruments is necessary for (a) conducting objective analyses, or (b) providing risk notifications associated with a surgical procedure in real time. We propose a new real-time detection algorithm for detection of surgical instruments using convolutional neural networks (CNNs). This algorithm is based on an object detection system YOLO9000 and ensures continuity of detection of the surgical tools in successive imaging frames based on motion vector prediction. This method exhibits a constant performance irrespective of a surgical instrument class, while the mean average precision (mAP) of all the tools is 84.7, with a speed of 38 frames per second (FPS).

Multiple instruments motion trajectory tracking in optical surgical navigation

Optical surgical navigation system has been a hot research topic because of its high accuracy. This paper focuses on identifying and tracking multiple surgical instruments to meet the requirements of applying multiple surgical instruments in clinical medicine. The methods of instrument identification based on the marker's geometrical arrangement and instrument tracking based on markers' motion vector were applied in the proposed algorithm. The experiments of multiple instruments' identification and tracking, the instruments' stability, the space distance and rotation of a pair of instruments at the same tracking time were performed to verify the proposed algorithm. The stereoscopic camera is applied to capture images, and two 850 nm filters are added in front of the binocular camera. The tracking experiment shows that ten instruments can be fully and accurately identified, and then all of them can be quickly and accurately tracked at the same time. A pair of instruments is simultaneously measured in the stability test, such as the typical surgical instrument (TSI) and the miniature surgical instrument (MSI). The ranges of standard deviations (SD) of the stability test for the TSI in the X-, Y-, and Z- axes are from 0.016 mm to 0.127 mm, from 0.011 mm to 0.090 mm, and from 0.124 mm to 0.901 mm, respectively. And the ranges of SDs for the MSI's stability test are from 0.011 mm to 0.133 mm in the X-axis, from 0.010 mm to 0.106 mm in the Y-axis, and from 0.093 mm to 0.932 mm in the Z-axis. The sub-millimeter SDs show that the proposed algorithm has a high stability. Moreover, the space distance test and the rotation test were performed for simultaneously tracking TSI and MSI. All experimental results indicate the proposed algorithm is able to meet the clinical accuracy requirements.

A novel finite element model-based navigation system-supported workflow for breast tumor excision

In the case of female breast cancer, a breast-conserving excision is often desirable. This surgery is based on preoperatively gathered MRI, mammography, and sonography images. These images are recorded in multiple patient positions, e. g., 2D mammography images in standing position with a compressed breast and 3D MRI images in prone position. In contrast, the surgery happens in supine or beach chair position. Due to these different perspectives and the flexible, thus challenging, breast tissue, the excision puts high demands on the physician. Therefore, this publication presents a novel eight-step excision support workflow that can be used to include information captured preoperatively through medical imaging based on a finite element (FE) model. In addition, an indoor positioning system is integrated in the workflow in order to track surgical devices and the sonography transducer during surgery. The preoperative part of the navigation system-supported workflow is outlined exemplarily based on first experimental results including 3D scans of a patient in different patient positions and her MRI images. Graphical Abstract Finite Element model based navigation system supported workflow for breast tumor excision is based on eight steps and allows inclusion of information from medical images recorded in multiple patient positions.