Electromagnetic tracking in image-guided laparoscopic surgery: Comparison with optical tracking and feasibility study of a combined laparoscope and laparoscopic ultrasound system

Distortion-Free Magnetic Tracking of Metal Instruments in Image-Guided Interventions

Electromagnetic tracking (EMT) can benefit image-guided interventions in cases where line of sight is unavailable. However, EMT can suffer from electromagnetic distortion in the presence of metal instruments. Metal instruments are widely used in laparoscopic surgery, ENT surgery, arthroscopy and many other clinical applications. In this work, we investigate the feasibility of tracking such metal instruments by placing the inductive sensor within the instrument shaft. We propose a magnetostatic model of the field within the instrument, and verify the results experimentally for frequencies from 6 kHz to 60 kHz. The impact of the instrument's dimensions, conductivity and transmitting field frequency is quantified for ranges representative of typical metal instruments used in image-guided interventions. We then performed tracking using the open-source Anser EMT system and quantify the error caused by the presence of the rod as a function of the frequency of the eight emitting coils for the system. The work clearly demonstrates why smaller tool diameters (less than 8 mm) are less susceptible to distortion, as well as identifying optimal frequencies (1 kHz to 2 kHz) for transmitter design to minimise for distortion in larger instruments.

Hybrid optical-vision tracking in laparoscopy: accuracy of navigation and ultrasound reconstruction

INTRODUCTION

The use of laparoscopic and robotic liver surgery is increasing. However, it presents challenges such as limited field of view and organ deformations. Surgeons rely on laparoscopic ultrasound (LUS) for guidance, but mentally correlating ultrasound images with pre-operative volumes can be difficult. In this direction, surgical navigation systems are being developed to assist with intra-operative understanding. One approach is performing intra-operative ultrasound 3D reconstructions. The accuracy of these reconstructions depends on tracking the LUS probe.

MATERIAL AND METHODS

This study evaluates the accuracy of LUS probe tracking and ultrasound 3D reconstruction using a hybrid tracking approach. The LUS probe is tracked from laparoscope images, while an optical tracker tracks the laparoscope. The accuracy of hybrid tracking is compared to full optical tracking using a dual-modality tool. Ultrasound 3D reconstruction accuracy is assessed on an abdominal phantom with CT transformed into the optical tracker's coordinate system.

RESULTS

Hybrid tracking achieves a tracking error < 2 mm within 10 cm between the laparoscope and the LUS probe. The ultrasound reconstruction accuracy is approximately 2 mm.

CONCLUSION

Hybrid tracking shows promising results that can meet the required navigation accuracy for laparoscopic liver surgery.

The value of Augmented Reality in surgery - A usability study on laparoscopic liver surgery

Augmented Reality (AR) is considered to be a promising technology for the guidance of laparoscopic liver surgery. By overlaying pre-operative 3D information of the liver and internal blood vessels on the laparoscopic view, surgeons can better understand the location of critical structures. In an effort to enable AR, several authors have focused on the development of methods to obtain an accurate alignment between the laparoscopic video image and the pre-operative 3D data of the liver, without assessing the benefit that the resulting overlay can provide during surgery. In this paper, we present a study that aims to assess quantitatively and qualitatively the value of an AR overlay in laparoscopic surgery during a simulated surgical task on a phantom setup. We design a study where participants are asked to physically localise pre-operative tumours in a liver phantom using three image guidance conditions - a baseline condition without any image guidance, a condition where the 3D surfaces of the liver are aligned to the video and displayed on a black background, and a condition where video see-through AR is displayed on the laparoscopic video. Using data collected from a cohort of 24 participants which include 12 surgeons, we observe that compared to the baseline, AR decreases the median localisation error of surgeons on non-peripheral targets from 25.8 mm to 9.2 mm. Using subjective feedback, we also identify that AR introduces usability improvements in the surgical task and increases the perceived confidence of the users. Between the two tested displays, the majority of participants preferred to use the AR overlay instead of navigated view of the 3D surfaces on a separate screen. We conclude that AR has the potential to improve performance and decision making in laparoscopic surgery, and that improvements in overlay alignment accuracy and depth perception should be pursued in the future.

Evaluation of a calibration rig for stereo laparoscopes

BACKGROUND

Accurate camera and hand-eye calibration are essential to ensure high-quality results in image-guided surgery applications. The process must also be able to be undertaken by a nonexpert user in a surgical setting.

PURPOSE

This work seeks to identify a suitable method for tracked stereo laparoscope calibration within theater.

METHODS

A custom calibration rig, to enable rapid calibration in a surgical setting, was designed. The rig was compared against freehand calibration. Stereo reprojection, stereo reconstruction, tracked stereo reprojection, and tracked stereo reconstruction error metrics were used to evaluate calibration quality.

RESULTS

Use of the calibration rig reduced mean errors: reprojection (1.47 mm [SD 0.13] vs. 3.14 mm [SD 2.11], p-value 1e-8), reconstruction (1.37 px [SD 0.10] vs. 10.10 px [SD 4.54], p-value 6e-7), and tracked reconstruction (1.38 mm [SD 0.10] vs. 12.64 mm [SD 4.34], p-value 1e-6) compared with freehand calibration. The use of a ChArUco pattern yielded slightly lower reprojection errors, while a dot grid produced lower reconstruction errors and was more robust under strong global illumination.

CONCLUSION

The use of the calibration rig results in a statistically significant decrease in calibration error metrics, versus freehand calibration, and represents the preferred approach for use in the operating theater.

Development of Single-Channel Dual-Element Custom-Made Ultrasound Scanner with Miniature Optical Position Tracker for Freehand Imaging

Handheld ultrasound has great potential in resource-limited areas, and can improve healthcare for rural populations. Single-channel ultrasound has been widely used in many clinical ultrasound applications, and optical tracking is considered accurate and reliable. In this study, we developed a 10 MHz lead magnesium niobate-lead titanate (PMN-PT) dual-element ultrasound transducer combined with a miniature optical position tracker, and then measured the rectus femoris of the thigh, upper arm, and cheek muscles. Compared to single-element transducers, dual-element transducers improve the contrast of near-field signals, effectively reduce noise, and are suitable for measuring curved surfaces. The purpose of position tracking is to calculate the location of the ultrasound transducer during the measurement process. By utilizing positioning information, 2D ultrasound imaging can be achieved while maintaining structural integrity. The dual-element ultrasound scanner presented in this study can enable continuous scanning over a large area without a scanning width limitation. The custom-made dual-element ultrasound scanner has the advantage of being a portable, reliable, and low-cost ultrasound device, and is helpful in popularizing medical care for remote villages.

Measuring Respiratory Motion for Supporting the Minimally Invasive Destruction of Liver Tumors

OBJECTIVE

Destroying liver tumors is a challenge for contemporary interventional radiology. The aim of this work is to compare different techniques used for the measurement of respiratory motion, as this is the main hurdle to the effective implementation of this therapy.

METHODS

Laparoscopic stereoscopic reconstruction of point displacements on the surface of the liver, observation of breathing using external markers placed on the surface of the abdominal cavity, and methods for registration of the surface of the abdominal cavity during breathing were implemented and evaluated.

RESULTS

The following accuracies were obtained: above 4 mm and 0.5 mm, and below 8 mm for laparoscopic, skin markers, and skin surface registration methods, respectively.

CONCLUSIONS

The clinical techniques and accompanying imaging modalities employed to destroy liver tumors, as well as the advantages and limitations of the proposed methods, are presented. Further directions for their development are also indicated.

Shape Reconstruction Processes for Interventional Application Devices: State of the Art, Progress, and Future Directions

Soft and continuum robots are transforming medical interventions thanks to their flexibility, miniaturization, and multidirectional movement abilities. Although flexibility enables reaching targets in unstructured and dynamic environments, it also creates challenges for control, especially due to interactions with the anatomy. Thus, in recent years lots of efforts have been devoted for the development of shape reconstruction methods, with the advancement of different kinematic models, sensors, and imaging techniques. These methods can increase the performance of the control action as well as provide the tip position of robotic manipulators relative to the anatomy. Each method, however, has its advantages and disadvantages and can be worthwhile in different situations. For example, electromagnetic (EM) and Fiber Bragg Grating (FBG) sensor-based shape reconstruction methods can be used in small-scale robots due to their advantages thanks to miniaturization, fast response, and high sensitivity. Yet, the problem of electromagnetic interference in the case of EM sensors, and poor response to high strains in the case of FBG sensors need to be considered. To help the reader make a suitable choice, this paper presents a review of recent progress on shape reconstruction methods, based on a systematic literature search, excluding pure kinematic models. Methods are classified into two categories. First, sensor-based techniques are presented that discuss the use of various sensors such as FBG, EM, and passive stretchable sensors for reconstructing the shape of the robots. Second, imaging-based methods are discussed that utilize images from different imaging systems such as fluoroscopy, endoscopy cameras, and ultrasound for the shape reconstruction process. The applicability, benefits, and limitations of each method are discussed. Finally, the paper draws some future promising directions for the enhancement of the shape reconstruction methods by discussing open questions and alternative methods.

Hybrid electromagnetic-ArUco tracking of laparoscopic ultrasound transducer in laparoscopic video

Purpose: The purpose of this work was to develop a new method of tracking a laparoscopic ultrasound (LUS) transducer in laparoscopic video by combining the hardware [e.g., electromagnetic (EM)] and the computer vision-based (e.g., ArUco) tracking methods. Approach: We developed a special tracking mount for the imaging tip of the LUS transducer. The mount incorporated an EM sensor and an ArUco pattern registered to it. The hybrid method used ArUco tracking for ArUco-success frames (i.e., frames where ArUco succeeds in detecting the pattern) and used corrected EM tracking for the ArUco-failure frames. The corrected EM tracking result was obtained by applying correction matrices to the original EM tracking result. The correction matrices were calculated in previous ArUco-success frames by comparing the ArUco result and the original EM tracking result. Results: We performed phantom and animal studies to evaluate the performance of our hybrid tracking method. The corrected EM tracking results showed significant improvements over the original EM tracking results. In the animal study, 59.2% frames were ArUco-success frames. For the ArUco-failure frames, mean reprojection errors for the original EM tracking method and for the corrected EM tracking method were 30.8 pixel and 10.3 pixel, respectively. Conclusions: The new hybrid method is more reliable than using ArUco tracking alone and more accurate and practical than using EM tracking alone for tracking the LUS transducer in the laparoscope camera image. The proposed method has the potential to significantly improve tracking performance for LUS-based augmented reality applications.

Visuo-Haptic Mixed Reality Simulation Using Unbound Handheld Tools

Visuo-haptic mixed reality (VHMR) adds virtual objects to a real scene and enables users to see and also touch them via a see-through display and a haptic device. Most studies with kinesthetic feedback use general-purpose haptic devices, which require the user to continuously hold an attached stylus. This approach constrains users to the mechanical limits of the device even when it is not needed. In this paper, we propose a novel VHMR concept with an encountered-type haptic display (ETHD), which consists of a precision hexapod positioner and a six-axis force/torque transducer. The main contribution is that the users work with unbound real-life tools with tracking markers. ETHD’s end-effector remains inside the virtual object and follows the tooltip to engage only during an interaction. We have developed a simulation setup and experimentally evaluated the relative accuracy and synchronization of the three major processes, namely tool tracking, haptic rendering, and visual rendering. The experiments successfully build-up to a simple simulation scenario where a tennis ball with a fixed center is deformed by the user.

Ultrasound Imaging Characterization of Soft Tissue Dynamics of the Seated Human Body

To characterize the dynamics of internal soft organs and external anatomical structures, this paper presents a system that combines medical ultrasound imaging with an optical tracker and a vertical exciter that imparts whole-body vibrations on seated subjects. The spatial and temporal accuracy of the system was validated using a phantom with calibrated internal structures, resulting in 0.224 mm maximum root-mean-square (r.m.s.) position error and 13 ms maximum synchronization error between sensors. In addition to the dynamics of the head and sternum, stomach dynamics were characterized by extracting the centroid of the stomach from the ultrasound images. The system was used to characterize the subject-specific body dynamics as well as the intrasubject variabilities caused by excitation pattern (frequency up-sweep, down-sweep, and white noise, 1-10 Hz), excitation amplitude (1 and 2 m/s2 r.m.s.), seat compliance (rigid and soft), and stomach filling (empty and 500 mL water). Human subjects experiments (n = 3) yielded preliminary results for the frequency response of the head, sternum, and stomach. The method presented here provides the first detailed in vivo characterization of internal and external human body dynamics. Tissue dynamics characterized by the system can inform design of vehicle structures and adaptive control of seat and suspension systems, as well as validate finite element models for predicting passenger comfort in the early stages of vehicle design.

Repurposing Molecular Imaging and Sensing for Cancer Image-Guided Surgery

Gone are the days when medical imaging was used primarily to visualize anatomic structures. The emergence of molecular imaging (MI), championed by radiolabeled ¹⁸F-FDG PET, has expanded the information content derived from imaging to include pathophysiologic and molecular processes. Cancer imaging, in particular, has leveraged advances in MI agents and technology to improve the accuracy of tumor detection, interrogate tumor heterogeneity, monitor treatment response, focus surgical resection, and enable image-guided biopsy. Surgeons are actively latching on to the incredible opportunities provided by medical imaging for preoperative planning, intraoperative guidance, and postoperative monitoring. From label-free techniques to enabling cancer-selective imaging agents, image-guided surgery provides surgical oncologists and interventional radiologists both macroscopic and microscopic views of cancer in the operating room. This review highlights the current state of MI and sensing approaches available for surgical guidance. Salient features of nuclear, optical, and multimodal approaches will be discussed, including their strengths, limitations, and clinical applications. To address the increasing complexity and diversity of methods available today, this review provides a framework to identify a contrast mechanism, suitable modality, and device. Emerging low-cost, portable, and user-friendly imaging systems make the case for adopting some of these technologies as the global standard of care in surgical practice.