Soft tissue navigation for laparoscopic partial nephrectomy

Are 3D Image Guidance Systems Ready for Use? A Comparative Analysis of 3D Image Guidance Implementations in Minimally Invasive Partial Nephrectomy

Introduction: Three-dimensional image-guided surgical (3D-IGS) systems for minimally invasive partial nephrectomy (MIPN) can potentially improve the efficiency and accuracy of intraoperative anatomical localization and tumor resection. This review seeks to analyze the current state of research regarding 3D-IGS, including the evaluation of clinical outcomes, system functionality, and qualitative insights regarding 3D-IGS's impact on surgical procedures. Methods: We have systematically reviewed the clinical literature pertaining to 3D-IGS deployed for MIPN. For inclusion, studies must produce a patient-specific 3D anatomical model from two-dimensional imaging. Data extracted from the studies include clinical results, registration (alignment of the 3D model to the surgical scene) method used, limitations, and data types reported. A subset of studies was qualitatively analyzed through an inductive coding approach to identify major themes and subthemes across the studies. Results: Twenty-five studies were included in the review. Eight (32%) studies reported clinical results that point to 3D-IGS improving multiple surgical outcomes. Manual registration was the most utilized (48%). Soft tissue deformation was the most cited limitation among the included studies. Many studies reported qualitative statements regarding surgeon accuracy improvement, but quantitative surgeon accuracy data were not reported. During the qualitative analysis, six major themes emerged across the nine applicable studies. They are as follows: 3D-IGS is necessary, 3D-IGS improved surgical outcomes, researcher/surgeon confidence in 3D-IGS system, enhanced surgeon ability/accuracy, anatomical explanation for qualitative assessment, and claims without data or reference to support. Conclusions: Currently, clinical outcomes are the main source of quantitative data available to point to 3D-IGS's efficacy. However, the literature qualitatively suggests the benefit of accurate 3D-IGS for robotic partial nephrectomy.

A Review of Cognitive Support Systems in the Operating Room

BACKGROUND

In recent years, numerous innovative yet challenging surgeries, such as minimally invasive procedures, have introduced an overwhelming amount of new technologies, increasing the cognitive load for surgeons and potentially diluting their attention. Cognitive support technologies (CSTs) have been in development to reduce surgeons' cognitive load and minimize errors. Despite its huge demands, it still lacks a systematic review.

METHODS

Literature was searched up until May 21st, 2021. Pubmed, Web of Science, and IEEExplore. Studies that aimed at reducing the cognitive load of surgeons were included. Additionally, studies that contained an experimental trial with real patients and real surgeons were prioritized, although phantom and animal studies were also included. Major outcomes that were assessed included surgical error, anatomical localization accuracy, total procedural time, and patient outcome.

RESULTS

A total of 37 studies were included. Overall, the implementation of CSTs had better surgical performance than the traditional methods. Most studies reported decreased error rate and increased efficiency. In terms of accuracy, most CSTs had over 90% accuracy in identifying anatomical markers with an error margin below 5 mm. Most studies reported a decrease in surgical time, although some were statistically insignificant.

DISCUSSION

CSTs have been shown to reduce the mental workload of surgeons. However, the limited ergonomic design of current CSTs has hindered their widespread use in the clinical setting. Overall, more clinical data on actual patients is needed to provide concrete evidence before the ubiquitous implementation of CSTs.

A survey of augmented reality methods to guide minimally invasive partial nephrectomy

INTRODUCTION

Minimally invasive partial nephrectomy (MIPN) has become the standard of care for localized kidney tumors over the past decade. The characteristics of each tumor, in particular its size and relationship with the excretory tract and vessels, allow one to judge its complexity and to attempt predicting the risk of complications. The recent development of virtual 3D model reconstruction and computer vision has opened the way to image-guided surgery and augmented reality (AR).

OBJECTIVE

Our objective was to perform a systematic review to list and describe the different AR techniques proposed to support PN.

MATERIALS AND METHODS

The systematic review of the literature was performed on 12/04/22, using the keywords "nephrectomy" and "augmented reality" on Embase and Medline. Articles were considered if they reported surgical outcomes when using AR with virtual image overlay on real vision, during ex vivo or in vivo MIPN. We classified them according to the registration technique they use.

RESULTS

We found 16 articles describing an AR technique during MIPN procedures that met the eligibility criteria. A moderate to high risk of bias was recorded for all the studies. We classified registration methods into three main families, of which the most promising one seems to be surface-based registration.

CONCLUSION

Despite promising results, there do not exist studies showing an improvement in clinical outcomes using AR. The ideal AR technique is probably yet to be established, as several designs are still being actively explored. More clinical data will be required to establish the potential contribution of this technology to MIPN.

Augmented Reality to Improve Surgical Simulation. Lessons Learned Towards the Design of a Hybrid Laparoscopic Simulator for Cholecystectomy

Hybrid surgical simulators based on Augmented Reality (AR) solutions benefit from the advantages of both the box trainers and the Virtual Reality simulators. This paper reports on the results of a long development stage of a hybrid simulator for laparoscopic cholecystectomy that integrates real and the virtual components. We first outline the specifications of the AR simulator and then we explain the strategy adopted for implementing it based on a careful selection of its simulated anatomical components, and characterized by a real-time tracking of both a target anatomy and of the laparoscope. The former is tracked by means of an electromagnetic field generator, while the latter requires an additional camera for video tracking. The new system was evaluated in terms of AR visualization accuracy, realism and hardware robustness. Obtained results show that the accuracy of AR visualization is adequate for training purposes. The qualitative evaluation confirms the robustness and the realism of the simulator. The AR simulator satisfies all the initial specifications in terms of anatomical appearance, modularity, reusability, minimization of spare parts cost, and ability to record surgical errors and to track in real-time the Calot's triangle and the laparoscope. The proposed system could be an effective training tool for learning the task of identification and isolation of Calot's triangle in laparoscopic cholecystectomy. Moreover, the presented strategy could be applied to simulate other surgical procedures involving the task of identification and isolation of generic tubular structures, such as blood vessels, biliary tree and nerves, which are not directly visible.

Mobile, real-time, and point-of-care augmented reality is robust, accurate, and feasible: a prospective pilot study

BACKGROUND

Augmented reality (AR) systems are currently being explored by a broad spectrum of industries, mainly for improving point-of-care access to data and images. Especially in surgery and especially for timely decisions in emergency cases, a fast and comprehensive access to images at the patient bedside is mandatory. Currently, imaging data are accessed at a distance from the patient both in time and space, i.e., at a specific workstation. Mobile technology and 3-dimensional (3D) visualization of radiological imaging data promise to overcome these restrictions by making bedside AR feasible.

METHODS

In this project, AR was realized in a surgical setting by fusing a 3D-representation of structures of interest with live camera images on a tablet computer using marker-based registration. The intent of this study was to focus on a thorough evaluation of AR. Feasibility, robustness, and accuracy were thus evaluated consecutively in a phantom model and a porcine model. Additionally feasibility was evaluated in one male volunteer.

RESULTS

In the phantom model (n = 10), AR visualization was feasible in 84% of the visualization space with high accuracy (mean reprojection error ± standard deviation (SD): 2.8 ± 2.7 mm; 95th percentile = 6.7 mm). In a porcine model (n = 5), AR visualization was feasible in 79% with high accuracy (mean reprojection error ± SD: 3.5 ± 3.0 mm; 95th percentile = 9.5 mm). Furthermore, AR was successfully used and proved feasible within a male volunteer.

CONCLUSIONS

Mobile, real-time, and point-of-care AR for clinical purposes proved feasible, robust, and accurate in the phantom, animal, and single-trial human model shown in this study. Consequently, AR following similar implementation proved robust and accurate enough to be evaluated in clinical trials assessing accuracy, robustness in clinical reality, as well as integration into the clinical workflow. If these further studies prove successful, AR might revolutionize data access at patient bedside.

Virtual and Augmented Reality Systems for Renal Interventions: A Systematic Review

PURPOSE

Many virtual and augmented reality systems have been proposed to support renal interventions. This paper reviews such systems employed in the treatment of renal cell carcinoma and renal stones.

METHODS

A systematic literature search was performed. Inclusion criteria were virtual and augmented reality systems for radical or partial nephrectomy and renal stone treatment, excluding systems solely developed or evaluated for training purposes.

RESULTS

In total, 52 research papers were identified and analyzed. Most of the identified literature (87%) deals with systems for renal cell carcinoma treatment. About 44% of the systems have already been employed in clinical practice, but only 20% in studies with ten or more patients. Main challenges remaining for future research include the consideration of organ movement and deformation, human factor issues, and the conduction of large clinical studies.

CONCLUSION

Augmented and virtual reality systems have the potential to improve safety and outcomes of renal interventions. In the last ten years, many technical advances have led to more sophisticated systems, which are already applied in clinical practice. Further research is required to cope with current limitations of virtual and augmented reality assistance in clinical environments.

Time-Of-Flight Camera, Optical Tracker and Computed Tomography in Pairwise Data Registration

Purpose

A growing number of medical applications, including minimal invasive surgery, depends on multi-modal or multi-sensors data processing. Fast and accurate 3D scene analysis, comprising data registration, seems to be crucial for the development of computer aided diagnosis and therapy. The advancement of surface tracking system based on optical trackers already plays an important role in surgical procedures planning. However, new modalities, like the time-of-flight (ToF) sensors, widely explored in non-medical fields are powerful and have the potential to become a part of computer aided surgery set-up. Connection of different acquisition systems promises to provide a valuable support for operating room procedures. Therefore, the detailed analysis of the accuracy of such multi-sensors positioning systems is needed.

Methods

We present the system combining pre-operative CT series with intra-operative ToF-sensor and optical tracker point clouds. The methodology contains: optical sensor set-up and the ToF-camera calibration procedures, data pre-processing algorithms, and registration technique. The data pre-processing yields a surface, in case of CT, and point clouds for ToF-sensor and marker-driven optical tracker representation of an object of interest. An applied registration technique is based on Iterative Closest Point algorithm.

Results

The experiments validate the registration of each pair of modalities/sensors involving phantoms of four various human organs in terms of Hausdorff distance and mean absolute distance metrics. The best surface alignment was obtained for CT and optical tracker combination, whereas the worst for experiments involving ToF-camera.

Conclusion

The obtained accuracies encourage to further develop the multi-sensors systems. The presented substantive discussion concerning the system limitations and possible improvements mainly related to the depth information produced by the ToF-sensor is useful for computer aided surgery developers.

Augmented reality visualization of deformable tubular structures for surgical simulation

BACKGROUND

Surgical simulation based on augmented reality (AR), mixing the benefits of physical and virtual simulation, represents a step forward in surgical training. However, available systems are unable to update the virtual anatomy following deformations impressed on actual anatomy.

METHODS

A proof-of-concept solution is described providing AR visualization of hidden deformable tubular structures using nitinol tubes sensorized with electromagnetic sensors. This system was tested in vitro on a setup comprised of sensorized cystic, left and right hepatic, and proper hepatic arteries. In the trial session, the surgeon deformed the tubular structures with surgical forceps in 10 positions.

RESULTS

The mean, standard deviation, and maximum misalignment between virtual and real arteries were 0.35, 0.22, and 0.99 mm, respectively.

CONCLUSION

The alignment accuracy obtained demonstrates the feasibility of the approach, which can be adopted in advanced AR simulations, in particular as an aid to the identification and isolation of tubular structures. Copyright © 2015 John Wiley & Sons, Ltd.

Augmented reality partial nephrectomy: examining the current status and future perspectives

A minimal access approach to partial nephrectomy has historically been under-utilized, but is now becoming more popular with the growth of robot-assisted laparoscopy. One of the criticisms of minimal access partial nephrectomy is the loss of haptic feedback. Augmented reality operating environments are forecast to play a major enabling role in the future of minimal access partial nephrectomy by integrating enhanced visual information to supplement this loss of haptic sensation. In this article, we systematically examine the current status of augmented reality in partial nephrectomy by identifying existing research challenges and exploring future agendas for this technology to achieve wider clinical translation.

Image-Guided Abdominal Surgery and Therapy Delivery

Image-Guided Surgery has become the standard of care in intracranial neurosurgery providing more exact resections while minimizing damage to healthy tissue. Moving that process to abdominal organs presents additional challenges in the form of image segmentation, image to physical space registration, organ motion and deformation. In this paper, we present methodologies and results for addressing these challenges in two specific organs: the liver and the kidney.

Augmented reality visualization during laparoscopic radical prostatectomy

PURPOSE

We present an augmented reality (AR) navigation system that conveys virtual organ models generated from transrectal ultrasonography (TRUS) onto a real laparoscopic video during radical prostatectomy. By providing this additional information about the actual anatomy, we can support surgeons in their working decisions. This work reports the system's first in-vivo application.

MATERIALS AND METHODS

The system uses custom-developed needles with colored heads that are inserted into the prostate as soon as the organ surface is uncovered. These navigation aids are once segmented in three-dimensional (3D) TRUS data that is acquired right after the placement of the needles and then continuously tracked in the laparoscopic video images by the surgical navigation system. The navigation system traces the navigation aids in real time and computes a registration between TRUS image and laparoscopic video based on the two-dimensional-three dimensional (2D-3D) point correspondences. With this registration, the system correctly superimposes TRUS-based 3D information on an additional AR monitor placed next to the normal laparoscopic screen. Surgical navigation guidance took place until the prostate was removed from the rectal wall. Finally, the navigation aids were removed together with the specimen inside the specimen bag.

RESULTS

The initial human in-vivo application of the surgical navigation system was successful. No complications occurred, the prostate was removed together with the navigation aids, and the system supported the surgeons as intended with an AR visualization in real time. In case of tissue deformations, changes in the spatial configuration of the navigation aids are detected, which preserves the system from erroneous navigation visualization.

CONCLUSIONS

Feasibility of the navigation system was shown in the first in-vivo application. TRUS information could be superimposed via AR in real time. To show the benefit for the patient, results obtained from a larger number of trials are needed.