Performance of image guided navigation in laparoscopic liver surgery - A systematic review

Tracking and mapping in medical computer vision: A review

As computer vision algorithms increase in capability, their applications in clinical systems will become more pervasive. These applications include: diagnostics, such as colonoscopy and bronchoscopy; guiding biopsies, minimally invasive interventions, and surgery; automating instrument motion; and providing image guidance using pre-operative scans. Many of these applications depend on the specific visual nature of medical scenes and require designing algorithms to perform in this environment. In this review, we provide an update to the field of camera-based tracking and scene mapping in surgery and diagnostics in medical computer vision. We begin with describing our review process, which results in a final list of 515 papers that we cover. We then give a high-level summary of the state of the art and provide relevant background for those who need tracking and mapping for their clinical applications. After which, we review datasets provided in the field and the clinical needs that motivate their design. Then, we delve into the algorithmic side, and summarize recent developments. This summary should be especially useful for algorithm designers and to those looking to understand the capability of off-the-shelf methods. We maintain focus on algorithms for deformable environments while also reviewing the essential building blocks in rigid tracking and mapping since there is a large amount of crossover in methods. With the field summarized, we discuss the current state of the tracking and mapping methods along with needs for future algorithms, needs for quantification, and the viability of clinical applications. We then provide some research directions and questions. We conclude that new methods need to be designed or combined to support clinical applications in deformable environments, and more focus needs to be put into collecting datasets for training and evaluation.

Current Status of Robotic Hepatobiliary and Pancreatic Surgery

Laparoscopic surgery is performed worldwide and has clear economic and social benefits in terms of patient recovery time. It is used for most gastrointestinal surgical procedures, but laparoscopic surgery for more complex procedures in the esophageal, hepatobiliary, and pancreatic regions remains challenging. Minimally invasive surgery that results in accurate tumor dissection is vital in surgical oncology, and development of surgical systems and instruments plays a key role in assisting surgeons to achieve this. A notable advance in the latter half of the 1990s was the da Vinci Surgical System, which involves master-slave surgical support robots. Featuring high-resolution three-dimensional (3D) imaging with magnification capabilities and forceps with multi-joint function, anti-shake function, and motion scaling, the system compensates for the drawbacks of conventional laparoscopic surgery. It is expected to be particularly useful in the field of hepato-biliary-pancreatic surgery, which requires delicate reconstruction involving complex liver anatomy with diverse vascular and biliary systems and anastomosis of the biliary tract, pancreas, and intestines. The learning curve is said to be short, and it is hoped that robotic surgery will be standardized in the near future. There is also a need for a standardized robotic surgery training system for young surgeons that can later be adapted to a wider range of surgeries. This systematic review describes trends and future prospects for robotic surgery in the hepatobiliary-pancreatic region.

Promising Results of Associating Liver Partition and Portal Vein Ligation for Staged Hepatectomy for Perihilar Cholangiocarcinoma in a Systematic Review and Single-Arm Meta-Analysis

BACKGROUND

ALPPS popularity is increasing among surgeons worldwide and its indications are expanding to cure patients with primarily unresectable liver tumors. Few reports recommended limitations or even contraindications of ALPPS in perihilar cholangiocarcinoma (phCC). Here, we discuss the results of ALPPS in patients with phCC in a systematic review as well as a pooled data analysis.

METHODS

MEDLINE and Web of Science databases were systematically searched for relevant literature up to December 2023. All studies reporting ALPPS in the management of phCC were included. A single-arm meta-analysis of proportions was carried out to estimate the overall rate of outcomes.

RESULTS

After obtaining 207 articles from the primary search, data of 18 studies containing 112 phCC patients were included in our systematic review. Rates of major morbidity and mortality were calculated to be 43% and 22%, respectively. The meta-analysis revealed a PHLF rate of 23%. One-year disease-free survival was 65% and one-year overall survival was 69%.

CONCLUSIONS

ALPPS provides a good chance of cure for patients with phCC in comparison to alternative treatment options, but at the expense of debatable morbidity and mortality. With refinement of the surgical technique and better perioperative patient management, the results of ALPPS in patients with phCC were improved.

Ultrasound guidance in navigated liver surgery: toward deep-learning enhanced compensation of deformation and organ motion

PURPOSE

Accuracy of image-guided liver surgery is challenged by deformation of the liver during the procedure. This study aims at improving navigation accuracy by using intraoperative deep learning segmentation and nonrigid registration of hepatic vasculature from ultrasound (US) images to compensate for changes in liver position and deformation.

METHODS

This was a single-center prospective study of patients with liver metastases from any origin. Electromagnetic tracking was used to follow US and liver movement. A preoperative 3D model of the liver, including liver lesions, and hepatic and portal vasculature, was registered with the intraoperative organ position. Hepatic vasculature was segmented using a reduced 3D U-Net and registered to preoperative imaging after initial alignment followed by nonrigid registration. Accuracy was assessed as Euclidean distance between the tumor center imaged in the intraoperative US and the registered preoperative image.

RESULTS

Median target registration error (TRE) after initial alignment was 11.6 mm in 25 procedures and improved to 6.9 mm after nonrigid registration (p = 0.0076). The number of TREs above 10 mm halved from 16 to 8 after nonrigid registration. In 9 cases, registration was performed twice after failure of the first attempt. The first registration cycle was completed in median 11 min (8:00-18:45 min) and a second in 5 min (2:30-10:20 min).

CONCLUSION

This novel registration workflow using automatic vascular detection and nonrigid registration allows to accurately localize liver lesions. Further automation in the workflow is required in initial alignment and classification accuracy.

Recent advances in computerized imaging and its vital roles in liver disease diagnosis, preoperative planning, and interventional liver surgery: A review

The earliest and most accurate detection of the pathological manifestations of hepatic diseases ensures effective treatments and thus positive prognostic outcomes. In clinical settings, screening and determining the extent of a pathology are prominent factors in preparing remedial agents and administering appropriate therapeutic procedures. Moreover, in a patient undergoing liver resection, a realistic preoperative simulation of the subject-specific anatomy and physiology also plays a vital part in conducting initial assessments, making surgical decisions during the procedure, and anticipating postoperative results. Conventionally, various medical imaging modalities, e.g., computed tomography, magnetic resonance imaging, and positron emission tomography, have been employed to assist in these tasks. In fact, several standardized procedures, such as lesion detection and liver segmentation, are also incorporated into prominent commercial software packages. Thus far, most integrated software as a medical device typically involves tedious interactions from the physician, such as manual delineation and empirical adjustments, as per a given patient. With the rapid progress in digital health approaches, especially medical image analysis, a wide range of computer algorithms have been proposed to facilitate those procedures. They include pattern recognition of a liver, its periphery, and lesion, as well as pre- and postoperative simulations. Prior to clinical adoption, however, software must conform to regulatory requirements set by the governing agency, for instance, valid clinical association and analytical and clinical validation. Therefore, this paper provides a detailed account and discussion of the state-of-the-art methods for liver image analyses, visualization, and simulation in the literature. Emphasis is placed upon their concepts, algorithmic classifications, merits, limitations, clinical considerations, and future research trends.

Robotic hepatectomy: current evidence and future directions

Minimally invasive hepatectomy continues to gain popularity and acceptance for treatment of benign and malignant liver disease. Robotic hepatectomy offers potential advantages over open and conventional laparoscopic approaches. Review of the literature on robotic hepatectomy was performed. Search terms included "robotic hepatectomy" and "minimally invasive hepatectomy." Search was further customized to include articles related to robotic surgical technology. Across many parameters in liver surgery, robotic liver resection appears to have comparable outcomes with respect to laparoscopic resection. The benefits over open resection are largely related to less morbidity and faster recovery times. There is evidence that the robotic approach may have a shorter learning curve and enable more difficult resections to be performed minimally invasively. The robotic platform may have the potential to achieve superior margin status or parenchymal sparing resection in oncologic resections, but numerous obstacles remain. The robotic platform has not been applied to liver surgery to the same extent as either laparoscopic or open surgery. Robotic surgical technology will need to continue developing to deliver on its potential advantages.

Ten years of Italian mini-invasiveness: the I Go MILS registry as a tool of dissemination, characterization and networking

Purposes of this study are to evaluate the main changes that have occurred in the Italian MILS activity in the last decade in terms of indications, approaches and outcomes as reported in the national registry and to provide specific details on the main areas of development of MILS. Data from patients undergoing minimally invasive liver resections at centers included in the I Go MILS Registry from its start-up (November 2014) to March 2023 were analyzed for the purposes of this study. The registry is intention-to-treat and prospective. Global recruitment trends stratified by indication to surgery and type of approach were analysed. 7413 MILS procedures were performed across all centers (median number of procedures per center: 63). Years (2020-2023) displayed a significantly higher proportion of treated patients diagnosed with hepatocellular carcinoma (HCC) (38.2% vs. 28.9% and 33.9%, p < 0.001) and cholangiocarcinoma (6.7% vs. 6.5% and 4.2%, p < 0.001) compared to the preceding triennial periods. Additionally, technical complexity demonstrated an increased prominence in Years (2019-2023) with a significantly higher percentage of grade III cases compared to the earlier periods (39.3% vs. 21.7% and 25.6%, p < 0.001). Annual case trends focusing on laparoscopic and robotic techniques demonstrated a steadily increase in the use of these techniques for complex case mix of indications. Overall, attitude and attention to MILS approach has evolved, so that currently indications to hepatic mini-invasiveness have expanded and surgical technique has been refined: Areas mainly involved in increasing growth trends are hepatocellular carcinoma, possible applications of MILS in transplant setting, intrahepatic cholangiocarcinoma and robotic approach.

Laparoscopic Microwave Ablation: Which Technologies Improve the Results

Liver resection is the best treatment for hepatocellular carcinoma (HCC) when resectable. Unfortunately, many patients with HCC cannot undergo liver resection. Percutaneous thermoablation represents a valid alternative for inoperable neoplasms and for small HCCs, but it is not always possible to accomplish it. In cases where the percutaneous approach is not feasible (not a visible lesion or in hazardous locations), laparoscopic thermoablation may be indicated. HCC diagnosis is commonly obtained from imaging modalities, such as CT and MRI, However, the interpretation of radiological images, which have a two-dimensional appearance, during the surgical procedure and in particular during laparoscopy, can be very difficult in many cases for the surgeon who has to treat the tumor in a three-dimensional environment. In recent years, more technologies have helped surgeons to improve the results after ablative treatments. The three-dimensional reconstruction of the radiological images has allowed the surgeon to assess the exact position of the tumor both before the surgery (virtual reality) and during the surgery with immersive techniques (augmented reality). Furthermore, indocyanine green (ICG) fluorescence imaging seems to be a valid tool to enhance the precision of laparoscopic thermoablation. Finally, the association with laparoscopic ultrasound with contrast media could improve the localization and characteristics of tumor lesions. This article describes the use of hepatic three-dimensional modeling, ICG fluorescence imaging and laparoscopic ultrasound examination, convenient for improving the preoperative surgical preparation for personalized laparoscopic approach.

Feasibility of a lung airway navigation system using fiber-Bragg shape sensing and artificial intelligence for early diagnosis of lung cancer

Currently early diagnosis of malignant lesions at the periphery of lung parenchyma requires guidance of the biopsy needle catheter from the bronchoscope into the smaller peripheral airways via harmful X-ray radiation. Previously, we developed an image-guided system, iMTECH which uses electromagnetic tracking and although it increases the precision of biopsy collection and minimizes the use of harmful X-ray radiation during the interventional procedures, it only traces the tip of the biopsy catheter leaving the remaining catheter untraceable in real time and therefore increasing image registration error. To address this issue, we developed a shape sensing guidance system containing a fiber-Bragg grating (FBG) catheter and an artificial intelligence (AI) software, AIrShape to track and guide the entire biopsy instrument inside the lung airways, without radiation or electromagnetic navigation. We used a FBG fiber with one central and three peripheral cores positioned at 120° from each other, an array of 25 draw tower gratings with 1cm/3nm spacing, 2 mm grating length, Ormocer-T coating, and a total outer diameter of 0.2 mm. The FBG fiber was placed in the working channel of a custom made three-lumen catheter with a tip bending mechanism (FBG catheter). The AIrShape software determines the position of the FBG catheter by superimposing its position to the lung airway center lines using an AI algorithm. The feasibility of the FBG system was tested in an anatomically accurate lung airway model and validated visually and with the iMTECH platform. The results prove a viable shape-sensing hardware and software navigation solution for flexible medical instruments to reach the peripheral airways. During future studies, the feasibility of FBG catheter will be tested in pre-clinical animal models.

Disappearing Colorectal Liver Metastases: Do We Really Need a Ghostbuster?

The development of new systemic treatment strategies has resulted in a significant increase in the response rates of colorectal liver metastases (CRLM) in the last few years. Although the radiological response is a favorable prognostic factor, complete shrinkage of CRLM, known as disappearing liver metastases (DLM), presents a therapeutic dilemma, and proper management is still debated in the literature. In fact, DLM is not necessarily equal to cure, and when resected, pathological examination reveals in more than 80% of patients a variable percentage of the tumor as residual disease or early recurrence in situ. Moreover, while a higher incidence of intrahepatic recurrence is documented in small series when surgery is avoided, its clinical significance for long-term OS is still under investigation. In light of this, a multidisciplinary approach and, in particular, radiologists’ role is needed to assist the surgeon in the management of DLM, thanks to emerging technology and strategy. Therefore, the aim of this review is to provide an overview of the DLM phenomenon and current management.

Leberchirurgie 4.0 - OP-Planung, Volumetrie, Navigation und Virtuelle Realität

Durch die Optimierung der konservativen Behandlung, die Verbesserung der bildgebenden Verfahren und die Weiterentwicklung der Operationstechniken haben sich das operative Spektrum sowie der Maßstab für die Resektabilität in Bezug auf die Leberchirurgie in den letzten Jahrzehnten deutlich verändert.

Dank zahlreicher technischer Entwicklungen, insbesondere der 3-dimensionalen Segmentierung, kann heutzutage die präoperative Planung und die Orientierung während der Operation selbst, vor allem bei komplexen Eingriffen, unter Berücksichtigung der patientenspezifischen Anatomie erleichtert werden.

Neue Technologien wie 3-D-Druck, virtuelle und augmentierte Realität bieten zusätzliche Darstellungsmöglichkeiten für die individuelle Anatomie. Verschiedene intraoperative Navigationsmöglichkeiten sollen die präoperative Planung im Operationssaal verfügbar machen, um so die Patientensicherheit zu erhöhen.

Dieser Übersichtsartikel soll einen Überblick über den gegenwärtigen Stand der verfügbaren Technologien sowie einen Ausblick in den Operationssaal der Zukunft geben.

Generation of Synthetic Data for the Comparison of Different 3D-3D Registration Approaches in Laparoscopic Surgery

In laparoscopic surgery image-guided navigation systems could support the surgeon by providing subsurface information such as the positions of tumors and vessels. For this purpose, one option is to perform a reliable registration of preoperative 3D data and a surface patch from laparo-scopic video data. A robust and automatic 3D-3D registration pipeline for the application during laparoscopic surgery has not yet been found due to application-specific challenges. To gain a better insight, we propose a framework enabling a qualitative and quantitative comparison of different registration approaches. The introduced framework is able to evaluate 3D feature descriptors and registration algorithms by generating and modifying synthetic data from clinical examples. Different confounding factors are considered and thus the reality can be reflected in any simplified or more complex way. Two exemplary experiments with a liver model, using the RANSAC algorithm, showed an increasing registration error for a decreasing size of the surface patch size and after introducing modifications. Moreover, the registration accuracy was dependent on the position and structure of the surface patch. The framework helps to quantitatively assess and optimize the registration pipeline, and hereby suggests future software improvements even with only few clinical examples. Clinical relevance- The introduced framework permits a quantitative and comprehensive comparison of different registration approaches which forms the basis for a supportive navigation tool in laparoscopic surgery.

Real-time stereo reconstruction of intra-operative scene and registration to pre-operative 3D models for augmenting surgeons' view during RAMIS

PURPOSE

During Minimally Invasive Surgery (MIS) procedures, there exists an ever-growing/apparent need for providing computer generated visual feedback to the surgeon(s), through a visualization device. While multiple solutions have been proposed in the literature, there is limited evidence of such a system performing reliably in practice, and when it does, it is often tailored to a specific operation type. Another important aspect is regarding the usability of such systems, which typically include complicated and time-consuming steps, and often require the assistance of specialized personnel. In this study, we propose an auxiliary visualization system for surgeons, which includes streamlined process to use pre-operative data of the patient, and apply it to two different MIS cases, namely Robot-assisted Partial Nephrectomy (RAPN) and Robot-assisted Partial Lateral Meniscectomy (RaPLM).

METHODS

The visualization and processing pipeline consists of an intra-operative 3D reconstruction of the surgical area, using an optimized version of the Quasi Dense method, aimed to perform with good accuracy while maintaining real-time speed. A set of pre-processing and post-processing techniques further contribute to the result by providing a smoother and more dense point cloud. DynamicFusion is used for the registration of the pre-operative model to the intra-operative scene. Two silicon kidney phantoms and an ex-vivo porcine meniscus are used for evaluation, representing subjects for the examined surgical cases.

RESULTS

Performance is evaluated qualitatively using the two datasets. The pre-operative model of the subject is projected on top of the actual 2D image and also in 3D space. The model is superimposed on top of the actual physical structure it represents, and remains in the correct position throughout the experiments, even when abrupt camera movements are taking place. Finally, when deformation is introduced, the model is deformed as well, resembling the real subject's structure.

CONCLUSIONS

Results demonstrate and validate the use of the presented algorithms for each separate task of the pipeline. A complete methodology to provide surgeon(s) with visual information during surgery is presented. Its operation is evaluated over two different surgical scenarios, paving the way for a single visualization methodology that can adapt and perform robustly for multiple cases, with minimal effort. This article is protected by copyright. All rights reserved.

Ultrasound-based navigation for open liver surgery using active liver tracking

PURPOSE

Despite extensive preoperative imaging, intraoperative localization of liver lesions after systemic treatment can be challenging. Therefore, an image-guided navigation setup is explored that links preoperative diagnostic scans and 3D models to intraoperative ultrasound (US), enabling overlay of detailed diagnostic images on intraoperative US. Aim of this study is to assess the workflow and accuracy of such a navigation system which compensates for liver motion.

METHODS

Electromagnetic (EM) tracking was used for organ tracking and movement of the transducer. After laparotomy, a sensor was attached to the liver surface while the EM-tracked US transducer enabled image acquisition and landmark digitization. Landmarks surrounding the lesion were selected during patient-specific preoperative 3D planning and identified for registration during surgery. Endpoints were accuracy and additional times of the investigative steps. Accuracy was computed at the center of the target lesion.

RESULTS

In total, 22 navigated procedures were performed. Navigation provided useful visualization of preoperative 3D models and their overlay on US imaging. Landmark-based registration resulted in a mean fiducial registration error of 10.3 ± 4.3 mm, and a mean target registration error of 8.5 ± 4.2 mm. Navigation was available after an average of 12.7 min.

CONCLUSION

We developed a navigation method combining ultrasound with active liver tracking for organ motion compensation, with an accuracy below 10 mm. Fixation of the liver sensor near the target lesion compensates for local movement and contributes to improved reliability during navigation. This represents an important step forward in providing surgical navigation throughout the procedure.

TRIAL REGISTRATION

This study is registered in the Netherlands Trial Register (number NL7951).