Organ surface deformation measurement and analysis in open hepatic surgery: method and preliminary results from 12 clinical cases

Registration accuracy comparing different rendering techniques on local vs external virtual 3D liver model reconstruction for vascular landmark setting by intraoperative ultrasound in augmented reality navigated liver resection

PURPOSE

Augmented reality navigation in liver surgery still faces technical challenges like insufficient registration accuracy. This study compared registration accuracy between local and external virtual 3D liver models (vir3DLivers) generated with different rendering techniques and the use of the left vs right main portal vein branch (LPV vs RPV) for landmark setting. The study should further examine how registration accuracy behaves with increasing distance from the ROI.

METHODS

Retrospective registration accuracy analysis of an optical intraoperative 3D navigation system, used in 13 liver tumor patients undergoing liver resection/thermal ablation.

RESULTS

109 measurements in 13 patients were performed. Registration accuracy with local and external vir3DLivers was comparable (8.76 ± 0.9 mm vs 7.85 ± 0.9 mm; 95% CI = -0.73 to 2.55 mm; p = 0.272). Registrations via the LPV demonstrated significantly higher accuracy than via the RPV (6.2 ± 0.85 mm vs 10.41 ± 0.99 mm, 95% CI = 2.39 to 6.03 mm, p < 0.001). There was a statistically significant positive but weak correlation between the accuracy (dFeature) and the distance from the ROI (dROI) (r = 0.298; p = 0.002).

CONCLUSION

Despite basing on different rendering techniques both local and external vir3DLivers have comparable registration accuracy, while LPV-based registrations significantly outperform RPV-based ones in accuracy. Higher accuracy can be assumed within distances of up to a few centimeters around the ROI.

Comparison study of intraoperative surface acquisition methods on registration accuracy for soft-tissue surgical navigation

Purpose

To study the difference between rigid registration and nonrigid registration using two forms of digitization (contact and noncontact) in human in vivo liver surgery.

Approach

A Conoprobe device attachment and sterilization process was developed to enable prospective noncontact intraoperative acquisition of organ surface data in the operating room (OR). The noncontact Conoprobe digitization method was compared against stylus-based acquisition in the context of image-to-physical registration for image-guided surgical navigation. Data from n = 10 patients undergoing liver resection were analyzed under an Institutional Review Board-approved study at Memorial Sloan Kettering Cancer Center. Organ surface coverage of each surface acquisition method was compared. Registration accuracies resulting from the acquisition techniques were compared for (1) rigid registration method (RRM), (2) model-based nonrigid registration method (NRM) using surface data only, and (3) NRM with one subsurface feature (vena cava) from tracked intraoperative ultrasound (NRM-VC). Novel vessel centerline and tumor targets were segmented and compared to their registered preoperative counterparts for accuracy validation.

Results

Surface data coverage collected by stylus and Conoprobe were 24.6 % ± 6.4 % and 19.6 % ± 5.0 % , respectively. The average difference between stylus data and Conoprobe data using NRM was - 1.05    mm and using NRM-VC was - 1.42    mm , indicating the registrations to Conoprobe data performed worse than to stylus data with both NRM approaches. However, using the stylus and Conoprobe acquisition methods led to significant improvement of NRM-VC over RRM by average differences of 4.48 and 3.66 mm, respectively.

Conclusion

The first use of a sterile-field amenable Conoprobe surface acquisition strategy in the OR is reported for open liver surgery. Under clinical conditions, the nonrigid registration significantly outperformed standard-of-care rigid registration, and acquisition by contact-based stylus and noncontact-based Conoprobe produced similar registration results. The accuracy benefits of noncontact surface acquisition with a Conoprobe are likely obscured by inferior data coverage and intrinsic noise within acquisition systems.

The Image-to-Physical Liver Registration Sparse Data Challenge: comparison of state-of-the-art using a common dataset

Purpose

Computational methods for image-to-physical registration during surgical guidance frequently rely on sparse point clouds obtained over a limited region of the organ surface. However, soft tissue deformations complicate the ability to accurately infer anatomical alignments from sparse descriptors of the organ surface. The Image-to-Physical Liver Registration Sparse Data Challenge introduced at SPIE Medical Imaging 2019 seeks to characterize the performance of sparse data registration methods on a common dataset to benchmark and identify effective tactics and limitations that will continue to inform the evolution of image-to-physical registration algorithms.

Approach

Three rigid and five deformable registration methods were contributed to the challenge. The deformable approaches consisted of two deep learning and three biomechanical boundary condition reconstruction methods. These algorithms were compared on a common dataset of 112 registration scenarios derived from a tissue-mimicking phantom with 159 subsurface validation targets. Target registration errors (TRE) were evaluated under varying conditions of data extent, target location, and measurement noise. Jacobian determinants and strain magnitudes were compared to assess displacement field consistency.

Results

Rigid registration algorithms produced significant differences in TRE ranging from 3.8 ± 2.4    mm to 7.7 ± 4.5    mm , depending on the choice of technique. Two biomechanical methods yielded TRE of 3.1 ± 1.8    mm and 3.3 ± 1.9    mm , which outperformed optimal rigid registration of targets. These methods demonstrated good performance under varying degrees of surface data coverage and across all anatomical segments of the liver. Deep learning methods exhibited TRE ranging from 4.3 ± 3.3    mm to 7.6 ± 5.3    mm but are likely to improve with continued development. TRE was weakly correlated among methods, with greatest agreement and field consistency observed among the biomechanical approaches.

Conclusions

The choice of registration algorithm significantly impacts registration accuracy and variability of deformation fields. Among current sparse data driven image-to-physical registration algorithms, biomechanical simulations that incorporate task-specific insight into boundary conditions seem to offer best performance.

A Real-Time 3D Laparoscopic Imaging System: Design, Method, and Validation

OBJECTIVE

This paper aims to propose a 3D laparoscopic imaging system that can realize dense 3D reconstruction in real time.

METHODS

Based on the active stereo technique which yields high-density, accurate and robust 3D reconstruction by combining structured light and stereo vision, we design a laparoscopic system consisting of two image feedback channels and one pattern projection channel. Remote high-speed image acquisition and pattern generation lay the foundation for the real-time dense 3D surface reconstruction and enable the miniaturization of the laparoscopic probe. To enhance the reconstruction efficiency and accuracy, we propose a novel active stereo method by which the dense 3D point cloud is obtained using only five patterns, while most existing multiple-shot structured light techniques require [Formula: see text] patterns. In our method, dual-frequency phase-shifting fringes are utilized to uniquely encode the pixels of the measured targets, and a dual-codeword matching scheme is developed to simplify the matching procedure and achieve high-precision reconstruction.

RESULTS

Compared with the existing structured light techniques, the proposed method shows better real-time efficiency and accuracy in both quantitative and qualitative ways. Ex-vivo experiments demonstrate the robustness of the proposed method to different biological organs and the effectiveness to lesions and deformations of the organs. Feasibility of the proposed system for real-time dense 3D reconstruction is verified in dynamic experiments. According to the experimental results, the system acquires 3D point clouds with a speed of 12 frames per second. Each frame contains more than 40,000 points, and the average errors tested on standard objects are less than 0.2 mm.

SIGNIFICANCE

This paper provides a new real-time dense 3D reconstruction method for 3D laparoscopic imaging. The established prototype system has shown good performance in reconstructing surface of biological tissues.

Ultrasound Based Planning and Navigation for Non-Anatomical Liver Resections – An Ex-Vivo Study

Goal: Non-anatomical resections of liver tumors can be very challenging as the surgeon cannot use anatomical landmarks on the liver surface or in the ultrasound image for guidance. This makes it difficult to achieve negative resection margins (R0) and still preserve as much healthy liver tissue as possible. Even though image-guided surgery systems have been introduced to overcome this challenge, they are still rarely used due to their inaccuracy, time-effort and complexity in usage and setup. Methods: We have developed a novel approach, which allows us to create an intra-operative resection plan using navigated ultrasound. First, the surface is scanned using a navigated ultrasound, followed by tumor segmentation on a midsection ultrasound image. Based on this information, the navigation system calculates an optimal resection strategy and displays it along with the tracked surgical instruments. In this study, this approach was evaluated by three experienced hepatobiliary surgeons on ex-vivo porcine models. Results: Using this technique, an R0 resection could be achieved in 22 out of 23 (95.7% R0 resection rate) cases with a median resection margin of 5.9 mm (IQR 3.5–7.7 mm). The resection margin between operators 1, 2 and 3 was 7.8 mm, 4.15 mm and 5.1 mm respectively (p = 0.054). Conclusions: This approach could represent a useful tool for intra-operative guidance in non-anatomical resection alongside conventional ultrasound guidance. However, instructions and training are essential especially if the operator has not used an image-guidance system before.

Multiphysics modeling toward enhanced guidance in hepatic microwave ablation: a preliminary framework

We compare a surface-driven, model-based deformation correction method to a clinically relevant rigid registration approach within the application of image-guided microwave ablation for the purpose of demonstrating improved localization and antenna placement in a deformable hepatic phantom. Furthermore, we present preliminary computational modeling of microwave ablation integrated within the navigational environment to lay the groundwork for a more comprehensive procedural planning and guidance framework. To achieve this, we employ a simple, retrospective model of microwave ablation after registration, which allows a preliminary evaluation of the combined therapeutic and navigational framework. When driving registrations with full organ surface data (i.e., as could be available in a percutaneous procedure suite), the deformation correction method improved average ablation antenna registration error by 58.9% compared to rigid registration (i.e., 2.5 ± 1.1    mm , 5.6 ± 2.3    mm of average target error for corrected and rigid registration, respectively) and on average improved volumetric overlap between the modeled and ground-truth ablation zones from 67.0 ± 11.8 % to 85.6 ± 5.0 % for rigid and corrected, respectively. Furthermore, when using sparse-surface data (i.e., as is available in an open surgical procedure), the deformation correction improved registration error by 38.3% and volumetric overlap from 64.8 ± 12.4 % to 77.1 ± 8.0 % for rigid and corrected, respectively. We demonstrate, in an initial phantom experiment, enhanced navigation in image-guided hepatic ablation procedures and identify a clear multiphysics pathway toward a more comprehensive thermal dose planning and deformation-corrected guidance framework.

Characterization and correction of intraoperative soft tissue deformation in image-guided laparoscopic liver surgery

Laparoscopic liver surgery is challenging to perform due to a compromised ability of the surgeon to localize subsurface anatomy in the constrained environment. While image guidance has the potential to address this barrier, intraoperative factors, such as insufflation and variable degrees of organ mobilization from supporting ligaments, may generate substantial deformation. The severity of laparoscopic deformation in humans has not been characterized, and current laparoscopic correction methods do not account for the mechanics of how intraoperative deformation is applied to the liver. We first measure the degree of laparoscopic deformation at two insufflation pressures over the course of laparoscopic-to-open conversion in 25 patients. With this clinical data alongside a mock laparoscopic phantom setup, we report a biomechanical correction approach that leverages anatomically load-bearing support surfaces from ligament attachments to iteratively reconstruct and account for intraoperative deformations. Laparoscopic deformations were significantly larger than deformations associated with open surgery, and our correction approach yielded subsurface target error of [Formula: see text] and surface error of [Formula: see text] using only sparse surface data with realistic surgical extent. Laparoscopic surface data extents were examined and found to impact registration accuracy. Finally, we demonstrate viability of the correction method with clinical data.

Deformation correction for image guided liver surgery: An intraoperative fidelity assessment

BACKGROUND

Although systems of 3-dimensional image-guided surgery are a valuable adjunct across numerous procedures, differences in organ shape between that reflected in the preoperative image data and the intraoperative state can compromise the fidelity of such guidance based on the image. In this work, we assessed in real time a novel, 3-dimensional image-guided operation platform that incorporates soft tissue deformation.

METHODS

A series of 125 alignment evaluations were performed across 20 patients. During the operation, the surgeon assessed the liver by swabbing an optically tracked stylus over the liver surface and viewing the image-guided operation display. Each patient had approximately 6 intraoperative comparative evaluations. For each assessment, 1 of only 2 types of alignments were considered: conventional rigid and novel deformable. The series of alignment types used was randomized and blinded to the surgeon. The surgeon provided a rating, R, from -3 to +3 for each display compared with the previous display, whereby a negative rating indicated degradation in fidelity and a positive rating an improvement.

RESULTS

A statistical analysis of the series of rating data by the clinician indicated that the surgeons were able to perceive an improvement (defined as a R > 1) of the model-based registration over the rigid registration (P = .01) as well as a degradation (defined as R < -1) when the rigid registration was compared with the novel deformable guidance information (P = .03).

CONCLUSION

This study provides evidence of the benefit of deformation correction in providing an accurate location for the liver for use in image-guided surgery systems.

Improving Registration Robustness for Image-Guided Liver Surgery in a Novel Human-to-Phantom Data Framework

In open image-guided liver surgery (IGLS), a sparse representation of the intraoperative organ surface can be acquired to drive image-to-physical registration. We hypothesize that uncharacterized error induced by variation in the collection patterns of organ surface data limits the accuracy and robustness of an IGLS registration. Clinical validation of such registration methods is challenged due to the difficulty in obtaining data representative of the true state of organ deformation. We propose a novel human-to-phantom validation framework that transforms surface collection patterns from in vivo IGLS procedures (n = 13) onto a well-characterized hepatic deformation phantom for the purpose of validating surface-driven, volumetric nonrigid registration methods. An important feature of the approach is that it centers on combining workflow-realistic data acquisition and surgical deformations that are appropriate in behavior and magnitude. Using the approach, we investigate volumetric target registration error (TRE) with both current rigid IGLS and our improved nonrigid registration methods. Additionally, we introduce a spatial data resampling approach to mitigate the workflow-sensitive sampling problem. Using our human-to-phantom approach, TRE after routine rigid registration was 10.9 ± 0.6 mm with a signed closest point distance associated with residual surface fit in the range of ±10 mm, highly representative of open liver resections. After applying our novel resampling strategy and improved deformation correction method, TRE was reduced by 51%, i.e., a TRE of 5.3 ± 0.5 mm. This paper reported herein realizes a novel tractable approach for the validation of image-to-physical registration methods and demonstrates promising results for our correction method.

Current technical aspects of oncological hepatic surgery

BACKGROUND

With the recent advances in oncological hepatic surgery, major liver resections became more widely utilized procedures. The era of modern hepatic surgery witnessed improvements in patients care in preoperative, intraoperative and postoperative aspects. This significantly improved surgical outcomes regarding morbidity and mortality. This review article focuses on the recent advances in oncological hepatic surgery.

DATA SOURCES

This review includes only data from peer-reviewed articles and journals. PubMed database was utilized as the primary source of the supporting literature to this review article on the latest advances in oncological hepatic surgery. Comprehensive and high sensitivity search strategies were performed to search related studies exhaustively up till June 2016. We critically and independently assessed over 50 recent publications written on this topic according to the selection criteria and quality assessment standard. We paid particular attention to the studies published in high impact journals that address the use of the surgical techniques mentioned in the articles in well-known institutions.

RESULTS

Among all utilized approaches aiming at the preoperative assessment of the liver function, Child-Turcotte-Pugh classification remains the most reliable tool correlating with survival outcome. Although the primary radiological tools including ultrasonography, computed tomography and magnetic resonance imaging remain on top of the menu of tests utilized in assessment of focal hepatic lesions, intraoperative ultrasonography projects to be a powerful additional tool in terms of sensitivity and specificity compared to the other conventional techniques in assessment of the liver in the operative setting, a procedure that can change the surgical strategy in 27.2% of the cases and consequently improve the oncological surgical outcome. In addition to the conventional surgical techniques of liver resection and portal vein embolization, associating liver partition and portal vein ligation for staged hepatectomy "ALPPS" projects to be an alternative option in patients with marginally resectable tumors with an inadequate size of future liver remnant with an accepted surgical oncological outcome.

CONCLUSIONS

Considering the clinicopathological nature of hepatic lesions, the comprehensive assessment and proper choice of the liver resection technique in highly selected patients is associated with improved surgical oncological outcome. Patients with underlying marginal future liver remnant volumes can now safely benefit from a wider range of surgical intervention, a breakthrough that significantly improved morbidity and mortality in this group of patients.

Intraoperative image-guided navigation system: development and applicability in 65 patients undergoing liver surgery

BACKGROUND

Image-guided systems have recently been introduced for their application in liver surgery. We aimed to identify and propose suitable indications for image-guided navigation systems in the domain of open oncologic liver surgery and, more specifically, in the setting of liver resection with and without microwave ablation.

METHOD

Retrospective analysis was conducted in patients undergoing liver resection with and without microwave ablation using an intraoperative image-guided stereotactic system during three stages of technological development (accuracy: 8.4 ± 4.4 mm in phase I and 8.4 ± 6.5 mm in phase II versus 4.5 ± 3.6 mm in phase III). It was evaluated, in which indications image-guided surgery was used according to the different stages of technical development.

RESULTS

Between 2009 and 2013, 65 patients underwent image-guided surgical treatment, resection alone (n = 38), ablation alone (n = 11), or a combination thereof (n = 16). With increasing accuracy of the system, image guidance was progressively used for atypical resections and combined microwave ablation and resection instead of formal liver resection (p < 0.0001).

CONCLUSION

Clinical application of image guidance is feasible, while its efficacy is subject to accuracy. The concept of image guidance has been shown to be increasingly efficient for selected indications in liver surgery. While accuracy of available technology is increasing pertaining to technological advancements, more and more previously untreatable scenarios such as multiple small, bilobar lesions and so-called vanishing lesions come within reach.

Evaluation of model-based deformation correction in image-guided liver surgery via tracked intraoperative ultrasound

Soft-tissue deformation represents a significant error source in current surgical navigation systems used for open hepatic procedures. While numerous algorithms have been proposed to rectify the tissue deformation that is encountered during open liver surgery, clinical validation of the proposed methods has been limited to surface-based metrics, and subsurface validation has largely been performed via phantom experiments. The proposed method involves the analysis of two deformation-correction algorithms for open hepatic image-guided surgery systems via subsurface targets digitized with tracked intraoperative ultrasound (iUS). Intraoperative surface digitizations were acquired via a laser range scanner and an optically tracked stylus for the purposes of computing the physical-to-image space registration and for use in retrospective deformation-correction algorithms. Upon completion of surface digitization, the organ was interrogated with a tracked iUS transducer where the iUS images and corresponding tracked locations were recorded. Mean closest-point distances between the feature contours delineated in the iUS images and corresponding three-dimensional anatomical model generated from preoperative tomograms were computed to quantify the extent to which the deformation-correction algorithms improved registration accuracy. The results for six patients, including eight anatomical targets, indicate that deformation correction can facilitate reduction in target error of [Formula: see text].

A Mechanics-Based Nonrigid Registration Method for Liver Surgery Using Sparse Intraoperative Data

In open abdominal image-guided liver surgery, sparse measurements of the organ surface can be taken intraoperatively via a laser-range scanning device or a tracked stylus with relatively little impact on surgical workflow. We propose a novel nonrigid registration method which uses sparse surface data to reconstruct a mapping between the preoperative CT volume and the intraoperative patient space. The mapping is generated using a tissue mechanics model subject to boundary conditions consistent with surgical supportive packing during liver resection therapy. Our approach iteratively chooses parameters which define these boundary conditions such that the deformed tissue model best fits the intraoperative surface data. Using two liver phantoms, we gathered a total of five deformation datasets with conditions comparable to open surgery. The proposed nonrigid method achieved a mean target registration error (TRE) of 3.3 mm for targets dispersed throughout the phantom volume, using a limited region of surface data to drive the nonrigid registration algorithm, while rigid registration resulted in a mean TRE of 9.5 mm. In addition, we studied the effect of surface data extent, the inclusion of subsurface data, the trade-offs of using a nonlinear tissue model, robustness to rigid misalignments, and the feasibility in five clinical datasets.

Automated scan prescription for MR imaging of deformed and normal livers

PURPOSE

We propose an automated scan prescription to assess normal and deformed livers and demonstrate its efficacy in normal volunteers and in simulated deformed livers.

METHODS

Our automated scan prescription can be used to identify the upper and lower edges of the liver enables in commonly used axial slice positioning. The liver's upper edge is detected by template matching and finally identified by applying an active shape model to a sagittal projection image. The lower edge is detected using a maximum a posteriori (MAP) probability estimate that utilizes statistical information from a region of interest (ROI) placed in the liver. This places no restraints on liver shape and is therefore effective in assessing a deformed liver. Following institutional review and approval, we tested our method in 45 healthy volunteers. We also used clinical information to simulate deformed livers and tested our method with those datasets offline.

RESULTS

We could detect the upper edges within an error range of -3 to 6 mm, even without intensity correction for normal volunteers. Similar detection of the lower edges with maximum 21-mm and 7.84-mm standard deviation for normal volunteers confirmed the superior efficacy of our modified approach for deformed livers to that using our previous method. Clinical use required approximately 10 s' computational time on a Core i5 laptop with 2-GB memory.

CONCLUSION

We propose a method for automated scan prescription in magnetic resonance (MR) imaging of the liver and demonstrate the efficacy of our algorithm for evaluating deformed livers within a practical computation time. Detection of liver edges of various shapes by applying the MAP estimate combined with statistical information from the ROI demonstrated the potential clinical utility of this technique.

Patient-specific treatment planning of electrochemotherapy: procedure design and possible pitfalls

Electrochemotherapy uses electroporation for enhancing chemotherapy. Electrochemotherapy can be performed using standard operating procedures with predefined electrode geometries, or using patient-specific treatment planning to predict electroporation. The latter relies on realistic computer models to provide optimal results (i.e. electric field distribution as well as electrodes' position and number) and is suitable for treatment of deep-seated tumors. Since treatment planning for deep-seated tumors has been used in radiotherapy, we expose parallelisms with radiotherapy in order to establish the procedure for electrochemotherapy of deep-seated tumors. We partitioned electrochemotherapy in the following phases: the mathematical model of electroporation, treatment planning, set-up verification, treatment delivery and monitoring, and response assessment. We developed a conceptual treatment planning software that incorporates mathematical models of electroporation. Preprocessing and segmentation of the patient's medical images are performed, and a 3D model is constructed which allows placement of electrodes and implementation of the mathematical model of electroporation. We demonstrated the feasibility of electrochemotherapy of deep-seated tumors treatment planning within a clinical study where treatment planning contributed to the effective electrochemotherapy treatment of deep-seated colorectal metastases in the liver. The described procedure can provide medical practitioners with information on using electrochemotherapy in the clinical setting. The main aims of this paper are: 1) to present the procedure for treating deep-seated tumors by electrochemotherapy based on patient-specific treatment planning, and 2) to identify gaps in knowledge and possible pitfalls of such procedure.