Evaluation of model-based deformation correction in image-guided liver surgery via tracked intraoperative ultrasound

Comparison study of intraoperative surface acquisition methods on registration accuracy for soft-tissue surgical navigation

Purpose

To study the difference between rigid registration and nonrigid registration using two forms of digitization (contact and noncontact) in human in vivo liver surgery.

Approach

A Conoprobe device attachment and sterilization process was developed to enable prospective noncontact intraoperative acquisition of organ surface data in the operating room (OR). The noncontact Conoprobe digitization method was compared against stylus-based acquisition in the context of image-to-physical registration for image-guided surgical navigation. Data from n = 10 patients undergoing liver resection were analyzed under an Institutional Review Board-approved study at Memorial Sloan Kettering Cancer Center. Organ surface coverage of each surface acquisition method was compared. Registration accuracies resulting from the acquisition techniques were compared for (1) rigid registration method (RRM), (2) model-based nonrigid registration method (NRM) using surface data only, and (3) NRM with one subsurface feature (vena cava) from tracked intraoperative ultrasound (NRM-VC). Novel vessel centerline and tumor targets were segmented and compared to their registered preoperative counterparts for accuracy validation.

Results

Surface data coverage collected by stylus and Conoprobe were 24.6 % ± 6.4 % and 19.6 % ± 5.0 % , respectively. The average difference between stylus data and Conoprobe data using NRM was - 1.05    mm and using NRM-VC was - 1.42    mm , indicating the registrations to Conoprobe data performed worse than to stylus data with both NRM approaches. However, using the stylus and Conoprobe acquisition methods led to significant improvement of NRM-VC over RRM by average differences of 4.48 and 3.66 mm, respectively.

Conclusion

The first use of a sterile-field amenable Conoprobe surface acquisition strategy in the OR is reported for open liver surgery. Under clinical conditions, the nonrigid registration significantly outperformed standard-of-care rigid registration, and acquisition by contact-based stylus and noncontact-based Conoprobe produced similar registration results. The accuracy benefits of noncontact surface acquisition with a Conoprobe are likely obscured by inferior data coverage and intrinsic noise within acquisition systems.

The Image-to-Physical Liver Registration Sparse Data Challenge: comparison of state-of-the-art using a common dataset

Purpose

Computational methods for image-to-physical registration during surgical guidance frequently rely on sparse point clouds obtained over a limited region of the organ surface. However, soft tissue deformations complicate the ability to accurately infer anatomical alignments from sparse descriptors of the organ surface. The Image-to-Physical Liver Registration Sparse Data Challenge introduced at SPIE Medical Imaging 2019 seeks to characterize the performance of sparse data registration methods on a common dataset to benchmark and identify effective tactics and limitations that will continue to inform the evolution of image-to-physical registration algorithms.

Approach

Three rigid and five deformable registration methods were contributed to the challenge. The deformable approaches consisted of two deep learning and three biomechanical boundary condition reconstruction methods. These algorithms were compared on a common dataset of 112 registration scenarios derived from a tissue-mimicking phantom with 159 subsurface validation targets. Target registration errors (TRE) were evaluated under varying conditions of data extent, target location, and measurement noise. Jacobian determinants and strain magnitudes were compared to assess displacement field consistency.

Results

Rigid registration algorithms produced significant differences in TRE ranging from 3.8 ± 2.4    mm to 7.7 ± 4.5    mm , depending on the choice of technique. Two biomechanical methods yielded TRE of 3.1 ± 1.8    mm and 3.3 ± 1.9    mm , which outperformed optimal rigid registration of targets. These methods demonstrated good performance under varying degrees of surface data coverage and across all anatomical segments of the liver. Deep learning methods exhibited TRE ranging from 4.3 ± 3.3    mm to 7.6 ± 5.3    mm but are likely to improve with continued development. TRE was weakly correlated among methods, with greatest agreement and field consistency observed among the biomechanical approaches.

Conclusions

The choice of registration algorithm significantly impacts registration accuracy and variability of deformation fields. Among current sparse data driven image-to-physical registration algorithms, biomechanical simulations that incorporate task-specific insight into boundary conditions seem to offer best performance.

Intraoperative Correction of Liver Deformation Using Sparse Surface and Vascular Features via Linearized Iterative Boundary Reconstruction

During image guided liver surgery, soft tissue deformation can cause considerable error when attempting to achieve accurate localization of the surgical anatomy through image-to-physical registration. In this paper, a linearized iterative boundary reconstruction technique is proposed to account for these deformations. The approach leverages a superposed formulation of boundary conditions to rapidly and accurately estimate the deformation applied to a preoperative model of the organ given sparse intraoperative data of surface and subsurface features. With this method, tracked intraoperative ultrasound (iUS) is investigated as a potential data source for augmenting registration accuracy beyond the capacity of conventional organ surface registration. In an expansive simulated dataset, features including vessel contours, vessel centerlines, and the posterior liver surface are extracted from iUS planes. Registration accuracy is compared across increasing data density to establish how iUS can be best employed to improve target registration error (TRE). From a baseline average TRE of 11.4 ± 2.2 mm using sparse surface data only, incorporating additional sparse features from three iUS planes improved average TRE to 6.4 ± 1.0 mm. Furthermore, increasing the sparse coverage to 16 tracked iUS planes improved average TRE to 3.9 ± 0.7 mm, exceeding the accuracy of registration based on complete surface data available with more cumbersome intraoperative CT without contrast. Additionally, the approach was applied to three clinical cases where on average error improved 67% over rigid registration and 56% over deformable surface registration when incorporating additional features from one independent tracked iUS plane.

Ultrasound Based Planning and Navigation for Non-Anatomical Liver Resections – An Ex-Vivo Study

Goal: Non-anatomical resections of liver tumors can be very challenging as the surgeon cannot use anatomical landmarks on the liver surface or in the ultrasound image for guidance. This makes it difficult to achieve negative resection margins (R0) and still preserve as much healthy liver tissue as possible. Even though image-guided surgery systems have been introduced to overcome this challenge, they are still rarely used due to their inaccuracy, time-effort and complexity in usage and setup. Methods: We have developed a novel approach, which allows us to create an intra-operative resection plan using navigated ultrasound. First, the surface is scanned using a navigated ultrasound, followed by tumor segmentation on a midsection ultrasound image. Based on this information, the navigation system calculates an optimal resection strategy and displays it along with the tracked surgical instruments. In this study, this approach was evaluated by three experienced hepatobiliary surgeons on ex-vivo porcine models. Results: Using this technique, an R0 resection could be achieved in 22 out of 23 (95.7% R0 resection rate) cases with a median resection margin of 5.9 mm (IQR 3.5–7.7 mm). The resection margin between operators 1, 2 and 3 was 7.8 mm, 4.15 mm and 5.1 mm respectively (p = 0.054). Conclusions: This approach could represent a useful tool for intra-operative guidance in non-anatomical resection alongside conventional ultrasound guidance. However, instructions and training are essential especially if the operator has not used an image-guidance system before.

[Navigated liver surgery : Current state and importance in the future]

The preoperative computer-assisted resection planning is the basis for every navigation. Thanks to modern algorithms, the prerequisites have been created to carry out a virtual resection planning and a risk analysis. Thus, individual segment resections can be precisely planned in any conceivable combination. The transfer of planning information and resection suggestions to the operating theater is still problematic. The so-called stereotactic liver navigation supports the exact intraoperative implementation of the planned resection strategy and provides the surgeon with real-time three-dimensional information on resection margins and critical structures during the resection. This is made possible by a surgical navigation system that measures the position of surgical instruments and then presents them together with the preoperative surgical planning data. Although surgical navigation systems have been indispensable in neurosurgery and spinal surgery for many years, these procedures have not yet become established as standard in liver surgery. This is mainly due to the technical challenge of navigating a moving organ. As the liver is constantly moving and deforming during surgery due to respiration and surgical manipulation, the surgical navigation system must be able to measure these alterations in order to adapt the preoperative navigation data to the current situation. Despite these advances, further developments are required until navigated liver resection enters clinical routine; however, it is already clear that laparoscopic liver surgery and robotic surgery will benefit most from navigation technology.

Multiphysics modeling toward enhanced guidance in hepatic microwave ablation: a preliminary framework

We compare a surface-driven, model-based deformation correction method to a clinically relevant rigid registration approach within the application of image-guided microwave ablation for the purpose of demonstrating improved localization and antenna placement in a deformable hepatic phantom. Furthermore, we present preliminary computational modeling of microwave ablation integrated within the navigational environment to lay the groundwork for a more comprehensive procedural planning and guidance framework. To achieve this, we employ a simple, retrospective model of microwave ablation after registration, which allows a preliminary evaluation of the combined therapeutic and navigational framework. When driving registrations with full organ surface data (i.e., as could be available in a percutaneous procedure suite), the deformation correction method improved average ablation antenna registration error by 58.9% compared to rigid registration (i.e., 2.5 ± 1.1    mm , 5.6 ± 2.3    mm of average target error for corrected and rigid registration, respectively) and on average improved volumetric overlap between the modeled and ground-truth ablation zones from 67.0 ± 11.8 % to 85.6 ± 5.0 % for rigid and corrected, respectively. Furthermore, when using sparse-surface data (i.e., as is available in an open surgical procedure), the deformation correction improved registration error by 38.3% and volumetric overlap from 64.8 ± 12.4 % to 77.1 ± 8.0 % for rigid and corrected, respectively. We demonstrate, in an initial phantom experiment, enhanced navigation in image-guided hepatic ablation procedures and identify a clear multiphysics pathway toward a more comprehensive thermal dose planning and deformation-corrected guidance framework.

Signal dropout correction-based ultrasound segmentation for diastolic mitral valve modeling

Three-dimensional ultrasound segmentation of mitral valve (MV) at diastole is helpful for duplicating geometry and pathology in a patient-specific dynamic phantom. The major challenge is the signal dropout at leaflet regions in transesophageal echocardiography image data. Conventional segmentation approaches suffer from missing sonographic data leading to inaccurate MV modeling at leaflet regions. This paper proposes a signal dropout correction-based ultrasound segmentation method for diastolic MV modeling. The proposed method combines signal dropout correction, image fusion, continuous max-flow segmentation, and active contour segmentation techniques. The signal dropout correction approach is developed to recover the missing segmentation information. Once the signal dropout regions of TEE image data are recovered, the MV model can be accurately duplicated. Compared with other methods in current literature, the proposed algorithm exhibits lower computational cost. The experimental results show that the proposed algorithm gives competitive results for diastolic MV modeling compared with conventional segmentation algorithms, evaluated in terms of accuracy and efficiency.

Image registration assessment in radiotherapy image guidance based on control chart monitoring

Image guidance with cone beam computed tomography in radiotherapy can guarantee the precision and accuracy of patient positioning prior to treatment delivery. During the image guidance process, operators need to take great effort to evaluate the image guidance quality before correcting a patient's position. This work proposes an image registration assessment method based on control chart monitoring to reduce the effort taken by the operator. According to the control chart plotted by daily registration scores of each patient, the proposed method can quickly detect both alignment errors and image quality inconsistency. Therefore, the proposed method can provide a clear guideline for the operators to identify unacceptable image quality and unacceptable image registration with minimal effort. Experimental results demonstrate that by using control charts from a clinical database of 10 patients undergoing prostate radiotherapy, the proposed method can quickly identify out-of-control signals and find special cause of out-of-control registration events.

Toward quantitative quasistatic elastography with a gravity-induced deformation source for image-guided breast surgery

Biomechanical breast models have been employed for applications in image registration and diagnostic analysis, breast augmentation simulation, and for surgical and biopsy guidance. Accurate applications of stress-strain relationships of tissue within the breast can improve the accuracy of biomechanical models that attempt to simulate breast deformations. Reported stiffness values for adipose, glandular, and cancerous tissue types vary greatly. Variations in reported stiffness properties have been attributed to differences in testing methodologies and assumptions, measurement errors, and natural interpatient differences in tissue elasticity. Therefore, the ability to determine patient-specific in vivo breast tissue properties would be advantageous for these procedural applications. While some in vivo elastography methods are not quantitative and others do not measure material properties under deformation conditions that are appropriate to the application of concern, in this study, we developed an elasticity estimation method that is performed using deformations representative of supine therapeutic procedures. More specifically, reconstruction of mechanical properties appropriate for the standard-of-care supine lumpectomy was performed by iteratively fitting two anatomical images before and after deformations taking place in the supine breast configuration. The method proposed is workflow-friendly, quantitative, and uses a noncontact, gravity-induced deformation source.

Fast elastic registration of soft tissues under large deformations

A fast and accurate fusion of intra-operative images with a pre-operative data is a key component of computer-aided interventions which aim at improving the outcomes of the intervention while reducing the patient's discomfort. In this paper, we focus on the problematic of the intra-operative navigation during abdominal surgery, which requires an accurate registration of tissues undergoing large deformations. Such a scenario occurs in the case of partial hepatectomy: to facilitate the access to the pathology, e.g. a tumor located in the posterior part of the right lobe, the surgery is performed on a patient in lateral position. Due to the change in patient's position, the resection plan based on the pre-operative CT scan acquired in the supine position must be updated to account for the deformations. We suppose that an imaging modality, such as the cone-beam CT, provides the information about the intra-operative shape of an organ, however, due to the reduced radiation dose and contrast, the actual locations of the internal structures necessary to update the planning are not available. To this end, we propose a method allowing for fast registration of the pre-operative data represented by a detailed 3D model of the liver and its internal structure and the actual configuration given by the organ surface extracted from the intra-operative image. The algorithm behind the method combines the iterative closest point technique with a biomechanical model based on a co-rotational formulation of linear elasticity which accounts for large deformations of the tissue. The performance, robustness and accuracy of the method is quantitatively assessed on a control semi-synthetic dataset with known ground truth and a real dataset composed of nine pairs of abdominal CT scans acquired in supine and flank positions. It is shown that the proposed surface-matching method is capable of reducing the target registration error evaluated of the internal structures of the organ from more than 40 mm to less then 10 mm. Moreover, the control data is used to demonstrate the compatibility of the method with intra-operative clinical scenario, while the real datasets are utilized to study the impact of parametrization on the accuracy of the method. The method is also compared to a state-of-the art intensity-based registration technique in terms of accuracy and performance.

Characterization and correction of intraoperative soft tissue deformation in image-guided laparoscopic liver surgery

Laparoscopic liver surgery is challenging to perform due to a compromised ability of the surgeon to localize subsurface anatomy in the constrained environment. While image guidance has the potential to address this barrier, intraoperative factors, such as insufflation and variable degrees of organ mobilization from supporting ligaments, may generate substantial deformation. The severity of laparoscopic deformation in humans has not been characterized, and current laparoscopic correction methods do not account for the mechanics of how intraoperative deformation is applied to the liver. We first measure the degree of laparoscopic deformation at two insufflation pressures over the course of laparoscopic-to-open conversion in 25 patients. With this clinical data alongside a mock laparoscopic phantom setup, we report a biomechanical correction approach that leverages anatomically load-bearing support surfaces from ligament attachments to iteratively reconstruct and account for intraoperative deformations. Laparoscopic deformations were significantly larger than deformations associated with open surgery, and our correction approach yielded subsurface target error of [Formula: see text] and surface error of [Formula: see text] using only sparse surface data with realistic surgical extent. Laparoscopic surface data extents were examined and found to impact registration accuracy. Finally, we demonstrate viability of the correction method with clinical data.

Deformation correction for image guided liver surgery: An intraoperative fidelity assessment

BACKGROUND

Although systems of 3-dimensional image-guided surgery are a valuable adjunct across numerous procedures, differences in organ shape between that reflected in the preoperative image data and the intraoperative state can compromise the fidelity of such guidance based on the image. In this work, we assessed in real time a novel, 3-dimensional image-guided operation platform that incorporates soft tissue deformation.

METHODS

A series of 125 alignment evaluations were performed across 20 patients. During the operation, the surgeon assessed the liver by swabbing an optically tracked stylus over the liver surface and viewing the image-guided operation display. Each patient had approximately 6 intraoperative comparative evaluations. For each assessment, 1 of only 2 types of alignments were considered: conventional rigid and novel deformable. The series of alignment types used was randomized and blinded to the surgeon. The surgeon provided a rating, R, from -3 to +3 for each display compared with the previous display, whereby a negative rating indicated degradation in fidelity and a positive rating an improvement.

RESULTS

A statistical analysis of the series of rating data by the clinician indicated that the surgeons were able to perceive an improvement (defined as a R > 1) of the model-based registration over the rigid registration (P = .01) as well as a degradation (defined as R < -1) when the rigid registration was compared with the novel deformable guidance information (P = .03).

CONCLUSION

This study provides evidence of the benefit of deformation correction in providing an accurate location for the liver for use in image-guided surgery systems.

Improving Registration Robustness for Image-Guided Liver Surgery in a Novel Human-to-Phantom Data Framework

In open image-guided liver surgery (IGLS), a sparse representation of the intraoperative organ surface can be acquired to drive image-to-physical registration. We hypothesize that uncharacterized error induced by variation in the collection patterns of organ surface data limits the accuracy and robustness of an IGLS registration. Clinical validation of such registration methods is challenged due to the difficulty in obtaining data representative of the true state of organ deformation. We propose a novel human-to-phantom validation framework that transforms surface collection patterns from in vivo IGLS procedures (n = 13) onto a well-characterized hepatic deformation phantom for the purpose of validating surface-driven, volumetric nonrigid registration methods. An important feature of the approach is that it centers on combining workflow-realistic data acquisition and surgical deformations that are appropriate in behavior and magnitude. Using the approach, we investigate volumetric target registration error (TRE) with both current rigid IGLS and our improved nonrigid registration methods. Additionally, we introduce a spatial data resampling approach to mitigate the workflow-sensitive sampling problem. Using our human-to-phantom approach, TRE after routine rigid registration was 10.9 ± 0.6 mm with a signed closest point distance associated with residual surface fit in the range of ±10 mm, highly representative of open liver resections. After applying our novel resampling strategy and improved deformation correction method, TRE was reduced by 51%, i.e., a TRE of 5.3 ± 0.5 mm. This paper reported herein realizes a novel tractable approach for the validation of image-to-physical registration methods and demonstrates promising results for our correction method.