Using multiple images and contours for deformable 3D-2D registration of a preoperative CT in laparoscopic liver surgery

PURPOSE

Laparoscopic liver resection is a challenging procedure because of the difficulty to localise inner structures such as tumours and vessels. Augmented reality overcomes this problem by overlaying preoperative 3D models on the laparoscopic views. It requires deformable registration of the preoperative 3D models to the laparoscopic views, which is a challenging task due to the liver flexibility and partial visibility.

METHODS

We propose several multi-view registration methods exploiting information from multiple views simultaneously in order to improve registration accuracy. They are designed to work on two scenarios: on rigidly related views and on non-rigidly related views. These methods exploit the liver's anatomical landmarks and texture information available in all the views to constrain registration.

RESULTS

We evaluated the registration accuracy of our methods quantitatively on synthetic and phantom data, and qualitatively on patient data. We measured 3D target registration errors in mm for the whole liver for the quantitative case, and 2D reprojection errors in pixels for the qualitative case.

CONCLUSION

The proposed rigidly related multi-view methods improve registration accuracy compared to the baseline single-view method. They comply with the 1 cm oncologic resection margin advised for hepatocellular carcinoma interventions, depending on the available registration constraints. The non-rigidly related multi-view method does not provide a noticeable improvement. This means that using multiple views with the rigidity assumption achieves the best overall registration error.

Automatic liver tumor localization using deep learning-based liver boundary motion estimation and biomechanical modeling (DL-Bio)

PURPOSE

Recently, two-dimensional-to-three-dimensional (2D-3D) deformable registration has been applied to deform liver tumor contours from prior reference images onto estimated cone-beam computed tomography (CBCT) target images to automate on-board tumor localizations. Biomechanical modeling has also been introduced to fine-tune the intra-liver deformation-vector-fields (DVFs) solved by 2D-3D deformable registration, especially at low-contrast regions, using tissue elasticity information and liver boundary DVFs. However, the caudal liver boundary shows low contrast from surrounding tissues in the cone-beam projections, which degrades the accuracy of the intensity-based 2D-3D deformable registration there and results in less accurate boundary conditions for biomechanical modeling. We developed a deep-learning (DL)-based method to optimize the liver boundary DVFs after 2D-3D deformable registration to further improve the accuracy of subsequent biomechanical modeling and liver tumor localization.

METHODS

The DL-based network was built based on the U-Net architecture. The network was trained in a supervised fashion to learn motion correlation between cranial and caudal liver boundaries to optimize the liver boundary DVFs. Inputs of the network had three channels, and each channel featured the 3D DVFs estimated by the 2D-3D deformable registration along one Cartesian direction (x, y, z). To incorporate patient-specific liver boundary information into the DVFs, the DVFs were masked by a liver boundary ring structure generated from the liver contour of the prior reference image. The network outputs were the optimized DVFs along the liver boundary with higher accuracy. From these optimized DVFs, boundary conditions were extracted for biomechanical modeling to further optimize the solution of intra-liver tumor motion. We evaluated the method using 34 liver cancer patient cases, with 24 for training and 10 for testing. We evaluated and compared the performance of three methods: 2D-3D deformable registration, 2D-3D-Bio (2D-3D deformable registration with biomechanical modeling), and DL-Bio (DL model prediction with biomechanical modeling). The tumor localization errors were quantified through calculating the center-of-mass-errors (COMEs), DICE coefficients, and Hausdorff distance between deformed liver tumor contours and manually segmented "gold-standard" contours.

RESULTS

The predicted DVFs by the DL model showed improved accuracy at the liver boundary, which translated into more accurate liver tumor localizations through biomechanical modeling. On a total of 90 evaluated images and tumor contours, the average (± sd) liver tumor COMEs of the 2D-3D, 2D-3D-Bio, and DL-Bio techniques were 4.7 ± 1.9 mm, 2.9 ± 1.0 mm, and 1.7 ± 0.4 mm. The corresponding average (± sd) DICE coefficients were 0.60 ± 0.12, 0.71 ± 0.07, and 0.78 ± 0.03; and the average (± sd) Hausdorff distances were 7.0 ± 2.6 mm, 5.4 ± 1.5 mm, and 4.5 ± 1.3 mm, respectively.

CONCLUSION

DL-Bio solves a general correlation model to improve the accuracy of the DVFs at the liver boundary. With improved boundary conditions, the accuracy of biomechanical modeling can be further increased for accurate intra-liver low-contrast tumor localization.

Application of micro-computer tomography and inverse finite element analysis for characterizing the visco-hyperelastic response of bulk liver tissue using indentation

Abstract

In-vitro mechanical indentation experimentation is performed on bulk liver tissue of lamb to characterize its nonlinear material behaviour. The material response is characterized by a visco-hyperelastic material model by the use of 2-dimensional inverse finite element (FE) analysis. The time-dependent behaviour is characterized by the viscoelastic model represented by a 4-parameter Prony series, whereas the large deformations are modelled using the hyperelastic Neo-Hookean model. The shear response described by the initial and final shear moduli and the corresponding Prony series parameters are optimized using ANSYS with the Root Mean Square (RMS) error being the objective function. Optimized material properties are validated using experimental results obtained under different loading histories. To study the efficacy of a 2D model, a three dimensional (3D) model of the specimen is developed using Micro-CT of the specimen. The initial elastic modulus of the lamb liver obtained was found to 13.5 kPa for 5% indentation depth at a loading rate of 1 mm/sec for 1-cycle. These properties are able to predict the response at 8.33% depth and a loading rate of 5 mm/sec at multiple cycles with reasonable accuracy.

Article highlights

Liver Bioreactor Design Issues of Fluid Flow and Zonation, Fibrosis, and Mechanics: A Computational Perspective

Tissue engineering, with the goal of repairing or replacing damaged tissue and organs, has continued to make dramatic science-based advances since its origins in the late 1980’s and early 1990’s. Such advances are always multi-disciplinary in nature, from basic biology and chemistry through physics and mathematics to various engineering and computer fields. This review will focus its attention on two topics critical for tissue engineering liver development: (a) fluid flow, zonation, and drug screening, and (b) biomechanics, tissue stiffness, and fibrosis, all within the context of 3D structures. First, a general overview of various bioreactor designs developed to investigate fluid transport and tissue biomechanics is given. This includes a mention of computational fluid dynamic methods used to optimize and validate these designs. Thereafter, the perspective provided by computer simulations of flow, reactive transport, and biomechanics responses at the scale of the liver lobule and liver tissue is outlined, in addition to how bioreactor-measured properties can be utilized in these models. Here, the fundamental issues of tortuosity and upscaling are highlighted, as well as the role of disease and fibrosis in these issues. Some idealized simulations of the effects of fibrosis on lobule drug transport and mechanics responses are provided to further illustrate these concepts. This review concludes with an outline of some practical applications of tissue engineering advances and how efficient computational upscaling techniques, such as dual continuum modeling, might be used to quantify the transition of bioreactor results to the full liver scale.

Estimation of the elastic parameters of human liver biomechanical models by means of medical images and evolutionary computation

This paper presents a method to computationally estimate the elastic parameters of two biomechanical models proposed for the human liver. The method is aimed at avoiding the invasive measurement of its mechanical response. The chosen models are a second order Mooney-Rivlin model and an Ogden model. A novel error function, the geometric similarity function (GSF), is formulated using similarity coefficients widely applied in the field of medical imaging (Jaccard coefficient and Hausdorff coefficient). This function is used to compare two 3D images. One of them corresponds to a reference deformation carried out over a finite element (FE) mesh of a human liver from a computer tomography image, whilst the other one corresponds to the FE simulation of that deformation in which variations in the values of the model parameters are introduced. Several search strategies, based on GSF as cost function, are developed to accurately find the elastics parameters of the models, namely: two evolutionary algorithms (scatter search and genetic algorithm) and an iterative local optimization. The results show that GSF is a very appropriate function to estimate the elastic parameters of the biomechanical models since the mean of the relative mean absolute errors committed by the three algorithms is lower than 4%.

Analysis of several biomechanical models for the simulation of lamb liver behaviour using similarity coefficients from medical image

In this study, six biomechanical models for simulating lamb liver behaviour are presented. They are validated using similarity coefficients from Medical Image on reconstructed volumes from computerised tomography images. In particular, the Jaccard and Hausdorff coefficients are used. Loads of 20 and 40 g are applied to the livers and their deformation is simulated by means of the finite element method. The models used are a linear elastic model, a neo-Hookean model, a Mooney-Rivlin model, an Ogden model, a linear viscoelastic model and a viscohyperelastic model. The model that provided a behaviour that is closest to reality was the viscohyperelastic model, where the hyperelastic part was modelled with an Ogden model.