Model-Assisted Image-Guided Liver Surgery Using Sparse Intraoperative Data

Learning soft tissue behavior of organs for surgical navigation with convolutional neural networks

PURPOSE

In surgical navigation, pre-operative organ models are presented to surgeons during the intervention to help them in efficiently finding their target. In the case of soft tissue, these models need to be deformed and adapted to the current situation by using intra-operative sensor data. A promising method to realize this are real-time capable biomechanical models.

METHODS

We train a fully convolutional neural network to estimate a displacement field of all points inside an organ when given only the displacement of a part of the organ's surface. The network trains on entirely synthetic data of random organ-like meshes, which allows us to use much more data than is otherwise available. The input and output data are discretized into a regular grid, allowing us to fully utilize the capabilities of convolutional operators and to train and infer in a highly parallelized manner.

RESULTS

The system is evaluated on in-silico liver models, phantom liver data and human in-vivo breathing data. We test the performance with varying material parameters, organ shapes and amount of visible surface. Even though the network is only trained on synthetic data, it adapts well to the various cases and gives a good estimation of the internal organ displacement. The inference runs at over 50 frames per second.

CONCLUSION

We present a novel method for training a data-driven, real-time capable deformation model. The accuracy is comparable to other registration methods, it adapts very well to previously unseen organs and does not need to be re-trained for every patient. The high inferring speed makes this method useful for many applications such as surgical navigation and real-time simulation.

Image-Aligned Dynamic Liver Reconstruction Using Intra-Operative Field of Views for Minimal Invasive Surgery

During hepatic minimal invasive surgery (MIS), 3-D reconstruction of a liver surface by interpreting the geometry of its soft tissues is achieving attractions. One of the major issues to be addressed in MIS is liver deformation. Moreover, it severely inhibits free sight and dexterity of tissue manipulation, which causes its intra-operative morphology and soft tissue motion altered as compared to its pre-operative shape. While many applications focus on 3-D reconstruction of rigid or semi-rigid scenes, the techniques applied in hepatic MIS must be able to cope with a dynamic and deformable environment. We propose an efficient technique for liver surface reconstruction based on the structure from motion to handle liver deformation. The reconstructed liver will assist surgeons to visualize liver surface more efficiently with better depth perception. We use the intra-operative field of views to generate 3-D template mesh from a dense keypoint cloud. We estimate liver deformation by finding best correspondence between 3-D templates and reconstruct a liver image to calculate translation and rotational motions. Our technique then finely tunes deformed surface by adding smoothness using shading cues. Up till now, this technique is not used for solving the human liver deformation problem. Our approach is tested and validated with synthetic as well as real in vivo data, which reveal that the reconstruction accuracy can be enhanced using our approach even in challenging laparoscopic environments.

Computer-assisted liver tumor surgery using a novel semiautomatic and a hybrid semiautomatic segmentation algorithm

We developed a medical image segmentation and preoperative planning application which implements a semiautomatic and a hybrid semiautomatic liver segmentation algorithm. The aim of this study was to evaluate the feasibility of computer-assisted liver tumor surgery using these algorithms which are based on thresholding by pixel intensity value from initial seed points. A random sample of 12 patients undergoing elective high-risk hepatectomies at our institution was prospectively selected to undergo computer-assisted surgery using our algorithms (June 2013-July 2014). Quantitative and qualitative evaluation was performed. The average computer analysis time (segmentation, resection planning, volumetry, visualization) was 45 min/dataset. The runtime for the semiautomatic algorithm was <0.2 s/slice. Liver volumetric segmentation using the hybrid method was achieved in 12.9 s/dataset (SD ± 6.14). Mean similarity index was 96.2 % (SD ± 1.6). The future liver remnant volume calculated by the application showed a correlation of 0.99 to that calculated using manual boundary tracing. The 3D liver models and the virtual liver resections had an acceptable coincidence with the real intraoperative findings. The patient-specific 3D models produced using our semiautomatic and hybrid semiautomatic segmentation algorithms proved to be accurate for the preoperative planning in liver tumor surgery and effectively enhanced the intraoperative medical image guidance.