1
|
Chen Z, Zou B, Kui X, Shi Y, Lv D, Chen L. Points of interest linear attention network for real-time non-rigid liver volume to surface registration. Med Phys 2024. [PMID: 38758744 DOI: 10.1002/mp.17108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2023] [Revised: 04/04/2024] [Accepted: 04/14/2024] [Indexed: 05/19/2024] Open
Abstract
BACKGROUND In laparoscopic liver surgery, accurately predicting the displacement of key intrahepatic anatomical structures is crucial for informing the doctor's intraoperative decision-making. However, due to the constrained surgical perspective, only a partial surface of the liver is typically visible. Consequently, the utilization of non-rigid volume to surface registration methods becomes essential. But traditional registration methods lack the necessary accuracy and cannot meet real-time requirements. PURPOSE To achieve high-precision liver registration with only partial surface information and estimate the displacement of internal liver tissues in real-time. METHODS We propose a novel neural network architecture tailored for real-time non-rigid liver volume to surface registration. The network utilizes a voxel-based method, integrating sparse convolution with the newly proposed points of interest (POI) linear attention module. POI linear attention module specifically calculates attention on the previously extracted POI. Additionally, we identified the most suitable normalization method RMSINorm. RESULTS We evaluated our proposed network and other networks on a dataset generated from real liver models and two real datasets. Our method achieves an average error of 4.23 mm and a mean frame rate of 65.4 fps in the generation dataset. It also achieves an average error of 8.29 mm in the human breathing motion dataset. CONCLUSIONS Our network outperforms CNN-based networks and other attention networks in terms of accuracy and inference speed.
Collapse
Affiliation(s)
- Zeming Chen
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
- Hunan Engineering Research Center of Machine Vision and Intelligent Medicine, Changsha, Hunan, China
| | - Beiji Zou
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
- Hunan Engineering Research Center of Machine Vision and Intelligent Medicine, Changsha, Hunan, China
| | - Xiaoyan Kui
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
- Hunan Engineering Research Center of Machine Vision and Intelligent Medicine, Changsha, Hunan, China
| | - Yangyang Shi
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
- Hunan Engineering Research Center of Machine Vision and Intelligent Medicine, Changsha, Hunan, China
| | - Ding Lv
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
- Hunan Engineering Research Center of Machine Vision and Intelligent Medicine, Changsha, Hunan, China
| | - Liming Chen
- Ecole Centrale de Lyon, University of Lyon, Lyon, France
| |
Collapse
|
2
|
Yang Z, Simon R, Linte CA. Learning feature descriptors for pre- and intra-operative point cloud matching for laparoscopic liver registration. Int J Comput Assist Radiol Surg 2023:10.1007/s11548-023-02893-3. [PMID: 37079248 DOI: 10.1007/s11548-023-02893-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Accepted: 03/29/2023] [Indexed: 04/21/2023]
Abstract
PURPOSE In laparoscopic liver surgery, preoperative information can be overlaid onto the intra-operative scene by registering a 3D preoperative model to the intra-operative partial surface reconstructed from the laparoscopic video. To assist with this task, we explore the use of learning-based feature descriptors, which, to our best knowledge, have not been explored for use in laparoscopic liver registration. Furthermore, a dataset to train and evaluate the use of learning-based descriptors does not exist. METHODS We present the LiverMatch dataset consisting of 16 preoperative models and their simulated intra-operative 3D surfaces. We also propose the LiverMatch network designed for this task, which outputs per-point feature descriptors, visibility scores, and matched points. RESULTS We compare the proposed LiverMatch network with a network closest to LiverMatch and a histogram-based 3D descriptor on the testing split of the LiverMatch dataset, which includes two unseen preoperative models and 1400 intra-operative surfaces. Results suggest that our LiverMatch network can predict more accurate and dense matches than the other two methods and can be seamlessly integrated with a RANSAC-ICP-based registration algorithm to achieve an accurate initial alignment. CONCLUSION The use of learning-based feature descriptors in laparoscopic liver registration (LLR) is promising, as it can help achieve an accurate initial rigid alignment, which, in turn, serves as an initialization for subsequent non-rigid registration.
Collapse
Affiliation(s)
- Zixin Yang
- Center for Imaging Science, Rochester Institute of Technology, 1 Lomb Memorial Dr, Rochester, 14623, NY, USA.
| | - Richard Simon
- Biomedical Engineering, Rochester Institute of Technology, 1 Lomb Memorial Dr, Rochester, 14623, NY, USA
| | - Cristian A Linte
- Center for Imaging Science, Rochester Institute of Technology, 1 Lomb Memorial Dr, Rochester, 14623, NY, USA
- Biomedical Engineering, Rochester Institute of Technology, 1 Lomb Memorial Dr, Rochester, 14623, NY, USA
| |
Collapse
|
3
|
Nakao M, Nakamura M, Matsuda T. Image-to-Graph Convolutional Network for 2D/3D Deformable Model Registration of Low-Contrast Organs. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:3747-3761. [PMID: 35901001 DOI: 10.1109/tmi.2022.3194517] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Organ shape reconstruction based on a single-projection image during treatment has wide clinical scope, e.g., in image-guided radiotherapy and surgical guidance. We propose an image-to-graph convolutional network that achieves deformable registration of a three-dimensional (3D) organ mesh for a low-contrast two-dimensional (2D) projection image. This framework enables simultaneous training of two types of transformation: from the 2D projection image to a displacement map, and from the sampled per-vertex feature to a 3D displacement that satisfies the geometrical constraint of the mesh structure. Assuming application to radiation therapy, the 2D/3D deformable registration performance is verified for multiple abdominal organs that have not been targeted to date, i.e., the liver, stomach, duodenum, and kidney, and for pancreatic cancer. The experimental results show shape prediction considering relationships among multiple organs can be used to predict respiratory motion and deformation from digitally reconstructed radiographs with clinically acceptable accuracy.
Collapse
|
4
|
Krames L, Suppa P, Nahm W. Generation of Synthetic Data for the Comparison of Different 3D-3D Registration Approaches in Laparoscopic Surgery. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:1871-1874. [PMID: 36086378 DOI: 10.1109/embc48229.2022.9871580] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
In laparoscopic surgery image-guided navigation systems could support the surgeon by providing subsurface information such as the positions of tumors and vessels. For this purpose, one option is to perform a reliable registration of preoperative 3D data and a surface patch from laparo-scopic video data. A robust and automatic 3D-3D registration pipeline for the application during laparoscopic surgery has not yet been found due to application-specific challenges. To gain a better insight, we propose a framework enabling a qualitative and quantitative comparison of different registration approaches. The introduced framework is able to evaluate 3D feature descriptors and registration algorithms by generating and modifying synthetic data from clinical examples. Different confounding factors are considered and thus the reality can be reflected in any simplified or more complex way. Two exemplary experiments with a liver model, using the RANSAC algorithm, showed an increasing registration error for a decreasing size of the surface patch size and after introducing modifications. Moreover, the registration accuracy was dependent on the position and structure of the surface patch. The framework helps to quantitatively assess and optimize the registration pipeline, and hereby suggests future software improvements even with only few clinical examples. Clinical relevance- The introduced framework permits a quantitative and comprehensive comparison of different registration approaches which forms the basis for a supportive navigation tool in laparoscopic surgery.
Collapse
|
5
|
Bardozzo F, Collins T, Forgione A, Hostettler A, Tagliaferri R. StaSiS-Net: a stacked and siamese disparity estimation network for depth reconstruction in modern 3D laparoscopy. Med Image Anal 2022; 77:102380. [DOI: 10.1016/j.media.2022.102380] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2021] [Revised: 01/26/2022] [Accepted: 01/27/2022] [Indexed: 10/19/2022]
|
6
|
Rabbani N, Calvet L, Espinel Y, Le Roy B, Ribeiro M, Buc E, Bartoli A. A methodology and clinical dataset with ground-truth to evaluate registration accuracy quantitatively in computer-assisted Laparoscopic Liver Resection. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2021. [DOI: 10.1080/21681163.2021.1997642] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- N. Rabbani
- EnCoV, Institut Pascal, Clermont-Ferrand, France
| | - L. Calvet
- EnCoV, Institut Pascal, Clermont-Ferrand, France
- CHU, Clermont-Ferrand, France
- IRIT, University of Toulouse
| | - Y. Espinel
- EnCoV, Institut Pascal, Clermont-Ferrand, France
| | - B. Le Roy
- EnCoV, Institut Pascal, Clermont-Ferrand, France
- CHU, Saint-Etienne, France
| | - M. Ribeiro
- EnCoV, Institut Pascal, Clermont-Ferrand, France
- CHU, Clermont-Ferrand, France
| | - E. Buc
- EnCoV, Institut Pascal, Clermont-Ferrand, France
- CHU, Clermont-Ferrand, France
| | - A. Bartoli
- EnCoV, Institut Pascal, Clermont-Ferrand, France
| |
Collapse
|
7
|
Heiselman JS, Miga MI. Strain Energy Decay Predicts Elastic Registration Accuracy From Intraoperative Data Constraints. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:1290-1302. [PMID: 33460370 PMCID: PMC8117369 DOI: 10.1109/tmi.2021.3052523] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Image-guided intervention for soft tissue organs depends on the accuracy of deformable registration methods to achieve effective results. While registration techniques based on elastic theory are prevalent, no methods yet exist that can prospectively estimate registration uncertainty to regulate sources and mitigate consequences of localization error in deforming organs. This paper introduces registration uncertainty metrics based on dispersion of strain energy from boundary constraints to predict the proportion of target registration error (TRE) remaining after nonrigid elastic registration. These uncertainty metrics depend on the spatial distribution of intraoperative constraints provided to registration with relation to patient-specific organ geometry. Predictive linear and bivariate gamma models are fit and cross-validated using an existing dataset of 6291 simulated registration examples, plus 699 novel simulated registrations withheld for independent validation. Average uncertainty and average proportion of TRE remaining after elastic registration are strongly correlated ( r = 0.78 ), with mean absolute difference in predicted TRE equivalent to 0.9 ± 0.6 mm (cross-validation) and 0.9 ± 0.5 mm (independent validation). Spatial uncertainty maps also permit localized TRE estimates accurate to an equivalent of 3.0 ± 3.1 mm (cross-validation) and 1.6 ± 1.2 mm (independent validation). Additional clinical evaluation of vascular features yields localized TRE estimates accurate to 3.4 ± 3.2 mm. This work formalizes a lower bound for the inherent uncertainty of nonrigid elastic registrations given coverage of intraoperative data constraints, and demonstrates a relation to TRE that can be predictively leveraged to inform data collection and provide a measure of registration confidence for elastic methods.
Collapse
|
8
|
Heiselman JS, Jarnagin WR, Miga MI. Intraoperative Correction of Liver Deformation Using Sparse Surface and Vascular Features via Linearized Iterative Boundary Reconstruction. IEEE TRANSACTIONS ON MEDICAL IMAGING 2020; 39:2223-2234. [PMID: 31976882 PMCID: PMC7314378 DOI: 10.1109/tmi.2020.2967322] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
During image guided liver surgery, soft tissue deformation can cause considerable error when attempting to achieve accurate localization of the surgical anatomy through image-to-physical registration. In this paper, a linearized iterative boundary reconstruction technique is proposed to account for these deformations. The approach leverages a superposed formulation of boundary conditions to rapidly and accurately estimate the deformation applied to a preoperative model of the organ given sparse intraoperative data of surface and subsurface features. With this method, tracked intraoperative ultrasound (iUS) is investigated as a potential data source for augmenting registration accuracy beyond the capacity of conventional organ surface registration. In an expansive simulated dataset, features including vessel contours, vessel centerlines, and the posterior liver surface are extracted from iUS planes. Registration accuracy is compared across increasing data density to establish how iUS can be best employed to improve target registration error (TRE). From a baseline average TRE of 11.4 ± 2.2 mm using sparse surface data only, incorporating additional sparse features from three iUS planes improved average TRE to 6.4 ± 1.0 mm. Furthermore, increasing the sparse coverage to 16 tracked iUS planes improved average TRE to 3.9 ± 0.7 mm, exceeding the accuracy of registration based on complete surface data available with more cumbersome intraoperative CT without contrast. Additionally, the approach was applied to three clinical cases where on average error improved 67% over rigid registration and 56% over deformable surface registration when incorporating additional features from one independent tracked iUS plane.
Collapse
Affiliation(s)
| | - William R. Jarnagin
- Department of Surgery at Memorial Sloan Kettering Cancer Center, New York, NY 10065 USA
| | - Michael I. Miga
- Department of Biomedical Engineering at Vanderbilt University, Nashville, TN 37235 USA
| |
Collapse
|
9
|
A case study: impact of target surface mesh size and mesh quality on volume-to-surface registration performance in hepatic soft tissue navigation. Int J Comput Assist Radiol Surg 2020; 15:1235-1245. [PMID: 32221798 PMCID: PMC7351822 DOI: 10.1007/s11548-020-02123-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2019] [Accepted: 02/10/2020] [Indexed: 11/30/2022]
Abstract
Purpose Soft tissue deformation severely impacts the registration of pre- and intra-operative image data during computer-assisted navigation in laparoscopic liver surgery. However, quantifying the impact of target surface size, surface orientation, and mesh quality on non-rigid registration performance remains an open research question. This paper aims to uncover how these affect volume-to-surface registration performance. Methods To find such evidence, we design three experiments that are evaluated using a three-step pipeline: (1) volume-to-surface registration using the physics-based shape matching method or PBSM, (2) voxelization of the deformed surface to a \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$1024^3$$\end{document}10243 voxel grid, and (3) computation of similarity (e.g., mutual information), distance (i.e., Hausdorff distance), and classical metrics (i.e., mean squared error or MSE). Results Using the Hausdorff distance, we report a statistical significance for the different partial surfaces. We found that removing non-manifold geometry and noise improved registration performance, and a target surface size of only 16.5% was necessary. Conclusion By investigating three different factors and improving registration results, we defined a generalizable evaluation pipeline and automatic post-processing strategies that were deemed helpful. All source code, reference data, models, and evaluation results are openly available for download: https://github.com/ghattab/EvalPBSM/.
Collapse
|
10
|
Schneider C, Thompson S, Totz J, Song Y, Allam M, Sodergren MH, Desjardins AE, Barratt D, Ourselin S, Gurusamy K, Stoyanov D, Clarkson MJ, Hawkes DJ, Davidson BR. Comparison of manual and semi-automatic registration in augmented reality image-guided liver surgery: a clinical feasibility study. Surg Endosc 2020; 34:4702-4711. [PMID: 32780240 PMCID: PMC7524854 DOI: 10.1007/s00464-020-07807-x] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2020] [Accepted: 07/10/2020] [Indexed: 02/06/2023]
Abstract
BACKGROUND The laparoscopic approach to liver resection may reduce morbidity and hospital stay. However, uptake has been slow due to concerns about patient safety and oncological radicality. Image guidance systems may improve patient safety by enabling 3D visualisation of critical intra- and extrahepatic structures. Current systems suffer from non-intuitive visualisation and a complicated setup process. A novel image guidance system (SmartLiver), offering augmented reality visualisation and semi-automatic registration has been developed to address these issues. A clinical feasibility study evaluated the performance and usability of SmartLiver with either manual or semi-automatic registration. METHODS Intraoperative image guidance data were recorded and analysed in patients undergoing laparoscopic liver resection or cancer staging. Stereoscopic surface reconstruction and iterative closest point matching facilitated semi-automatic registration. The primary endpoint was defined as successful registration as determined by the operating surgeon. Secondary endpoints were system usability as assessed by a surgeon questionnaire and comparison of manual vs. semi-automatic registration accuracy. Since SmartLiver is still in development no attempt was made to evaluate its impact on perioperative outcomes. RESULTS The primary endpoint was achieved in 16 out of 18 patients. Initially semi-automatic registration failed because the IGS could not distinguish the liver surface from surrounding structures. Implementation of a deep learning algorithm enabled the IGS to overcome this issue and facilitate semi-automatic registration. Mean registration accuracy was 10.9 ± 4.2 mm (manual) vs. 13.9 ± 4.4 mm (semi-automatic) (Mean difference - 3 mm; p = 0.158). Surgeon feedback was positive about IGS handling and improved intraoperative orientation but also highlighted the need for a simpler setup process and better integration with laparoscopic ultrasound. CONCLUSION The technical feasibility of using SmartLiver intraoperatively has been demonstrated. With further improvements semi-automatic registration may enhance user friendliness and workflow of SmartLiver. Manual and semi-automatic registration accuracy were comparable but evaluation on a larger patient cohort is required to confirm these findings.
Collapse
Affiliation(s)
- C. Schneider
- Division of Surgery & Interventional Science, Royal Free Campus, University College London, Pond Street, London, NW3 2QG UK
| | - S. Thompson
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - J. Totz
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - Y. Song
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - M. Allam
- Division of Surgery & Interventional Science, Royal Free Campus, University College London, Pond Street, London, NW3 2QG UK
| | - M. H. Sodergren
- Centre for Medical Image Computing (CMIC), University College London, London, UK
| | - A. E. Desjardins
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - D. Barratt
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - S. Ourselin
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - K. Gurusamy
- Division of Surgery & Interventional Science, Royal Free Campus, University College London, Pond Street, London, NW3 2QG UK ,Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Department of Hepatopancreatobiliary and Liver Transplant Surgery, Royal Free Hospital, London, UK
| | - D. Stoyanov
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Computer Science, University College London, London, UK
| | - M. J. Clarkson
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - D. J. Hawkes
- Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Centre for Medical Image Computing (CMIC), University College London, London, UK ,Department of Medical Physics and Bioengineering, University College London, London, UK
| | - B. R. Davidson
- Division of Surgery & Interventional Science, Royal Free Campus, University College London, Pond Street, London, NW3 2QG UK ,Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK ,Department of Hepatopancreatobiliary and Liver Transplant Surgery, Royal Free Hospital, London, UK
| |
Collapse
|
11
|
Nakamura M, Nakao M, Hirashima H, Iramina H, Mizowaki T. Performance evaluation of a newly developed three-dimensional model-based global-to-local registration in prostate cancer. JOURNAL OF RADIATION RESEARCH 2019; 60:595-602. [PMID: 31135904 PMCID: PMC6805968 DOI: 10.1093/jrr/rrz031] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/04/2019] [Revised: 03/26/2019] [Indexed: 06/09/2023]
Abstract
We evaluated the performance of a newly developed three-dimensional (3D) model-based global-to-local registration of multiple organs, by comparing it with a 3D model-based global registration in the prostate region. This study included 220 prostate cancer patients who underwent intensity-modulated radiotherapy or volumetric-modulated arc therapy. Our registration proceeded sequentially, i.e. global registration including affine and piece-wise affine transformation followed by local registration. As a local registration, Laplacian-based and finite element method-based registration was implemented in Algorithm A and B, respectively. Algorithm C was for global registration alone. The template models for the prostate, seminal vesicles, rectum and bladder were constructed from the first 20 patients, and then three different registrations were performed on these organs for the remaining 200 patients, to assess registration accuracy. The 75th percentile Hausdorff distance was <1 mm in Algorithm A; it was >1 mm in Algorithm B, except for the prostate; and 3.9 mm for the prostate and >7.8 mm for other organs in Algorithm C. The median computation time to complete registration was <101, 30 and 16 s in Algorithms A, B and C, respectively. Analysis of variance revealed significant differences among Algorithms A-C in the Hausdorff distance and computation time. In addition, no significant difference was observed in the difference of Hausdorff distance between Algorithm A and B with Tukey's multiple comparison test. The 3D model-based global-to-local registration, especially that implementing Laplacian-based registration, completed surface registration rapidly and provided sufficient registration accuracy in the prostate region.
Collapse
Affiliation(s)
- Mitsuhiro Nakamura
- Division of Medical Physics, Department of Information Technology and Medical Engineering, Human Health Sciences, Graduate School of Medicine, Kyoto University, Japan
- Department of Radiation Oncology and Image-Applied Therapy, Graduate School of Medicine, Kyoto University, Japan
| | - Megumi Nakao
- Department of Systems Science, Graduate School of Informatics, Kyoto University, Japan
| | - Hideaki Hirashima
- Department of Radiation Oncology and Image-Applied Therapy, Graduate School of Medicine, Kyoto University, Japan
| | - Hiraku Iramina
- Department of Radiation Oncology and Image-Applied Therapy, Graduate School of Medicine, Kyoto University, Japan
| | - Takashi Mizowaki
- Department of Radiation Oncology and Image-Applied Therapy, Graduate School of Medicine, Kyoto University, Japan
| |
Collapse
|
12
|
Oldhafer KJ, Peterhans M, Kantas A, Schenk A, Makridis G, Pelzl S, Wagner KC, Weber S, Stavrou GA, Donati M. [Navigated liver surgery : Current state and importance in the future]. Chirurg 2019; 89:769-776. [PMID: 30225532 DOI: 10.1007/s00104-018-0713-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
Abstract
The preoperative computer-assisted resection planning is the basis for every navigation. Thanks to modern algorithms, the prerequisites have been created to carry out a virtual resection planning and a risk analysis. Thus, individual segment resections can be precisely planned in any conceivable combination. The transfer of planning information and resection suggestions to the operating theater is still problematic. The so-called stereotactic liver navigation supports the exact intraoperative implementation of the planned resection strategy and provides the surgeon with real-time three-dimensional information on resection margins and critical structures during the resection. This is made possible by a surgical navigation system that measures the position of surgical instruments and then presents them together with the preoperative surgical planning data. Although surgical navigation systems have been indispensable in neurosurgery and spinal surgery for many years, these procedures have not yet become established as standard in liver surgery. This is mainly due to the technical challenge of navigating a moving organ. As the liver is constantly moving and deforming during surgery due to respiration and surgical manipulation, the surgical navigation system must be able to measure these alterations in order to adapt the preoperative navigation data to the current situation. Despite these advances, further developments are required until navigated liver resection enters clinical routine; however, it is already clear that laparoscopic liver surgery and robotic surgery will benefit most from navigation technology.
Collapse
Affiliation(s)
- K J Oldhafer
- Klinik für Allgemein- und Viszeralchirurgie, Asklepios Klinik Barmbek, Hamburg, Deutschland. .,Semmelweis Universität Budapest, Campus Hamburg, Hamburg, Deutschland.
| | | | - A Kantas
- Klinik für Allgemein- und Viszeralchirurgie, Asklepios Klinik Barmbek, Hamburg, Deutschland.,Semmelweis Universität Budapest, Campus Hamburg, Hamburg, Deutschland
| | - A Schenk
- Fraunhofer-Institut für Bildgestützte Medizin MEVIS, Bremen, Deutschland
| | - G Makridis
- Klinik für Allgemein- und Viszeralchirurgie, Asklepios Klinik Barmbek, Hamburg, Deutschland.,Semmelweis Universität Budapest, Campus Hamburg, Hamburg, Deutschland
| | - S Pelzl
- apoQlar, Hamburg, Deutschland
| | - K C Wagner
- Klinik für Allgemein- und Viszeralchirurgie, Asklepios Klinik Barmbek, Hamburg, Deutschland.,Semmelweis Universität Budapest, Campus Hamburg, Hamburg, Deutschland
| | - S Weber
- University of Bern, ARTORG Center for Biomedical Engineering Research, Bern, Schweiz
| | - G A Stavrou
- Klinik für Allgemein‑, Viszeralchirurgie, Thorax- und Kinderchirurgie, Klinikum Saarbrücken, Saarbrücken, Deutschland
| | - M Donati
- Semmelweis Universität Budapest, Campus Hamburg, Hamburg, Deutschland.,Department of Surgery and Medical Surgical Specialties, University of Catania, Catania, Italien
| |
Collapse
|
13
|
Adagolodjo Y, Goffin L, De Mathelin M, Courtecuisse H. Robotic Insertion of Flexible Needle in Deformable Structures Using Inverse Finite-Element Simulation. IEEE T ROBOT 2019. [DOI: 10.1109/tro.2019.2897858] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
14
|
An in vivo porcine dataset and evaluation methodology to measure soft-body laparoscopic liver registration accuracy with an extended algorithm that handles collisions. Int J Comput Assist Radiol Surg 2019; 14:1237-1245. [PMID: 31147817 DOI: 10.1007/s11548-019-02001-4] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2019] [Accepted: 05/15/2019] [Indexed: 10/26/2022]
Abstract
PURPOSE The registration of preoperative 3D images to intra-operative laparoscopic 2D images is one of the main concerns for augmented reality in computer-assisted surgery. For laparoscopic liver surgery, while several algorithms have been proposed, there is neither a public dataset nor a systematic evaluation methodology to quantitatively evaluate registration accuracy. METHOD Our main contribution is to provide such a dataset with an in vivo porcine model. It is used to evaluate a state-of-the-art registration algorithm that is capable of simultaneous registration and soft-body collision reasoning. RESULTS The dataset consists of 13 deformed liver states, with corresponding exploration videos and interventional CT acquisitions with 60 small artificial fiducials located on the surface of the liver and distributed within the parenchyma, where a precise registration is crucial for augmented reality. This dataset will be made public. Using this dataset, we show that collision reasoning improves performance of registration for strong deformation and independent lobe motion. CONCLUSION This dataset addresses the lack of public datasets in this field. As an example of use, we present and evaluate a state-of-the-art energy-based approach and a novel extension that handles self-collisions.
Collapse
|
15
|
Pfeiffer M, Riediger C, Weitz J, Speidel S. Learning soft tissue behavior of organs for surgical navigation with convolutional neural networks. Int J Comput Assist Radiol Surg 2019; 14:1147-1155. [PMID: 30993520 DOI: 10.1007/s11548-019-01965-7] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2019] [Accepted: 04/02/2019] [Indexed: 12/12/2022]
Abstract
PURPOSE In surgical navigation, pre-operative organ models are presented to surgeons during the intervention to help them in efficiently finding their target. In the case of soft tissue, these models need to be deformed and adapted to the current situation by using intra-operative sensor data. A promising method to realize this are real-time capable biomechanical models. METHODS We train a fully convolutional neural network to estimate a displacement field of all points inside an organ when given only the displacement of a part of the organ's surface. The network trains on entirely synthetic data of random organ-like meshes, which allows us to use much more data than is otherwise available. The input and output data are discretized into a regular grid, allowing us to fully utilize the capabilities of convolutional operators and to train and infer in a highly parallelized manner. RESULTS The system is evaluated on in-silico liver models, phantom liver data and human in-vivo breathing data. We test the performance with varying material parameters, organ shapes and amount of visible surface. Even though the network is only trained on synthetic data, it adapts well to the various cases and gives a good estimation of the internal organ displacement. The inference runs at over 50 frames per second. CONCLUSION We present a novel method for training a data-driven, real-time capable deformation model. The accuracy is comparable to other registration methods, it adapts very well to previously unseen organs and does not need to be re-trained for every patient. The high inferring speed makes this method useful for many applications such as surgical navigation and real-time simulation.
Collapse
Affiliation(s)
- Micha Pfeiffer
- National Center for Tumor Diseases (NCT), Partner Site Dresden, Dresden, Germany.
| | - Carina Riediger
- Department for Visceral, Thoracic and Vascular Surgery, University Hospital, Technical University Dresden, Dresden, Germany
| | - Jürgen Weitz
- Department for Visceral, Thoracic and Vascular Surgery, University Hospital, Technical University Dresden, Dresden, Germany
| | - Stefanie Speidel
- National Center for Tumor Diseases (NCT), Partner Site Dresden, Dresden, Germany
| |
Collapse
|
16
|
Brunet JN, Mendizabal A, Petit A, Golse N, Vibert E, Cotin S. Physics-Based Deep Neural Network for Augmented Reality During Liver Surgery. LECTURE NOTES IN COMPUTER SCIENCE 2019. [DOI: 10.1007/978-3-030-32254-0_16] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/04/2022]
|
17
|
Cheema MN, Nazir A, Sheng B, Li P, Qin J, Kim J, Feng DD. Image-Aligned Dynamic Liver Reconstruction Using Intra-Operative Field of Views for Minimal Invasive Surgery. IEEE Trans Biomed Eng 2018; 66:2163-2173. [PMID: 30507524 DOI: 10.1109/tbme.2018.2884319] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
During hepatic minimal invasive surgery (MIS), 3-D reconstruction of a liver surface by interpreting the geometry of its soft tissues is achieving attractions. One of the major issues to be addressed in MIS is liver deformation. Moreover, it severely inhibits free sight and dexterity of tissue manipulation, which causes its intra-operative morphology and soft tissue motion altered as compared to its pre-operative shape. While many applications focus on 3-D reconstruction of rigid or semi-rigid scenes, the techniques applied in hepatic MIS must be able to cope with a dynamic and deformable environment. We propose an efficient technique for liver surface reconstruction based on the structure from motion to handle liver deformation. The reconstructed liver will assist surgeons to visualize liver surface more efficiently with better depth perception. We use the intra-operative field of views to generate 3-D template mesh from a dense keypoint cloud. We estimate liver deformation by finding best correspondence between 3-D templates and reconstruct a liver image to calculate translation and rotational motions. Our technique then finely tunes deformed surface by adding smoothness using shading cues. Up till now, this technique is not used for solving the human liver deformation problem. Our approach is tested and validated with synthetic as well as real in vivo data, which reveal that the reconstruction accuracy can be enhanced using our approach even in challenging laparoscopic environments.
Collapse
|
18
|
Eppenga R, Kuhlmann K, Ruers T, Nijkamp J. Accuracy assessment of wireless transponder tracking in the operating room environment. Int J Comput Assist Radiol Surg 2018; 13:1937-1948. [PMID: 30099659 DOI: 10.1007/s11548-018-1838-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2018] [Accepted: 07/27/2018] [Indexed: 01/23/2023]
Abstract
PURPOSE To evaluate the applicability of the Calypso® wireless transponder tracking system (Varian Medical Systems Inc., USA) for real-time tumor motion tracking during surgical procedures on tumors in non-rigid target areas. An accuracy assessment was performed for an extended electromagnetic field of view (FoV) of 27.5 × 27.5 × 22.5 cm (which included the standard FoV of 14 × 14 × 19 cm) in which 5DOF wireless Beacon® transponders can be tracked. METHODS Using a custom-made measurement setup, we assessed single transponder relative accuracy, absolute accuracy and jitter throughout the extended FoV at 1440 locations interspaced with 2.5 cm in each orthogonal direction. The NDI Polaris Spectra optical tracking system (OTS) was used as a reference. Measurements were taken in a room without surrounding distorting factors and repeated in an operating room (OR). In the OR, the influence of a carbon fiber and regular stainless steel OR tabletop was investigated. RESULTS The calibration of the OTS and transponder system resulted in an average root-mean-square error (RMSE) vector of 0.03 cm. For both the standard and extended FoV, all accuracy measures were dependent on transponder to tracking array (TA) distances and the absolute accuracy was also dependent on TA to OR tabletop distances. This latter influence was reproducible, and after calibrating this, the residual error was below 0.1 cm RMSE within the entire standard FoV. Within the extended FoV, this residual RMSE did not exceed 0.1 cm for transponder to TA distances up to 25 cm. CONCLUSION This study shows that transponder tracking is promising for accurate tumor tracking in the operating room. This applies when using the standard FoV, but also when using the extended FoV up to 25 cm above the TA, substantially increasing flexibility.
Collapse
Affiliation(s)
- Roeland Eppenga
- Department of Surgical Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Koert Kuhlmann
- Department of Surgical Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Theo Ruers
- Department of Surgical Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
- Nanobiophysics Group, MIRA Institute, University of Twente, Enschede, The Netherlands
| | - Jasper Nijkamp
- Department of Surgical Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands.
- Department of Surgery, The Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX, Amsterdam, The Netherlands.
| |
Collapse
|
19
|
Özgür E, Koo B, Le Roy B, Buc E, Bartoli A. Preoperative liver registration for augmented monocular laparoscopy using backward–forward biomechanical simulation. Int J Comput Assist Radiol Surg 2018; 13:1629-1640. [DOI: 10.1007/s11548-018-1842-3] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2018] [Accepted: 07/31/2018] [Indexed: 12/01/2022]
|
20
|
Haouchine N, Kuang W, Cotin S, Yip M. Vision-Based Force Feedback Estimation for Robot-Assisted Surgery Using Instrument-Constrained Biomechanical Three-Dimensional Maps. IEEE Robot Autom Lett 2018. [DOI: 10.1109/lra.2018.2810948] [Citation(s) in RCA: 32] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
21
|
Robu MR, Ramalhinho J, Thompson S, Gurusamy K, Davidson B, Hawkes D, Stoyanov D, Clarkson MJ. Global rigid registration of CT to video in laparoscopic liver surgery. Int J Comput Assist Radiol Surg 2018; 13:947-956. [PMID: 29736801 PMCID: PMC5974008 DOI: 10.1007/s11548-018-1781-z] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2018] [Accepted: 04/27/2018] [Indexed: 11/09/2022]
Abstract
PURPOSE Image-guidance systems have the potential to aid in laparoscopic interventions by providing sub-surface structure information and tumour localisation. The registration of a preoperative 3D image with the intraoperative laparoscopic video feed is an important component of image guidance, which should be fast, robust and cause minimal disruption to the surgical procedure. Most methods for rigid and non-rigid registration require a good initial alignment. However, in most research systems for abdominal surgery, the user has to manually rotate and translate the models, which is usually difficult to perform quickly and intuitively. METHODS We propose a fast, global method for the initial rigid alignment between a 3D mesh derived from a preoperative CT of the liver and a surface reconstruction of the intraoperative scene. We formulate the shape matching problem as a quadratic assignment problem which minimises the dissimilarity between feature descriptors while enforcing geometrical consistency between all the feature points. We incorporate a novel constraint based on the liver contours which deals specifically with the challenges introduced by laparoscopic data. RESULTS We validate our proposed method on synthetic data, on a liver phantom and on retrospective clinical data acquired during a laparoscopic liver resection. We show robustness over reduced partial size and increasing levels of deformation. Our results on the phantom and on the real data show good initial alignment, which can successfully converge to the correct position using fine alignment techniques. Furthermore, since we can pre-process the CT scan before surgery, the proposed method runs faster than current algorithms. CONCLUSION The proposed shape matching method can provide a fast, global initial registration, which can be further refined by fine alignment methods. This approach will lead to a more usable and intuitive image-guidance system for laparoscopic liver surgery.
Collapse
Affiliation(s)
- Maria R Robu
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK.
- Centre For Medical Image Computing, University College London, London, UK.
| | - João Ramalhinho
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
- Centre For Medical Image Computing, University College London, London, UK
| | - Stephen Thompson
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
- Centre For Medical Image Computing, University College London, London, UK
| | - Kurinchi Gurusamy
- Division of Surgery and Interventional Science, University College London, London, UK
| | - Brian Davidson
- Division of Surgery and Interventional Science, University College London, London, UK
| | - David Hawkes
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
- Centre For Medical Image Computing, University College London, London, UK
| | - Danail Stoyanov
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
- Centre For Medical Image Computing, University College London, London, UK
| | - Matthew J Clarkson
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
- Centre For Medical Image Computing, University College London, London, UK
| |
Collapse
|
22
|
Plantefève R, Peterlik I, Cotin S. Intraoperative Biomechanical Registration of the Liver: Does the Heterogeneity of the Liver Matter? Ing Rech Biomed 2018. [DOI: 10.1016/j.irbm.2017.10.005] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
23
|
Peterlík I, Courtecuisse H, Rohling R, Abolmaesumi P, Nguan C, Cotin S, Salcudean S. Fast elastic registration of soft tissues under large deformations. Med Image Anal 2017; 45:24-40. [PMID: 29414434 DOI: 10.1016/j.media.2017.12.006] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2016] [Revised: 12/07/2017] [Accepted: 12/07/2017] [Indexed: 12/21/2022]
Abstract
A fast and accurate fusion of intra-operative images with a pre-operative data is a key component of computer-aided interventions which aim at improving the outcomes of the intervention while reducing the patient's discomfort. In this paper, we focus on the problematic of the intra-operative navigation during abdominal surgery, which requires an accurate registration of tissues undergoing large deformations. Such a scenario occurs in the case of partial hepatectomy: to facilitate the access to the pathology, e.g. a tumor located in the posterior part of the right lobe, the surgery is performed on a patient in lateral position. Due to the change in patient's position, the resection plan based on the pre-operative CT scan acquired in the supine position must be updated to account for the deformations. We suppose that an imaging modality, such as the cone-beam CT, provides the information about the intra-operative shape of an organ, however, due to the reduced radiation dose and contrast, the actual locations of the internal structures necessary to update the planning are not available. To this end, we propose a method allowing for fast registration of the pre-operative data represented by a detailed 3D model of the liver and its internal structure and the actual configuration given by the organ surface extracted from the intra-operative image. The algorithm behind the method combines the iterative closest point technique with a biomechanical model based on a co-rotational formulation of linear elasticity which accounts for large deformations of the tissue. The performance, robustness and accuracy of the method is quantitatively assessed on a control semi-synthetic dataset with known ground truth and a real dataset composed of nine pairs of abdominal CT scans acquired in supine and flank positions. It is shown that the proposed surface-matching method is capable of reducing the target registration error evaluated of the internal structures of the organ from more than 40 mm to less then 10 mm. Moreover, the control data is used to demonstrate the compatibility of the method with intra-operative clinical scenario, while the real datasets are utilized to study the impact of parametrization on the accuracy of the method. The method is also compared to a state-of-the art intensity-based registration technique in terms of accuracy and performance.
Collapse
Affiliation(s)
- Igor Peterlík
- MIMESIS, Inria Nancy, France; ICube, University of Strasbourg, CNRS, Strasbourg, France; Institute of Computer Science, Masaryk University, Brno, Czech Republic.
| | - Hadrien Courtecuisse
- ICube, University of Strasbourg, CNRS, Strasbourg, France; MIMESIS, Inria Nancy, France
| | - Robert Rohling
- Department of Electrical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Purang Abolmaesumi
- Department of Electrical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Christopher Nguan
- Urology Department, Vancouver General Hospital, Vancouver, BC, Canada
| | - Stéphane Cotin
- MIMESIS, Inria Nancy, France; ICube, University of Strasbourg, CNRS, Strasbourg, France
| | - Septimiu Salcudean
- Department of Electrical Engineering, University of British Columbia, Vancouver, BC, Canada
| |
Collapse
|
24
|
Heiselman JS, Clements LW, Collins JA, Weis JA, Simpson AL, Geevarghese SK, Kingham TP, Jarnagin WR, Miga MI. Characterization and correction of intraoperative soft tissue deformation in image-guided laparoscopic liver surgery. J Med Imaging (Bellingham) 2017; 5:021203. [PMID: 29285519 DOI: 10.1117/1.jmi.5.2.021203] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2017] [Accepted: 11/21/2017] [Indexed: 12/12/2022] Open
Abstract
Laparoscopic liver surgery is challenging to perform due to a compromised ability of the surgeon to localize subsurface anatomy in the constrained environment. While image guidance has the potential to address this barrier, intraoperative factors, such as insufflation and variable degrees of organ mobilization from supporting ligaments, may generate substantial deformation. The severity of laparoscopic deformation in humans has not been characterized, and current laparoscopic correction methods do not account for the mechanics of how intraoperative deformation is applied to the liver. We first measure the degree of laparoscopic deformation at two insufflation pressures over the course of laparoscopic-to-open conversion in 25 patients. With this clinical data alongside a mock laparoscopic phantom setup, we report a biomechanical correction approach that leverages anatomically load-bearing support surfaces from ligament attachments to iteratively reconstruct and account for intraoperative deformations. Laparoscopic deformations were significantly larger than deformations associated with open surgery, and our correction approach yielded subsurface target error of [Formula: see text] and surface error of [Formula: see text] using only sparse surface data with realistic surgical extent. Laparoscopic surface data extents were examined and found to impact registration accuracy. Finally, we demonstrate viability of the correction method with clinical data.
Collapse
Affiliation(s)
- Jon S Heiselman
- Vanderbilt University, Department of Biomedical Engineering, Nashville, Tennessee, United States.,Vanderbilt University, Vanderbilt Institute for Surgery and Engineering, Nashville, Tennessee, United States
| | - Logan W Clements
- Vanderbilt University, Department of Biomedical Engineering, Nashville, Tennessee, United States.,Vanderbilt University, Vanderbilt Institute for Surgery and Engineering, Nashville, Tennessee, United States
| | - Jarrod A Collins
- Vanderbilt University, Department of Biomedical Engineering, Nashville, Tennessee, United States.,Vanderbilt University, Vanderbilt Institute for Surgery and Engineering, Nashville, Tennessee, United States
| | - Jared A Weis
- Wake Forest School of Medicine, Department of Biomedical Engineering, Winston-Salem, North Carolina, United States
| | - Amber L Simpson
- Memorial Sloan-Kettering Cancer Center, Hepatopancreatobiliary Service, Department of Surgery, New York, New York, United States
| | - Sunil K Geevarghese
- Vanderbilt University Medical Center, Division of Hepatobiliary Surgery and Liver Transplantation, Nashville, Tennessee, United States
| | - T Peter Kingham
- Memorial Sloan-Kettering Cancer Center, Hepatopancreatobiliary Service, Department of Surgery, New York, New York, United States
| | - William R Jarnagin
- Memorial Sloan-Kettering Cancer Center, Hepatopancreatobiliary Service, Department of Surgery, New York, New York, United States
| | - Michael I Miga
- Vanderbilt University, Department of Biomedical Engineering, Nashville, Tennessee, United States.,Vanderbilt University, Vanderbilt Institute for Surgery and Engineering, Nashville, Tennessee, United States
| |
Collapse
|
25
|
Robu MR, Edwards P, Ramalhinho J, Thompson S, Davidson B, Hawkes D, Stoyanov D, Clarkson MJ. Intelligent viewpoint selection for efficient CT to video registration in laparoscopic liver surgery. Int J Comput Assist Radiol Surg 2017; 12:1079-1088. [PMID: 28401399 PMCID: PMC5509843 DOI: 10.1007/s11548-017-1584-7] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2017] [Accepted: 03/30/2017] [Indexed: 01/27/2023]
Abstract
PURPOSE Minimally invasive surgery offers advantages over open surgery due to a shorter recovery time, less pain and trauma for the patient. However, inherent challenges such as lack of tactile feedback and difficulty in controlling bleeding lower the percentage of suitable cases. Augmented reality can show a better visualisation of sub-surface structures and tumour locations by fusing pre-operative CT data with real-time laparoscopic video. Such augmented reality visualisation requires a fast and robust video to CT registration that minimises interruption to the surgical procedure. METHODS We propose to use view planning for efficient rigid registration. Given the trocar position, a set of camera positions are sampled and scored based on the corresponding liver surface properties. We implement a simulation framework to validate the proof of concept using a segmented CT model from a human patient. Furthermore, we apply the proposed method on clinical data acquired during a human liver resection. RESULTS The first experiment motivates the viewpoint scoring strategy and investigates reliable liver regions for accurate registrations in an intuitive visualisation. The second experiment shows wider basins of convergence for higher scoring viewpoints. The third experiment shows that a comparable registration performance can be achieved by at least two merged high scoring views and four low scoring views. Hence, the focus could change from the acquisition of a large liver surface to a small number of distinctive patches, thereby giving a more explicit protocol for surface reconstruction. We discuss the application of the proposed method on clinical data and show initial results. CONCLUSION The proposed simulation framework shows promising results to motivate more research into a comprehensive view planning method for efficient registration in laparoscopic liver surgery.
Collapse
Affiliation(s)
- Maria R Robu
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK.
| | - Philip Edwards
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| | - João Ramalhinho
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| | - Stephen Thompson
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| | - Brian Davidson
- Royal Free Campus, UCL Medical School, 9th Floor, Royal Free Hospital, Rowland Hill Street, London, UK
| | - David Hawkes
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| | - Danail Stoyanov
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| | - Matthew J Clarkson
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| |
Collapse
|
26
|
Schoob A, Kundrat D, Kahrs LA, Ortmaier T. Stereo vision-based tracking of soft tissue motion with application to online ablation control in laser microsurgery. Med Image Anal 2017. [PMID: 28624755 DOI: 10.1016/j.media.2017.06.004] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
Recent research has revealed that image-based methods can enhance accuracy and safety in laser microsurgery. In this study, non-rigid tracking using surgical stereo imaging and its application to laser ablation is discussed. A recently developed motion estimation framework based on piecewise affine deformation modeling is extended by a mesh refinement step and considering texture information. This compensates for tracking inaccuracies potentially caused by inconsistent feature matches or drift. To facilitate online application of the method, computational load is reduced by concurrent processing and affine-invariant fusion of tracking and refinement results. The residual latency-dependent tracking error is further minimized by Kalman filter-based upsampling, considering a motion model in disparity space. Accuracy is assessed in laparoscopic, beating heart, and laryngeal sequences with challenging conditions, such as partial occlusions and significant deformation. Performance is compared with that of state-of-the-art methods. In addition, the online capability of the method is evaluated by tracking two motion patterns performed by a high-precision parallel-kinematic platform. Related experiments are discussed for tissue substitute and porcine soft tissue in order to compare performances in an ideal scenario and in a setup mimicking clinical conditions. Regarding the soft tissue trial, the tracking error can be significantly reduced from 0.72 mm to below 0.05 mm with mesh refinement. To demonstrate online laser path adaptation during ablation, the non-rigid tracking framework is integrated into a setup consisting of a surgical Er:YAG laser, a three-axis scanning unit, and a low-noise stereo camera. Regardless of the error source, such as laser-to-camera registration, camera calibration, image-based tracking, and scanning latency, the ablation root mean square error is kept below 0.21 mm when the sample moves according to the aforementioned patterns. Final experiments regarding motion-compensated laser ablation of structurally deforming tissue highlight the potential of the method for vision-guided laser surgery.
Collapse
Affiliation(s)
- Andreas Schoob
- Leibniz Universität Hannover, Institute of Mechatronic Systems, Appelstr. 11a, 30167 Hanover, Germany.
| | - Dennis Kundrat
- Leibniz Universität Hannover, Institute of Mechatronic Systems, Appelstr. 11a, 30167 Hanover, Germany
| | - Lüder A Kahrs
- Leibniz Universität Hannover, Institute of Mechatronic Systems, Appelstr. 11a, 30167 Hanover, Germany
| | - Tobias Ortmaier
- Leibniz Universität Hannover, Institute of Mechatronic Systems, Appelstr. 11a, 30167 Hanover, Germany
| |
Collapse
|
27
|
Reichard D, Häntsch D, Bodenstedt S, Suwelack S, Wagner M, Kenngott H, Müller-Stich B, Maier-Hein L, Dillmann R, Speidel S. Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery. Int J Comput Assist Radiol Surg 2017; 12:1101-1110. [PMID: 28550405 DOI: 10.1007/s11548-017-1613-6] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2017] [Accepted: 05/15/2017] [Indexed: 10/19/2022]
Abstract
PURPOSE A key component of computer- assisted surgery systems is the accurate and robust registration of preoperative planning data with intraoperative sensor data. In laparoscopic surgery, this image-based registration remains challenging due to soft tissue deformations. This paper presents a novel approach for biomechanical soft tissue registration of preoperative CT data with stereo endoscopic image data. METHODS The proposed method consists of two registrations steps. First, we use a 3D surface mosaic from partial surfaces reconstructed from stereo endoscopic images to initially align the biomechanical model with the intraoperative position and shape of the organ. After this initialization, the biomechanical model is projected onto newly captured surfaces, resulting in displacement boundary conditions, which in turn are used to update the biomechanical model. RESULTS The method is evaluated in silico, using a human liver model, and in vivo, using porcine data. The quantitative in silico data shows a stable behaviour of the biomechanical model and root-mean-square deviation of volume vertices of under 3 mm with adjusted biomechanical parameters. CONCLUSION This work contributes a fully automatic featureless non-rigid registration approach. The results of the in silico and in vivo experiments suggest that our method is able to handle dynamic deformations during surgery. Additional experiments, especially regarding human tissue behaviour, are an important next step towards clinical applications.
Collapse
Affiliation(s)
- Daniel Reichard
- Karlsruhe Institute of Technology, Adenauerring 2, Bldg. 50.20, Karlsruhe, Germany.
| | - Dominik Häntsch
- Karlsruhe Institute of Technology, Adenauerring 2, Bldg. 50.20, Karlsruhe, Germany
| | - Sebastian Bodenstedt
- Karlsruhe Institute of Technology, Adenauerring 2, Bldg. 50.20, Karlsruhe, Germany
| | - Stefan Suwelack
- Karlsruhe Institute of Technology, Adenauerring 2, Bldg. 50.20, Karlsruhe, Germany
| | - Martin Wagner
- Department of General, Abdominal and Transplantation Surgery, University of Heidelberg, Heidelberg, Germany
| | - Hannes Kenngott
- Department of General, Abdominal and Transplantation Surgery, University of Heidelberg, Heidelberg, Germany
| | - Beat Müller-Stich
- Department of General, Abdominal and Transplantation Surgery, University of Heidelberg, Heidelberg, Germany
| | - Lena Maier-Hein
- Junior Group Computer-Assisted Interventions, Division of Medical and Biological Informatics, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Rüdiger Dillmann
- Karlsruhe Institute of Technology, Adenauerring 2, Bldg. 50.20, Karlsruhe, Germany
| | - Stefanie Speidel
- Karlsruhe Institute of Technology, Adenauerring 2, Bldg. 50.20, Karlsruhe, Germany
| |
Collapse
|
28
|
The status of augmented reality in laparoscopic surgery as of 2016. Med Image Anal 2017; 37:66-90. [DOI: 10.1016/j.media.2017.01.007] [Citation(s) in RCA: 183] [Impact Index Per Article: 26.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2016] [Revised: 01/16/2017] [Accepted: 01/23/2017] [Indexed: 12/27/2022]
|
29
|
Image Guidance in Endoscopic Sinus Surgery: Where Are We Heading? CURRENT OTORHINOLARYNGOLOGY REPORTS 2017. [DOI: 10.1007/s40136-017-0140-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
30
|
Si W, Liao X, Wang Q, Heng PA. Personalized heterogeneous deformable model for fast volumetric registration. Biomed Eng Online 2017; 16:30. [PMID: 28219432 PMCID: PMC5319060 DOI: 10.1186/s12938-017-0321-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2016] [Accepted: 02/10/2017] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Biomechanical deformable volumetric registration can help improve safety of surgical interventions by ensuring the operations are extremely precise. However, this technique has been limited by the accuracy and the computational efficiency of patient-specific modeling. METHODS This study presents a tissue-tissue coupling strategy based on penalty method to model the heterogeneous behavior of deformable body, and estimate the personalized tissue-tissue coupling parameters in a data-driven way. Moreover, considering that the computational efficiency of biomechanical model is highly dependent on the mechanical resolution, a practical coarse-to-fine scheme is proposed to increase runtime efficiency. Particularly, a detail enrichment database is established in an offline fashion to represent the mapping relationship between the deformation results of high-resolution hexahedral mesh extracted from the raw medical data and a newly constructed low-resolution hexahedral mesh. At runtime, the mechanical behavior of human organ under interactions is simulated with this low-resolution hexahedral mesh, then the microstructures are synthesized in virtue of the detail enrichment database. RESULTS The proposed method is validated by volumetric registration in an abdominal phantom compression experiments. Our personalized heterogeneous deformable model can well describe the coupling effects between different tissues of the phantom. Compared with high-resolution heterogeneous deformable model, the low-resolution deformable model with our detail enrichment database can achieve 9.4× faster, and the average target registration error is 3.42 mm, which demonstrates that the proposed method shows better volumetric registration performance than state-of-the-art. CONCLUSIONS Our framework can well balance the precision and efficiency, and has great potential to be adopted in the practical augmented reality image-guided robotic systems.
Collapse
Affiliation(s)
- Weixin Si
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong.,Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, 503644, Shenzhen, China
| | - Xiangyun Liao
- Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, 503644, Shenzhen, China
| | - Qiong Wang
- Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, 503644, Shenzhen, China.
| | - Pheng Ann Heng
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong.,Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, 503644, Shenzhen, China
| |
Collapse
|
31
|
Wang KF, Mo LQ, Kong DX. Role of mathematical medicine in gastrointestinal carcinoma: Current status and perspectives. Shijie Huaren Xiaohua Zazhi 2017; 25:114-121. [DOI: 10.11569/wcjd.v25.i2.114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/06/2023] Open
Abstract
Mathematical medicine has already played an important role in clinical and basic research as a major interdisciplinary branch of medicine. Mathematical medicine has an important role not only in imaging diagnosis, image storage and transmission in gastrointestinal (GI) cancer, but also in tumor precision therapy. Specifically, in the field of minimally invasive treatment such as precise ablation, 3-dimension modeling, navigation, and surgical simulation significantly improve the therapeutic safety and efficiency in GI cancer. In addition, in the era of big data, data analysis and individualized therapy using mathematical medicine will become a trend in the future, offering an effective method for diagnosing and treating GI cancer and promoting clinical and scientific research.
Collapse
|
32
|
Paulus CJ, Haouchine N, Kong SH, Soares RV, Cazier D, Cotin S. Handling topological changes during elastic registration. Int J Comput Assist Radiol Surg 2016; 12:461-470. [DOI: 10.1007/s11548-016-1502-4] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2016] [Accepted: 11/04/2016] [Indexed: 02/07/2023]
|
33
|
Robust augmented reality guidance with fluorescent markers in laparoscopic surgery. Int J Comput Assist Radiol Surg 2016; 11:899-907. [DOI: 10.1007/s11548-016-1385-4] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2016] [Accepted: 03/14/2016] [Indexed: 11/25/2022]
|
34
|
Haouchine N, Dequidt J, Berger MO, Cotin S. Monocular 3D Reconstruction and Augmentation of Elastic Surfaces with Self-Occlusion Handling. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2015; 21:1363-1376. [PMID: 26529459 DOI: 10.1109/tvcg.2015.2452905] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
This paper focuses on the 3D shape recovery and augmented reality on elastic objects with self-occlusions handling, using only single view images. Shape recovery from a monocular video sequence is an underconstrained problem and many approaches have been proposed to enforce constraints and resolve the ambiguities. State-of-the art solutions enforce smoothness or geometric constraints, consider specific deformation properties such as inextensibility or resort to shading constraints. However, few of them can handle properly large elastic deformations. We propose in this paper a real-time method that uses a mechanical model and able to handle highly elastic objects. The problem is formulated as an energy minimization problem accounting for a non-linear elastic model constrained by external image points acquired from a monocular camera. This method prevents us from formulating restrictive assumptions and specific constraint terms in the minimization. In addition, we propose to handle self-occluded regions thanks to the ability of mechanical models to provide appropriate predictions of the shape. Our method is compared to existing techniques with experiments conducted on computer-generated and real data that show the effectiveness of recovering and augmenting 3D elastic objects. Additionally, experiments in the context of minimally invasive liver surgery are also provided and results on deformations with the presence of self-occlusions are exposed.
Collapse
|
35
|
Song Y, Totz J, Thompson S, Johnsen S, Barratt D, Schneider C, Gurusamy K, Davidson B, Ourselin S, Hawkes D, Clarkson MJ. Locally rigid, vessel-based registration for laparoscopic liver surgery. Int J Comput Assist Radiol Surg 2015; 10:1951-61. [PMID: 26092658 PMCID: PMC4642598 DOI: 10.1007/s11548-015-1236-8] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2015] [Accepted: 05/30/2015] [Indexed: 12/05/2022]
Abstract
PURPOSE Laparoscopic liver resection has significant advantages over open surgery due to less patient trauma and faster recovery times, yet is difficult for most lesions due to the restricted field of view and lack of haptic feedback. Image guidance provides a potential solution but is challenging in a soft deforming organ such as the liver. In this paper, we therefore propose a laparoscopic ultrasound (LUS) image guidance system and study the feasibility of a locally rigid registration for laparoscopic liver surgery. METHODS We developed a real-time segmentation method to extract vessel centre points from calibrated, freehand, electromagnetically tracked, 2D LUS images. Using landmark-based initial registration and an optional iterative closest point (ICP) point-to-line registration, a vessel centre-line model extracted from preoperative computed tomography (CT) is registered to the ultrasound data during surgery. RESULTS Using the locally rigid ICP method, the RMS residual error when registering to a phantom was 0.7 mm, and the mean target registration error (TRE) for two in vivo porcine studies was 3.58 and 2.99 mm, respectively. Using the locally rigid landmark-based registration method gave a mean TRE of 4.23 mm using vessel centre lines derived from CT scans taken with pneumoperitoneum and 6.57 mm without pneumoperitoneum. CONCLUSION In this paper we propose a practical image-guided surgery system based on locally rigid registration of a CT-derived model to vascular structures located with LUS. In a physical phantom and during porcine laparoscopic liver resection, we demonstrate accuracy of target location commensurate with surgical requirements. We conclude that locally rigid registration could be sufficient for practically useful image guidance in the near future.
Collapse
Affiliation(s)
- Yi Song
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK.
| | - Johannes Totz
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| | - Steve Thompson
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| | - Stian Johnsen
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| | - Dean Barratt
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| | - Crispin Schneider
- Royal Free Campus, 9th Floor, Royal Free Hospital, UCL Medical School, Rowland Hill Street, London, UK
| | - Kurinchi Gurusamy
- Royal Free Campus, 9th Floor, Royal Free Hospital, UCL Medical School, Rowland Hill Street, London, UK
| | - Brian Davidson
- Royal Free Campus, 9th Floor, Royal Free Hospital, UCL Medical School, Rowland Hill Street, London, UK
| | - Sébastien Ourselin
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| | - David Hawkes
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK
| | - Matthew J Clarkson
- Centre For Medical Image Computing, Engineering Front Building, University College London, Malet Place, London, UK.
| |
Collapse
|
36
|
Paulus CJ, Haouchine N, Cazier D, Cotin S. Surgical Augmented Reality with Topological Changes. ACTA ACUST UNITED AC 2015. [DOI: 10.1007/978-3-319-24553-9_51] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/09/2023]
|
37
|
Patient-Specific Biomechanical Modeling for Guidance During Minimally-Invasive Hepatic Surgery. Ann Biomed Eng 2015; 44:139-53. [DOI: 10.1007/s10439-015-1419-z] [Citation(s) in RCA: 79] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2015] [Accepted: 08/05/2015] [Indexed: 11/26/2022]
|
38
|
Haouchine N, Cotin S, Peterlik I, Dequidt J, Lopez MS, Kerrien E, Berger MO. Impact of Soft Tissue Heterogeneity on Augmented Reality for Liver Surgery. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2015; 21:584-597. [PMID: 26357206 DOI: 10.1109/tvcg.2014.2377772] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
This paper presents a method for real-time augmented reality of internal liver structures during minimally invasive hepatic surgery. Vessels and tumors computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Compared to current methods, our method is able to locate the in-depth positions of the tumors based on partial three-dimensional liver tissue motion using a real-time biomechanical model. This model permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Experimentations conducted on phantom liver permits to measure the accuracy of the augmentation while real-time augmentation on in vivo human liver during real surgery shows the benefits of such an approach for minimally invasive surgery.
Collapse
|