1
|
Guan P, Luo H, Guo J, Zhang Y, Jia F. Intraoperative laparoscopic liver surface registration with preoperative CT using mixing features and overlapping region masks. Int J Comput Assist Radiol Surg 2023:10.1007/s11548-023-02846-w. [PMID: 36787037 DOI: 10.1007/s11548-023-02846-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Accepted: 01/27/2023] [Indexed: 02/15/2023]
Abstract
PURPOSE Laparoscopic liver resection is a minimal invasive surgery. Augmented reality can map preoperative anatomy information extracted from computed tomography to the intraoperative liver surface reconstructed from stereo 3D laparoscopy. However, liver surface registration is particularly challenging as the intraoperative surface is only partially visible and suffers from large liver deformations due to pneumoperitoneum. This study proposes a deep learning-based robust point cloud registration network. METHODS This study proposed a low overlap liver surface registration algorithm combining local mixed features and global features of point clouds. A learned overlap mask is used to filter the non-overlapping region of the point cloud, and a network is used to predict the overlapping region threshold to regulate the training process. RESULTS We validated the algorithm on the DePoLL (the Deformable Porcine Laparoscopic Liver) dataset. Compared with the baseline method and other state-of-the-art registration methods, our method achieves minimum target registration error (TRE) of 19.9 ± 2.7 mm. CONCLUSION The proposed point cloud registration method uses the learned overlapping mask to filter the non-overlapping areas in the point cloud, then the extracted overlapping area point cloud is registered according to the mixed features and global features, and this method is robust and efficient in low-overlap liver surface registration.
Collapse
Affiliation(s)
- Peidong Guan
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.,Shenzhen College of Advanced Technology, University of Chinese Academy and Sciences, Shenzhen, China
| | - Huoling Luo
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Jianxi Guo
- Department of Interventional Radiology, Shenzhen People's Hospital, Shenzhen, China
| | - Yanfang Zhang
- Department of Interventional Radiology, Shenzhen People's Hospital, Shenzhen, China.
| | - Fucang Jia
- Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. .,Shenzhen College of Advanced Technology, University of Chinese Academy and Sciences, Shenzhen, China. .,Pazhou Lab, Guangzhou, China.
| |
Collapse
|
2
|
Marahrens N, Scaglioni B, Jones D, Prasad R, Biyani CS, Valdastri P. Towards Autonomous Robotic Minimally Invasive Ultrasound Scanning and Vessel Reconstruction on Non-Planar Surfaces. Front Robot AI 2022; 9:940062. [PMID: 36304794 PMCID: PMC9594548 DOI: 10.3389/frobt.2022.940062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Accepted: 06/09/2022] [Indexed: 11/13/2022] Open
Abstract
Autonomous robotic Ultrasound (US) scanning has been the subject of research for more than 2 decades. However, little work has been done to apply this concept into a minimally invasive setting, in which accurate force sensing is generally not available and robot kinematics are unreliable due to the tendon-driven, compliant robot structure. As a result, the adequate orientation of the probe towards the tissue surface remains unknown and the anatomy reconstructed from scan may become highly inaccurate. In this work we present solutions to both of these challenges: an attitude sensor fusion scheme for improved kinematic sensing and a visual, deep learning based algorithm to establish and maintain contact between the organ surface and the US probe. We further introduce a novel scheme to estimate and orient the probe perpendicular to the center line of a vascular structure. Our approach enables, for the first time, to autonomously scan across a non-planar surface and navigate along an anatomical structure with a robotically guided minimally invasive US probe. Our experiments on a vessel phantom with a convex surface confirm a significant improvement of the reconstructed curved vessel geometry, with our approach strongly reducing the mean positional error and variance. In the future, our approach could help identify vascular structures more effectively and help pave the way towards semi-autonomous assistance during partial hepatectomy and the potential to reduce procedure length and complication rates.
Collapse
Affiliation(s)
- Nils Marahrens
- Storm Lab UK, Institute for Robotics, Autonomous Systems and Sensing, School of Electronic and Electrical Engineering, University of Leeds, Leeds, United Kingdom
- *Correspondence: Nils Marahrens,
| | - Bruno Scaglioni
- Storm Lab UK, Institute for Robotics, Autonomous Systems and Sensing, School of Electronic and Electrical Engineering, University of Leeds, Leeds, United Kingdom
| | - Dominic Jones
- Storm Lab UK, Institute for Robotics, Autonomous Systems and Sensing, School of Electronic and Electrical Engineering, University of Leeds, Leeds, United Kingdom
| | - Raj Prasad
- Department for Hepatobiliary Surgery, James’s University Hospital, Leeds Teachings Hospitals NHS Trust, Leeds, United Kingdom
| | - Chandra Shekhar Biyani
- Department for Urology, James’s University Hospital, Leeds Teachings Hospitals NHS Trust, Leeds, United Kingdom
| | - Pietro Valdastri
- Storm Lab UK, Institute for Robotics, Autonomous Systems and Sensing, School of Electronic and Electrical Engineering, University of Leeds, Leeds, United Kingdom
| |
Collapse
|
3
|
Robotic Surgery: Rediscovering Human Anatomy. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:ijerph182312744. [PMID: 34886470 PMCID: PMC8657036 DOI: 10.3390/ijerph182312744] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Revised: 11/24/2021] [Accepted: 12/01/2021] [Indexed: 11/17/2022]
Abstract
Since its advent, robotic surgery has redefined the operating room experience. It directly addressed and resolved many of the shortcomings of laparoscopic methods while maintaining a minimally invasive approach that brought benefits in cosmesis and healing for patients but also benefits in ergonomics and precision for surgeons. This new platform has brought with it changes in surgical training and education, principally through the utilization of virtual reality. Accurate depictions of human anatomy seen through augmented reality allow the surgeon-in-training to learn, practice and perfect their skills before they operate on their first patient. However, the anatomical knowledge required for minimally invasive surgery (MIS) is distinct from current methods of dissection and prosection that inherently cater towards open surgery with large cuts and unobstructed field. It is integral that robotic surgeons are also equipped with accurate anatomical information, heralding a new era in which anatomists can work alongside those developing virtual reality technology to create anatomical training curricula for MIS. As the field of surgery and medicine in general moves to include more and more technology, it is only fitting that the building blocks of medical education follow suit and rediscover human anatomy in a modern context.
Collapse
|
4
|
He B, Yin D, Chen X, Luo H, Xiao D, He M, Wang G, Fang C, Liu L, Jia F. A study of generalization and compatibility performance of 3D U-Net segmentation on multiple heterogeneous liver CT datasets. BMC Med Imaging 2021; 21:178. [PMID: 34819022 PMCID: PMC8611902 DOI: 10.1186/s12880-021-00708-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2021] [Accepted: 11/15/2021] [Indexed: 12/22/2022] Open
Abstract
BACKGROUND Most existing algorithms have been focused on the segmentation from several public Liver CT datasets scanned regularly (no pneumoperitoneum and horizontal supine position). This study primarily segmented datasets with unconventional liver shapes and intensities deduced by contrast phases, irregular scanning conditions, different scanning objects of pigs and patients with large pathological tumors, which formed the multiple heterogeneity of datasets used in this study. METHODS The multiple heterogeneous datasets used in this paper includes: (1) One public contrast-enhanced CT dataset and one public non-contrast CT dataset; (2) A contrast-enhanced dataset that has abnormal liver shape with very long left liver lobes and large-sized liver tumors with abnormal presets deduced by microvascular invasion; (3) One artificial pneumoperitoneum dataset under the pneumoperitoneum and three scanning profiles (horizontal/left/right recumbent position); (4) Two porcine datasets of Bama type and domestic type that contains pneumoperitoneum cases but with large anatomy discrepancy with humans. The study aimed to investigate the segmentation performances of 3D U-Net in: (1) generalization ability between multiple heterogeneous datasets by cross-testing experiments; (2) the compatibility when hybrid training all datasets in different sampling and encoder layer sharing schema. We further investigated the compatibility of encoder level by setting separate level for each dataset (i.e., dataset-wise convolutions) while sharing the decoder. RESULTS Model trained on different datasets has different segmentation performance. The prediction accuracy between LiTS dataset and Zhujiang dataset was about 0.955 and 0.958 which shows their good generalization ability due to that they were all contrast-enhanced clinical patient datasets scanned regularly. For the datasets scanned under pneumoperitoneum, their corresponding datasets scanned without pneumoperitoneum showed good generalization ability. Dataset-wise convolution module in high-level can improve the dataset unbalance problem. The experimental results will facilitate researchers making solutions when segmenting those special datasets. CONCLUSIONS (1) Regularly scanned datasets is well generalized to irregularly ones. (2) The hybrid training is beneficial but the dataset imbalance problem always exits due to the multi-domain homogeneity. The higher levels encoded more domain specific information than lower levels and thus were less compatible in terms of our datasets.
Collapse
Affiliation(s)
- Baochun He
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China
| | - Dalong Yin
- Department of Hepatobiliary Surgery, The First Affiliated Hospital, Harbin Medical University, Harbin, China
- Department of Hepatobiliary Surgery, The First Affiliated Hospital, University of Science and Technology of China, Hefei, China
| | - Xiaoxia Chen
- Department of Radiology, The Third Medical Center, General Hospital of PLA, Beijing, China
| | - Huoling Luo
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China
| | - Deqiang Xiao
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China
| | - Mu He
- First Hepatobiliary Surgery, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Guisheng Wang
- Department of Radiology, The Third Medical Center, General Hospital of PLA, Beijing, China
| | - Chihua Fang
- First Hepatobiliary Surgery, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Lianxin Liu
- Department of Hepatobiliary Surgery, The First Affiliated Hospital, Harbin Medical University, Harbin, China.
- Department of Hepatobiliary Surgery, The First Affiliated Hospital, University of Science and Technology of China, Hefei, China.
| | - Fucang Jia
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China.
- Pazhou Lab, Guangzhou, China.
| |
Collapse
|