1
|
Wang C, Guo L, Zhu J, Zhu L, Li C, Zhu H, Song A, Lu L, Teng GJ, Navab N, Jiang Z. Review of robotic systems for thoracoabdominal puncture interventional surgery. APL Bioeng 2024; 8:021501. [PMID: 38572313 PMCID: PMC10987197 DOI: 10.1063/5.0180494] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Accepted: 03/11/2024] [Indexed: 04/05/2024] Open
Abstract
Cancer, with high morbidity and high mortality, is one of the major burdens threatening human health globally. Intervention procedures via percutaneous puncture have been widely used by physicians due to its minimally invasive surgical approach. However, traditional manual puncture intervention depends on personal experience and faces challenges in terms of precisely puncture, learning-curve, safety and efficacy. The development of puncture interventional surgery robotic (PISR) systems could alleviate the aforementioned problems to a certain extent. This paper attempts to review the current status and prospective of PISR systems for thoracic and abdominal application. In this review, the key technologies related to the robotics, including spatial registration, positioning navigation, puncture guidance feedback, respiratory motion compensation, and motion control, are discussed in detail.
Collapse
Affiliation(s)
- Cheng Wang
- Hanglok-Tech Co. Ltd., Hengqin 519000, People's Republic of China
| | - Li Guo
- Hanglok-Tech Co. Ltd., Hengqin 519000, People's Republic of China
| | | | - Lifeng Zhu
- State Key Laboratory of Digital Medical Engineering, Jiangsu Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, People's Republic of China
| | - Chichi Li
- School of Computer Science and Engineering, Macau University of Science and Technology, Macau, 999078, People's Republic of China
| | - Haidong Zhu
- Center of Interventional Radiology and Vascular Surgery, Department of Radiology, Zhongda Hospital, Medical School, Southeast University, Nanjing 210009, People's Republic of China
| | - Aiguo Song
- State Key Laboratory of Digital Medical Engineering, Jiangsu Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, People's Republic of China
| | | | - Gao-Jun Teng
- Center of Interventional Radiology and Vascular Surgery, Department of Radiology, Zhongda Hospital, Medical School, Southeast University, Nanjing 210009, People's Republic of China
| | | | - Zhongliang Jiang
- Computer Aided Medical Procedures, Technical University of Munich, Munich 80333, Germany
| |
Collapse
|
2
|
Fang H, Li H, Song S, Pang K, Ai D, Fan J, Song H, Yu Y, Yang J. Motion-flow-guided recurrent network for respiratory signal estimation of x-ray angiographic image sequences. Phys Med Biol 2020; 65:245020. [PMID: 32590382 DOI: 10.1088/1361-6560/aba087] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
Motion compensation can eliminate inconsistencies of respiratory movement during image acquisitions for precise vascular reconstruction in the clinical diagnosis of vascular disease from x-ray angiographic image sequences. In x-ray-based vascular interventional therapy, motion modeling can simulate the process of organ deformation driven by motion signals to display a dynamic organ on angiograms without contrast agent injection. Automatic respiratory signal estimation from x-ray angiographic image sequences is essential for motion compensation and modeling. The effects of respiratory motion, cardiac impulses, and tremors on structures in the chest and abdomen bring difficulty in extracting accurate respiratory signals individually. In this study, an end-to-end deep learning framework based on a motion-flow-guided recurrent network is proposed to address the aforementioned problem. The proposed method utilizes a convolutional neural network to learn the spatial features of every single frame, and a recurrent neural network to learn the temporal features of the entire sequence. The combination of the two networks can effectively analyze the image sequence to realize respiratory signal estimation. In addition, the motion-flow between consecutive frames is introduced to provide a dynamic constraint of spatial features, which enables the recurrent network to learn better temporal features from dynamic spatial features than from static spatial features. We demonstrate the advantages of our approach on designed datasets which contain coronary and hepatic angiographic sequences with diaphragm structures, and coronary angiographic sequences without diaphragm structures. Our method improves over state-of-the-art manifold-learning-based methods by 85.7%, 81.5% and 75.3% in respiratory signal accuracy metric on these datasets. The results demonstrate that the proposed method can effectively estimate respiratory signals from multiple motion patterns.
Collapse
Affiliation(s)
- Huihui Fang
- Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, People's Republic of China
| | | | | | | | | | | | | | | | | |
Collapse
|
3
|
Liu H, Lin Y, Ibragimov B, Zhang C. Low dose 4D-CT super-resolution reconstruction via inter-plane motion estimation based on optical flow. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2020.102085] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
4
|
A machine learning pipeline for internal anatomical landmark embedding based on a patient surface model. Int J Comput Assist Radiol Surg 2018; 14:53-61. [DOI: 10.1007/s11548-018-1871-y] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2018] [Accepted: 10/04/2018] [Indexed: 11/27/2022]
|
5
|
Fahmi S, Simonis FFJ, Abayazid M. Respiratory motion estimation of the liver with abdominal motion as a surrogate. Int J Med Robot 2018; 14:e1940. [PMID: 30112864 PMCID: PMC6282606 DOI: 10.1002/rcs.1940] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2018] [Revised: 06/08/2018] [Accepted: 06/10/2018] [Indexed: 12/25/2022]
Abstract
Background: Respiratory‐induced motion (RIM) causes uncertainties in localizing hepatic lesions, which could lead to inaccurate targeting during interventions. One approach to mitigate the problem is respiratory motion estimation (RME), in which the liver motion is estimated by measuring external signals called surrogates. Methods: A learning‐based approach has been developed and validated to estimate the RIM of hepatic lesions. External markers placed on the human's abdomen were chosen as surrogates. Accordingly, appropriate motion models (multivariate, Ridge and Lasso regression models) were designed to correlate the liver motion with the abdominal motion, and trained to estimate the superior–inferior (SI) motion of the liver. Three subjects volunteered for 6 sessions of such that liver images acquired by magnetic resonance imaging (MRI) were recorded alongside camera‐tracked external markers. Results and conclusions: The proposed machine learning approach was validated in MRI on human subjects and the results show that the approach could estimate the respiratory‐induced SI motion of the liver with a mean absolute error (MAE) accuracy below 2 mm.
Collapse
Affiliation(s)
- Shamel Fahmi
- Robotics and Mechatronics group (RaM), the faculty of Electrical Engineering Mathematics and Computer Science, Technical Medical Centre, University of Twente, Enschede, 7500AE, the Netherlands.,Advanced Robotics Department, Istituto Italiano di Tecnologia, Genova, 16163, Italy
| | - Frank F J Simonis
- Magnetic Detection and Imaging Department, Faculty of Science and Technology, University of Twente, Enschede, 7500AE, the Netherlands
| | - Momen Abayazid
- Robotics and Mechatronics group (RaM), the faculty of Electrical Engineering Mathematics and Computer Science, Technical Medical Centre, University of Twente, Enschede, 7500AE, the Netherlands
| |
Collapse
|
6
|
Wilms M, Werner R, Yamamoto T, Handels H, Ehrhardt J. Subpopulation-based correspondence modelling for improved respiratory motion estimation in the presence of inter-fraction motion variations. ACTA ACUST UNITED AC 2017; 62:5823-5839. [DOI: 10.1088/1361-6560/aa70cc] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
|
7
|
Fischer P, Pohl T, Faranesh A, Maier A, Hornegger J. Unsupervised Learning for Robust Respiratory Signal Estimation From X-Ray Fluoroscopy. IEEE TRANSACTIONS ON MEDICAL IMAGING 2017; 36:865-877. [PMID: 27654320 PMCID: PMC5489115 DOI: 10.1109/tmi.2016.2609888] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
Respiratory signals are required for image gating and motion compensation in minimally invasive interventions. In X-ray fluoroscopy, extraction of a respiratory signal can be challenging due to characteristics of interventional imaging, in particular injection of contrast agent and automatic exposure control. We present a novel method for respiratory signal extraction based on dimensionality reduction that can tolerate these events. Images are divided into patches of multiple sizes. Low-dimensional embeddings are generated for each patch using illumination-invariant kernel PCA. Patches with respiratory information are selected automatically by agglomerative clustering. The signals from this respiratory cluster are combined robustly to a single respiratory signal. In the experiments, we evaluate our method on a variety of scenarios. If the diaphragm is visible, we track its superior-inferior motion as ground truth. Our method has a correlation coefficient of more than 91% with the ground truth irrespective of whether or not contrast agent injection or automatic exposure control occur. Additionally, we show that very similar signals are estimated from biplane sequences and from sequences without visible diaphragm. Since all these cases are handled automatically, the method is robust enough to be considered for use in a clinical setting.
Collapse
|