1
|
Xiao H, Xue X, Zhu M, Jiang X, Xia Q, Chen K, Li H, Long L, Peng K. Deep learning-based lung image registration: A review. Comput Biol Med 2023; 165:107434. [PMID: 37696177 DOI: 10.1016/j.compbiomed.2023.107434] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Revised: 08/13/2023] [Accepted: 08/28/2023] [Indexed: 09/13/2023]
Abstract
Lung image registration can effectively describe the relative motion of lung tissues, thereby helping to solve series problems in clinical applications. Since the lungs are soft and fairly passive organs, they are influenced by respiration and heartbeat, resulting in discontinuity of lung motion and large deformation of anatomic features. This poses great challenges for accurate registration of lung image and its applications. The recent application of deep learning (DL) methods in the field of medical image registration has brought promising results. However, a versatile registration framework has not yet emerged due to diverse challenges of registration for different regions of interest (ROI). DL-based image registration methods used for other ROI cannot achieve satisfactory results in lungs. In addition, there are few review articles available on DL-based lung image registration. In this review, the development of conventional methods for lung image registration is briefly described and a more comprehensive survey of DL-based methods for lung image registration is illustrated. The DL-based methods are classified according to different supervision types, including fully-supervised, weakly-supervised and unsupervised. The contributions of researchers in addressing various challenges are described, as well as the limitations of these approaches. This review also presents a comprehensive statistical analysis of the cited papers in terms of evaluation metrics and loss functions. In addition, publicly available datasets for lung image registration are also summarized. Finally, the remaining challenges and potential trends in DL-based lung image registration are discussed.
Collapse
Affiliation(s)
- Hanguang Xiao
- College of Artificial Intelligent, Chongqing University of Technology, Chongqing 401135, China
| | - Xufeng Xue
- College of Artificial Intelligent, Chongqing University of Technology, Chongqing 401135, China
| | - Mi Zhu
- College of Artificial Intelligent, Chongqing University of Technology, Chongqing 401135, China.
| | - Xin Jiang
- College of Artificial Intelligent, Chongqing University of Technology, Chongqing 401135, China
| | - Qingling Xia
- College of Artificial Intelligent, Chongqing University of Technology, Chongqing 401135, China
| | - Kai Chen
- College of Artificial Intelligent, Chongqing University of Technology, Chongqing 401135, China
| | - Huanqi Li
- College of Artificial Intelligent, Chongqing University of Technology, Chongqing 401135, China
| | - Li Long
- College of Artificial Intelligent, Chongqing University of Technology, Chongqing 401135, China
| | - Ke Peng
- College of Artificial Intelligent, Chongqing University of Technology, Chongqing 401135, China.
| |
Collapse
|
2
|
Krass S, Lassen-Schmidt B, Schenk A. Computer-assisted image-based risk analysis and planning in lung surgery - a review. Front Surg 2022; 9:920457. [PMID: 36211288 PMCID: PMC9535081 DOI: 10.3389/fsurg.2022.920457] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Accepted: 09/08/2022] [Indexed: 11/16/2022] Open
Abstract
In this paper, we give an overview on current trends in computer-assisted image-based methods for risk analysis and planning in lung surgery and present our own developments with a focus on computed tomography (CT) based algorithms and applications. The methods combine heuristic, knowledge based image processing algorithms for segmentation, quantification and visualization based on CT images of the lung. Impact for lung surgery is discussed regarding risk assessment, quantitative assessment of resection strategies, and surgical guiding. In perspective, we discuss the role of deep-learning based AI methods for further improvements.
Collapse
Affiliation(s)
- Stefan Krass
- Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
- Correspondence: Stefan Krass
| | | | - Andrea Schenk
- Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
- Department of Diagnostic and Interventional Radiology, Hannover Medical School, Hannover, Germany
| |
Collapse
|
3
|
Nakao M, Kobayashi K, Tokuno J, Chen-Yoshikawa T, Date H, Matsuda T. Deformation analysis of surface and bronchial structures in intraoperative pneumothorax using deformable mesh registration. Med Image Anal 2021; 73:102181. [PMID: 34303889 DOI: 10.1016/j.media.2021.102181] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2020] [Revised: 07/08/2021] [Accepted: 07/13/2021] [Indexed: 11/30/2022]
Abstract
The positions of nodules can change because of intraoperative lung deflation, and the modeling of pneumothorax-associated deformation remains a challenging issue for intraoperative tumor localization. In this study, we introduce spatial and geometric analysis methods for inflated/deflated lungs and discuss heterogeneity in pneumothorax-associated lung deformation. Contrast-enhanced CT images simulating intraoperative conditions were acquired from live Beagle dogs. The images contain the overall shape of the lungs, including all lobes and internal bronchial structures, and were analyzed to provide a statistical deformation model that could be used as prior knowledge to predict pneumothorax. To address the difficulties of mapping pneumothorax CT images with topological changes and CT intensity shifts, we designed deformable mesh registration techniques for mixed data structures including the lobe surfaces and the bronchial centerlines. Three global-to-local registration steps were performed under the constraint that the deformation was spatially continuous and smooth, while matching visible bronchial tree structures as much as possible. The developed framework achieved stable registration with a Hausdorff distance of less than 1 mm and a target registration error of less than 5 mm, and visualized deformation fields that demonstrate per-lobe contractions and rotations with high variability between subjects. The deformation analysis results show that the strain of lung parenchyma was 35% higher than that of bronchi, and that deformation in the deflated lung is heterogeneous.
Collapse
Affiliation(s)
- Megumi Nakao
- Graduate School of Informatics, Kyoto University, Yoshida-Honmachi, Sakyo, Kyoto, 606-8501, Japan.
| | - Kotaro Kobayashi
- Graduate School of Informatics, Kyoto University, Yoshida-Honmachi, Sakyo, Kyoto, 606-8501, Japan
| | - Junko Tokuno
- Kyoto University Hospital, 54 Kawaharacho, Shogoin, Sakyo, Kyoto, 606-8507, Japan
| | | | - Hiroshi Date
- Kyoto University Hospital, 54 Kawaharacho, Shogoin, Sakyo, Kyoto, 606-8507, Japan
| | - Tetsuya Matsuda
- Graduate School of Informatics, Kyoto University, Yoshida-Honmachi, Sakyo, Kyoto, 606-8501, Japan
| |
Collapse
|
4
|
Alvarez P, Rouzé S, Miga MI, Payan Y, Dillenseger JL, Chabanas M. A hybrid, image-based and biomechanics-based registration approach to markerless intraoperative nodule localization during video-assisted thoracoscopic surgery. Med Image Anal 2021; 69:101983. [PMID: 33588119 DOI: 10.1016/j.media.2021.101983] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2020] [Revised: 01/16/2021] [Accepted: 01/26/2021] [Indexed: 12/09/2022]
Abstract
The resection of small, low-dense or deep lung nodules during video-assisted thoracoscopic surgery (VATS) is surgically challenging. Nodule localization methods in clinical practice typically rely on the preoperative placement of markers, which may lead to clinical complications. We propose a markerless lung nodule localization framework for VATS based on a hybrid method combining intraoperative cone-beam CT (CBCT) imaging, free-form deformation image registration, and a poroelastic lung model with allowance for air evacuation. The difficult problem of estimating intraoperative lung deformations is decomposed into two more tractable sub-problems: (i) estimating the deformation due the change of patient pose from preoperative CT (supine) to intraoperative CBCT (lateral decubitus); and (ii) estimating the pneumothorax deformation, i.e. a collapse of the lung within the thoracic cage. We were able to demonstrate the feasibility of our localization framework with a retrospective validation study on 5 VATS clinical cases. Average initial errors in the range of 22 to 38 mm were reduced to the range of 4 to 14 mm, corresponding to an error correction in the range of 63 to 85%. To our knowledge, this is the first markerless lung deformation compensation method dedicated to VATS and validated on actual clinical data.
Collapse
Affiliation(s)
- Pablo Alvarez
- Univ. Rennes 1, Inserm, LTSI - UMR 1099, Rennes F-35000, France; Univ. Grenoble Alpes, CNRS, Grenoble INP, TIMC-IMAG, Grenoble F-38000, France.
| | - Simon Rouzé
- Univ. Rennes 1, Inserm, LTSI - UMR 1099, Rennes F-35000, France; CHU Rennes, Department of Cardio-Thoracic and Vascular Surgery, Rennes F-35000, France.
| | - Michael I Miga
- Vanderbilt Institute for Surgery and Engineering, Vanderbilt University, Nashville, TN, USA.
| | - Yohan Payan
- Univ. Grenoble Alpes, CNRS, Grenoble INP, TIMC-IMAG, Grenoble F-38000, France.
| | | | - Matthieu Chabanas
- Univ. Grenoble Alpes, CNRS, Grenoble INP, TIMC-IMAG, Grenoble F-38000, France; Vanderbilt Institute for Surgery and Engineering, Vanderbilt University, Nashville, TN, USA.
| |
Collapse
|