1
|
Zhang L, Ning G, Liang H, Han B, Liao H. One-shot neuroanatomy segmentation through online data augmentation and confidence aware pseudo label. Med Image Anal 2024; 95:103182. [PMID: 38688039 DOI: 10.1016/j.media.2024.103182] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Revised: 11/26/2023] [Accepted: 04/18/2024] [Indexed: 05/02/2024]
Abstract
Recently, deep learning-based brain segmentation methods have achieved great success. However, most approaches focus on supervised segmentation, which requires many high-quality labeled images. In this paper, we pay attention to one-shot segmentation, aiming to learn from one labeled image and a few unlabeled images. We propose an end-to-end unified network that joints deformation modeling and segmentation tasks. Our network consists of a shared encoder, a deformation modeling head, and a segmentation head. In the training phase, the atlas and unlabeled images are input to the encoder to get multi-scale features. The features are then fed to the multi-scale deformation modeling module to estimate the atlas-to-image deformation field. The deformation modeling module implements the estimation at the feature level in a coarse-to-fine manner. Then, we employ the field to generate the augmented image pair through online data augmentation. We do not apply any appearance transformations cause the shared encoder could capture appearance variations. Finally, we adopt supervised segmentation loss for the augmented image. Considering that the unlabeled images still contain rich information, we introduce confidence aware pseudo label for them to further boost the segmentation performance. We validate our network on three benchmark datasets. Experimental results demonstrate that our network significantly outperforms other deep single-atlas-based and traditional multi-atlas-based segmentation methods. Notably, the second dataset is collected from multi-center, and our network still achieves promising segmentation performance on both the seen and unseen test sets, revealing its robustness. The source code will be available at https://github.com/zhangliutong/brainseg.
Collapse
Affiliation(s)
- Liutong Zhang
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Guochen Ning
- School of Clinical Medicine, Tsinghua University, Beijing, China
| | - Hanying Liang
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Boxuan Han
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Hongen Liao
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China; School of Biomedical Engineering, Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China.
| |
Collapse
|
2
|
Li A, Ying Y, Gao T, Zhang L, Zhao X, Zhao Y, Song G, Zhang H. MF-Net: multi-scale feature extraction-integration network for unsupervised deformable registration. Front Neurosci 2024; 18:1364409. [PMID: 38680447 PMCID: PMC11045908 DOI: 10.3389/fnins.2024.1364409] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2024] [Accepted: 03/20/2024] [Indexed: 05/01/2024] Open
Abstract
Deformable registration plays a fundamental and crucial role in scenarios such as surgical navigation and image-assisted analysis. While deformable registration methods based on unsupervised learning have shown remarkable success in predicting displacement fields with high accuracy, many existing registration networks are limited by the lack of multi-scale analysis, restricting comprehensive utilization of global and local features in the images. To address this limitation, we propose a novel registration network called multi-scale feature extraction-integration network (MF-Net). First, we propose a multiscale analysis strategy that enables the model to capture global and local semantic information in the image, thus facilitating accurate texture and detail registration. Additionally, we introduce grouped gated inception block (GI-Block) as the basic unit of the feature extractor, enabling the feature extractor to selectively extract quantitative features from images at various resolutions. Comparative experiments demonstrate the superior accuracy of our approach over existing methods.
Collapse
Affiliation(s)
- Andi Li
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Yuhan Ying
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Tian Gao
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
- School of Automation and Electrical Engineering, Shenyang Ligong University, Shenyang, China
| | - Lei Zhang
- Spine Surgery Unit, Shengjing Hospital of China Medical University, Shenyang, China
| | - Xingang Zhao
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
| | - Yiwen Zhao
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
| | - Guoli Song
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
| | - He Zhang
- Orthopedic Department, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| |
Collapse
|
3
|
Ou Z, Lu X, Gu Y. HCS-Net: Multi-level deformation strategy combined with quadruple attention for image registration. Comput Biol Med 2024; 168:107832. [PMID: 38071839 DOI: 10.1016/j.compbiomed.2023.107832] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2023] [Revised: 11/09/2023] [Accepted: 12/04/2023] [Indexed: 01/10/2024]
Abstract
BACKGROUND AND OBJECTIVE Non-rigid image registration plays a significant role in computer-aided diagnosis and surgical navigation for brain diseases. Registration methods that utilize convolutional neural networks (CNNs) have shown excellent accuracy when applied to brain magnetic resonance images (MRI). However, CNNs have limitations in understanding long-range spatial relationships in images, which makes it challenging to incorporate contextual information. And in intricate image registration tasks, it is difficult to achieve a satisfactory dense prediction field, resulting in poor registration performance. METHODS This paper proposes a multi-level deformable unsupervised registration model that combines Transformer and CNN to achieve non-rigid registration of brain MRI. Firstly, utilizing a dual encoder structure to establish the dependency relationship between the global features of two images and to merge features of varying scales, as well as to preserve the relative spatial position information of feature maps at different scales. Then the proposed multi-level deformation strategy utilizes different deformable fields of varying resolutions generated by the decoding structure to progressively deform the moving image. Ultimately, the proposed quadruple attention module is incorporated into the decoding structure to merge feature information from various directions and emphasize the spatial features in the dominant channels. RESULTS The experimental results on multiple brain MR datasets demonstrate that the promising network could provide accurate registration and is comparable to state-of-the-art methods. CONCLUSION The proposed registration model can generate superior deformable fields and achieve more precise registration effects, enhancing the auxiliary role of medical image registration in various fields and advancing the development of computer-aided diagnosis, surgical navigation, and related domains.
Collapse
Affiliation(s)
- Zhuolin Ou
- Inner Mongolia Key Laboratory of Pattern Recognition and Intelligent Image Processing, School of Information Engineering, Inner Mongolia University of Science and Technology, Baotou, 014010, China
| | - Xiaoqi Lu
- Inner Mongolia Key Laboratory of Pattern Recognition and Intelligent Image Processing, School of Information Engineering, Inner Mongolia University of Science and Technology, Baotou, 014010, China; School of Information Engineering, Inner Mongolia University of Technology, Hohhot, 010051, China.
| | - Yu Gu
- Inner Mongolia Key Laboratory of Pattern Recognition and Intelligent Image Processing, School of Information Engineering, Inner Mongolia University of Science and Technology, Baotou, 014010, China.
| |
Collapse
|
4
|
Ma X, He J, Liu X, Liu Q, Chen G, Yuan B, Li C, Xia Y. Hierarchical cumulative network for unsupervised medical image registration. Comput Biol Med 2023; 167:107598. [PMID: 37913614 DOI: 10.1016/j.compbiomed.2023.107598] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 09/23/2023] [Accepted: 10/17/2023] [Indexed: 11/03/2023]
Abstract
Unsupervised deep learning techniques have gained increasing popularity in deformable medical image registration However, existing methods usually overlook the optimal similarity position between moving and fixed images To tackle this issue, we propose a novel hierarchical cumulative network (HCN), which explicitly considers the optimal similarity position with an effective Bidirectional Asymmetric Registration Module (BARM). The BARM simultaneously learns two asymmetric displacement vector fields (DVFs) to optimally warp both moving images and fixed images to their optimal similar shape along the geodesic path. Furthermore, we incorporate the BARM into a Laplacian pyramid network with hierarchical recursion, in which the moving image at the lowest level of the pyramid is warped successively for aligning to the fixed image at the lowest level of the pyramid to capture multiple DVFs. We then accumulate these DVFs and up-sample them to warp the moving images at higher levels of the pyramid to align to the fixed image of the top level. The entire system is end-to-end and jointly trained in an unsupervised manner. Extensive experiments were conducted on two public 3D Brain MRI datasets to demonstrate that our HCN outperforms both the traditional and state-of-the-art registration methods. To further evaluate the performance of our HCN, we tested it on the validation set of the MICCAI Learn2Reg 2021 challenge. Additionally, a cross-dataset evaluation was conducted to assess the generalization of our HCN. Experimental results showed that our HCN is an effective deformable registration method and achieves excellent generalization performance.
Collapse
Affiliation(s)
- Xinke Ma
- National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science and Engineering, Northwestern Polytechnical University, Xi'an 710072, China.
| | - Jiang He
- Huiying Medical Technology Co., Ltd., Room A206, B2, Dongsheng Science and Technology Park, Haidian District, Beijing 100192, China.
| | - Xing Liu
- National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science and Engineering, Northwestern Polytechnical University, Xi'an 710072, China.
| | - Qin Liu
- National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science and Engineering, Northwestern Polytechnical University, Xi'an 710072, China.
| | - Geng Chen
- National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science and Engineering, Northwestern Polytechnical University, Xi'an 710072, China.
| | - Bo Yuan
- Sichuan Provincial Health Information Center (Sichuan Provincial Health and Medical Big Data Center), Chengdu 610041, China.
| | - Changyang Li
- Sydney Polytechnic Institute, NSW 2000, Australia.
| | - Yong Xia
- National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science and Engineering, Northwestern Polytechnical University, Xi'an 710072, China.
| |
Collapse
|
5
|
Tang K, Wang L, Huang X, Cheng X, Zhu YM. MD-SGT: Multi-dilation spherical graph transformer for unsupervised medical image registration. Comput Med Imaging Graph 2023; 108:102281. [PMID: 37579555 DOI: 10.1016/j.compmedimag.2023.102281] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Revised: 06/16/2023] [Accepted: 07/22/2023] [Indexed: 08/16/2023]
Abstract
Deformable medical image registration is an essential preprocess step for several clinical applications. Even though the existing convolutional neural network and transformer based methods achieved the promising results, the limited long-range spatial dependence and non-uniform attention span of these models prohibit further improving the registration performance. To deal with this issue, we proposed a multi-dilation spherical graph transformer (MD-SGT), in which the encoder combined the advantages of convolutional and graph transformer blocks to distinguish effectively the differences between the reference and the template images at various scales. Specifically, the features of each voxel were obtained by aggregating the information from its neighbors sampled from different spherical regions with different dilation rates. The implicit convolution inductive bias and long-range uniform attention span induced by such information aggregation manner made the features more representative for registration. Through the qualitative and quantitative comparisons with state-of-the-art methods on two datasets, we demonstrated that combining long-range uniform attention span and inductive bias are beneficial for promoting the image registration performance, with the Dice score, ASD and HD95 being improved at least by 0.5%, 2.2% and 1.1%, respectively.
Collapse
Affiliation(s)
- Kun Tang
- Engineering Research Center of Text Computing & Cognitive Intelligence, Ministry of Education, Key Laboratory of Intelligent Medical Image Analysis and Precise Diagnosis of Guizhou Province, State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, China
| | - Lihui Wang
- Engineering Research Center of Text Computing & Cognitive Intelligence, Ministry of Education, Key Laboratory of Intelligent Medical Image Analysis and Precise Diagnosis of Guizhou Province, State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, China.
| | - Xingyu Huang
- Engineering Research Center of Text Computing & Cognitive Intelligence, Ministry of Education, Key Laboratory of Intelligent Medical Image Analysis and Precise Diagnosis of Guizhou Province, State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, China
| | - Xinyu Cheng
- Engineering Research Center of Text Computing & Cognitive Intelligence, Ministry of Education, Key Laboratory of Intelligent Medical Image Analysis and Precise Diagnosis of Guizhou Province, State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, China
| | - Yue-Min Zhu
- University Lyon, INSA Lyon, CNRS, Inserm, IRP Metislab CREATIS UMR5220, U1206, Lyon 69621, France
| |
Collapse
|