1
|
Pan J, Chang D, Wu W, Chen Y, Wang S. Self-supervised tomographic image noise suppression via residual image prior network. Comput Biol Med 2024; 179:108837. [PMID: 38991317 DOI: 10.1016/j.compbiomed.2024.108837] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Revised: 05/29/2024] [Accepted: 06/29/2024] [Indexed: 07/13/2024]
Abstract
Computed tomography (CT) denoising is a challenging task in medical imaging that has garnered considerable attention. Supervised networks require a lot of noisy-clean image pairs, which are always unavailable in clinical settings. Existing self-supervised algorithms for suppressing noise with paired noisy images have limitations, such as ignoring the residual between similar image pairs during training and insufficiently learning the spectrum information of images. In this study, we propose a Residual Image Prior Network (RIP-Net) to sufficiently model the residual between the paired similar noisy images. Our approach offers new insights into the field by addressing the limitations of existing methods. We first establish a mathematical theorem clarifying the non-equivalence between similar-image-based self-supervised learning and supervised learning. It helps us better understand the strengths and limitations of self-supervised learning. Secondly, we introduce a novel regularization term to model a low-frequency residual image prior. This can improve the accuracy and robustness of our model. Finally, we design a well-structured denoising network capable of exploring spectrum information while simultaneously sensing context messages. The network has dual paths for modeling high and low-frequency compositions in the raw noisy image. Additionally, context perception modules capture local and global interactions to produce high-quality images. The comprehensive experiments on preclinical photon-counting CT, clinical brain CT, and low-dose CT datasets, demonstrate that our RIP-Net is superior to other unsupervised denoising methods.
Collapse
Affiliation(s)
- Jiayi Pan
- School of Biomedical Engineering, Sun Yat-Sen University, Shenzhen, Guangdong, China.
| | - Dingyue Chang
- Institute of Materials, China Academy of Engineering Physics, Mianyang, Sichuan, China
| | - Weiwen Wu
- School of Biomedical Engineering, Sun Yat-Sen University, Shenzhen, Guangdong, China.
| | - Yang Chen
- School of Computer Science and Engineering, Southeast University, Nanjing, Jiangsu, China.
| | - Shaoyu Wang
- School of Information Engineering, Nanchang University, Nanchang, Jiangxi, China.
| |
Collapse
|
2
|
Liu T, Huang S, Li R, Gao P, Li W, Lu H, Song Y, Rong J. Dual and Multi-Target Cone-Beam X-ray Luminescence Computed Tomography Based on the DeepCB-XLCT Network. Bioengineering (Basel) 2024; 11:874. [PMID: 39329616 PMCID: PMC11428951 DOI: 10.3390/bioengineering11090874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2024] [Revised: 08/24/2024] [Accepted: 08/27/2024] [Indexed: 09/28/2024] Open
Abstract
BACKGROUND AND OBJECTIVE Emerging as a hybrid imaging modality, cone-beam X-ray luminescence computed tomography (CB-XLCT) has been developed using X-ray-excitable nanoparticles. In contrast to conventional bio-optical imaging techniques like bioluminescence tomography (BLT) and fluorescence molecular tomography (FMT), CB-XLCT offers the advantage of greater imaging depth while significantly reducing interference from autofluorescence and background fluorescence, owing to its utilization of X-ray-excited nanoparticles. However, due to the intricate excitation process and extensive light scattering within biological tissues, the inverse problem of CB-XLCT is fundamentally ill-conditioned. METHODS An end-to-end three-dimensional deep encoder-decoder network, termed DeepCB-XLCT, is introduced to improve the quality of CB-XLCT reconstructions. This network directly establishes a nonlinear mapping between the distribution of internal X-ray-excitable nanoparticles and the corresponding boundary fluorescent signals. To improve the fidelity of target shape restoration, the structural similarity loss (SSIM) was incorporated into the objective function of the DeepCB-XLCT network. Additionally, a loss term specifically for target regions was introduced to improve the network's emphasis on the areas of interest. As a result, the inaccuracies in reconstruction caused by the simplified linear model used in conventional methods can be effectively minimized by the proposed DeepCB-XLCT method. RESULTS AND CONCLUSIONS Numerical simulations, phantom experiments, and in vivo experiments with two targets were performed, revealing that the DeepCB-XLCT network enhances reconstruction accuracy regarding contrast-to-noise ratio and shape similarity when compared to traditional methods. In addition, the findings from the XLCT tomographic images involving three targets demonstrate its potential for multi-target CB-XLCT imaging.
Collapse
Affiliation(s)
- Tianshuai Liu
- Biomedical Engineering Department, Fourth Military Medical University, Xi’an 710032, China; (T.L.); (S.H.); (R.L.); (P.G.); (W.L.)
- Shaanxi Provincial Key Laboratory of Bioelectromagnetic Detection and Intelligent Perception, Xi’an 710032, China
| | - Shien Huang
- Biomedical Engineering Department, Fourth Military Medical University, Xi’an 710032, China; (T.L.); (S.H.); (R.L.); (P.G.); (W.L.)
- School of Software Engineering, Xi’an Jiaotong University, Xi’an 710049, China
| | - Ruijing Li
- Biomedical Engineering Department, Fourth Military Medical University, Xi’an 710032, China; (T.L.); (S.H.); (R.L.); (P.G.); (W.L.)
- Shaanxi Provincial Key Laboratory of Bioelectromagnetic Detection and Intelligent Perception, Xi’an 710032, China
| | - Peng Gao
- Biomedical Engineering Department, Fourth Military Medical University, Xi’an 710032, China; (T.L.); (S.H.); (R.L.); (P.G.); (W.L.)
- Shaanxi Provincial Key Laboratory of Bioelectromagnetic Detection and Intelligent Perception, Xi’an 710032, China
| | - Wangyang Li
- Biomedical Engineering Department, Fourth Military Medical University, Xi’an 710032, China; (T.L.); (S.H.); (R.L.); (P.G.); (W.L.)
- Shaanxi Provincial Key Laboratory of Bioelectromagnetic Detection and Intelligent Perception, Xi’an 710032, China
| | - Hongbing Lu
- Biomedical Engineering Department, Fourth Military Medical University, Xi’an 710032, China; (T.L.); (S.H.); (R.L.); (P.G.); (W.L.)
- Shaanxi Provincial Key Laboratory of Bioelectromagnetic Detection and Intelligent Perception, Xi’an 710032, China
| | - Yonghong Song
- School of Software Engineering, Xi’an Jiaotong University, Xi’an 710049, China
| | - Junyan Rong
- Biomedical Engineering Department, Fourth Military Medical University, Xi’an 710032, China; (T.L.); (S.H.); (R.L.); (P.G.); (W.L.)
- Shaanxi Provincial Key Laboratory of Bioelectromagnetic Detection and Intelligent Perception, Xi’an 710032, China
| |
Collapse
|
3
|
Marcos L, Babyn P, Alirezaie J. Pure Vision Transformer (CT-ViT) with Noise2Neighbors Interpolation for Low-Dose CT Image Denoising. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01108-8. [PMID: 38622385 DOI: 10.1007/s10278-024-01108-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Revised: 03/26/2024] [Accepted: 03/26/2024] [Indexed: 04/17/2024]
Abstract
Convolutional neural networks (CNN) have been used for a wide variety of deep learning applications, especially in computer vision. For medical image processing, researchers have identified certain challenges associated with CNNs. These challenges encompass the generation of less informative features, limitations in capturing both high and low-frequency information within feature maps, and the computational cost incurred when enhancing receptive fields by deepening the network. Transformers have emerged as an approach aiming to address and overcome these specific limitations of CNNs in the context of medical image analysis. Preservation of all spatial details of medical images is necessary to ensure accurate patient diagnosis. Hence, this research introduced the use of a pure Vision Transformer (ViT) for a denoising artificial neural network for medical image processing specifically for low-dose computed tomography (LDCT) image denoising. The proposed model follows a U-Net framework that contains ViT modules with the integration of Noise2Neighbor (N2N) interpolation operation. Five different datasets containing LDCT and normal-dose CT (NDCT) image pairs were used to carry out this experiment. To test the efficacy of the proposed model, this experiment includes comparisons between the quantitative and visual results among CNN-based (BM3D, RED-CNN, DRL-E-MP), hybrid CNN-ViT-based (TED-Net), and the proposed pure ViT-based denoising model. The findings of this study showed that there is about 15-20% increase in SSIM and PSNR when using self-attention transformers than using the typical pure CNN. Visual results also showed improvements especially when it comes to showing fine structural details of CT images.
Collapse
Affiliation(s)
- Luella Marcos
- Department of Electrical, Biomedical and Computer Engineering, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, M5B 2K3, Ontario, Canada
| | - Paul Babyn
- Department of Medical Imaging, University of Saskatchewan, 105 Administration Pl, Saskatoon, SK S7N0W8, Saskatchewan, Canada
| | - Javad Alirezaie
- Department of Electrical, Biomedical and Computer Engineering, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, M5B 2K3, Ontario, Canada.
| |
Collapse
|
4
|
Zhang H, Zhang P, Cheng W, Li S, Yan R, Hou R, Gui Z, Liu Y, Chen Y. Learnable PM diffusion coefficients and reformative coordinate attention network for low dose CT denoising. Phys Med Biol 2023; 68:245017. [PMID: 37536336 DOI: 10.1088/1361-6560/aced33] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Accepted: 08/03/2023] [Indexed: 08/05/2023]
Abstract
Objective.Various deep learning methods have recently been used for low dose CT (LDCT) denoising. Aggressive denoising may destroy the edge and fine anatomical structures of CT images. Therefore a key issue in LDCT denoising tasks is the difficulty of balancing noise/artifact suppression and edge/structure preservation.Approach.We proposed an LDCT denoising network based on the encoder-decoder structure, namely the Learnable PM diffusion coefficient and efficient attention network (PMA-Net). First, using the powerful feature modeling capability of partial differential equations, we constructed a multiple learnable edge module to generate precise edge information, incorporating the anisotropic image processing idea of Perona-Malik (PM) model into the neural network. Second, a multiscale reformative coordinate attention module was designed to extract multiscale information. Non-overlapping dilated convolution capturing abundant contextual content was combined with coordinate attention which could embed the spatial location information of important features into the channel attention map. Finally, we imposed additional constraints on the edge information using edge-enhanced multiscale perceptual loss to avoid structure loss and over-smoothing.Main results.Experiments are conducted on simulated and real datasets. The quantitative and qualitative results show that the proposed method has better performance in suppressing noise/artifacts and preserving edges/structures.Significance.This work proposes a novel edge feature extraction method that unfolds partial differential equation into neural networks, which contributes to the interpretability and clinical application value of neural network.
Collapse
Affiliation(s)
- Haowen Zhang
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan 030051, People's Republic of China
| | - Pengcheng Zhang
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan 030051, People's Republic of China
| | - Weiting Cheng
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan 030051, People's Republic of China
| | - Shu Li
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan 030051, People's Republic of China
| | - Rongbiao Yan
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan 030051, People's Republic of China
| | - Ruifeng Hou
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan 030051, People's Republic of China
| | - Zhiguo Gui
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan 030051, People's Republic of China
| | - Yi Liu
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan 030051, People's Republic of China
- Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing 210096, People's Republic of China
| | - Yang Chen
- Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing 210096, People's Republic of China
- Laboratory of Image Science and Technology, Southeast University, Nanjing 210096, People's Republic of China
- Centre de Recherche en Information Biomedicale Sino-Francais (LIA CRIBs), F-3500 Rennes, France
- Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing, Southeast University, Nanjing, People's Republic of China
| |
Collapse
|
5
|
Wang D, Fan F, Wu Z, Liu R, Wang F, Yu H. CTformer: convolution-free Token2Token dilated vision transformer for low-dose CT denoising. Phys Med Biol 2023; 68:065012. [PMID: 36854190 DOI: 10.1088/1361-6560/acc000] [Citation(s) in RCA: 19] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2022] [Accepted: 02/28/2023] [Indexed: 03/02/2023]
Abstract
Objective. Low-dose computed tomography (LDCT) denoising is an important problem in CT research. Compared to the normal dose CT, LDCT images are subjected to severe noise and artifacts. Recently in many studies, vision transformers have shown superior feature representation ability over the convolutional neural networks (CNNs). However, unlike CNNs, the potential of vision transformers in LDCT denoising was little explored so far. Our paper aims to further explore the power of transformer for the LDCT denoising problem.Approach. In this paper, we propose a Convolution-free Token2Token Dilated Vision Transformer (CTformer) for LDCT denoising. The CTformer uses a more powerful token rearrangement to encompass local contextual information and thus avoids convolution. It also dilates and shifts feature maps to capture longer-range interaction. We interpret the CTformer by statically inspecting patterns of its internal attention maps and dynamically tracing the hierarchical attention flow with an explanatory graph. Furthermore, overlapped inference mechanism is employed to effectively eliminate the boundary artifacts that are common for encoder-decoder-based denoising models.Main results. Experimental results on Mayo dataset suggest that the CTformer outperforms the state-of-the-art denoising methods with a low computational overhead.Significance. The proposed model delivers excellent denoising performance on LDCT. Moreover, low computational cost and interpretability make the CTformer promising for clinical applications.
Collapse
Affiliation(s)
- Dayang Wang
- Department of Electrical and Computer Engineering, University of Massachusetts, Lowell, MA, United States of America
| | - Fenglei Fan
- Weill Cornell Medicine, Cornell University, New York City, NY, United States of America
| | - Zhan Wu
- School of Cyberspace Security, Southeast University, Nanjing, Jiangsu, People's Republic of China
| | - Rui Liu
- 3920 Mystic Valley Parkway, Medford, MA, United States of America
| | - Fei Wang
- Weill Cornell Medicine, Cornell University, New York City, NY, United States of America
| | - Hengyong Yu
- Department of Electrical and Computer Engineering, University of Massachusetts, Lowell, MA, United States of America
| |
Collapse
|