1
|
Chen X, Zhou B, Guo X, Xie H, Liu Q, Duncan JS, Sinusas AJ, Liu C. DuDoCFNet: Dual-Domain Coarse-to-Fine Progressive Network for Simultaneous Denoising, Limited-View Reconstruction, and Attenuation Correction of Cardiac SPECT. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:3110-3125. [PMID: 38578853 DOI: 10.1109/tmi.2024.3385650] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/07/2024]
Abstract
Single-Photon Emission Computed Tomography (SPECT) is widely applied for the diagnosis of coronary artery diseases. Low-dose (LD) SPECT aims to minimize radiation exposure but leads to increased image noise. Limited-view (LV) SPECT, such as the latest GE MyoSPECT ES system, enables accelerated scanning and reduces hardware expenses but degrades reconstruction accuracy. Additionally, Computed Tomography (CT) is commonly used to derive attenuation maps ( μ -maps) for attenuation correction (AC) of cardiac SPECT, but it will introduce additional radiation exposure and SPECT-CT misalignments. Although various methods have been developed to solely focus on LD denoising, LV reconstruction, or CT-free AC in SPECT, the solution for simultaneously addressing these tasks remains challenging and under-explored. Furthermore, it is essential to explore the potential of fusing cross-domain and cross-modality information across these interrelated tasks to further enhance the accuracy of each task. Thus, we propose a Dual-Domain Coarse-to-Fine Progressive Network (DuDoCFNet), a multi-task learning method for simultaneous LD denoising, LV reconstruction, and CT-free μ -map generation of cardiac SPECT. Paired dual-domain networks in DuDoCFNet are cascaded using a multi-layer fusion mechanism for cross-domain and cross-modality feature fusion. Two-stage progressive learning strategies are applied in both projection and image domains to achieve coarse-to-fine estimations of SPECT projections and CT-derived μ -maps. Our experiments demonstrate DuDoCFNet's superior accuracy in estimating projections, generating μ -maps, and AC reconstructions compared to existing single- or multi-task learning methods, under various iterations and LD levels. The source code of this work is available at https://github.com/XiongchaoChen/DuDoCFNet-MultiTask.
Collapse
|
2
|
Wu J, Jiang X, Zhong L, Zheng W, Li X, Lin J, Li Z. Linear diffusion noise boosted deep image prior for unsupervised sparse-view CT reconstruction. Phys Med Biol 2024; 69:165029. [PMID: 39119998 DOI: 10.1088/1361-6560/ad69f7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2024] [Accepted: 07/31/2024] [Indexed: 08/10/2024]
Abstract
Objective.Deep learning has markedly enhanced the performance of sparse-view computed tomography reconstruction. However, the dependence of these methods on supervised training using high-quality paired datasets, and the necessity for retraining under varied physical acquisition conditions, constrain their generalizability across new imaging contexts and settings.Approach.To overcome these limitations, we propose an unsupervised approach grounded in the deep image prior framework. Our approach advances beyond the conventional single noise level input by incorporating multi-level linear diffusion noise, significantly mitigating the risk of overfitting. Furthermore, we embed non-local self-similarity as a deep implicit prior within a self-attention network structure, improving the model's capability to identify and utilize repetitive patterns throughout the image. Additionally, leveraging imaging physics, gradient backpropagation is performed between the image domain and projection data space to optimize network weights.Main Results.Evaluations with both simulated and clinical cases demonstrate our method's effective zero-shot adaptability across various projection views, highlighting its robustness and flexibility. Additionally, our approach effectively eliminates noise and streak artifacts while significantly restoring intricate image details.Significance. Our method aims to overcome the limitations in current supervised deep learning-based sparse-view CT reconstruction, offering improved generalizability and adaptability without the need for extensive paired training data.
Collapse
Affiliation(s)
- Jia Wu
- School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, People's Republic of China
- School of Medical Information and Engineering, Southwest Medical University, Luzhou 646000, People's Republic of China
| | - Xiaoming Jiang
- Chongqing Engineering Research Center of Medical Electronics and Information Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, People's Republic of China
| | - Lisha Zhong
- School of Medical Information and Engineering, Southwest Medical University, Luzhou 646000, People's Republic of China
| | - Wei Zheng
- Key Laboratory of Big Data Intelligent Computing, Chongqing University of Posts and Telecommunications, Chongqing 400065, People's Republic of China
| | - Xinwei Li
- Chongqing Engineering Research Center of Medical Electronics and Information Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, People's Republic of China
| | - Jinzhao Lin
- School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, People's Republic of China
| | - Zhangyong Li
- Chongqing Engineering Research Center of Medical Electronics and Information Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, People's Republic of China
| |
Collapse
|
3
|
Cam RM, Villa U, Anastasio MA. Learning a stable approximation of an existing but unknown inverse mapping: application to the half-time circular Radon transform. INVERSE PROBLEMS 2024; 40:085002. [PMID: 38933410 PMCID: PMC11197394 DOI: 10.1088/1361-6420/ad4f0a] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 04/05/2024] [Accepted: 05/22/2024] [Indexed: 06/28/2024]
Abstract
Supervised deep learning-based methods have inspired a new wave of image reconstruction methods that implicitly learn effective regularization strategies from a set of training data. While they hold potential for improving image quality, they have also raised concerns regarding their robustness. Instabilities can manifest when learned methods are applied to find approximate solutions to ill-posed image reconstruction problems for which a unique and stable inverse mapping does not exist, which is a typical use case. In this study, we investigate the performance of supervised deep learning-based image reconstruction in an alternate use case in which a stable inverse mapping is known to exist but is not yet analytically available in closed form. For such problems, a deep learning-based method can learn a stable approximation of the unknown inverse mapping that generalizes well to data that differ significantly from the training set. The learned approximation of the inverse mapping eliminates the need to employ an implicit (optimization-based) reconstruction method and can potentially yield insights into the unknown analytic inverse formula. The specific problem addressed is image reconstruction from a particular case of radially truncated circular Radon transform (CRT) data, referred to as 'half-time' measurement data. For the half-time image reconstruction problem, we develop and investigate a learned filtered backprojection method that employs a convolutional neural network to approximate the unknown filtering operation. We demonstrate that this method behaves stably and readily generalizes to data that differ significantly from training data. The developed method may find application to wave-based imaging modalities that include photoacoustic computed tomography.
Collapse
Affiliation(s)
- Refik Mert Cam
- Department of Electrical and Computer Engineering, University of Illinois Urbana–Champaign, Urbana, IL 61801, United States of America
| | - Umberto Villa
- Oden Institute for Computational Engineering & Sciences, The University of Texas at Austin, Austin, TX 78712, United States of America
| | - Mark A Anastasio
- Department of Electrical and Computer Engineering, University of Illinois Urbana–Champaign, Urbana, IL 61801, United States of America
- Department of Bioengineering, University of Illinois Urbana–Champaign, Urbana, IL 61801, United States of America
| |
Collapse
|
4
|
Konovalov AB. Compressed-sensing-inspired reconstruction algorithms in low-dose computed tomography: A review. Phys Med 2024; 124:104491. [PMID: 39079308 DOI: 10.1016/j.ejmp.2024.104491] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/02/2024] [Revised: 07/13/2024] [Accepted: 07/23/2024] [Indexed: 08/10/2024] Open
Abstract
BACKGROUND Optimization of the dose the patient receives during scanning is an important problem in modern medical X-ray computed tomography (CT). One of the basic ways to its solution is to reduce the number of views. Compressed sensing theory helped promote the development of a new class of effective reconstruction algorithms for limited data CT. These compressed-sensing-inspired (CSI) algorithms optimize the Lp (0 ≤ p ≤ 1) norm of images and can accurately reconstruct CT tomograms from a very few views. The paper presents a review of the CSI algorithms and discusses prospects for their further use in commercial low-dose CT. METHODS Many literature references with the CSI algorithms have been were searched. To structure the material collected the author gives a classification framework within which he describes Lp regularization methods, the basic CSI algorithms that are used most often in few-view CT, and some of their derivatives. Lots of examples are provided to illustrate the use of the CSI algorithms in few-view and low-dose CT. RESULTS A list of the CSI algorithms is compiled from the literature search. For better demonstrativeness they are summarized in a table. The inference is done that already today some of the algorithms are capable of reconstruction from 20 to 30 views with acceptable quality and dose reduction by a factor of 10. DISCUSSION In conclusion the author discusses how soon the CSI reconstruction algorithms can be introduced in the practice of medical diagnosis and used in commercial CT scanners.
Collapse
Affiliation(s)
- Alexander B Konovalov
- FSUE "Russian Federal Nuclear Center - Zababakhin All-Russia Research Institute of Technical Physics", Snezhinsk, Chelyabinsk Region 456770, Russia.
| |
Collapse
|
5
|
Li G, Deng Z, Ge Y, Luo S. HEAL: High-Frequency Enhanced and Attention-Guided Learning Network for Sparse-View CT Reconstruction. Bioengineering (Basel) 2024; 11:646. [PMID: 39061728 PMCID: PMC11273693 DOI: 10.3390/bioengineering11070646] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2024] [Revised: 06/08/2024] [Accepted: 06/18/2024] [Indexed: 07/28/2024] Open
Abstract
X-ray computed tomography (CT) imaging technology has become an indispensable diagnostic tool in clinical examination. However, it poses a risk of ionizing radiation, making the reduction of radiation dose one of the current research hotspots in CT imaging. Sparse-view imaging, as one of the main methods for reducing radiation dose, has made significant progress in recent years. In particular, sparse-view reconstruction methods based on deep learning have shown promising results. Nevertheless, efficiently recovering image details under ultra-sparse conditions remains a challenge. To address this challenge, this paper proposes a high-frequency enhanced and attention-guided learning Network (HEAL). HEAL includes three optimization strategies to achieve detail enhancement: Firstly, we introduce a dual-domain progressive enhancement module, which leverages fidelity constraints within each domain and consistency constraints across domains to effectively narrow the solution space. Secondly, we incorporate both channel and spatial attention mechanisms to improve the network's feature-scaling process. Finally, we propose a high-frequency component enhancement regularization term that integrates residual learning with direction-weighted total variation, utilizing directional cues to effectively distinguish between noise and textures. The HEAL network is trained, validated and tested under different ultra-sparse configurations of 60 views and 30 views, demonstrating its advantages in reconstruction accuracy and detail enhancement.
Collapse
Affiliation(s)
- Guang Li
- Jiangsu Key Laboratory for Biomaterials and Devices, School of Biological Science and Medical Engineering, Southeast University, Nanjing 210096, China; (G.L.); (Z.D.)
| | - Zhenhao Deng
- Jiangsu Key Laboratory for Biomaterials and Devices, School of Biological Science and Medical Engineering, Southeast University, Nanjing 210096, China; (G.L.); (Z.D.)
| | - Yongshuai Ge
- Research Center for Medical Artificial Intelligence, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Key Laboratory of Biomedical Imaging Science and System, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Shouhua Luo
- Jiangsu Key Laboratory for Biomaterials and Devices, School of Biological Science and Medical Engineering, Southeast University, Nanjing 210096, China; (G.L.); (Z.D.)
| |
Collapse
|
6
|
Qiao Z, Liu P, Fang C, Redler G, Epel B, Halpern H. Directional TV algorithm for image reconstruction from sparse-view projections in EPR imaging. Phys Med Biol 2024; 69:115051. [PMID: 38729205 DOI: 10.1088/1361-6560/ad4a1b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Accepted: 05/10/2024] [Indexed: 05/12/2024]
Abstract
Objective.Electron paramagnetic resonance (EPR) imaging is an advanced in vivo oxygen imaging modality. The main drawback of EPR imaging is the long scanning time. Sparse-view projections collection is an effective fast scanning pattern. However, the commonly-used filtered back projection (FBP) algorithm is not competent to accurately reconstruct images from sparse-view projections because of the severe streak artifacts. The aim of this work is to develop an advanced algorithm for sparse reconstruction of 3D EPR imaging.Methods.The optimization based algorithms including the total variation (TV) algorithm have proven to be effective in sparse reconstruction in EPR imaging. To further improve the reconstruction accuracy, we propose the directional TV (DTV) model and derive its Chambolle-Pock solving algorithm.Results.After the algorithm correctness validation on simulation data, we explore the sparse reconstruction capability of the DTV algorithm via a simulated six-sphere phantom and two real bottle phantoms filled with OX063 trityl solution and scanned by an EPR imager with a magnetic field strength of 250 G.Conclusion.Both the simulated and real data experiments show that the DTV algorithm is superior to the existing FBP and TV-type algorithms and a deep learning based method according to visual inspection and quantitative evaluations in sparse reconstruction of EPR imaging.Significance.These insights gained in this work may be used in the development of fast EPR imaging workflow of practical significance.
Collapse
Affiliation(s)
- Zhiwei Qiao
- School of Computer and Information Technology, Shanxi University, Taiyuan, Shanxi, People's Republic of China
| | - Peng Liu
- School of Computer and Information Technology, Shanxi University, Taiyuan, Shanxi, People's Republic of China
- Department of Big Data and Intelligent Engineering, Shanxi Institute of Technology, Yangquan, Shanxi, People's Republic of China
| | - Chenyun Fang
- School of Computer and Information Technology, Shanxi University, Taiyuan, Shanxi, People's Republic of China
| | - Gage Redler
- Department of Radiation Oncology, Moffitt Cancer Center, Tampa, FL, United States of America
| | - Boris Epel
- Department of Radiation and Cellular Oncology, University of Chicago, Chicago, IL, United States of America
| | - Howard Halpern
- Department of Radiation and Cellular Oncology, University of Chicago, Chicago, IL, United States of America
| |
Collapse
|
7
|
Li X, Jing K, Yang Y, Wang Y, Ma J, Zheng H, Xu Z. Noise-Generating and Imaging Mechanism Inspired Implicit Regularization Learning Network for Low Dose CT Reconstrution. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:1677-1689. [PMID: 38145543 DOI: 10.1109/tmi.2023.3347258] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/27/2023]
Abstract
Low-dose computed tomography (LDCT) helps to reduce radiation risks in CT scanning while maintaining image quality, which involves a consistent pursuit of lower incident rays and higher reconstruction performance. Although deep learning approaches have achieved encouraging success in LDCT reconstruction, most of them treat the task as a general inverse problem in either the image domain or the dual (sinogram and image) domains. Such frameworks have not considered the original noise generation of the projection data and suffer from limited performance improvement for the LDCT task. In this paper, we propose a novel reconstruction model based on noise-generating and imaging mechanism in full-domain, which fully considers the statistical properties of intrinsic noises in LDCT and prior information in sinogram and image domains. To solve the model, we propose an optimization algorithm based on the proximal gradient technique. Specifically, we derive the approximate solutions of the integer programming problem on the projection data theoretically. Instead of hand-crafting the sinogram and image regularizers, we propose to unroll the optimization algorithm to be a deep network. The network implicitly learns the proximal operators of sinogram and image regularizers with two deep neural networks, providing a more interpretable and effective reconstruction procedure. Numerical results demonstrate our proposed method improvements of > 2.9 dB in peak signal to noise ratio, > 1.4% promotion in structural similarity metric, and > 9 HU decrements in root mean square error over current state-of-the-art LDCT methods.
Collapse
|
8
|
Wang L, Meng M, Chen S, Bian Z, Zeng D, Meng D, Ma J. Semi-supervised iterative adaptive network for low-dose CT sinogram recovery. Phys Med Biol 2024; 69:085013. [PMID: 38422540 DOI: 10.1088/1361-6560/ad2ee7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 02/29/2024] [Indexed: 03/02/2024]
Abstract
Background.Concern has been expressed regarding the risk of carcinogenesis from medical computed tomography (CT) radiation. Lowering radiation in CT without appropriate modifications often leads to severe noise-induced artifacts in the images. The utilization of deep learning (DL) techniques has achieved promising reconstruction performance in low-dose CT (LDCT) imaging. However, most DL-based algorithms require the pre-collection of a large set of image pairs (low-dose/standard-dose) and the training of networks in an end-to-end supervised manner. Meanwhile, securing such a large volume of paired, well-registered training data in clinical practice is challenging. Moreover, these algorithms often overlook the potential to utilize the abundant information in a large collection of LDCT-only images/sinograms.Methods.In this paper, we introduce a semi-supervised iterative adaptive network (SIA-Net) for LDCT imaging, utilizing both labeled and unlabeled sinograms in a cohesive network framework, integrating supervised and unsupervised learning processes. Specifically, the supervised process captures critical features (i.e. noise distribution and tissue characteristics) latent in the paired sinograms, while the unsupervised process effectively learns these features in the unlabeled low-dose sinograms, employing a conventional weighted least-squares model with a regularization term. Furthermore, the SIA-Net method is designed to adaptively transfer the learned feature distribution from the supervised to the unsupervised process, thereby obtaining a high-fidelity sinogram through iterative adaptive learning. Finally, high-quality CT images can be reconstructed from the refined sinogram using the filtered back-projection algorithm.Results.Experimental results on two clinical datasets indicate that the proposed SIA-Net method achieves competitive performance in terms of noise reduction and structure preservation in LDCT imaging, when compared to traditional supervised learning methods.
Collapse
Affiliation(s)
- Lei Wang
- School of Future Technology, Xi'an Jiaotong University, Xi'an 710049, People's Republic of China
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
| | - Mingqiang Meng
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
- Pazhou Lab (Huangpu), Guangdong, People's Republic of China
| | - Shixuan Chen
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
| | - Zhaoying Bian
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
- Pazhou Lab (Huangpu), Guangdong, People's Republic of China
| | - Dong Zeng
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
- Pazhou Lab (Huangpu), Guangdong, People's Republic of China
- Department of Radiology, Zhujiang Hospital, Southern Medical University, Guangdong, People's Republic of China
| | - Deyu Meng
- School of Mathematics and Statistics, Xi'an Jiaotong University, Xi'an 710049, People's Republic of China
| | - Jianhua Ma
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
- School of Life Science and Technology, Xi'an Jiaotong University, Xi'an, Shaanxi, People's Republic of China
| |
Collapse
|
9
|
Lang Y, Jiang Z, Sun L, Xiang L, Ren L. Hybrid-supervised deep learning for domain transfer 3D protoacoustic image reconstruction. Phys Med Biol 2024; 69:10.1088/1361-6560/ad3327. [PMID: 38471184 PMCID: PMC11076107 DOI: 10.1088/1361-6560/ad3327] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Accepted: 03/12/2024] [Indexed: 03/14/2024]
Abstract
Objective. Protoacoustic imaging showed great promise in providing real-time 3D dose verification of proton therapy. However, the limited acquisition angle in protoacoustic imaging induces severe artifacts, which impairs its accuracy for dose verification. In this study, we developed a hybrid-supervised deep learning method for protoacoustic imaging to address the limited view issue.Approach. We proposed a Recon-Enhance two-stage deep learning method. In the Recon-stage, a transformer-based network was developed to reconstruct initial pressure maps from raw acoustic signals. The network is trained in a hybrid-supervised approach, where it is first trained using supervision by the iteratively reconstructed pressure map and then fine-tuned using transfer learning and self-supervision based on the data fidelity constraint. In the enhance-stage, a 3D U-net is applied to further enhance the image quality with supervision from the ground truth pressure map. The final protoacoustic images are then converted to dose for proton verification.Main results. The results evaluated on a dataset of 126 prostate cancer patients achieved an average root mean squared errors (RMSE) of 0.0292, and an average structural similarity index measure (SSIM) of 0.9618, out-performing related start-of-the-art methods. Qualitative results also demonstrated that our approach addressed the limit-view issue with more details reconstructed. Dose verification achieved an average RMSE of 0.018, and an average SSIM of 0.9891. Gamma index evaluation demonstrated a high agreement (94.7% and 95.7% for 1%/3 mm and 1%/5 mm) between the predicted and the ground truth dose maps. Notably, the processing time was reduced to 6 s, demonstrating its feasibility for online 3D dose verification for prostate proton therapy.Significance. Our study achieved start-of-the-art performance in the challenging task of direct reconstruction from radiofrequency signals, demonstrating the great promise of PA imaging as a highly efficient and accurate tool forinvivo3D proton dose verification to minimize the range uncertainties of proton therapy to improve its precision and outcomes.
Collapse
Affiliation(s)
- Yankun Lang
- Department of Radiation Oncology Physics, University of Maryland, Baltimore, Baltimore, MD 21201, United States of America
| | - Zhuoran Jiang
- Department of Radiation Oncology, Duke University, Durham, NC 27710, United States of America
| | - Leshan Sun
- Department of Biomedical Engineering and Radiology, University of California, Irvine, Irnive, CA, 92617, United States of America
| | - Liangzhong Xiang
- Department of Biomedical Engineering and Radiology, University of California, Irvine, Irnive, CA, 92617, United States of America
| | - Lei Ren
- Department of Radiation Oncology Physics, University of Maryland, Baltimore, Baltimore, MD 21201, United States of America
| |
Collapse
|
10
|
Lu B, Fu L, Pan Y, Dong Y. SWISTA-Nets: Subband-adaptive wavelet iterative shrinkage thresholding networks for image reconstruction. Comput Med Imaging Graph 2024; 113:102345. [PMID: 38330636 DOI: 10.1016/j.compmedimag.2024.102345] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2023] [Revised: 01/29/2024] [Accepted: 01/29/2024] [Indexed: 02/10/2024]
Abstract
Robust and interpretable image reconstruction is central to imageology applications in clinical practice. Prevalent deep networks, with strong learning ability to extract implicit information from data manifold, are still lack of prior knowledge introduced from mathematics or physics, leading to instability, poor structure interpretability and high computation cost. As to this issue, we propose two prior knowledge-driven networks to combine the good interpretability of mathematical methods and the powerful learnability of deep learning methods. Incorporating different kinds of prior knowledge, we propose subband-adaptive wavelet iterative shrinkage thresholding networks (SWISTA-Nets), where almost every network module is in one-to-one correspondence with each step involved in the iterative algorithm. By end-to-end training of proposed SWISTA-Nets, implicit information can be extracted from training data and guide the tuning process of key parameters that possess mathematical definition. The inverse problems associated with two medical imaging modalities, i.e., electromagnetic tomography and X-ray computational tomography are applied to validate the proposed networks. Both visual and quantitative results indicate that the SWISTA-Nets outperform mathematical methods and state-of-the-art prior knowledge-driven networks, especially with fewer training parameters, interpretable network structures and well robustness. We assume that our analysis will support further investigation of prior knowledge-driven networks in the field of ill-posed image reconstruction.
Collapse
Affiliation(s)
- Binchun Lu
- Department of Precision Instrument, Tsinghua University, Beijing 100084, China.
| | - Lidan Fu
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China.
| | - Yixuan Pan
- Department of Precision Instrument, Tsinghua University, Beijing 100084, China.
| | - Yonggui Dong
- Department of Precision Instrument, Tsinghua University, Beijing 100084, China.
| |
Collapse
|
11
|
Li Y, Feng J, Xiang J, Li Z, Liang D. AIRPORT: A Data Consistency Constrained Deep Temporal Extrapolation Method To Improve Temporal Resolution In Contrast Enhanced CT Imaging. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:1605-1618. [PMID: 38133967 DOI: 10.1109/tmi.2023.3344712] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/24/2023]
Abstract
Typical tomographic image reconstruction methods require that the imaged object is static and stationary during the time window to acquire a minimally complete data set. The violation of this requirement leads to temporal-averaging errors in the reconstructed images. For a fixed gantry rotation speed, to reduce the errors, it is desired to reconstruct images using data acquired over a narrower angular range, i.e., with a higher temporal resolution. However, image reconstruction with a narrower angular range violates the data sufficiency condition, resulting in severe data-insufficiency-induced errors. The purpose of this work is to decouple the trade-off between these two types of errors in contrast-enhanced computed tomography (CT) imaging. We demonstrated that using the developed data consistency constrained deep temporal extrapolation method (AIRPORT), the entire time-varying imaged object can be accurately reconstructed with 40 frames-per-second temporal resolution, the time window needed to acquire a single projection view data using a typical C-arm cone-beam CT system. AIRPORT is applicable to general non-sparse imaging tasks using a single short-scan data acquisition.
Collapse
|
12
|
Li M, Niu C, Wang G, Amma MR, Chapagain KM, Gabrielson S, Li A, Jonker K, de Ruiter N, Clark JA, Butler P, Butler A, Yu H. Deep Few-view High-resolution Photon-counting Extremity CT at Halved Dose for a Clinical Trial. ARXIV 2024:arXiv:2403.12331v1. [PMID: 38562444 PMCID: PMC10984006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
The latest X-ray photon-counting computed tomography (PCCT) for extremity allows multi-energy high-resolution (HR) imaging for tissue characterization and material decomposition. However, both radiation dose and imaging speed need improvement for contrast-enhanced and other studies. Despite the success of deep learning methods for 2D few-view reconstruction, applying them to HR volumetric reconstruction of extremity scans for clinical diagnosis has been limited due to GPU memory constraints, training data scarcity, and domain gap issues. In this paper, we propose a deep learning-based approach for PCCT image reconstruction at halved dose and doubled speed in a New Zealand clinical trial. Particularly, we present a patch-based volumetric refinement network to alleviate the GPU memory limitation, train network with synthetic data, and use model-based iterative refinement to bridge the gap between synthetic and real-world data. The simulation and phantom experiments demonstrate consistently improved results under different acquisition conditions on both in- and off-domain structures using a fixed network. The image quality of 8 patients from the clinical trial are evaluated by three radiologists in comparison with the standard image reconstruction with a full-view dataset. It is shown that our proposed approach is essentially identical to or better than the clinical benchmark in terms of diagnostic image quality scores. Our approach has a great potential to improve the safety and efficiency of PCCT without compromising image quality.
Collapse
Affiliation(s)
- Mengzhou Li
- Biomedical Imaging Center, Rensselaer Polytechnic, Troy, NY, 12180 USA
| | - Chuang Niu
- Biomedical Imaging Center, Rensselaer Polytechnic, Troy, NY, 12180 USA
| | - Ge Wang
- Biomedical Imaging Center, Rensselaer Polytechnic, Troy, NY, 12180 USA
| | - Maya R Amma
- MARS Bioimaging Limited, Christchurch, New Zealand, 8041
| | - Krishna M Chapagain
- MARS Bioimaging Limited, Christchurch, New Zealand, 8041
- Department of Radiology, University of Otago, Christchurch, New Zealand, 8011
| | | | - Andrew Li
- Pacific Radiology, Christchurch, New Zealand, 8013
| | - Kevin Jonker
- MARS Bioimaging Limited, Christchurch, New Zealand, 8041
- University of Canterbury, Christchurch, New Zealand, 8041
| | | | - Jennifer A Clark
- MARS Bioimaging Limited, Christchurch, New Zealand, 8041
- Department of Radiology, University of Otago, Christchurch, New Zealand, 8011
| | - Phil Butler
- MARS Bioimaging Limited, Christchurch, New Zealand, 8041
| | - Anthony Butler
- MARS Bioimaging Limited, Christchurch, New Zealand, 8041
- Department of Radiology, University of Otago, Christchurch, New Zealand, 8011
- Canterbury District Health Board, Christchurch, New Zealand, 8011
| | - Hengyong Yu
- Department of ECE, University of Massachusetts Lowell, Lowell, MA, USA, 01854
| |
Collapse
|
13
|
Zhou H, Zhang H, Zhao X, Zhang P, Zhu Y. A model-based direct inversion network (MDIN) for dual spectral computed tomography. Phys Med Biol 2024; 69:055005. [PMID: 38271738 DOI: 10.1088/1361-6560/ad229f] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 01/25/2024] [Indexed: 01/27/2024]
Abstract
Objective. Dual spectral computed tomography (DSCT) is a very challenging problem in the field of imaging. Due to the nonlinearity of its mathematical model, the images reconstructed by the conventional CT usually suffer from the beam hardening artifacts. Additionally, several existing DSCT methods rely heavily on the information of the spectra, which is often not readily available in applications. To address this problem, in this study, we aim to develop a novel approach to improve the DSCT reconstruction performance.Approach. A model-based direct inversion network (MDIN) is proposed for DSCT, which can directly predict the basis material images from the collected polychromatic projections. The all operations are performed in the network, requiring neither the conventional algorithms nor the information of the spectra. It can be viewed as an approximation to the inverse procedure of DSCT imaging model. The MDIN is composed of projection pre-decomposition module (PD-module), domain transformation layer (DT-layer), and image post-decomposition module (ID-module). The PD-module first performs the pre-decomposition on the polychromatic projections that consists of a series of stacked one-dimensional convolution layers. The DT-layer is designed to obtain the preliminary decomposed results, which has the characteristics of sparsely connected and learnable parameters. And the ID-module uses a deep neural network to further decompose the reconstructed results of the DT-layer so as to achieve higher-quality basis material images.Main results. Numerical experiments demonstrate that the proposed MDIN has significant advantages in substance decomposition, artifact reduction and noise suppression compared to other methods in the DSCT reconstruction.Significance. The proposed method has a flexible applicability, which can be extended to other CT problems, such as multi-spectral CT and low dose CT.
Collapse
Affiliation(s)
- Haichuan Zhou
- School of Mathematical Sciences, Capital Normal University, Beijing, 100048, People's Republic of China
- School of Medical Technology and Engineering, Henan University of Science and Technology, Luoyang, 471000, People's Republic of China
| | - Huitao Zhang
- School of Mathematical Sciences, Capital Normal University, Beijing, 100048, People's Republic of China
- Shenzhen National Applied Mathematics Center, Southern University of Science and Technology, Shenzhen, 518055, People's Republic of China
| | - Xing Zhao
- School of Mathematical Sciences, Capital Normal University, Beijing, 100048, People's Republic of China
- Shenzhen National Applied Mathematics Center, Southern University of Science and Technology, Shenzhen, 518055, People's Republic of China
| | - Peng Zhang
- School of Mathematical Sciences, Capital Normal University, Beijing, 100048, People's Republic of China
| | - Yining Zhu
- School of Mathematical Sciences, Capital Normal University, Beijing, 100048, People's Republic of China
- Shenzhen National Applied Mathematics Center, Southern University of Science and Technology, Shenzhen, 518055, People's Republic of China
| |
Collapse
|
14
|
Kang Y, Liu J, Wu F, Wang K, Qiang J, Hu D, Zhang Y. Deep convolutional dictionary learning network for sparse view CT reconstruction with a group sparse prior. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 244:108010. [PMID: 38199137 DOI: 10.1016/j.cmpb.2024.108010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Revised: 12/25/2023] [Accepted: 01/05/2024] [Indexed: 01/12/2024]
Abstract
Purpose Numerous techniques based on deep learning have been utilized in sparse view computed tomography (CT) imaging. Nevertheless, the majority of techniques are instinctively constructed utilizing state-of-the-art opaque convolutional neural networks (CNNs) and lack interpretability. Moreover, CNNs tend to focus on local receptive fields and neglect nonlocal self-similarity prior information. Obtaining diagnostically valuable images from sparsely sampled projections is a challenging and ill-posed task. Method To address this issue, we propose a unique and understandable model named DCDL-GS for sparse view CT imaging. This model relies on a network comprised of convolutional dictionary learning and a nonlocal group sparse prior. To enhance the quality of image reconstruction, we utilize a neural network in conjunction with a statistical iterative reconstruction framework and perform a set number of iterations. Inspired by group sparsity priors, we adopt a novel group thresholding operation to improve the feature representation and constraint ability and obtain a theoretical interpretation. Furthermore, our DCDL-GS model incorporates filtered backprojection (FBP) reconstruction, fast sliding window nonlocal self-similarity operations, and a lightweight and interpretable convolutional dictionary learning network to enhance the applicability of the model. Results The efficiency of our proposed DCDL-GS model in preserving edges and recovering features is demonstrated by the visual results obtained on the LDCT-P and UIH datasets. Compared to the results of the most advanced techniques, the quantitative results are enhanced, with increases of 0.6-0.8 dB for the peak signal-to-noise ratio (PSNR), 0.005-0.01 for the structural similarity index measure (SSIM), and 1-1.3 for the regulated Fréchet inception distance (rFID) on the test dataset. The quantitative results also show the effectiveness of our proposed deep convolution iterative reconstruction module and nonlocal group sparse prior. Conclusion In this paper, we create a consolidated and enhanced mathematical model by integrating projection data and prior knowledge of images into a deep iterative model. The model is more practical and interpretable than existing approaches. The results from the experiment show that the proposed model performs well in comparison to the others.
Collapse
Affiliation(s)
- Yanqin Kang
- College of Computer and Information, Anhui Polytechnic University, Wuhu, China; Key Laboratory of Computer Network and Information Integration (Southeast University) Ministry of Education Nanjing, China
| | - Jin Liu
- College of Computer and Information, Anhui Polytechnic University, Wuhu, China; Key Laboratory of Computer Network and Information Integration (Southeast University) Ministry of Education Nanjing, China.
| | - Fan Wu
- College of Computer and Information, Anhui Polytechnic University, Wuhu, China
| | - Kun Wang
- College of Computer and Information, Anhui Polytechnic University, Wuhu, China
| | - Jun Qiang
- College of Computer and Information, Anhui Polytechnic University, Wuhu, China
| | - Dianlin Hu
- Key Laboratory of Computer Network and Information Integration (Southeast University) Ministry of Education Nanjing, China; School of Computer Science and Engineering, Southeast University, Nanjing, China
| | - Yikun Zhang
- Key Laboratory of Computer Network and Information Integration (Southeast University) Ministry of Education Nanjing, China; School of Computer Science and Engineering, Southeast University, Nanjing, China
| |
Collapse
|
15
|
Bousse A, Kandarpa VSS, Rit S, Perelli A, Li M, Wang G, Zhou J, Wang G. Systematic Review on Learning-based Spectral CT. IEEE TRANSACTIONS ON RADIATION AND PLASMA MEDICAL SCIENCES 2024; 8:113-137. [PMID: 38476981 PMCID: PMC10927029 DOI: 10.1109/trpms.2023.3314131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/14/2024]
Abstract
Spectral computed tomography (CT) has recently emerged as an advanced version of medical CT and significantly improves conventional (single-energy) CT. Spectral CT has two main forms: dual-energy computed tomography (DECT) and photon-counting computed tomography (PCCT), which offer image improvement, material decomposition, and feature quantification relative to conventional CT. However, the inherent challenges of spectral CT, evidenced by data and image artifacts, remain a bottleneck for clinical applications. To address these problems, machine learning techniques have been widely applied to spectral CT. In this review, we present the state-of-the-art data-driven techniques for spectral CT.
Collapse
Affiliation(s)
- Alexandre Bousse
- LaTIM, Inserm UMR 1101, Université de Bretagne Occidentale, 29238 Brest, France
| | | | - Simon Rit
- Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Étienne, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69373, Lyon, France
| | - Alessandro Perelli
- Department of Biomedical Engineering, School of Science and Engineering, University of Dundee, DD1 4HN, UK
| | - Mengzhou Li
- Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, New York, USA
| | - Guobao Wang
- Department of Radiology, University of California Davis Health, Sacramento, USA
| | - Jian Zhou
- CTIQ, Canon Medical Research USA, Inc., Vernon Hills, 60061, USA
| | - Ge Wang
- Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, New York, USA
| |
Collapse
|
16
|
Nelson BJ, Kc P, Badal A, Jiang L, Masters SC, Zeng R. Pediatric evaluations for deep learning CT denoising. Med Phys 2024; 51:978-990. [PMID: 38127330 DOI: 10.1002/mp.16901] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Revised: 11/13/2023] [Accepted: 12/04/2023] [Indexed: 12/23/2023] Open
Abstract
BACKGROUND Deep learning (DL) CT denoising models have the potential to improve image quality for lower radiation dose exams. These models are generally trained with large quantities of adult patient image data. However, CT, and increasingly DL denoising methods, are used in both adult and pediatric populations. Pediatric body habitus and size can differ significantly from adults and vary dramatically from newborns to adolescents. Ensuring that pediatric subgroups of different body sizes are not disadvantaged by DL methods requires evaluations capable of assessing performance in each subgroup. PURPOSE To assess DL CT denoising in pediatric and adult-sized patients, we built a framework of computer simulated image quality (IQ) control phantoms and evaluation methodology. METHODS The computer simulated IQ phantoms in the framework featured pediatric-sized versions of standard CatPhan 600 and MITA-LCD phantoms with a range of diameters matching the mean effective diameters of pediatric patients ranging from newborns to 18 years old. These phantoms were used in simulating CT images that were then inputs for a DL denoiser to evaluate performance in different sized patients. Adult CT test images were simulated using standard-sized phantoms scanned with adult scan protocols. Pediatric CT test images were simulated with pediatric-sized phantoms and adjusted pediatric protocols. The framework's evaluation methodology consisted of denoising both adult and pediatric test images then assessing changes in image quality, including noise, image sharpness, CT number accuracy, and low contrast detectability. To demonstrate the use of the framework, a REDCNN denoising model trained on adult patient images was evaluated. To validate that the DL model performance measured with the proposed pediatric IQ phantoms was representative of performance in more realistic patient anatomy, anthropomorphic pediatric XCAT phantoms of the same age range were also used to compare noise reduction performance. RESULTS Using the proposed pediatric-sized IQ phantom framework, size differences between adult and pediatric-sized phantoms were observed to substantially influence the adult trained DL denoising model's performance. When applied to adult images, the DL model achieved a 60% reduction in noise standard deviation without substantial loss in sharpness in mid or high spatial frequencies. However, in smaller phantoms the denoising performance dropped due to different image noise textures resulting from the smaller field of view (FOV) between adult and pediatric protocols. In the validation study, noise reduction trends in the pediatric-sized IQ phantoms were found to be consistent with those found in anthropomorphic phantoms. CONCLUSION We developed a framework of using pediatric-sized IQ phantoms for pediatric subgroup evaluation of DL denoising models. Using the framework, we found the performance of an adult trained DL denoiser did not generalize well in the smaller diameter phantoms corresponding to younger pediatric patient sizes. Our work suggests noise texture differences from FOV changes between adult and pediatric protocols can contribute to poor generalizability in DL denoising and that the proposed framework is an effective means to identify these performance disparities for a given model.
Collapse
Affiliation(s)
- Brandon J Nelson
- Center for Devices and Radiological Health, Office of Science and Engineering Labs, Division of Imaging, Diagnostics, and Software Reliability, U.S. Food and Drug Administration, Silver Spring, Maryland, USA
| | - Prabhat Kc
- Center for Devices and Radiological Health, Office of Science and Engineering Labs, Division of Imaging, Diagnostics, and Software Reliability, U.S. Food and Drug Administration, Silver Spring, Maryland, USA
| | - Andreu Badal
- Center for Devices and Radiological Health, Office of Science and Engineering Labs, Division of Imaging, Diagnostics, and Software Reliability, U.S. Food and Drug Administration, Silver Spring, Maryland, USA
| | - Lu Jiang
- Center for Devices and Radiological Health, Office of Product Evaluation and Quality, Office of Radiological Health, U.S. Food and Drug Administration, Silver Spring, Maryland, USA
| | - Shane C Masters
- Center for Drug Evaluation and Research, Office of Specialty Medicine, Division of Imaging and Radiation Medicine, U.S. Food and Drug Administration, Silver Spring, Maryland, USA
| | - Rongping Zeng
- Center for Devices and Radiological Health, Office of Science and Engineering Labs, Division of Imaging, Diagnostics, and Software Reliability, U.S. Food and Drug Administration, Silver Spring, Maryland, USA
| |
Collapse
|
17
|
Liu A, Gang GJ, Stayman JW. Fourier Diffusion for Sparse CT Reconstruction. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12925:1292516. [PMID: 39247536 PMCID: PMC11378968 DOI: 10.1117/12.3008622] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/10/2024]
Abstract
Sparse CT reconstruction continues to be an area of interest in a number of novel imaging systems. Many different approaches have been tried including model-based methods, compressed sensing approaches, and most recently deep-learning-based processing. Diffusion models, in particular, have become extremely popular due to their ability to effectively encode rich information about images and to allow for posterior sampling to generate many possible outputs. One drawback of diffusion models is that their recurrent structure tends to be computationally expensive. In this work we apply a new Fourier diffusion approach that permits processing with many fewer time steps than the standard scalar diffusion model. We present an extension of the Fourier diffusion technique and evaluate it in a simulated breast cone-beam CT system with a sparse view acquisition.
Collapse
Affiliation(s)
- Anqi Liu
- Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Grace J Gang
- Radiology, University of Pennsylvania, Philadelphia, PA, USA
| | - J Webster Stayman
- Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
18
|
Bousse A, Kandarpa VSS, Rit S, Perelli A, Li M, Wang G, Zhou J, Wang G. Systematic Review on Learning-based Spectral CT. ARXIV 2024:arXiv:2304.07588v8. [PMID: 37461421 PMCID: PMC10350100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Indexed: 07/24/2023]
Abstract
Spectral computed tomography (CT) has recently emerged as an advanced version of medical CT and significantly improves conventional (single-energy) CT. Spectral CT has two main forms: dual-energy computed tomography (DECT) and photon-counting computed tomography (PCCT), which offer image improvement, material decomposition, and feature quantification relative to conventional CT. However, the inherent challenges of spectral CT, evidenced by data and image artifacts, remain a bottleneck for clinical applications. To address these problems, machine learning techniques have been widely applied to spectral CT. In this review, we present the state-of-the-art data-driven techniques for spectral CT.
Collapse
Affiliation(s)
| | | | - Simon Rit
- Univ. Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Étienne, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69373, Lyon, France
| | - Alessandro Perelli
- School of Science and Engineering, University of Dundee, DD1 4HN Dundee, U.K
| | - Mengzhou Li
- Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY 12180 USA
| | - Guobao Wang
- Department of Radiology, University of California Davis Health, Sacramento, CA 95817 USA
| | - Jian Zhou
- CTIQ, Canon Medical Research USA, Inc., Vernon Hills, IL 60061 USA
| | - Ge Wang
- Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, NY 12180 USA
| |
Collapse
|
19
|
Lin J, Li J, Dou J, Zhong L, Di J, Qin Y. Dual-Domain Reconstruction Network Incorporating Multi-Level Wavelet Transform and Recurrent Convolution for Sparse View Computed Tomography Imaging. Tomography 2024; 10:133-158. [PMID: 38250957 PMCID: PMC11154272 DOI: 10.3390/tomography10010011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 01/08/2024] [Accepted: 01/10/2024] [Indexed: 01/23/2024] Open
Abstract
Sparse view computed tomography (SVCT) aims to reduce the number of X-ray projection views required for reconstructing the cross-sectional image of an object. While SVCT significantly reduces X-ray radiation dose and speeds up scanning, insufficient projection data give rise to issues such as severe streak artifacts and blurring in reconstructed images, thereby impacting the diagnostic accuracy of CT detection. To address this challenge, a dual-domain reconstruction network incorporating multi-level wavelet transform and recurrent convolution is proposed in this paper. The dual-domain network is composed of a sinogram domain network (SDN) and an image domain network (IDN). Multi-level wavelet transform is employed in both IDN and SDN to decompose sinograms and CT images into distinct frequency components, which are then processed through separate network branches to recover detailed information within their respective frequency bands. To capture global textures, artifacts, and shallow features in sinograms and CT images, a recurrent convolution unit (RCU) based on convolutional long and short-term memory (Conv-LSTM) is designed, which can model their long-range dependencies through recurrent calculation. Additionally, a self-attention-based multi-level frequency feature normalization fusion (MFNF) block is proposed to assist in recovering high-frequency components by aggregating low-frequency components. Finally, an edge loss function based on the Laplacian of Gaussian (LoG) is designed as the regularization term for enhancing the recovery of high-frequency edge structures. The experimental results demonstrate the effectiveness of our approach in reducing artifacts and enhancing the reconstruction of intricate structural details across various sparse views and noise levels. Our method excels in both performance and robustness, as evidenced by its superior outcomes in numerous qualitative and quantitative assessments, surpassing contemporary state-of-the-art CNNs or Transformer-based reconstruction methods.
Collapse
Affiliation(s)
- Juncheng Lin
- Institute of Advanced Photonics Technology, School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China; (J.L.); (J.L.); (J.D.); (L.Z.); (Y.Q.)
- Guangdong Provincial Key Laboratory of Information Photonics Technology, Guangdong University of Technology, Guangzhou 510006, China
- Key Laboratory of Photonic Technology for Integrated Sensing and Communication, Ministry of Education, Guangdong University of Technology, Guangzhou 510006, China
| | - Jialin Li
- Institute of Advanced Photonics Technology, School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China; (J.L.); (J.L.); (J.D.); (L.Z.); (Y.Q.)
- Guangdong Provincial Key Laboratory of Information Photonics Technology, Guangdong University of Technology, Guangzhou 510006, China
- Key Laboratory of Photonic Technology for Integrated Sensing and Communication, Ministry of Education, Guangdong University of Technology, Guangzhou 510006, China
| | - Jiazhen Dou
- Institute of Advanced Photonics Technology, School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China; (J.L.); (J.L.); (J.D.); (L.Z.); (Y.Q.)
- Guangdong Provincial Key Laboratory of Information Photonics Technology, Guangdong University of Technology, Guangzhou 510006, China
- Key Laboratory of Photonic Technology for Integrated Sensing and Communication, Ministry of Education, Guangdong University of Technology, Guangzhou 510006, China
| | - Liyun Zhong
- Institute of Advanced Photonics Technology, School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China; (J.L.); (J.L.); (J.D.); (L.Z.); (Y.Q.)
- Guangdong Provincial Key Laboratory of Information Photonics Technology, Guangdong University of Technology, Guangzhou 510006, China
- Key Laboratory of Photonic Technology for Integrated Sensing and Communication, Ministry of Education, Guangdong University of Technology, Guangzhou 510006, China
| | - Jianglei Di
- Institute of Advanced Photonics Technology, School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China; (J.L.); (J.L.); (J.D.); (L.Z.); (Y.Q.)
- Guangdong Provincial Key Laboratory of Information Photonics Technology, Guangdong University of Technology, Guangzhou 510006, China
- Key Laboratory of Photonic Technology for Integrated Sensing and Communication, Ministry of Education, Guangdong University of Technology, Guangzhou 510006, China
| | - Yuwen Qin
- Institute of Advanced Photonics Technology, School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China; (J.L.); (J.L.); (J.D.); (L.Z.); (Y.Q.)
- Guangdong Provincial Key Laboratory of Information Photonics Technology, Guangdong University of Technology, Guangzhou 510006, China
- Key Laboratory of Photonic Technology for Integrated Sensing and Communication, Ministry of Education, Guangdong University of Technology, Guangzhou 510006, China
| |
Collapse
|
20
|
Liu P, Fang C, Qiao Z. A dense and U-shaped transformer with dual-domain multi-loss function for sparse-view CT reconstruction. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2024; 32:207-228. [PMID: 38306086 DOI: 10.3233/xst-230184] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/03/2024]
Abstract
OBJECTIVE CT image reconstruction from sparse-view projections is an important imaging configuration for low-dose CT, as it can reduce radiation dose. However, the CT images reconstructed from sparse-view projections by traditional analytic algorithms suffer from severe sparse artifacts. Therefore, it is of great value to develop advanced methods to suppress these artifacts. In this work, we aim to use a deep learning (DL)-based method to suppress sparse artifacts. METHODS Inspired by the good performance of DenseNet and Transformer architecture in computer vision tasks, we propose a Dense U-shaped Transformer (D-U-Transformer) to suppress sparse artifacts. This architecture exploits the advantages of densely connected convolutions in capturing local context and Transformer in modelling long-range dependencies, and applies channel attention to fusion features. Moreover, we design a dual-domain multi-loss function with learned weights for the optimization of the model to further improve image quality. RESULTS Experimental results of our proposed D-U-Transformer yield performance improvements on the well-known Mayo Clinic LDCT dataset over several representative DL-based models in terms of artifact suppression and image feature preservation. Extensive internal ablation experiments demonstrate the effectiveness of the components in the proposed model for sparse-view computed tomography (SVCT) reconstruction. SIGNIFICANCE The proposed method can effectively suppress sparse artifacts and achieve high-precision SVCT reconstruction, thus promoting clinical CT scanning towards low-dose radiation and high-quality imaging. The findings of this work can be applied to denoising and artifact removal tasks in CT and other medical images.
Collapse
Affiliation(s)
- Peng Liu
- School of Computer and Information Technology, Shanxi University, Taiyuan, China
- Department of Big Data and Intelligent Engineering, Shanxi Institute of Technology, Yangquan, China
| | - Chenyun Fang
- School of Computer and Information Technology, Shanxi University, Taiyuan, China
| | - Zhiwei Qiao
- School of Computer and Information Technology, Shanxi University, Taiyuan, China
| |
Collapse
|
21
|
Ylisiurua S, Sipola A, Nieminen MT, Brix MAK. Deep learning enables time-efficient soft tissue enhancement in CBCT: Proof-of-concept study for dentomaxillofacial applications. Phys Med 2024; 117:103184. [PMID: 38016216 DOI: 10.1016/j.ejmp.2023.103184] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Revised: 10/06/2023] [Accepted: 11/19/2023] [Indexed: 11/30/2023] Open
Abstract
PURPOSE The use of iterative and deep learning reconstruction methods, which would allow effective noise reduction, is limited in cone-beam computed tomography (CBCT). As a consequence, the visibility of soft tissues is limited with CBCT. The study aimed to improve this issue through time-efficient deep learning enhancement (DLE) methods. METHODS Two DLE networks, UNIT and U-Net, were trained with simulated CBCT data. The performance of the networks was tested with three different test data sets. The quantitative evaluation measured the structural similarity index measure (SSIM) and the peak signal-to-noise ratio (PSNR) of the DLE reconstructions with respect to the ground truth iterative reconstruction method. In the second assessment, a dentomaxillofacial radiologist assessed the resolution of hard tissue structures, visibility of soft tissues, and overall image quality of real patient data using the Likert scale. Finally, the technical image quality was determined using modulation transfer function, noise power spectrum, and noise magnitude analyses. RESULTS The study demonstrated that deep learning CBCT denoising is feasible and time efficient. The DLE methods, trained with simulated CBCT data, generalized well, and DLE provided quantitatively (SSIM/PSNR) and visually similar noise-reduction as conventional IR, but with faster processing time. The DLE methods improved soft tissue visibility compared to the conventional Feldkamp-Davis-Kress (FDK) algorithm through noise reduction. However, in hard tissue quantification tasks, the radiologist preferred the FDK over the DLE methods. CONCLUSION Post-reconstruction DLE allowed feasible reconstruction times while yielding improvements in soft tissue visibility in each dataset.
Collapse
Affiliation(s)
- Sampo Ylisiurua
- Research Unit of Health Sciences and Technology, University of Oulu, Oulu 90220, Finland; Department of Diagnostic Radiology, Oulu University Hospital, Oulu 90220, Finland.
| | - Annina Sipola
- Medical Research Center, University of Oulu and Oulu University Hospital, Oulu 90220, Finland; Department of Dental Imaging, Oulu University Hospital, Oulu 90220, Finland; Research Unit of Oral Health Sciences, University of Oulu, Oulu 90220, Finland.
| | - Miika T Nieminen
- Research Unit of Health Sciences and Technology, University of Oulu, Oulu 90220, Finland; Department of Diagnostic Radiology, Oulu University Hospital, Oulu 90220, Finland; Medical Research Center, University of Oulu and Oulu University Hospital, Oulu 90220, Finland.
| | - Mikael A K Brix
- Research Unit of Health Sciences and Technology, University of Oulu, Oulu 90220, Finland; Department of Diagnostic Radiology, Oulu University Hospital, Oulu 90220, Finland; Medical Research Center, University of Oulu and Oulu University Hospital, Oulu 90220, Finland.
| |
Collapse
|
22
|
Kim S, Kim B, Lee J, Baek J. Sparsier2Sparse: Self-supervised convolutional neural network-based streak artifacts reduction in sparse-view CT images. Med Phys 2023; 50:7731-7747. [PMID: 37303108 DOI: 10.1002/mp.16552] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2022] [Revised: 05/26/2023] [Accepted: 05/29/2023] [Indexed: 06/13/2023] Open
Abstract
BACKGROUND Sparse-view computed tomography (CT) has attracted a lot of attention for reducing both scanning time and radiation dose. However, sparsely-sampled projection data generate severe streak artifacts in the reconstructed images. In recent decades, many sparse-view CT reconstruction techniques based on fully-supervised learning have been proposed and have shown promising results. However, it is not feasible to acquire pairs of full-view and sparse-view CT images in real clinical practice. PURPOSE In this study, we propose a novel self-supervised convolutional neural network (CNN) method to reduce streak artifacts in sparse-view CT images. METHODS We generate the training dataset using only sparse-view CT data and train CNN based on self-supervised learning. Since the streak artifacts can be estimated using prior images under the same CT geometry system, we acquire prior images by iteratively applying the trained network to given sparse-view CT images. We then subtract the estimated steak artifacts from given sparse-view CT images to produce the final results. RESULTS We validated the imaging performance of the proposed method using extended cardiac-torso (XCAT) and the 2016 AAPM Low-Dose CT Grand Challenge dataset from Mayo Clinic. From the results of visual inspection and modulation transfer function (MTF), the proposed method preserved the anatomical structures effectively and showed higher image resolution compared to the various streak artifacts reduction methods for all projection views. CONCLUSIONS We propose a new framework for streak artifacts reduction when only the sparse-view CT data are given. Although we do not use any information of full-view CT data for CNN training, the proposed method achieved the highest performance in preserving fine details. By overcoming the limitation of dataset requirements on fully-supervised-based methods, we expect that our framework can be utilized in the medical imaging field.
Collapse
Affiliation(s)
- Seongjun Kim
- School of Integrated Technology, Yonsei University, Incheon, South Korea
| | - Byeongjoon Kim
- Department of Artificial Intelligence, College of Computing, Yonsei University, Seoul, South Korea
| | - Jooho Lee
- Department of Artificial Intelligence, College of Computing, Yonsei University, Seoul, South Korea
| | - Jongduk Baek
- Department of Artificial Intelligence, College of Computing, Yonsei University, Seoul, South Korea
- Bareunex Imaging, Inc., Seoul, South Korea
| |
Collapse
|
23
|
Ge X, Yang P, Wu Z, Luo C, Jin P, Wang Z, Wang S, Huang Y, Niu T. Virtual differential phase-contrast and dark-field imaging of x-ray absorption images via deep learning. Bioeng Transl Med 2023; 8:e10494. [PMID: 38023711 PMCID: PMC10658538 DOI: 10.1002/btm2.10494] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 12/22/2022] [Accepted: 01/04/2023] [Indexed: 01/21/2023] Open
Abstract
Weak absorption contrast in biological tissues has hindered x-ray computed tomography from accessing biological structures. Recently, grating-based imaging has emerged as a promising solution to biological low-contrast imaging, providing complementary and previously unavailable structural information of the specimen. Although it has been successfully applied to work with conventional x-ray sources, grating-based imaging is time-consuming and requires a sophisticated experimental setup. In this work, we demonstrate that a deep convolutional neural network trained with a generative adversarial network can directly convert x-ray absorption images into differential phase-contrast and dark-field images that are comparable to those obtained at both a synchrotron beamline and a laboratory facility. By smearing back all of the virtual projections, high-quality tomographic images of biological test specimens deliver the differential phase-contrast- and dark-field-like contrast and quantitative information, broadening the horizon of x-ray image contrast generation.
Collapse
Affiliation(s)
- Xin Ge
- School of Science, Shenzhen Campus of Sun Yat‐sen UniversityShenzhenGuangdongChina
- Institute of Biomedical EngineeringShenzhen Bay LaboratoryShenzhenGuangdongChina
| | - Pengfei Yang
- College of Biomedical Engineering and Instrument Science, Zhejiang UniversityHangzhouZhejiangChina
| | - Zhao Wu
- National Synchrotron Radiation LaboratoryUniversity of Science and Technology of ChinaHefeiAnhuiChina
| | - Chen Luo
- Institute of Biomedical EngineeringShenzhen Bay LaboratoryShenzhenGuangdongChina
| | - Peng Jin
- Institute of Biomedical EngineeringShenzhen Bay LaboratoryShenzhenGuangdongChina
| | - Zhili Wang
- Department of Optical EngineeringSchool of Physics, Hefei University of TechnologyHefeiAnhuiChina
| | - Shengxiang Wang
- Spallation Neutron Source Science CenterDongguanGuangdongChina
- Institute of High Energy Physics, Chinese Academy of SciencesBeijingChina
| | - Yongsheng Huang
- School of Science, Shenzhen Campus of Sun Yat‐sen UniversityShenzhenGuangdongChina
| | - Tianye Niu
- Institute of Biomedical EngineeringShenzhen Bay LaboratoryShenzhenGuangdongChina
- Peking University Aerospace School of Clinical Medicine, Aerospace Center HospitalBeijingChina
| |
Collapse
|
24
|
Gao Y, Tan J, Shi Y, Zhang H, Lu S, Gupta A, Li H, Reiter M, Liang Z. Machine Learned Texture Prior From Full-Dose CT Database via Multi-Modality Feature Selection for Bayesian Reconstruction of Low-Dose CT. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:3129-3139. [PMID: 34968178 PMCID: PMC9243192 DOI: 10.1109/tmi.2021.3139533] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
In our earlier study, we proposed a regional Markov random field type tissue-specific texture prior from previous full-dose computed tomography (FdCT) scan for current low-dose CT (LdCT) imaging, which showed clinical benefits through task-based evaluation. Nevertheless, two assumptions were made for early study. One assumption is that the center pixel has a linear relationship with its nearby neighbors and the other is previous FdCT scans of the same subject are available. To eliminate the two assumptions, we proposed a database assisted end-to-end LdCT reconstruction framework which includes a deep learning texture prior model and a multi-modality feature based candidate selection model. A convolutional neural network-based texture prior is proposed to eliminate the linear relationship assumption. And for scenarios in which the concerned subject has no previous FdCT scans, we propose to select one proper prior candidate from the FdCT database using multi-modality features. Features from three modalities are used including the subjects' physiological factors, the CT scan protocol, and a novel feature named Lung Mark which is deliberately proposed to reflect the z-axial property of human anatomy. Moreover, a majority vote strategy is designed to overcome the noise effect from LdCT scans. Experimental results showed the effectiveness of Lung Mark. The selection model has accuracy of 84% testing on 1,470 images from 49 subjects. The learned texture prior from FdCT database provided reconstruction comparable to the subjects having corresponding FdCT. This study demonstrated the feasibility of bringing clinically relevant textures from available FdCT database to perform Bayesian reconstruction of any current LdCT scan.
Collapse
|
25
|
Li Q, Li R, Wang T, Cheng Y, Qiang Y, Wu W, Zhao J, Zhang D. A cascade-based dual-domain data correction network for sparse view CT image reconstruction. Comput Biol Med 2023; 165:107345. [PMID: 37603960 DOI: 10.1016/j.compbiomed.2023.107345] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Revised: 07/18/2023] [Accepted: 08/07/2023] [Indexed: 08/23/2023]
Abstract
Computed tomography (CT) provides non-invasive anatomical structures of the human body and is also widely used for clinical diagnosis, but excessive ionizing radiation in X-rays can cause harm to the human body. Therefore, the researchers obtained sparse sinograms reconstructed sparse view CT images (SVCT) by reducing the amount of X-ray projection, thereby reducing the radiological effects caused by radiation. This paper proposes a cascade-based dual-domain data correction network (CDDCN), which can effectively combine the complementary information contained in the sinogram domain and the image domain to reconstruct high-quality CT images from sparse view sinograms. Specifically, several encoder-decoder subnets are cascaded in the sinogram domain to reconstruct artifact-free and noise-free CT images. In the encoder-decoder subnets, spatial-channel domain learning is designed to achieve efficient feature fusion through a group merging structure, providing continuous and elaborate pixel-level features and improving feature extraction efficiency. At the same time, to ensure that the original sinogram data collected can be retained, a sinogram data consistency layer is proposed to ensure the fidelity of the sinogram data. To further maintain the consistency between the reconstructed image and the reference image, a multi-level composite loss function is designed for regularization to compensate for excessive smoothing and distortion of the image caused by pixel loss and preserve image details and texture. Quantitative and qualitative analysis shows that CDDCN achieves competitive results in artifact removal, edge preservation, detail restoration, and visual improvement for sparsely sampled data under different views.
Collapse
Affiliation(s)
- Qing Li
- College of Computer Science and Technology (College of Data Science), Taiyuan University of Technology, Taiyuan, 030024, China
| | - Runrui Li
- College of Computer Science and Technology (College of Data Science), Taiyuan University of Technology, Taiyuan, 030024, China
| | - Tao Wang
- College of Computer Science and Technology (College of Data Science), Taiyuan University of Technology, Taiyuan, 030024, China
| | - Yubin Cheng
- College of Computer Science and Technology (College of Data Science), Taiyuan University of Technology, Taiyuan, 030024, China
| | - Yan Qiang
- College of Computer Science and Technology (College of Data Science), Taiyuan University of Technology, Taiyuan, 030024, China.
| | - Wei Wu
- Department of Clinical Laboratory, Affiliated People's Hospital of Shanxi Medical University, Shanxi Provincial People's Hospital, Taiyuan, 030012, China
| | - Juanjuan Zhao
- College of Computer Science and Technology (College of Data Science), Taiyuan University of Technology, Taiyuan, 030024, China; School of Information Engineering, Jinzhong College of Information, Jinzhong, 030800, China
| | - Dongxu Zhang
- College of Computer Science and Technology (College of Data Science), Taiyuan University of Technology, Taiyuan, 030024, China
| |
Collapse
|
26
|
Cheng W, He J, Liu Y, Zhang H, Wang X, Liu Y, Zhang P, Chen H, Gui Z. CAIR: Combining integrated attention with iterative optimization learning for sparse-view CT reconstruction. Comput Biol Med 2023; 163:107161. [PMID: 37311381 DOI: 10.1016/j.compbiomed.2023.107161] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Revised: 05/21/2023] [Accepted: 06/07/2023] [Indexed: 06/15/2023]
Abstract
Sparse-view CT is an efficient way for low dose scanning but degrades image quality. Inspired by the successful use of non-local attention in natural image denoising and compression artifact removal, we proposed a network combining integrated attention and iterative optimization learning for sparse-view CT reconstruction (CAIR). Specifically, we first unrolled the proximal gradient descent into a deep network and added an enhanced initializer between the gradient term and the approximation term. It can enhance the information flow between different layers, fully preserve the image details, and improve the network convergence speed. Secondly, the integrated attention module was introduced into the reconstruction process as a regularization term. It adaptively fuses the local and non-local features of the image which are used to reconstruct the complex texture and repetitive details of the image, respectively. Note that we innovatively designed a one-shot iteration strategy to simplify the network structure and reduce the reconstruction time while maintaining image quality. Experiments showed that the proposed method is very robust and outperforms state-of-the-art methods in terms of both quantitative and qualitative, greatly improving the preservation of structures and the removal of artifacts.
Collapse
Affiliation(s)
- Weiting Cheng
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, 030051, China
| | - Jichun He
- School of Medical and BioInformation Engineering, Northeastern University, Shenyang, 110000, China
| | - Yi Liu
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, 030051, China
| | - Haowen Zhang
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, 030051, China
| | - Xiang Wang
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, 030051, China
| | - Yuhang Liu
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, 030051, China
| | - Pengcheng Zhang
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, 030051, China
| | - Hao Chen
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, 030051, China
| | - Zhiguo Gui
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, 030051, China.
| |
Collapse
|
27
|
Ernst P, Chatterjee S, Rose G, Speck O, Nürnberger A. Sinogram upsampling using Primal-Dual UNet for undersampled CT and radial MRI reconstruction. Neural Netw 2023; 166:704-721. [PMID: 37604079 DOI: 10.1016/j.neunet.2023.08.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 08/03/2023] [Accepted: 08/04/2023] [Indexed: 08/23/2023]
Abstract
Computed tomography (CT) and magnetic resonance imaging (MRI) are two widely used clinical imaging modalities for non-invasive diagnosis. However, both of these modalities come with certain problems. CT uses harmful ionising radiation, and MRI suffers from slow acquisition speed. Both problems can be tackled by undersampling, such as sparse sampling. However, such undersampled data leads to lower resolution and introduces artefacts. Several techniques, including deep learning based methods, have been proposed to reconstruct such data. However, the undersampled reconstruction problem for these two modalities was always considered as two different problems and tackled separately by different research works. This paper proposes a unified solution for both sparse CT and undersampled radial MRI reconstruction, achieved by applying Fourier transform-based pre-processing on the radial MRI and then finally reconstructing both modalities using sinogram upsampling combined with filtered back-projection. The Primal-Dual network is a deep learning based method for reconstructing sparsely-sampled CT data. This paper introduces Primal-Dual UNet, which improves the Primal-Dual network in terms of accuracy and reconstruction speed. The proposed method resulted in an average SSIM of 0.932±0.021 while performing sparse CT reconstruction for fan-beam geometry with a sparsity level of 16, achieving a statistically significant improvement over the previous model, which resulted in 0.919±0.016. Furthermore, the proposed model resulted in 0.903±0.019 and 0.957±0.023 average SSIM while reconstructing undersampled brain and abdominal MRI data with an acceleration factor of 16, respectively - statistically significant improvements over the original model, which resulted in 0.867±0.025 and 0.949±0.025. Finally, this paper shows that the proposed network not only improves the overall image quality, but also improves the image quality for the regions-of-interest: liver, kidneys, and spleen; as well as generalises better than the baselines in presence the of a needle.
Collapse
Affiliation(s)
- Philipp Ernst
- Data and Knowledge Engineering Group, Faculty of Computer Science, Otto von Guericke University Magdeburg, Germany; Research Campus STIMULATE, Otto von Guericke University Magdeburg, Germany
| | - Soumick Chatterjee
- Data and Knowledge Engineering Group, Faculty of Computer Science, Otto von Guericke University Magdeburg, Germany; Genomics Research Centre, Human Technopole, Milan, Italy.
| | - Georg Rose
- Institute of Medical Engineering, Faculty of Electrical Engineering and Information Technology, Otto von Guericke University Magdeburg, Germany; Research Campus STIMULATE, Otto von Guericke University Magdeburg, Germany
| | - Oliver Speck
- Biomedical Magnetic Resonance, Faculty of Natural Sciences, Otto von Guericke University Magdeburg, Germany; Research Campus STIMULATE, Otto von Guericke University Magdeburg, Germany; German Centre for Neurodegenerative Disease, Magdeburg, Germany; Centre for Behavioural Brain Sciences, Magdeburg, Germany
| | - Andreas Nürnberger
- Data and Knowledge Engineering Group, Faculty of Computer Science, Otto von Guericke University Magdeburg, Germany; Centre for Behavioural Brain Sciences, Magdeburg, Germany
| |
Collapse
|
28
|
Li M, Wang J, Chen Y, Tang Y, Wu Z, Qi Y, Jiang H, Zheng J, Tsui BMW. Low-Dose CT Image Synthesis for Domain Adaptation Imaging Using a Generative Adversarial Network With Noise Encoding Transfer Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:2616-2630. [PMID: 37030685 DOI: 10.1109/tmi.2023.3261822] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Deep learning (DL) based image processing methods have been successfully applied to low-dose x-ray images based on the assumption that the feature distribution of the training data is consistent with that of the test data. However, low-dose computed tomography (LDCT) images from different commercial scanners may contain different amounts and types of image noise, violating this assumption. Moreover, in the application of DL based image processing methods to LDCT, the feature distributions of LDCT images from simulation and clinical CT examination can be quite different. Therefore, the network models trained with simulated image data or LDCT images from one specific scanner may not work well for another CT scanner and image processing task. To solve such domain adaptation problem, in this study, a novel generative adversarial network (GAN) with noise encoding transfer learning (NETL), or GAN-NETL, is proposed to generate a paired dataset with a different noise style. Specifically, we proposed a method to perform noise encoding operator and incorporate it into the generator to extract a noise style. Meanwhile, with a transfer learning (TL) approach, the image noise encoding operator transformed the noise type of the source domain to that of the target domain for realistic noise generation. One public and two private datasets are used to evaluate the proposed method. Experiment results demonstrated the feasibility and effectiveness of our proposed GAN-NETL model in LDCT image synthesis. In addition, we conduct additional image denoising study using the synthesized clinical LDCT data, which verified the merit of the proposed synthesis in improving the performance of the DL based LDCT processing method.
Collapse
|
29
|
Liu J, Zhang T, Kang Y, Wang Y, Zhang Y, Hu D, Chen Y. Deep residual constrained reconstruction via learned convolutional sparse coding for low-dose CT imaging. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104868] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/28/2023]
|
30
|
Yu Z, Rahman A, Laforest R, Schindler TH, Gropler RJ, Wahl RL, Siegel BA, Jha AK. Need for objective task-based evaluation of deep learning-based denoising methods: A study in the context of myocardial perfusion SPECT. Med Phys 2023; 50:4122-4137. [PMID: 37010001 PMCID: PMC10524194 DOI: 10.1002/mp.16407] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2022] [Revised: 01/20/2023] [Accepted: 03/01/2023] [Indexed: 04/04/2023] Open
Abstract
BACKGROUND Artificial intelligence-based methods have generated substantial interest in nuclear medicine. An area of significant interest has been the use of deep-learning (DL)-based approaches for denoising images acquired with lower doses, shorter acquisition times, or both. Objective evaluation of these approaches is essential for clinical application. PURPOSE DL-based approaches for denoising nuclear-medicine images have typically been evaluated using fidelity-based figures of merit (FoMs) such as root mean squared error (RMSE) and structural similarity index measure (SSIM). However, these images are acquired for clinical tasks and thus should be evaluated based on their performance in these tasks. Our objectives were to: (1) investigate whether evaluation with these FoMs is consistent with objective clinical-task-based evaluation; (2) provide a theoretical analysis for determining the impact of denoising on signal-detection tasks; and (3) demonstrate the utility of virtual imaging trials (VITs) to evaluate DL-based methods. METHODS A VIT to evaluate a DL-based method for denoising myocardial perfusion SPECT (MPS) images was conducted. To conduct this evaluation study, we followed the recently published best practices for the evaluation of AI algorithms for nuclear medicine (the RELAINCE guidelines). An anthropomorphic patient population modeling clinically relevant variability was simulated. Projection data for this patient population at normal and low-dose count levels (20%, 15%, 10%, 5%) were generated using well-validated Monte Carlo-based simulations. The images were reconstructed using a 3-D ordered-subsets expectation maximization-based approach. Next, the low-dose images were denoised using a commonly used convolutional neural network-based approach. The impact of DL-based denoising was evaluated using both fidelity-based FoMs and area under the receiver operating characteristic curve (AUC), which quantified performance on the clinical task of detecting perfusion defects in MPS images as obtained using a model observer with anthropomorphic channels. We then provide a mathematical treatment to probe the impact of post-processing operations on signal-detection tasks and use this treatment to analyze the findings of this study. RESULTS Based on fidelity-based FoMs, denoising using the considered DL-based method led to significantly superior performance. However, based on ROC analysis, denoising did not improve, and in fact, often degraded detection-task performance. This discordance between fidelity-based FoMs and task-based evaluation was observed at all the low-dose levels and for different cardiac-defect types. Our theoretical analysis revealed that the major reason for this degraded performance was that the denoising method reduced the difference in the means of the reconstructed images and of the channel operator-extracted feature vectors between the defect-absent and defect-present cases. CONCLUSIONS The results show the discrepancy between the evaluation of DL-based methods with fidelity-based metrics versus the evaluation on clinical tasks. This motivates the need for objective task-based evaluation of DL-based denoising approaches. Further, this study shows how VITs provide a mechanism to conduct such evaluations computationally, in a time and resource-efficient setting, and avoid risks such as radiation dose to the patient. Finally, our theoretical treatment reveals insights into the reasons for the limited performance of the denoising approach and may be used to probe the effect of other post-processing operations on signal-detection tasks.
Collapse
Affiliation(s)
- Zitong Yu
- Department of Biomedical Engineering, Washington University in St. Louis, St. Louis, Missouri, USA
| | - Ashequr Rahman
- Department of Biomedical Engineering, Washington University in St. Louis, St. Louis, Missouri, USA
| | - Richard Laforest
- Mallinckrodt Institute of Radiology, Washington University in St. Louis, St. Louis, Missouri, USA
| | - Thomas H. Schindler
- Mallinckrodt Institute of Radiology, Washington University in St. Louis, St. Louis, Missouri, USA
| | - Robert J. Gropler
- Mallinckrodt Institute of Radiology, Washington University in St. Louis, St. Louis, Missouri, USA
| | - Richard L. Wahl
- Mallinckrodt Institute of Radiology, Washington University in St. Louis, St. Louis, Missouri, USA
| | - Barry A. Siegel
- Mallinckrodt Institute of Radiology, Washington University in St. Louis, St. Louis, Missouri, USA
| | - Abhinav K. Jha
- Department of Biomedical Engineering, Washington University in St. Louis, St. Louis, Missouri, USA
- Mallinckrodt Institute of Radiology, Washington University in St. Louis, St. Louis, Missouri, USA
| |
Collapse
|
31
|
Guo Z, Liu Z, Barbastathis G, Zhang Q, Glinsky ME, Alpert BK, Levine ZH. Noise-resilient deep learning for integrated circuit tomography. OPTICS EXPRESS 2023; 31:15355-15371. [PMID: 37157639 DOI: 10.1364/oe.486213] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
X-ray tomography is a non-destructive imaging technique that reveals the interior of an object from its projections at different angles. Under sparse-view and low-photon sampling, regularization priors are required to retrieve a high-fidelity reconstruction. Recently, deep learning has been used in X-ray tomography. The prior learned from training data replaces the general-purpose priors in iterative algorithms, achieving high-quality reconstructions with a neural network. Previous studies typically assume the noise statistics of test data are acquired a priori from training data, leaving the network susceptible to a change in the noise characteristics under practical imaging conditions. In this work, we propose a noise-resilient deep-reconstruction algorithm and apply it to integrated circuit tomography. By training the network with regularized reconstructions from a conventional algorithm, the learned prior shows strong noise resilience without the need for additional training with noisy examples, and allows us to obtain acceptable reconstructions with fewer photons in test data. The advantages of our framework may further enable low-photon tomographic imaging where long acquisition times limit the ability to acquire a large training set.
Collapse
|
32
|
Li Y, Sun X, Wang S, Li X, Qin Y, Pan J, Chen P. MDST: multi-domain sparse-view CT reconstruction based on convolution and swin transformer. Phys Med Biol 2023; 68:095019. [PMID: 36889004 DOI: 10.1088/1361-6560/acc2ab] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2022] [Accepted: 03/08/2023] [Indexed: 03/10/2023]
Abstract
Objective.Sparse-view computed tomography (SVCT), which can reduce the radiation doses administered to patients and hasten data acquisition, has become an area of particular interest to researchers. Most existing deep learning-based image reconstruction methods are based on convolutional neural networks (CNNs). Due to the locality of convolution and continuous sampling operations, existing approaches cannot fully model global context feature dependencies, which makes the CNN-based approaches less efficient in modeling the computed tomography (CT) images with various structural information.Approach.To overcome the above challenges, this paper develops a novel multi-domain optimization network based on convolution and swin transformer (MDST). MDST uses swin transformer block as the main building block in both projection (residual) domain and image (residual) domain sub-networks, which models global and local features of the projections and reconstructed images. MDST consists of two modules for initial reconstruction and residual-assisted reconstruction, respectively. The sparse sinogram is first expanded in the initial reconstruction module with a projection domain sub-network. Then, the sparse-view artifacts are effectively suppressed by an image domain sub-network. Finally, the residual assisted reconstruction module to correct the inconsistency of the initial reconstruction, further preserving image details.Main results. Extensive experiments on CT lymph node datasets and real walnut datasets show that MDST can effectively alleviate the loss of fine details caused by information attenuation and improve the reconstruction quality of medical images.Significance.MDST network is robust and can effectively reconstruct images with different noise level projections. Different from the current prevalent CNN-based networks, MDST uses transformer as the main backbone, which proves the potential of transformer in SVCT reconstruction.
Collapse
Affiliation(s)
- Yu Li
- Department of Information and Communication Engineering, North University of China, Taiyuan, People's Republic of China
- The State Key Lab for Electronic Testing Technology, North University of China, People's Republic of China
| | - XueQin Sun
- Department of Information and Communication Engineering, North University of China, Taiyuan, People's Republic of China
- The State Key Lab for Electronic Testing Technology, North University of China, People's Republic of China
| | - SuKai Wang
- Department of Information and Communication Engineering, North University of China, Taiyuan, People's Republic of China
- The State Key Lab for Electronic Testing Technology, North University of China, People's Republic of China
| | - XuRu Li
- Department of Information and Communication Engineering, North University of China, Taiyuan, People's Republic of China
- The State Key Lab for Electronic Testing Technology, North University of China, People's Republic of China
| | - YingWei Qin
- Department of Information and Communication Engineering, North University of China, Taiyuan, People's Republic of China
- The State Key Lab for Electronic Testing Technology, North University of China, People's Republic of China
| | - JinXiao Pan
- Department of Information and Communication Engineering, North University of China, Taiyuan, People's Republic of China
- The State Key Lab for Electronic Testing Technology, North University of China, People's Republic of China
| | - Ping Chen
- Department of Information and Communication Engineering, North University of China, Taiyuan, People's Republic of China
- The State Key Lab for Electronic Testing Technology, North University of China, People's Republic of China
| |
Collapse
|
33
|
Shen J, Luo M, Liu H, Liao P, Chen H, Zhang Y. MLF-IOSC: Multi-Level Fusion Network With Independent Operation Search Cell for Low-Dose CT Denoising. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:1145-1158. [PMID: 36423311 DOI: 10.1109/tmi.2022.3224396] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Computed tomography (CT) is widely used in clinical medicine, and low-dose CT (LDCT) has become popular to reduce potential patient harm during CT acquisition. However, LDCT aggravates the problem of noise and artifacts in CT images, increasing diagnosis difficulty. Through deep learning, denoising CT images by artificial neural network has aroused great interest for medical imaging and has been hugely successful. We propose a framework to achieve excellent LDCT noise reduction using independent operation search cells, inspired by neural architecture search, and introduce the Laplacian to further improve image quality. Employing patch-based training, the proposed method can effectively eliminate CT image noise while retaining the original structures and details, hence significantly improving diagnosis efficiency and promoting LDCT clinical applications.
Collapse
|
34
|
Du C, Qiao Z. EPRI sparse reconstruction method based on deep learning. Magn Reson Imaging 2023; 97:24-30. [PMID: 36493992 DOI: 10.1016/j.mri.2022.12.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Revised: 11/03/2022] [Accepted: 12/04/2022] [Indexed: 12/12/2022]
Abstract
Electron paramagnetic resonance imaging (EPRI) is an advanced tumor oxygen concentration imaging method. Now, the bottleneck problem of EPRI is that the scanning time is too long. Sparse reconstruction is an effective and fast imaging method, which means reconstructing images from sparse-view projections. However, the EPRI images sparsely reconstructed by the classic filtered back projection (FBP) algorithm often contain severe streak artifacts, which affect subsequent image processing. In this work, we propose a feature pyramid attention-based, residual, dense, deep convolutional network (FRD-Net) to suppress the streak artifacts in the FBP-reconstructed images. This network combines residual connection, attention mechanism, dense connections and introduces perceptual loss. The EPRI image with streak artifacts is used as the input of the network and the output-label is the corresponding high-quality image densely reconstructed by the FBP algorithm. After training, the FRD-Net gets the capability of suppressing streak artifacts. The real data reconstruction experiments show that the FRD-Net can better improve the sparse reconstruction accuracy, compared with three existing representative deep networks.
Collapse
Affiliation(s)
- Congcong Du
- School of Computer and Information Technology, Shanxi University, Taiyuan, Shanxi 030006, China
| | - Zhiwei Qiao
- School of Computer and Information Technology, Shanxi University, Taiyuan, Shanxi 030006, China.
| |
Collapse
|
35
|
Xia W, Shan H, Wang G, Zhang Y. Physics-/Model-Based and Data-Driven Methods for Low-Dose Computed Tomography: A survey. IEEE SIGNAL PROCESSING MAGAZINE 2023; 40:89-100. [PMID: 38404742 PMCID: PMC10883591 DOI: 10.1109/msp.2022.3204407] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/27/2024]
Abstract
Since 2016, deep learning (DL) has advanced tomographic imaging with remarkable successes, especially in low-dose computed tomography (LDCT) imaging. Despite being driven by big data, the LDCT denoising and pure end-to-end reconstruction networks often suffer from the black box nature and major issues such as instabilities, which is a major barrier to apply deep learning methods in low-dose CT applications. An emerging trend is to integrate imaging physics and model into deep networks, enabling a hybridization of physics/model-based and data-driven elements. In this paper, we systematically review the physics/model-based data-driven methods for LDCT, summarize the loss functions and training strategies, evaluate the performance of different methods, and discuss relevant issues and future directions.
Collapse
Affiliation(s)
- Wenjun Xia
- School of Cyber Science and Engineering, Sichuan University, Chengdu 610065, China
| | - Hongming Shan
- Institute of Science and Technology for Brain-Inspired Intelligence and MOE Frontiers Center for Brain Science, Fudan University, Shanghai, and also with Shanghai Center for Brain Science and Brain-Inspired Technology, Shanghai 200433, China
| | - Ge Wang
- Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY 12180 USA
| | - Yi Zhang
- School of Cyber Science and Engineering, Sichuan University, Chengdu 610065, China
| |
Collapse
|
36
|
Yang M, Wang J, Zhang Z, Li J, Liu L. Transfer learning framework for low-dose CT reconstruction based on marginal distribution adaptation in multiscale. Med Phys 2023; 50:1450-1465. [PMID: 36321246 DOI: 10.1002/mp.16027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Revised: 09/05/2022] [Accepted: 09/09/2022] [Indexed: 11/05/2022] Open
Abstract
BACKGROUND With the increasing use of computed tomography (CT) in clinical practice, limiting CT radiation exposure to reduce potential cancer risks has become one of the important directions of medical imaging research. As the dose decreases, the reconstructed CT image will be severely degraded by projection noise. PURPOSE As an important method of image processing, supervised deep learning has been widely used in the restoration of low-dose CT (LDCT) in recent years. However, the normal-dose CT (NDCT) corresponding to a specific LDCT (it is regarded as the label of the LDCT, which is necessary for supervised learning) is very difficult to obtain so that the application of supervised learning methods in LDCT reconstruction is limited. It is necessary to construct a unsupervised deep learning framework for LDCT reconstruction that does not depend on paired LDCT-NDCT datasets. METHODS We presented an unsupervised learning framework for the transferring from the identity mapping to the low-dose reconstruction task, called marginal distribution adaptation in multiscale (MDAM). For NDCTs as source domain data, MDAM is an identity map with two parts: firstly, it establishes a dimensionality reduction mapping, which can obtain the same feature distribution from NDCTs and LDCTs; and then NDCTs is retrieved by reconstructing the image overview and details from the low-dimensional features. For the purpose of the feature transfer between source domain and target domain (LDCTs), we introduce the multiscale feature extraction in the MDAM, and then eliminate differences in probability distributions of these multiscale features between NDCTs and LDCTs through wavelet decomposition and domain adaptation learning. RESULTS Image quality evaluation metrics and subjective quality scores show that, as an unsupervised method, the performance of the MDAM approaches or even surpasses some state-of-the-art supervised methods. Especially, MDAM has been favorably evaluated in terms of noise suppression, structural preservation, and lesion detection. CONCLUSIONS We demonstrated that, the MDAM framework can reconstruct corresponding NDCTs from LDCTs with high accuracy, and without relying on any labeles. Moreover, it is more suitable for clinical application compared with supervised learning methods.
Collapse
Affiliation(s)
- Minghan Yang
- Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, Anhui, China
| | - Jianye Wang
- Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, Anhui, China
| | - Ziheng Zhang
- Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, Anhui, China
| | - Jie Li
- Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, Anhui, China
- Hefei Cancer Hospital, Chinese Academy of Sciences, Hefei, Anhui, China
| | - Lingling Liu
- Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, Anhui, China
- Hefei Cancer Hospital, Chinese Academy of Sciences, Hefei, Anhui, China
| |
Collapse
|
37
|
Lu Z, Xia W, Huang Y, Hou M, Chen H, Zhou J, Shan H, Zhang Y. M 3NAS: Multi-Scale and Multi-Level Memory-Efficient Neural Architecture Search for Low-Dose CT Denoising. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:850-863. [PMID: 36327187 DOI: 10.1109/tmi.2022.3219286] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Lowering the radiation dose in computed tomography (CT) can greatly reduce the potential risk to public health. However, the reconstructed images from dose-reduced CT or low-dose CT (LDCT) suffer from severe noise which compromises the subsequent diagnosis and analysis. Recently, convolutional neural networks have achieved promising results in removing noise from LDCT images. The network architectures that are used are either handcrafted or built on top of conventional networks such as ResNet and U-Net. Recent advances in neural network architecture search (NAS) have shown that the network architecture has a dramatic effect on the model performance. This indicates that current network architectures for LDCT may be suboptimal. Therefore, in this paper, we make the first attempt to apply NAS to LDCT and propose a multi-scale and multi-level memory-efficient NAS for LDCT denoising, termed M3NAS. On the one hand, the proposed M3NAS fuses features extracted by different scale cells to capture multi-scale image structural details. On the other hand, the proposed M3NAS can search a hybrid cell- and network-level structure for better performance. In addition, M3NAS can effectively reduce the number of model parameters and increase the speed of inference. Extensive experimental results on two different datasets demonstrate that the proposed M3NAS can achieve better performance and fewer parameters than several state-of-the-art methods. In addition, we also validate the effectiveness of the multi-scale and multi-level architecture for LDCT denoising, and present further analysis for different configurations of super-net.
Collapse
|
38
|
Yi Z, Wang J, Li M. Deep image and feature prior algorithm based on U-ConformerNet structure. Phys Med 2023; 107:102535. [PMID: 36764130 DOI: 10.1016/j.ejmp.2023.102535] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/11/2022] [Revised: 01/04/2023] [Accepted: 01/25/2023] [Indexed: 02/10/2023] Open
Abstract
PURPOSE The reconstruction performance of the deep image prior (DIP) approach is limited by the conventional convolutional layer structure and it is difficult to enhance its potential. In order to improve the quality of image reconstruction and suppress artifacts, we propose a DIP algorithm with better performance, and verify its superiority in the latest case. METHODS We construct a new U-ConformerNet structure as the DIP algorithm's network, replacing the traditional convolutional layer-based U-net structure, and introduce the 'lpips' deep network based feature distance regularization method. Our algorithm can switch between supervised and unsupervised modes at will to meet different needs. RESULTS The reconstruction was performed on the low dose CT dataset (LoDoPaB). Our algorithm attained a PSNR of more than 35 dB under unsupervised conditions, and the PSNR under the supervised condition is greater than 36 dB. Both of which are better than the performance of the DIP-TV. Furthermore, the accuracy of this method is positively connected with the quality of the a priori image with the help of deep networks. In terms of noise eradication and artifact suppression, the DIP algorithm with U-ConformerNet structure outperforms the standard DIP method based on convolutional structure. CONCLUSIONS It is known by experimental verification that, in unsupervised mode, the algorithm improves the output PSNR by at least 2-3 dB when compared to the DIP-TV algorithm (proposed in 2020). In supervised mode, our algorithm approaches that of the state-of-the-art end-to-end deep learning algorithms.
Collapse
Affiliation(s)
- Zhengming Yi
- The State Key Laboratory of Refractories and Metallurgy, Wuhan University of Science and Technology, Wuhan 430081, Hubei, China; Key Laboratory for Ferrous Metallurgy and Resources Utilization of Metallurgy and Resources Utilization of Education, Wuhan University of Science and Technology, Wuhan 430081, Hubei, China
| | - Junjie Wang
- The State Key Laboratory of Refractories and Metallurgy, Wuhan University of Science and Technology, Wuhan 430081, Hubei, China; Key Laboratory for Ferrous Metallurgy and Resources Utilization of Metallurgy and Resources Utilization of Education, Wuhan University of Science and Technology, Wuhan 430081, Hubei, China
| | - Mingjie Li
- The State Key Laboratory of Refractories and Metallurgy, Wuhan University of Science and Technology, Wuhan 430081, Hubei, China; Key Laboratory for Ferrous Metallurgy and Resources Utilization of Metallurgy and Resources Utilization of Education, Wuhan University of Science and Technology, Wuhan 430081, Hubei, China.
| |
Collapse
|
39
|
Zhang P, Ren S, Liu Y, Gui Z, Shangguan H, Wang Y, Shu H, Chen Y. A total variation prior unrolling approach for computed tomography reconstruction. Med Phys 2023; 50:2816-2834. [PMID: 36791315 DOI: 10.1002/mp.16307] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Revised: 01/09/2023] [Accepted: 02/06/2023] [Indexed: 02/17/2023] Open
Abstract
BACKGROUND With the rapid development of deep learning technology, deep neural networks can effectively enhance the performance of computed tomography (CT) reconstructions. One kind of commonly used method to construct CT reconstruction networks is to unroll the conventional iterative reconstruction (IR) methods to convolutional neural networks (CNNs). However, most unrolling methods primarily unroll the fidelity term of IR methods to CNNs, without unrolling the prior terms. The prior terms are always directly replaced by neural networks. PURPOSE In conventional IR methods, the prior terms play a vital role in improving the visual quality of reconstructed images. Unrolling the hand-crafted prior terms to CNNs may provide a more specialized unrolling approach to further improve the performance of CT reconstruction. In this work, a primal-dual network (PD-Net) was proposed by unrolling both the data fidelity term and the total variation (TV) prior term, which effectively preserves the image edges and textures in the reconstructed images. METHODS By further deriving the Chambolle-Pock (CP) algorithm instance for CT reconstruction, we discovered that the TV prior updates the reconstructed images with its divergences in each iteration of the solution process. Based on this discovery, CNNs were applied to yield the divergences of the feature maps for the reconstructed image generated in each iteration. Additionally, a loss function was applied to the predicted divergences of the reconstructed image to guarantee that the CNNs' results were the divergences of the corresponding feature maps in the iteration. In this manner, the proposed CNNs seem to play the same roles in the PD-Net as the TV prior in the IR methods. Thus, the TV prior in the CP algorithm instance can be directly unrolled to CNNs. RESULTS The datasets from the Low-Dose CT Image and Projection Data and the Piglet dataset were employed to assess the effectiveness of our proposed PD-Net. Compared with conventional CT reconstruction methods, our proposed method effectively preserves the structural and textural information in reference to ground truth. CONCLUSIONS The experimental results show that our proposed PD-Net framework is feasible for the implementation of CT reconstruction tasks. Owing to the promising results yielded by our proposed neural network, this study is intended to inspire further development of unrolling approaches by enabling the direct unrolling of hand-crafted prior terms to CNNs.
Collapse
Affiliation(s)
- Pengcheng Zhang
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, China
| | - Shuhui Ren
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, China
| | - Yi Liu
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, China
| | - Zhiguo Gui
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, China
| | - Hong Shangguan
- School of Electronic Information Engineering, Taiyuan University of Science and Technology, Taiyuan, China
| | - Yanling Wang
- School of Information, Shanxi University of Finance and Economics, Taiyuan, China
| | - Huazhong Shu
- Laboratory of Image Science and Technology, Southeast University, Nanjing, China
| | - Yang Chen
- Laboratory of Image Science and Technology, Southeast University, Nanjing, China.,Centre de Recherche en Information Biomedicale Sino-Francais (LIA CRIBs), Rennes, France.,Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing, China.,Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing, Southeast University, Nanjing, China
| |
Collapse
|
40
|
Xie H, Thorn S, Liu YH, Lee S, Liu Z, Wang G, Sinusas AJ, Liu C. Deep-Learning-Based Few-Angle Cardiac SPECT Reconstruction Using Transformer. IEEE TRANSACTIONS ON RADIATION AND PLASMA MEDICAL SCIENCES 2023; 7:33-40. [PMID: 37397179 PMCID: PMC10312390 DOI: 10.1109/trpms.2022.3187595] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2023]
Abstract
Convolutional neural networks (CNNs) have been extremely successful in various medical imaging tasks. However, because the size of the convolutional kernel used in a CNN is much smaller than the image size, CNN has a strong spatial inductive bias and lacks a global understanding of the input images. Vision Transformer, a recently emerged network structure in computer vision, can potentially overcome the limitations of CNNs for image-reconstruction tasks. In this work, we proposed a slice-by-slice Transformer network (SSTrans-3D) to reconstruct cardiac SPECT images from 3D few-angle data. To be specific, the network reconstructs the whole 3D volume using a slice-by-slice scheme. By doing so, SSTrans-3D alleviates the memory burden required by 3D reconstructions using Transformer. The network can still obtain a global understanding of the image volume with the Transformer attention blocks. Lastly, already reconstructed slices are used as the input to the network so that SSTrans-3D can potentially obtain more informative features from these slices. Validated on porcine, phantom, and human studies acquired using a GE dedicated cardiac SPECT scanner, the proposed method produced images with clearer heart cavity, higher cardiac defect contrast, and more accurate quantitative measurements on the testing data as compared with a deep U-net.
Collapse
Affiliation(s)
| | - Stephanie Thorn
- Department of Internal Medicine (Cardiology) at Yale University
| | - Yi-Hwa Liu
- Department of Internal Medicine (Cardiology) at Yale University
| | - Supum Lee
- Department of Internal Medicine (Cardiology) at Yale University
| | - Zhao Liu
- Department of Radiology and Biomedical Imaging at Yale University
| | - Ge Wang
- Department of Biomedical Engineering at Rensselaer Polytechnic Institute
| | - Albert J Sinusas
- Department of Biomedical Engineering
- Department of Internal Medicine (Cardiology) at Yale University
- Department of Radiology and Biomedical Imaging at Yale University
| | - Chi Liu
- Department of Biomedical Engineering
- Department of Radiology and Biomedical Imaging at Yale University
| |
Collapse
|
41
|
Wang J, Tang Y, Wu Z, Tsui BMW, Chen W, Yang X, Zheng J, Li M. Domain-adaptive denoising network for low-dose CT via noise estimation and transfer learning. Med Phys 2023; 50:74-88. [PMID: 36018732 DOI: 10.1002/mp.15952] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2022] [Revised: 08/19/2022] [Accepted: 08/19/2022] [Indexed: 01/25/2023] Open
Abstract
BACKGROUND In recent years, low-dose computed tomography (LDCT) has played an important role in the diagnosis CT to reduce the potential adverse effects of X-ray radiation on patients, while maintaining the same diagnostic image quality. PURPOSE Deep learning (DL)-based methods have played an increasingly important role in the field of LDCT imaging. However, its performance is highly dependent on the consistency of feature distributions between training data and test data. Due to patient's breathing movements during data acquisition, the paired LDCT and normal dose CT images are difficult to obtain from realistic imaging scenarios. Moreover, LDCT images from simulation or clinical CT examination often have different feature distributions due to the pollution by different amounts and types of image noises. If a network model trained with a simulated dataset is used to directly test clinical patients' LDCT data, its denoising performance may be degraded. Based on this, we propose a novel domain-adaptive denoising network (DADN) via noise estimation and transfer learning to resolve the out-of-distribution problem in LDCT imaging. METHODS To overcome the previous adaptation issue, a novel network model consisting of a reconstruction network and a noise estimation network was designed. The noise estimation network based on a double branch structure is used for image noise extraction and adaptation. Meanwhile, the U-Net-based reconstruction network uses several spatially adaptive normalization modules to fuse multi-scale noise input. Moreover, to facilitate the adaptation of the proposed DADN network to new imaging scenarios, we set a two-stage network training plan. In the first stage, the public simulated dataset is used for training. In the second transfer training stage, we will continue to fine-tune the network model with a torso phantom dataset, while some parameters are frozen. The main reason using the two-stage training scheme is based on the fact that the feature distribution of image content from the public dataset is complex and diverse, whereas the feature distribution of noise pattern from the torso phantom dataset is closer to realistic imaging scenarios. RESULTS In an evaluation study, the trained DADN model is applied to both the public and clinical patient LDCT datasets. Through the comparison of visual inspection and quantitative results, it is shown that the proposed DADN network model can perform well in terms of noise and artifact suppression, while effectively preserving image contrast and details. CONCLUSIONS In this paper, we have proposed a new DL network to overcome the domain adaptation problem in LDCT image denoising. Moreover, the results demonstrate the feasibility and effectiveness of the application of our proposed DADN network model as a new DL-based LDCT image denoising method.
Collapse
Affiliation(s)
- Jiping Wang
- Institute of Electronic Information Engineering, Changchun University of Science and Technology, Changchun, China.,Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China
| | - Yufei Tang
- Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China.,School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
| | - Zhongyi Wu
- Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China.,School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
| | - Benjamin M W Tsui
- Department of Radiology, School of Medicine, Johns Hopkins University, Baltimore, Maryland, USA
| | - Wei Chen
- Minfound Medical Systems Co. Ltd., Shaoxing, Zhejiang, China
| | - Xiaodong Yang
- Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China
| | - Jian Zheng
- Institute of Electronic Information Engineering, Changchun University of Science and Technology, Changchun, China.,Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China.,School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
| | - Ming Li
- Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China.,School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
| |
Collapse
|
42
|
TIME-Net: Transformer-Integrated Multi-Encoder Network for limited-angle artifact removal in dual-energy CBCT. Med Image Anal 2023; 83:102650. [PMID: 36334394 DOI: 10.1016/j.media.2022.102650] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2021] [Revised: 08/25/2022] [Accepted: 10/07/2022] [Indexed: 11/08/2022]
Abstract
Dual-energy cone-beam computed tomography (DE-CBCT) is a promising imaging technique with foreseeable clinical applications. DE-CBCT images acquired with two different spectra can provide material-specific information. Meanwhile, the anatomical consistency and energy-domain correlation result in significant information redundancy, which could be exploited to improve image quality. In this context, this paper develops the Transformer-Integrated Multi-Encoder Network (TIME-Net) for DE-CBCT to remove the limited-angle artifacts. TIME-Net comprises three encoders (image encoder, prior encoder, and transformer encoder), two decoders (low- and high-energy decoders), and one feature fusion module. Three encoders extract various features for image restoration. The feature fusion module compresses these features into more compact shared features and feeds them to the decoders. Two decoders perform differential learning for DE-CBCT images. By design, TIME-Net could obtain high-quality DE-CBCT images using two complementary quarter-scans, holding great potential to reduce radiation dose and shorten the acquisition time. Qualitative and quantitative analyses based on simulated data and real rat data have demonstrated the promising performance of TIME-Net in artifact removal, subtle structure restoration, and reconstruction accuracy preservation. Two clinical applications, virtual non-contrast (VNC) imaging and iodine quantification, have proved the potential utility of the DE-CBCT images provided by TIME-Net.
Collapse
|
43
|
X-ray CT image denoising with MINF: A modularized iterative network framework for data from multiple dose levels. Comput Biol Med 2023; 152:106419. [PMID: 36527781 DOI: 10.1016/j.compbiomed.2022.106419] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 11/30/2022] [Accepted: 12/10/2022] [Indexed: 12/14/2022]
Abstract
In clinical applications, multi-dose scan protocols will cause the noise levels of computed tomography (CT) images to fluctuate widely. The popular low-dose CT (LDCT) denoising network outputs denoised images through an end-to-end mapping between an LDCT image and its corresponding ground truth. The limitation of this method is that the reduced noise level of the image may not meet the diagnostic needs of doctors. To establish a denoising model adapted to the multi-noise levels robustness, we proposed a novel and efficient modularized iterative network framework (MINF) to learn the feature of the original LDCT and the outputs of the previous modules, which can be reused in each following module. The proposed network can achieve the goal of gradual denoising, outputting clinical images with different denoising levels, and providing the reviewing physicians with increased confidence in their diagnosis. Moreover, a multi-scale convolutional neural network (MCNN) module is designed to extract as much feature information as possible during the network's training. Extensive experiments on public and private clinical datasets were carried out, and comparisons with several state-of-the-art methods show that the proposed method can achieve satisfactory results for noise suppression of LDCT images. In further comparisons with modularized adaptive processing neural network (MAP-NN), the proposed network shows superior step-by-step or gradual denoising performance. Considering the high quality of gradual denoising results, the proposed method can obtain satisfactory performance in terms of image contrast and detail protection as the level of denoising increases, which shows its potential to be suitable for a multi-dose levels denoising task.
Collapse
|
44
|
CT-Net: Cascaded T-shape network using spectral redundancy for dual-energy CT limited-angle reconstruction. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104072] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
45
|
Zhu M, Mao Z, Li D, Wang Y, Zeng D, Bian Z, Ma J. Structure-preserved meta-learning uniting network for improving low-dose CT quality. Phys Med Biol 2022; 67. [PMID: 36351294 DOI: 10.1088/1361-6560/aca194] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Accepted: 11/09/2022] [Indexed: 11/10/2022]
Abstract
Objective.Deep neural network (DNN) based methods have shown promising performances for low-dose computed tomography (LDCT) imaging. However, most of the DNN-based methods are trained on simulated labeled datasets, and the low-dose simulation algorithms are usually designed based on simple statistical models which deviate from the real clinical scenarios, which could lead to issues of overfitting, instability and poor robustness. To address these issues, in this work, we present a structure-preserved meta-learning uniting network (shorten as 'SMU-Net') to suppress noise-induced artifacts and preserve structure details in the unlabeled LDCT imaging task in real scenarios.Approach.Specifically, the presented SMU-Net contains two networks, i.e., teacher network and student network. The teacher network is trained on simulated labeled dataset and then helps the student network train with the unlabeled LDCT images via the meta-learning strategy. The student network is trained on real LDCT dataset with the pseudo-labels generated by the teacher network. Moreover, the student network adopts the Co-teaching strategy to improve the robustness of the presented SMU-Net.Main results.We validate the proposed SMU-Net method on three public datasets and one real low-dose dataset. The visual image results indicate that the proposed SMU-Net has superior performance on reducing noise-induced artifacts and preserving structure details. And the quantitative results exhibit that the presented SMU-Net method generally obtains the highest signal-to-noise ratio (PSNR), the highest structural similarity index measurement (SSIM), and the lowest root-mean-square error (RMSE) values or the lowest natural image quality evaluator (NIQE) scores.Significance.We propose a meta learning strategy to obtain high-quality CT images in the LDCT imaging task, which is designed to take advantage of unlabeled CT images to promote the reconstruction performance in the LDCT environments.
Collapse
Affiliation(s)
- Manman Zhu
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
| | - Zerui Mao
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
| | - Danyang Li
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
| | - Yongbo Wang
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
| | - Dong Zeng
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
| | - Zhaoying Bian
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
| | - Jianhua Ma
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, People's Republic of China
| |
Collapse
|
46
|
Deep learning tomographic reconstruction through hierarchical decomposition of domain transforms. Vis Comput Ind Biomed Art 2022; 5:30. [PMID: 36484980 PMCID: PMC9733764 DOI: 10.1186/s42492-022-00127-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Accepted: 11/15/2022] [Indexed: 12/13/2022] Open
Abstract
Deep learning (DL) has shown unprecedented performance for many image analysis and image enhancement tasks. Yet, solving large-scale inverse problems like tomographic reconstruction remains challenging for DL. These problems involve non-local and space-variant integral transforms between the input and output domains, for which no efficient neural network models are readily available. A prior attempt to solve tomographic reconstruction problems with supervised learning relied on a brute-force fully connected network and only allowed reconstruction with a 1284 system matrix size. This cannot practically scale to realistic data sizes such as 5124 and 5126 for three-dimensional datasets. Here we present a novel framework to solve such problems with DL by casting the original problem as a continuum of intermediate representations between the input and output domains. The original problem is broken down into a sequence of simpler transformations that can be well mapped onto an efficient hierarchical network architecture, with exponentially fewer parameters than a fully connected network would need. We applied the approach to computed tomography (CT) image reconstruction for a 5124 system matrix size. This work introduces a new kind of data-driven DL solver for full-size CT reconstruction without relying on the structure of direct (analytical) or iterative (numerical) inversion techniques. This work presents a feasibility demonstration of full-scale learnt reconstruction, whereas more developments will be needed to demonstrate superiority relative to traditional reconstruction approaches. The proposed approach is also extendable to other imaging problems such as emission and magnetic resonance reconstruction. More broadly, hierarchical DL opens the door to a new class of solvers for general inverse problems, which could potentially lead to improved signal-to-noise ratio, spatial resolution and computational efficiency in various areas.
Collapse
|
47
|
Jia Y, McMichael N, Mokarzel P, Thompson B, Si D, Humphries T. Superiorization-inspired unrolled SART algorithm with U-Net generated perturbations for sparse-view and limited-angle CT reconstruction. Phys Med Biol 2022; 67. [PMID: 36541524 DOI: 10.1088/1361-6560/aca513] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Accepted: 11/22/2022] [Indexed: 11/23/2022]
Abstract
Objective.Unrolled algorithms are a promising approach for reconstruction of CT images in challenging scenarios, such as low-dose, sparse-view and limited-angle imaging. In an unrolled algorithm, a fixed number of iterations of a reconstruction method are unrolled into multiple layers of a neural network, and interspersed with trainable layers. The entire network is then trained end-to-end in a supervised fashion, to learn an appropriate regularizer from training data. In this paper we propose a novel unrolled algorithm, and compare its performance with several other approaches on sparse-view and limited-angle CT.Approach.The proposed algorithm is inspired by the superiorization methodology, an optimization heuristic in which iterates of a feasibility-seeking method are perturbed between iterations, typically using descent directions of a model-based penalty function. Our algorithm instead uses a modified U-net architecture to introduce the perturbations, allowing a network to learn beneficial perturbations to the image at various stages of the reconstruction, based on the training data.Main Results.In several numerical experiments modeling sparse-view and limited angle CT scenarios, the algorithm provides excellent results. In particular, it outperforms several competing unrolled methods in limited-angle scenarios, while providing comparable or better performance on sparse-view scenarios.Significance.This work represents a first step towards exploiting the power of deep learning within the superiorization methodology. Additionally, it studies the effect of network architecture on the performance of unrolled methods, as well as the effectiveness of the unrolled approach on both limited-angle CT, where previous studies have primarily focused on the sparse-view and low-dose cases.
Collapse
Affiliation(s)
- Yiran Jia
- School of STEM, University of Washington Bothell, Bothell, WA 98011, United States of America
| | - Noah McMichael
- School of STEM, University of Washington Bothell, Bothell, WA 98011, United States of America
| | - Pedro Mokarzel
- School of STEM, University of Washington Bothell, Bothell, WA 98011, United States of America
| | - Brandon Thompson
- School of STEM, University of Washington Bothell, Bothell, WA 98011, United States of America
| | - Dong Si
- School of STEM, University of Washington Bothell, Bothell, WA 98011, United States of America
| | - Thomas Humphries
- School of STEM, University of Washington Bothell, Bothell, WA 98011, United States of America
| |
Collapse
|
48
|
Qiao Z, Du C. RAD-UNet: a Residual, Attention-Based, Dense UNet for CT Sparse Reconstruction. J Digit Imaging 2022; 35:1748-1758. [PMID: 35882689 PMCID: PMC9712860 DOI: 10.1007/s10278-022-00685-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2021] [Revised: 07/06/2022] [Accepted: 07/08/2022] [Indexed: 10/16/2022] Open
Abstract
To suppress the streak artifacts in images reconstructed from sparse-view projections in computed tomography (CT), a residual, attention-based, dense UNet (RAD-UNet) deep network is proposed to achieve accurate sparse reconstruction. The filtered back projection (FBP) algorithm is used to reconstruct the CT image with streak artifacts from sparse-view projections. Then, the image is processed by the RAD-UNet to suppress streak artifacts and obtain high-quality CT image. Those images with streak artifacts are used as the input of the RAD-UNet, and the output-label images are the corresponding high-quality images. Through training via the large-scale training data, the RAD-UNet can obtain the capability of suppressing streak artifacts. This network combines residual connection, attention mechanism, dense connection and perceptual loss. This network can improve the nonlinear fitting capability and the performance of suppressing streak artifacts. The experimental results show that the RAD-UNet can improve the reconstruction accuracy compared with three existing representative deep networks. It may not only suppress streak artifacts but also better preserve image details. The proposed networks may be readily applied to other image processing tasks including image denoising, image deblurring, and image super-resolution.
Collapse
Affiliation(s)
- Zhiwei Qiao
- School of Computer and Information Technology, Shanxi University, Taiyuan, 030006 Shanxi China
| | - Congcong Du
- School of Computer and Information Technology, Shanxi University, Taiyuan, 030006 Shanxi China
| |
Collapse
|
49
|
Li D, Bian Z, Li S, He J, Zeng D, Ma J. Noise Characteristics Modeled Unsupervised Network for Robust CT Image Reconstruction. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:3849-3861. [PMID: 35939459 DOI: 10.1109/tmi.2022.3197400] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Deep learning (DL)-based methods show great potential in computed tomography (CT) imaging field. The DL-based reconstruction methods are usually evaluated on the training and testing datasets which are obtained from the same distribution, i.e., the same CT scan protocol (i.e., the region setting, kVp, mAs, etc.). In this work, we focus on analyzing the robustness of the DL-based methods against protocol-specific distribution shifts (i.e., the training and testing datasets are from different region settings, different kVp settings, or different mAs settings, respectively). The results show that the DL-based reconstruction methods are sensitive to the protocol-specific perturbations which can be attributed to the noise distribution shift between the training and testing datasets. Based on these findings, we presented a low-dose CT reconstruction method using an unsupervised strategy with the consideration of noise distribution to address the issue of protocol-specific perturbations. Specifically, unpaired sinogram data is enrolled into the network training, which represents unique information for specific imaging protocol, and a Gaussian mixture model (GMM) is introduced to characterize the noise distribution in CT images. It can be termed as GMM based unsupervised CT reconstruction network (GMM-unNet) method. Moreover, an expectation-maximization algorithm is designed to optimize the presented GMM-unNet method. Extensive experiments are performed on three datasets from different scan protocols, which demonstrate that the presented GMM-unNet method outperforms the competing methods both qualitatively and quantitatively.
Collapse
|
50
|
Kim B, Shim H, Baek J. A streak artifact reduction algorithm in sparse-view CT using a self-supervised neural representation. Med Phys 2022; 49:7497-7515. [PMID: 35880806 DOI: 10.1002/mp.15885] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Revised: 07/13/2022] [Accepted: 07/17/2022] [Indexed: 12/27/2022] Open
Abstract
PURPOSE Sparse-view computed tomography (CT) has been attracting attention for its reduced radiation dose and scanning time. However, analytical image reconstruction methods suffer from streak artifacts due to insufficient projection views. Recently, various deep learning-based methods have been developed to solve this ill-posed inverse problem. Despite their promising results, they are easily overfitted to the training data, showing limited generalizability to unseen systems and patients. In this work, we propose a novel streak artifact reduction algorithm that provides a system- and patient-specific solution. METHODS Motivated by the fact that streak artifacts are deterministic errors, we regenerate the same artifacts from a prior CT image under the same system geometry. This prior image need not be perfect but should contain patient-specific information and be consistent with full-view projection data for accurate regeneration of the artifacts. To this end, we use a coordinate-based neural representation that often causes image blur but can greatly suppress the streak artifacts while having multiview consistency. By employing techniques in neural radiance fields originally proposed for scene representations, the neural representation is optimized to the measured sparse-view projection data via self-supervised learning. Then, we subtract the regenerated artifacts from the analytically reconstructed original image to obtain the final corrected image. RESULTS To validate the proposed method, we used simulated data of extended cardiac-torso phantoms and the 2016 NIH-AAPM-Mayo Clinic Low-Dose CT Grand Challenge and experimental data of physical pediatric and head phantoms. The performance of the proposed method was compared with a total variation-based iterative reconstruction method, naive application of the neural representation, and a convolutional neural network-based method. In visual inspection, it was observed that the small anatomical features were best preserved by the proposed method. The proposed method also achieved the best scores in the visual information fidelity, modulation transfer function, and lung nodule segmentation. CONCLUSIONS The results on both simulated and experimental data suggest that the proposed method can effectively reduce the streak artifacts while preserving small anatomical structures that are easily blurred or replaced with misleading features by the existing methods. Since the proposed method does not require any additional training datasets, it would be useful in clinical practice where the large datasets cannot be collected.
Collapse
Affiliation(s)
- Byeongjoon Kim
- School of Integrated Technology, Yonsei University, Incheon, South Korea
| | - Hyunjung Shim
- School of Integrated Technology, Yonsei University, Incheon, South Korea
| | - Jongduk Baek
- School of Integrated Technology, Yonsei University, Incheon, South Korea
| |
Collapse
|