3-D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning From a 2-D Trained Network

Domain adaptive noise reduction with iterative knowledge transfer and style generalization learning

Low-dose computed tomography (LDCT) denoising tasks face significant challenges in practical imaging scenarios. Supervised methods encounter difficulties in real-world scenarios as there are no paired data for training. Moreover, when applied to datasets with varying noise patterns, these methods may experience decreased performance owing to the domain gap. Conversely, unsupervised methods do not require paired data and can be directly trained on real-world data. However, they often exhibit inferior performance compared to supervised methods. To address this issue, it is necessary to leverage the strengths of these supervised and unsupervised methods. In this paper, we propose a novel domain adaptive noise reduction framework (DANRF), which integrates both knowledge transfer and style generalization learning to effectively tackle the domain gap problem. Specifically, an iterative knowledge transfer method with knowledge distillation is selected to train the target model using unlabeled target data and a pre-trained source model trained with paired simulation data. Meanwhile, we introduce the mean teacher mechanism to update the source model, enabling it to adapt to the target domain. Furthermore, an iterative style generalization learning process is also designed to enrich the style diversity of the training dataset. We evaluate the performance of our approach through experiments conducted on multi-source datasets. The results demonstrate the feasibility and effectiveness of our proposed DANRF model in multi-source LDCT image processing tasks. Given its hybrid nature, which combines the advantages of supervised and unsupervised learning, and its ability to bridge domain gaps, our approach is well-suited for improving practical low-dose CT imaging in clinical settings. Code for our proposed approach is publicly available at https://github.com/tyfeiii/DANRF.

Dual and Multi-Target Cone-Beam X-ray Luminescence Computed Tomography Based on the DeepCB-XLCT Network

BACKGROUND AND OBJECTIVE

Emerging as a hybrid imaging modality, cone-beam X-ray luminescence computed tomography (CB-XLCT) has been developed using X-ray-excitable nanoparticles. In contrast to conventional bio-optical imaging techniques like bioluminescence tomography (BLT) and fluorescence molecular tomography (FMT), CB-XLCT offers the advantage of greater imaging depth while significantly reducing interference from autofluorescence and background fluorescence, owing to its utilization of X-ray-excited nanoparticles. However, due to the intricate excitation process and extensive light scattering within biological tissues, the inverse problem of CB-XLCT is fundamentally ill-conditioned.

METHODS

An end-to-end three-dimensional deep encoder-decoder network, termed DeepCB-XLCT, is introduced to improve the quality of CB-XLCT reconstructions. This network directly establishes a nonlinear mapping between the distribution of internal X-ray-excitable nanoparticles and the corresponding boundary fluorescent signals. To improve the fidelity of target shape restoration, the structural similarity loss (SSIM) was incorporated into the objective function of the DeepCB-XLCT network. Additionally, a loss term specifically for target regions was introduced to improve the network's emphasis on the areas of interest. As a result, the inaccuracies in reconstruction caused by the simplified linear model used in conventional methods can be effectively minimized by the proposed DeepCB-XLCT method.

RESULTS AND CONCLUSIONS

Numerical simulations, phantom experiments, and in vivo experiments with two targets were performed, revealing that the DeepCB-XLCT network enhances reconstruction accuracy regarding contrast-to-noise ratio and shape similarity when compared to traditional methods. In addition, the findings from the XLCT tomographic images involving three targets demonstrate its potential for multi-target CB-XLCT imaging.

IQAGPT: computed tomography image quality assessment with vision-language and ChatGPT models

Large language models (LLMs), such as ChatGPT, have demonstrated impressive capabilities in various tasks and attracted increasing interest as a natural language interface across many domains. Recently, large vision-language models (VLMs) that learn rich vision-language correlation from image-text pairs, like BLIP-2 and GPT-4, have been intensively investigated. However, despite these developments, the application of LLMs and VLMs in image quality assessment (IQA), particularly in medical imaging, remains unexplored. This is valuable for objective performance evaluation and potential supplement or even replacement of radiologists' opinions. To this end, this study introduces IQAGPT, an innovative computed tomography (CT) IQA system that integrates image-quality captioning VLM with ChatGPT to generate quality scores and textual reports. First, a CT-IQA dataset comprising 1,000 CT slices with diverse quality levels is professionally annotated and compiled for training and evaluation. To better leverage the capabilities of LLMs, the annotated quality scores are converted into semantically rich text descriptions using a prompt template. Second, the image-quality captioning VLM is fine-tuned on the CT-IQA dataset to generate quality descriptions. The captioning model fuses image and text features through cross-modal attention. Third, based on the quality descriptions, users verbally request ChatGPT to rate image-quality scores or produce radiological quality reports. Results demonstrate the feasibility of assessing image quality using LLMs. The proposed IQAGPT outperformed GPT-4 and CLIP-IQA, as well as multitask classification and regression models that solely rely on images.

A Hybrid Framework of Dual-Domain Signal Restoration and Multi-depth Feature Reinforcement for Low-Dose Lung CT Denoising

Low-dose computer tomography (LDCT) has been widely used in medical diagnosis. Various denoising methods have been presented to remove noise in LDCT scans. However, existing methods cannot achieve satisfactory results due to the difficulties in (1) distinguishing the characteristics of structures, textures, and noise confused in the image domain, and (2) representing local details and global semantics in the hierarchical features. In this paper, we propose a novel denoising method consisting of (1) a 2D dual-domain restoration framework to reconstruct noise-free structure and texture signals separately, and (2) a 3D multi-depth reinforcement U-Net model to further recover image details with enhanced hierarchical features. In the 2D dual-domain restoration framework, the convolutional neural networks are adopted in both the image domain where the image structures are well preserved through the spatial continuity, and the sinogram domain where the textures and noise are separately represented by different wavelet coefficients and processed adaptively. In the 3D multi-depth reinforcement U-Net model, the hierarchical features from the 3D U-Net are enhanced by the cross-resolution attention module (CRAM) and dual-branch graph convolution module (DBGCM). The CRAM preserves local details by integrating adjacent low-level features with different resolutions, while the DBGCM enhances global semantics by building graphs for high-level features in intra-feature and inter-feature dimensions. Experimental results on the LUNA16 dataset and 2016 NIH-AAPM-Mayo Clinic LDCT Grand Challenge dataset illustrate the proposed method outperforms the state-of-the-art methods on removing noise from LDCT images with clear structures and textures, proving its potential in clinical practice.

Image quality evaluation in deep-learning-based CT noise reduction using virtual imaging trial methods: Contrast-dependent spatial resolution

BACKGROUND

Deep-learning-based image reconstruction and noise reduction methods (DLIR) have been increasingly deployed in clinical CT. Accurate image quality assessment of these methods is challenging as the performance measured using physical phantoms may not represent the true performance of DLIR in patients since DLIR is trained mostly on patient images.

PURPOSE

In this work, we aim to develop a patient-data-based virtual imaging trial framework and, as a first application, use it to measure the spatial resolution properties of a DLIR method.

METHODS

The patient-data-based virtual imaging trial framework consists of five steps: (1) insertion of lesions into projection domain data using the acquisition geometry of the patient exam to simulate different lesion characteristics; (2) insertion of noise into projection domain data using a realistic photon statistical model of the CT system to simulate different dose levels; (3) creation of DLIR-processed images from projection or image data; (4) creation of ensembles of DLIR-processed patient images from a large number of noise and lesion realizations; and (5) evaluation of image quality using ensemble DLIR images. This framework was applied to measure the spatial resolution of a ResNet based deep convolutional neural network (DCNN) trained on patient images. Lesions in a cylindrical shape and different contrast levels (-500, -100, -50, -20, -10 HU) were inserted to the lower right lobe of the liver in a patient case. Multiple dose levels were simulated (50%, 25%, 12.5%). Each lesion and dose condition had 600 noise realizations. Multiple reconstruction and denoising methods were used on all the noise realizations, including the original filtered-backprojection (FBP), iterative reconstruction (IR), and the DCNN method with three different strength setting (DCNN-weak, DCNN-medium, and DCNN-strong). Mean lesion signal was calculated by performing ensemble averaging of all the noise realizations for each lesion and dose condition and then subtracting the lesion-present images from the lesion absent images. Modulation transfer functions (MTFs) both in-plane and along the z-axis were calculated based on the mean lesion signals. The standard deviations of MTFs at each condition were estimated with bootstrapping: randomly sampling (with replacement) all the DLIR/FBP/IR images from the ensemble data (600 samples) at each condition. The impact of varying lesion contrast, dose levels, and denoising strengths were evaluated. Statistical analysis with paired t-test was used to compare the z-axis and in-plane spatial resolution of five algorithms for five different contrasts and three dose levels.

RESULTS

The in-plane and z-axis spatial resolution degradation of DCNN becomes more severe as the contrast or radiation dose decreased, or DCNN denoising strength increased. In comparison with FBP, a 59.5% and 4.1% reduction of in-plane and z-axis MTF (in terms of spatial frequencies at 50% MTF), respectively, was observed at low contrast (-10 HU) for DCNN with the highest denoising strength at 25% routine dose level. When the dose level reduces from 50% to 12.5% of routine dose, the in-plane and z-axis MTFs reduces from 92.1% to 76.3%, and from 98.9% to 95.5%, respectively, at contrast of -100 HU, using FBP as the reference. For most conditions of contrasts and dose levels, significant differences were found among the five algorithms, with the following relationship in both in-plane and cross-plane spatial resolution: FBP > DCNN-Weak > IR > DCNN-Medium > DCNN-Strong. The spatial resolution difference among algorithms decreases at higher contrast or dose levels.

CONCLUSIONS

A patient-data-based virtual imaging trial framework was developed and applied to measuring the spatial resolution properties of a DCNN noise reduction method at different contrast and dose levels using real patient data. As with other non-linear image reconstruction and post-processing techniques, the evaluated DCNN method degraded the in-plane and z-axis spatial resolution at lower contrast levels, lower radiation dose, and higher denoising strength.

Streak artefact removal in x-ray dark-field computed tomography using a convolutional neural network

BACKGROUND

Computed tomography (CT) relies on the attenuation of x-rays, and is, hence, of limited use for weakly attenuating organs of the body, such as the lung. X-ray dark-field (DF) imaging is a recently developed technology that utilizes x-ray optical gratings to enable small-angle scattering as an alternative contrast mechanism. The DF signal provides structural information about the micromorphology of an object, complementary to the conventional attenuation signal. A first human-scale x-ray DF CT has been developed by our group. Despite specialized processing algorithms, reconstructed images remain affected by streaking artifacts, which often hinder image interpretation. In recent years, convolutional neural networks have gained popularity in the field of CT reconstruction, amongst others for streak artefact removal.

PURPOSE

Reducing streak artifacts is essential for the optimization of image quality in DF CT, and artefact free images are a prerequisite for potential future clinical application. The purpose of this paper is to demonstrate the feasibility of CNN post-processing for artefact reduction in x-ray DF CT and how multi-rotation scans can serve as a pathway for training data.

METHODS

We employed a supervised deep-learning approach using a three-dimensional dual-frame UNet in order to remove streak artifacts. Required training data were obtained from the experimental x-ray DF CT prototype at our institute. Two different operating modes were used to generate input and corresponding ground truth data sets. Clinically relevant scans at dose-compatible radiation levels were used as input data, and extended scans with substantially fewer artifacts were used as ground truth data. The latter is neither dose-, nor time-compatible and, therefore, unfeasible for clinical imaging of patients.

RESULTS

The trained CNN was able to greatly reduce streak artifacts in DF CT images. The network was tested against images with entirely different, previously unseen image characteristics. In all cases, CNN processing substantially increased the image quality, which was quantitatively confirmed by increased image quality metrics. Fine details are preserved during processing, despite the output images appearing smoother than the ground truth images.

CONCLUSIONS

Our results showcase the potential of a neural network to reduce streak artifacts in x-ray DF CT. The image quality is successfully enhanced in dose-compatible x-ray DF CT, which plays an essential role for the adoption of x-ray DF CT into modern clinical radiology.

Simulated deep CT characterization of liver metastases with high-resolution filtered back projection reconstruction

Early diagnosis and accurate prognosis of colorectal cancer is critical for determining optimal treatment plans and maximizing patient outcomes, especially as the disease progresses into liver metastases. Computed tomography (CT) is a frontline tool for this task; however, the preservation of predictive radiomic features is highly dependent on the scanning protocol and reconstruction algorithm. We hypothesized that image reconstruction with a high-frequency kernel could result in a better characterization of liver metastases features via deep neural networks. This kernel produces images that appear noisier but preserve more sinogram information. A simulation pipeline was developed to study the effects of imaging parameters on the ability to characterize the features of liver metastases. This pipeline utilizes a fractal approach to generate a diverse population of shapes representing virtual metastases, and then it superimposes them on a realistic CT liver region to perform a virtual CT scan using CatSim. Datasets of 10,000 liver metastases were generated, scanned, and reconstructed using either standard or high-frequency kernels. These data were used to train and validate deep neural networks to recover crafted metastases characteristics, such as internal heterogeneity, edge sharpness, and edge fractal dimension. In the absence of noise, models scored, on average, 12.2% ( α = 0.012 ) and 7.5% ( α = 0.049 ) lower squared error for characterizing edge sharpness and fractal dimension, respectively, when using high-frequency reconstructions compared to standard. However, the differences in performance were statistically insignificant when a typical level of CT noise was simulated in the clinical scan. Our results suggest that high-frequency reconstruction kernels can better preserve information for downstream artificial intelligence-based radiomic characterization, provided that noise is limited. Future work should investigate the information-preserving kernels in datasets with clinical labels.

Image fusion-based low-dose CBCT enhancement method for visualizing miniscrew insertion in the infrazygomatic crest

Digital dental technology covers oral cone-beam computed tomography (CBCT) image processing and low-dose CBCT dental applications. A low-dose CBCT image enhancement method based on image fusion is proposed to address the need for subzygomatic small screw insertion. Specifically, firstly, a sharpening correction module is proposed, where the CBCT image is sharpened to compensate for the loss of details in the underexposed/over-exposed region. Secondly, a visibility restoration module based on type II fuzzy sets is designed, and a contrast enhancement module using curve transformation is designed. In addition to this, we propose a perceptual fusion module that fuses visibility and contrast of oral CBCT images. As a result, the problems of overexposure/underexposure, low visibility, and low contrast that occur in oral CBCT images can be effectively addressed with consistent interpretability. The proposed algorithm was analyzed in comparison experiments with a variety of algorithms, as well as ablation experiments. After analysis, compared with advanced enhancement algorithms, this algorithm achieved excellent results in low-dose CBCT enhancement and effective observation of subzygomatic small screw implantation. Compared with the best performing method, the evaluation metric is 0.07-2 higher on both datasets. The project can be found at: https://github.com/sunpeipei2024/low-dose-CBCT .

FSformer: A combined frequency separation network and transformer for LDCT denoising

Low-dose computed tomography (LDCT) has been widely concerned in the field of medical imaging because of its low radiation hazard to humans. However, under low-dose radiation scenarios, a large amount of noise/artifacts are present in the reconstructed image, which reduces the clarity of the image and is not conducive to diagnosis. To improve the LDCT image quality, we proposed a combined frequency separation network and Transformer (FSformer) for LDCT denoising. Firstly, FSformer decomposes the LDCT images into low-frequency images and multi-layer high-frequency images by frequency separation blocks. Then, the low-frequency components are fused with the high-frequency components of different layers to remove the noise in the high-frequency components with the help of the potential texture of low-frequency parts. Next, the estimated noise images can be obtained by using Transformer stage in the frequency aggregation denoising block. Finally, they are fed into the reconstruction prediction block to obtain improved quality images. In addition, a compound loss function with frequency loss and Charbonnier loss is used to guide the training of the network. The performance of FSformer has been validated and evaluated on AAPM Mayo dataset, real Piglet dataset and clinical dataset. Compared with previous representative models in different architectures, FSformer achieves the optimal metrics with PSNR of 33.7714 dB and SSIM of 0.9254 on Mayo dataset, the testing time is 1.825 s. The experimental results show that FSformer is a state-of-the-art (SOTA) model with noise/artifact suppression and texture/organization preservation. Moreover, the model has certain robustness and can effectively improve LDCT image quality.

LIT-Former: Linking In-Plane and Through-Plane Transformers for Simultaneous CT Image Denoising and Deblurring

This paper studies 3D low-dose computed tomography (CT) imaging. Although various deep learning methods were developed in this context, typically they focus on 2D images and perform denoising due to low-dose and deblurring for super-resolution separately. Up to date, little work was done for simultaneous in-plane denoising and through-plane deblurring, which is important to obtain high-quality 3D CT images with lower radiation and faster imaging speed. For this task, a straightforward method is to directly train an end-to-end 3D network. However, it demands much more training data and expensive computational costs. Here, we propose to link in-plane and through-plane transformers for simultaneous in-plane denoising and through-plane deblurring, termed as LIT-Former, which can efficiently synergize in-plane and through-plane sub-tasks for 3D CT imaging and enjoy the advantages of both convolution and transformer networks. LIT-Former has two novel designs: efficient multi-head self-attention modules (eMSM) and efficient convolutional feed-forward networks (eCFN). First, eMSM integrates in-plane 2D self-attention and through-plane 1D self-attention to efficiently capture global interactions of 3D self-attention, the core unit of transformer networks. Second, eCFN integrates 2D convolution and 1D convolution to extract local information of 3D convolution in the same fashion. As a result, the proposed LIT-Former synergizes these two sub-tasks, significantly reducing the computational complexity as compared to 3D counterparts and enabling rapid convergence. Extensive experimental results on simulated and clinical datasets demonstrate superior performance over state-of-the-art models. The source code is made available at https://github.com/hao1635/LIT-Former.

Noise-Generating and Imaging Mechanism Inspired Implicit Regularization Learning Network for Low Dose CT Reconstrution

Low-dose computed tomography (LDCT) helps to reduce radiation risks in CT scanning while maintaining image quality, which involves a consistent pursuit of lower incident rays and higher reconstruction performance. Although deep learning approaches have achieved encouraging success in LDCT reconstruction, most of them treat the task as a general inverse problem in either the image domain or the dual (sinogram and image) domains. Such frameworks have not considered the original noise generation of the projection data and suffer from limited performance improvement for the LDCT task. In this paper, we propose a novel reconstruction model based on noise-generating and imaging mechanism in full-domain, which fully considers the statistical properties of intrinsic noises in LDCT and prior information in sinogram and image domains. To solve the model, we propose an optimization algorithm based on the proximal gradient technique. Specifically, we derive the approximate solutions of the integer programming problem on the projection data theoretically. Instead of hand-crafting the sinogram and image regularizers, we propose to unroll the optimization algorithm to be a deep network. The network implicitly learns the proximal operators of sinogram and image regularizers with two deep neural networks, providing a more interpretable and effective reconstruction procedure. Numerical results demonstrate our proposed method improvements of > 2.9 dB in peak signal to noise ratio, > 1.4% promotion in structural similarity metric, and > 9 HU decrements in root mean square error over current state-of-the-art LDCT methods.

A systematic literature review: deep learning techniques for synthetic medical image generation and their applications in radiotherapy

The aim of this systematic review is to determine whether Deep Learning (DL) algorithms can provide a clinically feasible alternative to classic algorithms for synthetic Computer Tomography (sCT). The following categories are presented in this study: ∙ MR-based treatment planning and synthetic CT generation techniques. ∙ Generation of synthetic CT images based on Cone Beam CT images. ∙ Low-dose CT to High-dose CT generation. ∙ Attenuation correction for PET images. To perform appropriate database searches, we reviewed journal articles published between January 2018 and June 2023. Current methodology, study strategies, and results with relevant clinical applications were analyzed as we outlined the state-of-the-art of deep learning based approaches to inter-modality and intra-modality image synthesis. This was accomplished by contrasting the provided methodologies with traditional research approaches. The key contributions of each category were highlighted, specific challenges were identified, and accomplishments were summarized. As a final step, the statistics of all the cited works from various aspects were analyzed, which revealed that DL-based sCTs have achieved considerable popularity, while also showing the potential of this technology. In order to assess the clinical readiness of the presented methods, we examined the current status of DL-based sCT generation.

Revealing the Denoising Principle of Zero-Shot N2N-Based Algorithm from 1D Spectrum to 2D Image

Denoising is a necessary step in image analysis to extract weak signals, especially those hardly identified by the naked eye. Unlike the data-driven deep-learning denoising algorithms relying on a clean image as the reference, Noise2Noise (N2N) was able to denoise the noise image, providing sufficiently noise images with the same subject but randomly distributed noise. Further, by introducing data augmentation to create a big data set and regularization to prevent model overfitting, zero-shot N2N-based denoising was proposed in which only a single noisy image was needed. Although various N2N-based denoising algorithms have been developed with high performance, their complicated black box operation prevented the lightweight. Therefore, to reveal the working function of the zero-shot N2N-based algorithm, we proposed a lightweight Peak2Peak algorithm (P2P) and qualitatively and quantitatively analyzed its denoising behavior on the 1D spectrum and 2D image. We found that the high-performance denoising originates from the trade-off balance between the loss function and regularization in the denoising module, where regularization is the switch of denoising. Meanwhile, the signal extraction is mainly from the self-supervised characteristic learning in the data augmentation module. Further, the lightweight P2P improved the denoising speed by at least ten times but with little performance loss, compared with that of the current N2N-based algorithms. In general, the visualization of P2P provides a reference for revealing the working function of zero-shot N2N-based algorithms, which would pave the way for the application of these algorithms toward real-time (in situ, in vivo, and operando) research improving both temporal and spatial resolutions. The P2P is open-source at https://github.com/3331822w/Peak2Peakand will be accessible online access at https://ramancloud.xmu.edu.cn/tutorial.

DeepmdQCT: A multitask network with domain invariant features and comprehensive attention mechanism for quantitative computer tomography diagnosis of osteoporosis

In the medical field, the application of machine learning technology in the automatic diagnosis and monitoring of osteoporosis often faces challenges related to domain adaptation in drug therapy research. The existing neural networks used for the diagnosis of osteoporosis may experience a decrease in model performance when applied to new data domains due to changes in radiation dose and equipment. To address this issue, in this study, we propose a new method for multi domain diagnostic and quantitative computed tomography (QCT) images, called DeepmdQCT. This method adopts a domain invariant feature strategy and integrates a comprehensive attention mechanism to guide the fusion of global and local features, effectively improving the diagnostic performance of multi domain CT images. We conducted experimental evaluations on a self-created OQCT dataset, and the results showed that for dose domain images, the average accuracy reached 91%, while for device domain images, the accuracy reached 90.5%. our method successfully estimated bone density values, with a fit of 0.95 to the gold standard. Our method not only achieved high accuracy in CT images in the dose and equipment fields, but also successfully estimated key bone density values, which is crucial for evaluating the effectiveness of osteoporosis drug treatment. In addition, we validated the effectiveness of our architecture in feature extraction using three publicly available datasets. We also encourage the application of the DeepmdQCT method to a wider range of medical image analysis fields to improve the performance of multi-domain images.

A Review of deep learning methods for denoising of medical low-dose CT images

To prevent patients from being exposed to excess of radiation in CT imaging, the most common solution is to decrease the radiation dose by reducing the X-ray, and thus the quality of the resulting low-dose CT images (LDCT) is degraded, as evidenced by more noise and streaking artifacts. Therefore, it is important to maintain high quality CT image while effectively reducing radiation dose. In recent years, with the rapid development of deep learning technology, deep learning-based LDCT denoising methods have become quite popular because of their data-driven and high-performance features to achieve excellent denoising results. However, to our knowledge, no relevant article has so far comprehensively introduced and reviewed advanced deep learning denoising methods such as Transformer structures in LDCT denoising tasks. Therefore, based on the literatures related to LDCT image denoising published from year 2016-2023, and in particular from 2020 to 2023, this study presents a systematic survey of current situation, and challenges and future research directions in LDCT image denoising field. Four types of denoising networks are classified according to the network structure: CNN-based, Encoder-Decoder-based, GAN-based, and Transformer-based denoising networks, and each type of denoising network is described and summarized from the perspectives of structural features and denoising performances. Representative deep-learning denoising methods for LDCT are experimentally compared and analyzed. The study results show that CNN-based denoising methods capture image details efficiently through multi-level convolution operation, demonstrating superior denoising effects and adaptivity. Encoder-decoder networks with MSE loss, achieve outstanding results in objective metrics. GANs based methods, employing innovative generators and discriminators, obtain denoised images that exhibit perceptually a closeness to NDCT. Transformer-based methods have potential for improving denoising performances due to their powerful capability in capturing global information. Challenges and opportunities for deep learning based LDCT denoising are analyzed, and future directions are also presented.

Systematic Review on Learning-based Spectral CT

Spectral computed tomography (CT) has recently emerged as an advanced version of medical CT and significantly improves conventional (single-energy) CT. Spectral CT has two main forms: dual-energy computed tomography (DECT) and photon-counting computed tomography (PCCT), which offer image improvement, material decomposition, and feature quantification relative to conventional CT. However, the inherent challenges of spectral CT, evidenced by data and image artifacts, remain a bottleneck for clinical applications. To address these problems, machine learning techniques have been widely applied to spectral CT. In this review, we present the state-of-the-art data-driven techniques for spectral CT.

An unsupervised two-step training framework for low-dose computed tomography denoising

BACKGROUND

Although low-dose computed tomography (CT) imaging has been more widely adopted in clinical practice to reduce radiation exposure to patients, the reconstructed CT images tend to have more noise, which impedes accurate diagnosis. Recently, deep neural networks using convolutional neural networks to reduce noise in the reconstructed low-dose CT images have shown considerable improvement. However, they need a large number of paired normal- and low-dose CT images to fully train the network via supervised learning methods.

PURPOSE

To propose an unsupervised two-step training framework for image denoising that uses low-dose CT images of one dataset and unpaired high-dose CT images from another dataset.

METHODS

Our proposed framework trains the denoising network in two steps. In the first training step, we train the network using 3D volumes of CT images and predict the center CT slice from them. This pre-trained network is used in the second training step to train the denoising network and is combined with the memory-efficient denoising generative adversarial network (DenoisingGAN), which further enhances both objective and perceptual quality.

RESULTS

The experimental results on phantom and clinical datasets show superior performance over the existing traditional machine learning and self-supervised deep learning methods, and the results are comparable to the fully supervised learning methods.

CONCLUSIONS

We proposed a new unsupervised learning framework for low-dose CT denoising, convincingly improving noisy CT images from both objective and perceptual quality perspectives. Because our denoising framework does not require physics-based noise models or system-dependent assumptions, our proposed method can be easily reproduced; consequently, it can also be generally applicable to various CT scanners or dose levels.

CT image denoising methods for image quality improvement and radiation dose reduction

With the ever-increasing use of computed tomography (CT), concerns about its radiation dose have become a significant public issue. To address the need for radiation dose reduction, CT denoising methods have been widely investigated and applied in low-dose CT images. Numerous noise reduction algorithms have emerged, such as iterative reconstruction and most recently, deep learning (DL)-based approaches. Given the rapid advancements in Artificial Intelligence techniques, we recognize the need for a comprehensive review that emphasizes the most recently developed methods. Hence, we have performed a thorough analysis of existing literature to provide such a review. Beyond directly comparing the performance, we focus on pivotal aspects, including model training, validation, testing, generalizability, vulnerability, and evaluation methods. This review is expected to raise awareness of the various facets involved in CT image denoising and the specific challenges in developing DL-based models.

Deep Filtered Back Projection for CT Reconstruction

Filtered back projection (FBP) is a classic analytical algorithm for computed tomography (CT) reconstruction, with high computational efficiency. However, images reconstructed by FBP often suffer from excessive noise and artifacts. The original FBP algorithm uses a window function to smooth signals and a linear interpolation to estimate projection values at un-sampled locations. In this study, we propose a novel framework named DeepFBP in which an optimized filter and an optimized nonlinear interpolation operator are learned with neural networks. Specifically, the learned filter can be considered as the product of an optimized window function and the ramp filter, and the learned interpolation can be considered as an optimized way to utilize projection information of nearby locations through nonlinear combination. The proposed method remains the high computational efficiency of the original FBP and achieves much better reconstruction quality at different noise levels. It also outperforms the TV-based statistical iterative algorithm, with computational time being reduced in an order of two, and state-of-the-art post-processing deep learning methods that have deeper and more complicated network structures.

Systematic Review on Learning-based Spectral CT

Spectral computed tomography (CT) has recently emerged as an advanced version of medical CT and significantly improves conventional (single-energy) CT. Spectral CT has two main forms: dual-energy computed tomography (DECT) and photon-counting computed tomography (PCCT), which offer image improvement, material decomposition, and feature quantification relative to conventional CT. However, the inherent challenges of spectral CT, evidenced by data and image artifacts, remain a bottleneck for clinical applications. To address these problems, machine learning techniques have been widely applied to spectral CT. In this review, we present the state-of-the-art data-driven techniques for spectral CT.

Transferring U-Net between low-dose CT denoising tasks: a validation study with varied spatial resolutions

Background

Recently, deep learning techniques have been widely used in low-dose computed tomography (LDCT) imaging applications for quickly generating high quality computed tomography (CT) images at lower radiation dose levels. The purpose of this study is to validate the reproducibility of the denoising performance of a given network that has been trained in advance across varied LDCT image datasets that are acquired from different imaging systems with different spatial resolutions.

Methods

Specifically, LDCT images with comparable noise levels but having different spatial resolutions were prepared to train the U-Net. The number of CT images used for the network training, validation and test was 2,400, 300 and 300, respectively. Afterwards, self- and cross-validations among six selected spatial resolutions (62.5, 125, 250, 375, 500, 625 µm) were studied and compared side by side. The residual variance, peak signal to noise ratio (PSNR), normalized root mean square error (NRMSE) and structural similarity (SSIM) were measured and compared. In addition, network retraining on a small number of image set was performed to fine tune the performance of transfer learning among LDCT tasks with varied spatial resolutions.

Results

Results demonstrated that the U-Net trained upon LDCT images having a certain spatial resolution can effectively reduce the noise of the other LDCT images having different spatial resolutions. Regardless, results showed that image artifacts would be generated during the above cross validations. For instance, noticeable residual artifacts were presented at the margin and central areas of the object as the resolution inconsistency increased. The retraining results showed that the artifacts caused by the resolution mismatch can be greatly reduced by utilizing about only 20% of the original training data size. This quantitative improvement led to a reduction in the NRMSE from 0.1898 to 0.1263 and an increase in the SSIM from 0.7558 to 0.8036.

Conclusions

In conclusion, artifacts would be generated when transferring the U-Net to a LDCT denoising task with different spatial resolution. To maintain the denoising performance, it is recommended to retrain the U-Net with a small amount of datasets having the same target spatial resolution.

GAN-based medical image small region forgery detection via a two-stage cascade framework

Using generative adversarial network (GAN) Goodfellow et al. (2014) for data enhancement of medical images is significantly helpful for many computer-aided diagnosis (CAD) tasks. A new GAN-based automated tampering attack, like CT-GAN Mirsky et al. (2019), has emerged. It can inject or remove lung cancer lesions to CT scans. Because the tampering region may even account for less than 1% of the original image, even state-of-the-art methods are challenging to detect the traces of such tampering. This paper proposes a two-stage cascade framework to detect GAN-based medical image small region forgery like CT-GAN. In the local detection stage, we train the detector network with small sub-images so that interference information in authentic regions will not affect the detector. We use depthwise separable convolution and residual networks to prevent the detector from over-fitting and enhance the ability to find forged regions through the attention mechanism. The detection results of all sub-images in the same image will be combined into a heatmap. In the global classification stage, using gray-level co-occurrence matrix (GLCM) can better extract features of the heatmap. Because the shape and size of the tampered region are uncertain, we use hyperplanes in an infinite-dimensional space for classification. Our method can classify whether a CT image has been tampered and locate the tampered position. Sufficient experiments show that our method can achieve excellent performance than the state-of-the-art detection methods.

Joint denoising and interpolating network for low-dose cone-beam CT reconstruction under hybrid dose-reduction strategy

Cone-beam computed tomography (CBCT) is generally reconstructed with hundreds of two-dimensional X-Ray projections through the FDK algorithm, and its excessive ionizing radiation of X-Ray may impair patients' health. Two common dose-reduction strategies are to either lower the intensity of X-Ray, i.e., low-intensity CBCT, or reduce the number of projections, i.e., sparse-view CBCT. Existing efforts improve the low-dose CBCT images only under a single dose-reduction strategy. In this paper, we argue that applying the two strategies simultaneously can reduce dose in a gentle manner and avoid the extreme degradation of the projection data in a single dose-reduction strategy, especially under ultra-low-dose situations. Therefore, we develop a Joint Denoising and Interpolating Network (JDINet) in projection domain to improve the CBCT quality with the hybrid low-intensity and sparse-view projections. Specifically, JDINet mainly includes two important components, i.e., denoising module and interpolating module, to respectively suppress the noise caused by the low-intensity strategy and interpolate the missing projections caused by the sparse-view strategy. Because FDK actually utilizes the projection information after ramp-filtering, we develop a filtered structural similarity constraint to help JDINet focus on the reconstruction-required information. Afterward, we employ a Postprocessing Network (PostNet) in the reconstruction domain to refine the CBCT images that are reconstructed with denoised and interpolated projections. In general, a complete CBCT reconstruction framework is built with JDINet, FDK, and PostNet. Experiments demonstrate that our framework decreases RMSE by approximately 8 %, 15 %, and 17 %, respectively, on the 1/8, 1/16, and 1/32 dose data, compared to the latest methods. In conclusion, our learning-based framework can be deeply imbedded into the CBCT systems to promote the development of CBCT. Source code is available at https://github.com/LianyingChao/FusionLowDoseCBCT.

Unsharp Structure Guided Filtering for Self-Supervised Low-Dose CT Imaging

Low-dose computed tomography (LDCT) imaging faces great challenges. Although supervised learning has revealed great potential, it requires sufficient and high-quality references for network training. Therefore, existing deep learning methods have been sparingly applied in clinical practice. To this end, this paper presents a novel Unsharp Structure Guided Filtering (USGF) method, which can reconstruct high-quality CT images directly from low-dose projections without clean references. Specifically, we first employ low-pass filters to estimate the structure priors from the input LDCT images. Then, inspired by classical structure transfer techniques, deep convolutional networks are adopted to implement our imaging method which combines guided filtering and structure transfer. Finally, the structure priors serve as the guidance images to alleviate over-smoothing, as they can transfer specific structural characteristics to the generated images. Furthermore, we incorporate traditional FBP algorithms into self-supervised training to enable the transformation of projection domain data to the image domain. Extensive comparisons and analyses on three datasets demonstrate that the proposed USGF has achieved superior performance in terms of noise suppression and edge preservation, and could have a significant impact on LDCT imaging in the future.

Reducing the risk of hallucinations with interpretable deep learning models for low-dose CT denoising: comparative performance analysis

Objective.Reducing CT radiation dose is an often proposed measure to enhance patient safety, which, however results in increased image noise, translating into degradation of clinical image quality. Several deep learning methods have been proposed for low-dose CT (LDCT) denoising. The high risks posed by possible hallucinations in clinical images necessitate methods which aid the interpretation of deep learning networks. In this study, we aim to use qualitative reader studies and quantitative radiomics studies to assess the perceived quality, signal preservation and statistical feature preservation of LDCT volumes denoised by deep learning. We aim to compare interpretable deep learning methods with classical deep neural networks in clinical denoising performance.Approach.We conducted an image quality analysis study to assess the image quality of the denoised volumes based on four criteria to assess the perceived image quality. We subsequently conduct a lesion detection/segmentation study to assess the impact of denoising on signal detectability. Finally, a radiomic analysis study was performed to observe the quantitative and statistical similarity of the denoised images to standard dose CT (SDCT) images.Main results.The use of specific deep learning based algorithms generate denoised volumes which are qualitatively inferior to SDCT volumes(p< 0.05). Contrary to previous literature, denoising the volumes did not reduce the accuracy of the segmentation (p> 0.05). The denoised volumes, in most cases, generated radiomics features which were statistically similar to those generated from SDCT volumes (p> 0.05).Significance.Our results show that the denoised volumes have a lower perceived quality than SDCT volumes. Noise and denoising do not significantly affect detectability of the abdominal lesions. Denoised volumes also contain statistically identical features to SDCT volumes.

Self-supervised denoising of projection data for low-dose cone-beam CT

BACKGROUND

Convolutional neural networks (CNNs) have shown promising results in image denoising tasks. While most existing CNN-based methods depend on supervised learning by directly mapping noisy inputs to clean targets, high-quality references are often unavailable for interventional radiology such as cone-beam computed tomography (CBCT).

PURPOSE

In this paper, we propose a novel self-supervised learning method that reduces noise in projections acquired by ordinary CBCT scans.

METHODS

With a network that partially blinds input, we are able to train the denoising model by mapping the partially blinded projections to the original projections. Additionally, we incorporate noise-to-noise learning into the self-supervised learning by mapping the adjacent projections to the original projections. With standard image reconstruction methods such as FDK-type algorithms, we can reconstruct high-quality CBCT images from the projections denoised by our projection-domain denoising method.

RESULTS

In the head phantom study, we measure peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) values of the proposed method along with the other denoising methods and uncorrected low-dose CBCT data for a quantitative comparison both in projection and image domains. The PSNR and SSIM values of our self-supervised denoising approach are 27.08 and 0.839, whereas those of uncorrected CBCT images are 15.68 and 0.103, respectively. In the retrospective study, we assess the quality of interventional patient CBCT images to evaluate the projection-domain and image-domain denoising methods. Both qualitative and quantitative results indicate that our approach can effectively produce high-quality CBCT images with low-dose projections in the absence of duplicate clean or noisy references.

CONCLUSIONS

Our self-supervised learning strategy is capable of restoring anatomical information while efficiently removing noise in CBCT projection data.

Sparse2Noise: Low-dose synchrotron X-ray tomography without high-quality reference data

BACKGROUND

Synchrotron radiation computed tomography (SR-CT) holds promise for high-resolution in vivo imaging. Notably, the reconstruction of SR-CT images necessitates a large set of data to be captured with sufficient photons from multiple angles, resulting in high radiation dose received by the object. Reducing the number of projections and/or photon flux is a straightforward means to lessen the radiation dose, however, compromises data completeness, thus introducing noises and artifacts. Deep learning (DL)-based supervised methods effectively denoise and remove artifacts, but they heavily depend on high-quality paired data acquired at high doses. Although algorithms exist for training without high-quality references, they struggle to effectively eliminate persistent artifacts present in real-world data.

METHODS

This work presents a novel low-dose imaging strategy namely Sparse2Noise, which combines the reconstruction data from paired sparse-view CT scan (normal-flux) and full-view CT scan (low-flux) using a convolutional neural network (CNN). Sparse2Noise does not require high-quality reconstructed data as references and allows for fresh training on data with very small size. Sparse2Noise was evaluated by both simulated and experimental data.

RESULTS

Sparse2Noise effectively reduces noise and ring artifacts while maintaining high image quality, outperforming state-of-the-art image denoising methods at same dose levels. Furthermore, Sparse2Noise produces impressive high image quality for ex vivo rat hindlimb imaging with the acceptable low radiation dose (i.e., 0.5 Gy with the isotropic voxel size of 26 μm).

CONCLUSIONS

This work represents a significant advance towards in vivo SR-CT imaging. It is noteworthy that Sparse2Noise can also be used for denoising in conventional CT and/or phase-contrast CT.

Self-supervised deep learning for joint 3D low-dose PET/CT image denoising

Deep learning (DL)-based denoising of low-dose positron emission tomography (LDPET) and low-dose computed tomography (LDCT) has been widely explored. However, previous methods have focused only on single modality denoising, neglecting the possibility of simultaneously denoising LDPET and LDCT using only one neural network, i.e., joint LDPET/LDCT denoising. Moreover, DL-based denoising methods generally require plenty of well-aligned LD-normal-dose (LD-ND) sample pairs, which can be difficult to obtain. To this end, we propose a self-supervised two-stage training framework named MAsk-then-Cycle (MAC), to achieve self-supervised joint LDPET/LDCT denoising. The first stage of MAC is masked autoencoder (MAE)-based pre-training and the second stage is self-supervised denoising training. Specifically, we propose a self-supervised denoising strategy named cycle self-recombination (CSR), which enables denoising without well-aligned sample pairs. Unlike other methods that treat noise as a homogeneous whole, CSR disentangles noise into signal-dependent and independent noises. This is more in line with the actual imaging process and allows for flexible recombination of noises and signals to generate new samples. These new samples contain implicit constraints that can improve the network's denoising ability. Based on these constraints, we design multiple loss functions to enable self-supervised training. Then we design a CSR-based denoising network to achieve joint 3D LDPET/LDCT denoising. Existing self-supervised methods generally lack pixel-level constraints on networks, which can easily lead to additional artifacts. Before denoising training, we perform MAE-based pre-training to indirectly impose pixel-level constraints on networks. Experiments on an LDPET/LDCT dataset demonstrate its superiority over existing methods. Our method is the first self-supervised joint LDPET/LDCT denoising method. It does not require any prior assumptions and is therefore more robust.

Structure-preserving low-dose computed tomography image denoising using a deep residual adaptive global context attention network

Background

Low-dose computed tomography (LDCT) scans can effectively reduce the radiation damage to patients, but this is highly detrimental to CT image quality. Deep convolutional neural networks (CNNs) have shown their potential in improving LDCT image quality. However, the conventional CNN-based approaches rely fundamentally on the convolution operations, which are ineffective for modeling the correlations among nonlocal similar structures and the regionally distinct statistical properties in CT images. This modeling deficiency hampers the denoising performance for CT images derived in this manner.

Methods

In this paper, we propose an adaptive global context (AGC) modeling scheme to describe the nonlocal correlations and the regionally distinct statistics in CT images with negligible computation load. We further propose an AGC-based long-short residual encoder-decoder (AGC-LSRED) network for efficient LDCT image noise artifact-suppression tasks. Specifically, stacks of residual AGC attention blocks (RAGCBs) with long and short skip connections are constructed in the AGC-LSRED network, which allows valuable structural and positional information to be bypassed through these identity-based skip connections and thus eases the training of the deep denoising network. For training the AGC-LSRED network, we propose a compound loss that combines the L1 loss, adversarial loss, and self-supervised multi-scale perceptual loss.

Results

Quantitative and qualitative experimental studies were performed to verify and validate the effectiveness of the proposed method. The simulation experiments demonstrated the proposed method exhibits the best result in terms of noise suppression [root-mean-square error (RMSE) =9.02; peak signal-to-noise ratio (PSNR) =33.17] and fine structure preservation [structural similarity index (SSIM) =0.925] compared with other competitive CNN-based methods. The experiments on real data illustrated that the proposed method has advantages over other methods in terms of radiologists' subjective assessment scores (averaged scores =4.34).

Conclusions

With the use of the AGC modeling scheme to characterize the structural information in CT images and of residual AGC-attention blocks with long and short skip connections to ease the network training, the proposed AGC-LSRED method achieves satisfactory results in preserving fine anatomical structures and suppressing noise in LDCT images.

Unsupervised learning-based dual-domain method for low-dose CT denoising

Objective. Low-dose CT (LDCT) is an important research topic in the field of CT imaging because of its ability to reduce radiation damage in clinical diagnosis. In recent years, deep learning techniques have been widely applied in LDCT imaging and a large number of denoising methods have been proposed. However, One major challenge of supervised deep learning-based methods is the exactly geometric pairing of datasets with different doses. Therefore, the aim of this study is to develop an unsupervised learning-based LDCT imaging method to address the aforementioned challenges.Approach. In this paper, we propose an unsupervised learning-based dual-domain method for LDCT denoising, which consists of two stages: the first stage is projection domain denoising, in which the unsupervised learning method Noise2Self is applied to denoise the projection data with statistically independent and zero-mean noise. The second stage is an iterative enhancement approach, which combines the prior information obtained from the generative model with an iterative reconstruction algorithm to enhance the details of the reconstructed image.Main results. Experimental results show that our proposed method outperforms the comparison method in terms of denoising effect. Particularly, in terms of SSIM, the denoised results obtained using our method achieve the highest SSIM.Significance. In conclusion, our unsupervised learning-based method can be a promising alternative to the traditional supervised methods for LDCT imaging, especially when the availability of the labeled datasets is limited.

MLNAN: Multi-level noise-aware network for low-dose CT imaging implemented with constrained cycle Wasserstein generative adversarial networks

Low-dose CT techniques attempt to minimize the radiation exposure of patients by estimating the high-resolution normal-dose CT images to reduce the risk of radiation-induced cancer. In recent years, many deep learning methods have been proposed to solve this problem by building a mapping function between low-dose CT images and their high-dose counterparts. However, most of these methods ignore the effect of different radiation doses on the final CT images, which results in large differences in the intensity of the noise observable in CT images. What'more, the noise intensity of low-dose CT images exists significantly differences under different medical devices manufacturers. In this paper, we propose a multi-level noise-aware network (MLNAN) implemented with constrained cycle Wasserstein generative adversarial networks to recovery the low-dose CT images under uncertain noise levels. Particularly, the noise-level classification is predicted and reused as a prior pattern in generator networks. Moreover, the discriminator network introduces noise-level determination. Under two dose-reduction strategies, experiments to evaluate the performance of proposed method are conducted on two datasets, including the simulated clinical AAPM challenge datasets and commercial CT datasets from United Imaging Healthcare (UIH). The experimental results illustrate the effectiveness of our proposed method in terms of noise suppression and structural detail preservation compared with several other deep-learning based methods. Ablation studies validate the effectiveness of the individual components regarding the afforded performance improvement. Further research for practical clinical applications and other medical modalities is required in future works.

CDDnet: Cross-domain denoising network for low-dose CT image via local and global information alignment

The domain shift problem has emerged as a challenge in cross-domain low-dose CT (LDCT) image denoising task, where the acquisition of a sufficient number of medical images from multiple sources may be constrained by privacy concerns. In this study, we propose a novel cross-domain denoising network (CDDnet) that incorporates both local and global information of CT images. To address the local component, a local information alignment module has been proposed to regularize the similarity between extracted target and source features from selected patches. To align the general information of the semantic structure from a global perspective, an autoencoder is adopted to learn the latent correlation between the source label and the estimated target label generated by the pre-trained denoiser. Experimental results demonstrate that our proposed CDDnet effectively alleviates the domain shift problem, outperforming other deep learning-based and domain adaptation-based methods under cross-domain scenarios.

Direct estimation of regional lung volume change from paired and single CT images using residual regression neural network

BACKGROUND

Chest computed tomography (CT) enables characterization of pulmonary diseases by producing high-resolution and high-contrast images of the intricate lung structures. Deformable image registration is used to align chest CT scans at different lung volumes, yielding estimates of local tissue expansion and contraction.

PURPOSE

We investigated the utility of deep generative models for directly predicting local tissue volume change from lung CT images, bypassing computationally expensive iterative image registration and providing a method that can be utilized in scenarios where either one or two CT scans are available.

METHODS

A residual regression convolutional neural network, called Reg3DNet+, is proposed for directly regressing high-resolution images of local tissue volume change (i.e., Jacobian) from CT images. Image registration was performed between lung volumes at total lung capacity (TLC) and functional residual capacity (FRC) using a tissue mass- and structure-preserving registration algorithm. The Jacobian image was calculated from the registration-derived displacement field and used as the ground truth for local tissue volume change. Four separate Reg3DNet+ models were trained to predict Jacobian images using a multifactorial study design to compare the effects of network input (i.e., single image vs. paired images) and output space (i.e., FRC vs. TLC). The models were trained and evaluated on image datasets from the COPDGene study. Models were evaluated against the registration-derived Jacobian images using local, regional, and global evaluation metrics.

RESULTS

Statistical analysis revealed that both factors - network input and output space - were significant determinants for change in evaluation metrics. Paired-input models performed better than single-input models, and model performance was better in the output space of FRC rather than TLC. Mean structural similarity index for paired-input models was 0.959 and 0.956 for FRC and TLC output spaces, respectively, and for single-input models was 0.951 and 0.937. Global evaluation metrics demonstrated correlation between registration-derived Jacobian mean and predicted Jacobian mean: coefficient of determination (r2 ) for paired-input models was 0.974 and 0.938 for FRC and TLC output spaces, respectively, and for single-input models was 0.598 and 0.346. After correcting for effort, registration-derived lobar volume change was strongly correlated with the predicted lobar volume change: for paired-input models r2 was 0.899 for both FRC and TLC output spaces, and for single-input models r2 was 0.803 and 0.862, respectively.

CONCLUSIONS

Convolutional neural networks can be used to directly predict local tissue mechanics, eliminating the need for computationally expensive image registration. Networks that use paired CT images acquired at TLC and FRC allow for more accurate prediction of local tissue expansion compared to networks that use a single image. Networks that only require a single input image still show promising results, particularly after correcting for effort, and allow for local tissue expansion estimation in cases where multiple CT scans are not available. For single-input networks, the FRC image is more predictive of local tissue volume change compared to the TLC image.

Low-Dose CT Image Synthesis for Domain Adaptation Imaging Using a Generative Adversarial Network With Noise Encoding Transfer Learning

Deep learning (DL) based image processing methods have been successfully applied to low-dose x-ray images based on the assumption that the feature distribution of the training data is consistent with that of the test data. However, low-dose computed tomography (LDCT) images from different commercial scanners may contain different amounts and types of image noise, violating this assumption. Moreover, in the application of DL based image processing methods to LDCT, the feature distributions of LDCT images from simulation and clinical CT examination can be quite different. Therefore, the network models trained with simulated image data or LDCT images from one specific scanner may not work well for another CT scanner and image processing task. To solve such domain adaptation problem, in this study, a novel generative adversarial network (GAN) with noise encoding transfer learning (NETL), or GAN-NETL, is proposed to generate a paired dataset with a different noise style. Specifically, we proposed a method to perform noise encoding operator and incorporate it into the generator to extract a noise style. Meanwhile, with a transfer learning (TL) approach, the image noise encoding operator transformed the noise type of the source domain to that of the target domain for realistic noise generation. One public and two private datasets are used to evaluate the proposed method. Experiment results demonstrated the feasibility and effectiveness of our proposed GAN-NETL model in LDCT image synthesis. In addition, we conduct additional image denoising study using the synthesized clinical LDCT data, which verified the merit of the proposed synthesis in improving the performance of the DL based LDCT processing method.

Impact of loss functions on the performance of a deep neural network designed to restore low-dose digital mammography

Digital mammography is currently the most common imaging tool for breast cancer screening. Although the benefits of using digital mammography for cancer screening outweigh the risks associated with the x-ray exposure, the radiation dose must be kept as low as possible while maintaining the diagnostic utility of the generated images, thus minimizing patient risks. Many studies investigated the feasibility of dose reduction by restoring low-dose images using deep neural networks. In these cases, choosing the appropriate training database and loss function is crucial and impacts the quality of the results. In this work, we used a standard residual network (ResNet) to restore low-dose digital mammography images and evaluated the performance of several loss functions. For training purposes, we extracted 256,000 image patches from a dataset of 400 images of retrospective clinical mammography exams, where dose reduction factors of 75% and 50% were simulated to generate low and standard-dose pairs. We validated the network in a real scenario by using a physical anthropomorphic breast phantom to acquire real low-dose and standard full-dose images in a commercially available mammography system, which were then processed through our trained model. We benchmarked our results against an analytical restoration model for low-dose digital mammography. Objective assessment was performed through the signal-to-noise ratio (SNR) and the mean normalized squared error (MNSE), decomposed into residual noise and bias. Statistical tests revealed that the use of the perceptual loss (PL4) resulted in statistically significant differences when compared to all other loss functions. Additionally, images restored using the PL4 achieved the closest residual noise to the standard dose. On the other hand, perceptual loss PL3, structural similarity index (SSIM) and one of the adversarial losses achieved the lowest bias for both dose reduction factors. The source code of our deep neural network is available at https://github.com/WANG-AXIS/LdDMDenoising.

Dynamic controllable residual generative adversarial network for low-dose computed tomography imaging

Background

Computed tomography (CT) imaging technology has become an indispensable auxiliary method in medical diagnosis and treatment. In mitigating the radiation damage caused by X-rays, low-dose computed tomography (LDCT) scanning is becoming more widely applied. However, LDCT scanning reduces the signal-to-noise ratio of the projection, and the resulting images suffer from serious streak artifacts and spot noise. In particular, the intensity of noise and artifacts varies significantly across different body parts under a single low-dose protocol.

Methods

To improve the quality of different degraded LDCT images in a unified framework, we developed a generative adversarial learning framework with a dynamic controllable residual. First, the generator network consists of the basic subnetwork and the conditional subnetwork. Inspired by the dynamic control strategy, we designed the basic subnetwork to adopt a residual architecture, with the conditional subnetwork providing weights to control the residual intensity. Second, we chose the Visual Geometry Group Network-128 (VGG-128) as the discriminator to improve the noise artifact suppression and feature retention ability of the generator. Additionally, a hybrid loss function was specifically designed, including the mean square error (MSE) loss, structural similarity index metric (SSIM) loss, adversarial loss, and gradient penalty (GP) loss.

Results

The results obtained on two datasets show the competitive performance of the proposed framework, with a 3.22 dB peak signal-to-noise ratio (PSNR) margin, 0.03 SSIM margin, and 0.2 contrast-to-noise ratio margin on the Challenge data and a 1.0 dB PSNR margin and 0.01 SSIM margin on the real data.

Conclusions

Experimental results demonstrated the competitive performance of the proposed method in terms of noise decrease, structural retention, and visual impression improvement.

Fast and low-dose medical imaging generation empowered by hybrid deep-learning and iterative reconstruction

Fast and low-dose reconstructions of medical images are highly desired in clinical routines. We propose a hybrid deep-learning and iterative reconstruction (hybrid DL-IR) framework and apply it for fast magnetic resonance imaging (MRI), fast positron emission tomography (PET), and low-dose computed tomography (CT) image generation tasks. First, in a retrospective MRI study (6,066 cases), we demonstrate its capability of handling 3- to 10-fold under-sampled MR data, enabling organ-level coverage with only 10- to 100-s scan time; second, a low-dose CT study (142 cases) shows that our framework can successfully alleviate the noise and streak artifacts in scans performed with only 10% radiation dose (0.61 mGy); and last, a fast whole-body PET study (131 cases) allows us to faithfully reconstruct tumor-induced lesions, including small ones (<4 mm), from 2- to 4-fold-accelerated PET acquisition (30-60 s/bp). This study offers a promising avenue for accurate and high-quality image reconstruction with broad clinical value.

Self-trained deep convolutional neural network for noise reduction in CT

Purpose

Supervised deep convolutional neural network (CNN)-based methods have been actively used in clinical CT to reduce image noise. The networks of these methods are typically trained using paired high- and low-quality data from a massive number of patients and/or phantom images. This training process is tedious, and the network trained under a given condition may not be generalizable to patient images acquired and reconstructed under different conditions. We propose a self-trained deep CNN (ST_CNN) method for noise reduction in CT that does not rely on pre-existing training datasets.

Approach

The ST_CNN training was accomplished using extensive data augmentation in the projection domain, and the inference was applied to the data itself. Specifically, multiple independent noise insertions were applied to the original patient projection data to generate multiple realizations of low-quality projection data. Then, rotation augmentation was adopted for both the original and low-quality projection data by applying the rotation angle directly on the projection data so that images were rotated at arbitrary angles without introducing additional bias. A large number of paired low- and high-quality images from the same patient were reconstructed and paired for training the ST_CNN model.

Results

No significant difference was found between the ST_CNN and conventional CNN models in terms of the peak signal-to-noise ratio and structural similarity index measure. The ST_CNN model outperformed the conventional CNN model in terms of noise texture and homogeneity in liver parenchyma as well as better subjective visualization of liver lesions. The ST_CNN may sacrifice the sharpness of vessels slightly compared to the conventional CNN model but without affecting the visibility of peripheral vessels or diagnosis of vascular pathology.

Conclusions

The proposed ST_CNN method trained from the data itself may achieve similar image quality in comparison with conventional deep CNN denoising methods pre-trained on external datasets.

Multimodal-Boost: Multimodal Medical Image Super-Resolution Using Multi-Attention Network With Wavelet Transform

Multimodal medical images are widely used by clinicians and physicians to analyze and retrieve complementary information from high-resolution images in a non-invasive manner. Loss of corresponding image resolution adversely affects the overall performance of medical image interpretation. Deep learning-based single image super resolution (SISR) algorithms have revolutionized the overall diagnosis framework by continually improving the architectural components and training strategies associated with convolutional neural networks (CNN) on low-resolution images. However, existing work lacks in two ways: i) the SR output produced exhibits poor texture details, and often produce blurred edges, ii) most of the models have been developed for a single modality, hence, require modification to adapt to a new one. This work addresses (i) by proposing generative adversarial network (GAN) with deep multi-attention modules to learn high-frequency information from low-frequency data. Existing approaches based on the GAN have yielded good SR results; however, the texture details of their SR output have been experimentally confirmed to be deficient for medical images particularly. The integration of wavelet transform (WT) and GANs in our proposed SR model addresses the aforementioned limitation concerning textons. While the WT divides the LR image into multiple frequency bands, the transferred GAN uses multi-attention and upsample blocks to predict high-frequency components. Additionally, we present a learning method for training domain-specific classifiers as perceptual loss functions. Using a combination of multi-attention GAN loss and a perceptual loss function results in an efficient and reliable performance. Applying the same model for medical images from diverse modalities is challenging, our work addresses (ii) by training and performing on several modalities via transfer learning. Using two medical datasets, we validate our proposed SR network against existing state-of-the-art approaches and achieve promising results in terms of structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR).

Medical image super-resolution reconstruction algorithms based on deep learning: A survey

BACKGROUND AND OBJECTIVE

With the high-resolution (HR) requirements of medical images in clinical practice, super-resolution (SR) reconstruction algorithms based on low-resolution (LR) medical images have become a research hotspot. This type of method can significantly improve image SR without improving hardware equipment, so it is of great significance to review it.

METHODS

Aiming at the unique SR reconstruction algorithms in the field of medical images, based on subdivided medical fields such as magnetic resonance (MR) images, computed tomography (CT) images, and ultrasound images. Firstly, we deeply analyzed the research progress of SR reconstruction algorithms, and summarized and compared the different types of algorithms. Secondly, we introduced the evaluation indicators corresponding to the SR reconstruction algorithms. Finally, we prospected the development trend of SR reconstruction technology in the medical field.

RESULTS

The medical image SR reconstruction technology based on deep learning can provide more abundant lesion information, relieve the expert's diagnosis pressure, and improve the diagnosis efficiency and accuracy.

CONCLUSION

The medical image SR reconstruction technology based on deep learning helps to improve the quality of medicine, provides help for the diagnosis of experts, and lays a solid foundation for the subsequent analysis and identification tasks of the computer, which is of great significance for improving the diagnosis efficiency of experts and realizing intelligent medical care.

Segmentation-Free PVC for Cardiac SPECT Using a Densely-Connected Multi-Dimensional Dynamic Network

In nuclear imaging, limited resolution causes partial volume effects (PVEs) that affect image sharpness and quantitative accuracy. Partial volume correction (PVC) methods incorporating high-resolution anatomical information from CT or MRI have been demonstrated to be effective. However, such anatomical-guided methods typically require tedious image registration and segmentation steps. Accurately segmented organ templates are also hard to obtain, particularly in cardiac SPECT imaging, due to the lack of hybrid SPECT/CT scanners with high-end CT and associated motion artifacts. Slight mis-registration/mis-segmentation would result in severe degradation in image quality after PVC. In this work, we develop a deep-learning-based method for fast cardiac SPECT PVC without anatomical information and associated organ segmentation. The proposed network involves a densely-connected multi-dimensional dynamic mechanism, allowing the convolutional kernels to be adapted based on the input images, even after the network is fully trained. Intramyocardial blood volume (IMBV) is introduced as an additional clinical-relevant loss function for network optimization. The proposed network demonstrated promising performance on 28 canine studies acquired on a GE Discovery NM/CT 570c dedicated cardiac SPECT scanner with a 64-slice CT using Technetium-99m-labeled red blood cells. This work showed that the proposed network with densely-connected dynamic mechanism produced superior results compared with the same network without such mechanism. Results also showed that the proposed network without anatomical information could produce images with statistically comparable IMBV measurements to the images generated by anatomical-guided PVC methods, which could be helpful in clinical translation.

A self-supervised guided knowledge distillation framework for unpaired low-dose CT image denoising

Low-dose computed tomography (LDCT) can significantly reduce the damage of X-ray to the human body, but the reduction of CT dose will produce images with severe noise and artifacts, which will affect the diagnosis of doctors. Recently, deep learning has attracted more and more attention from researchers. However, most of the denoising networks applied to deep learning-based LDCT imaging are supervised methods, which require paired data for network training. In a realistic imaging scenario, obtaining well-aligned image pairs is challenging due to the error in the table re-positioning and the patient's physiological movement during data acquisition. In contrast, the unpaired learning method can overcome the drawbacks of supervised learning, making it more feasible to collect unpaired training data in most real-world imaging applications. In this study, we develop a novel unpaired learning framework, Self-Supervised Guided Knowledge Distillation (SGKD), which enables the guidance of supervised learning using the results generated by self-supervised learning. The proposed SGKD scheme contains two stages of network training. First, we can achieve the LDCT image quality improvement by the designed self-supervised cycle network. Meanwhile, it can also produce two complementary training datasets from the unpaired LDCT and NDCT images. Second, a knowledge distillation strategy with the above two datasets is exploited to further improve the LDCT image denoising performance. To evaluate the effectiveness and feasibility of the proposed method, extensive experiments were performed on the simulated AAPM challenging and real-world clinical LDCT datasets. The qualitative and quantitative results show that the proposed SGKD achieves better performance in terms of noise suppression and detail preservation compared with some state-of-the-art network models.

Automated nonlinear registration of coronary PET to CT angiography using pseudo-CT generated from PET with generative adversarial networks

BACKGROUND

Coronary 18F-sodium-fluoride (18F-NaF) positron emission tomography (PET) showed promise in imaging coronary artery disease activity. Currently image processing remains subjective due to the need for manual registration of PET and computed tomography (CT) angiography data. We aimed to develop a novel fully automated method to register coronary 18F-NaF PET to CT angiography using pseudo-CT generated by generative adversarial networks (GAN).

METHODS

A total of 169 patients, 139 in the training and 30 in the testing sets were considered for generation of pseudo-CT from non-attenuation corrected (NAC) PET using GAN. Non-rigid registration was used to register pseudo-CT to CT angiography and the resulting transformation was used to align PET with CT angiography. We compared translations, maximal standard uptake value (SUVmax) and target to background ratio (TBRmax) at the location of plaques, obtained after observer and automated alignment.

RESULTS

Automatic end-to-end registration was performed for 30 patients with 88 coronary vessels and took 27.5 seconds per patient. Difference in displacement motion vectors between GAN-based and observer-based registration in the x-, y-, and z-directions was 0.8 ± 3.0, 0.7 ± 3.0, and 1.7 ± 3.9 mm, respectively. TBRmax had a coefficient of repeatability (CR) of 0.31, mean bias of 0.03 and narrow limits of agreement (LOA) (95% LOA: - 0.29 to 0.33). SUVmax had CR of 0.26, mean bias of 0 and narrow LOA (95% LOA: - 0.26 to 0.26).

CONCLUSION

Pseudo-CT generated by GAN are perfectly registered to PET can be used to facilitate quick and fully automated registration of PET and CT angiography.

MLF-IOSC: Multi-Level Fusion Network With Independent Operation Search Cell for Low-Dose CT Denoising

Computed tomography (CT) is widely used in clinical medicine, and low-dose CT (LDCT) has become popular to reduce potential patient harm during CT acquisition. However, LDCT aggravates the problem of noise and artifacts in CT images, increasing diagnosis difficulty. Through deep learning, denoising CT images by artificial neural network has aroused great interest for medical imaging and has been hugely successful. We propose a framework to achieve excellent LDCT noise reduction using independent operation search cells, inspired by neural architecture search, and introduce the Laplacian to further improve image quality. Employing patch-based training, the proposed method can effectively eliminate CT image noise while retaining the original structures and details, hence significantly improving diagnosis efficiency and promoting LDCT clinical applications.

M3NAS: Multi-Scale and Multi-Level Memory-Efficient Neural Architecture Search for Low-Dose CT Denoising

Lowering the radiation dose in computed tomography (CT) can greatly reduce the potential risk to public health. However, the reconstructed images from dose-reduced CT or low-dose CT (LDCT) suffer from severe noise which compromises the subsequent diagnosis and analysis. Recently, convolutional neural networks have achieved promising results in removing noise from LDCT images. The network architectures that are used are either handcrafted or built on top of conventional networks such as ResNet and U-Net. Recent advances in neural network architecture search (NAS) have shown that the network architecture has a dramatic effect on the model performance. This indicates that current network architectures for LDCT may be suboptimal. Therefore, in this paper, we make the first attempt to apply NAS to LDCT and propose a multi-scale and multi-level memory-efficient NAS for LDCT denoising, termed M3NAS. On the one hand, the proposed M3NAS fuses features extracted by different scale cells to capture multi-scale image structural details. On the other hand, the proposed M3NAS can search a hybrid cell- and network-level structure for better performance. In addition, M3NAS can effectively reduce the number of model parameters and increase the speed of inference. Extensive experimental results on two different datasets demonstrate that the proposed M3NAS can achieve better performance and fewer parameters than several state-of-the-art methods. In addition, we also validate the effectiveness of the multi-scale and multi-level architecture for LDCT denoising, and present further analysis for different configurations of super-net.

AIGAN: Attention-encoding Integrated Generative Adversarial Network for the reconstruction of low-dose CT and low-dose PET images

X-ray computed tomography (CT) and positron emission tomography (PET) are two of the most commonly used medical imaging technologies for the evaluation of many diseases. Full-dose imaging for CT and PET ensures the image quality but usually raises concerns about the potential health risks of radiation exposure. The contradiction between reducing the radiation exposure and remaining diagnostic performance can be addressed effectively by reconstructing the low-dose CT (L-CT) and low-dose PET (L-PET) images to the same high-quality ones as full-dose (F-CT and F-PET). In this paper, we propose an Attention-encoding Integrated Generative Adversarial Network (AIGAN) to achieve efficient and universal full-dose reconstruction for L-CT and L-PET images. AIGAN consists of three modules: the cascade generator, the dual-scale discriminator and the multi-scale spatial fusion module (MSFM). A sequence of consecutive L-CT (L-PET) slices is first fed into the cascade generator that integrates with a generation-encoding-generation pipeline. The generator plays the zero-sum game with the dual-scale discriminator for two stages: the coarse and fine stages. In both stages, the generator generates the estimated F-CT (F-PET) images as like the original F-CT (F-PET) images as possible. After the fine stage, the estimated fine full-dose images are then fed into the MSFM, which fully explores the inter- and intra-slice structural information, to output the final generated full-dose images. Experimental results show that the proposed AIGAN achieves the state-of-the-art performances on commonly used metrics and satisfies the reconstruction needs for clinical standards.

Deep learning based unpaired image-to-image translation applications for medical physics: a systematic review

Purpose. There is a growing number of publications on the application of unpaired image-to-image (I2I) translation in medical imaging. However, a systematic review covering the current state of this topic for medical physicists is lacking. The aim of this article is to provide a comprehensive review of current challenges and opportunities for medical physicists and engineers to apply I2I translation in practice.Methods and materials. The PubMed electronic database was searched using terms referring to unpaired (unsupervised), I2I translation, and medical imaging. This review has been reported in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. From each full-text article, we extracted information extracted regarding technical and clinical applications of methods, Transparent Reporting for Individual Prognosis Or Diagnosis (TRIPOD) study type, performance of algorithm and accessibility of source code and pre-trained models.Results. Among 461 unique records, 55 full-text articles were included in the review. The major technical applications described in the selected literature are segmentation (26 studies), unpaired domain adaptation (18 studies), and denoising (8 studies). In terms of clinical applications, unpaired I2I translation has been used for automatic contouring of regions of interest in MRI, CT, x-ray and ultrasound images, fast MRI or low dose CT imaging, CT or MRI only based radiotherapy planning, etc Only 5 studies validated their models using an independent test set and none were externally validated by independent researchers. Finally, 12 articles published their source code and only one study published their pre-trained models.Conclusion. I2I translation of medical images offers a range of valuable applications for medical physicists. However, the scarcity of external validation studies of I2I models and the shortage of publicly available pre-trained models limits the immediate applicability of the proposed methods in practice.

GraphSKT: Graph-Guided Structured Knowledge Transfer for Domain Adaptive Lesion Detection

Adversarial-based adaptation has dominated the area of domain adaptive detection over the past few years. Despite their general efficacy for various tasks, the learned representations may not capture the intrinsic topological structures of the whole images and thus are vulnerable to distributional shifts especially in real-world applications, such as geometric distortions across imaging devices in medical images. In this case, forcefully matching data distributions across domains cannot ensure precise knowledge transfer and are prone to result in the negative transfer. In this paper, we explore the problem of domain adaptive lesion detection from the perspective of relational reasoning, and propose a Graph-Structured Knowledge Transfer (GraphSKT) framework to perform hierarchical reasoning by modeling both the intra- and inter-domain topological structures. To be specific, we utilize cross-domain correspondence to mine meaningful foreground regions for representing graph nodes and explicitly endow each node with contextual information. Then, the intra- and inter-domain graphs are built on the top of instance-level features to achieve a high-level understanding of the lesion and whole medical image, and transfer the structured knowledge from source to target domains. The contextual and semantic information is propagated through graph nodes methodically, enhancing the expressive power of learned features for the lesion detection tasks. Extensive experiments on two types of challenging datasets demonstrate that the proposed GraphSKT significantly outperforms the state-of-the-art approaches for detection of polyps in colonoscopy images and of mass in mammographic images.

Generalizable transfer learning of automated tumor segmentation from cervical cancers toward a universal model for uterine malignancies in diffusion-weighted MRI

PURPOSE

To investigate the generalizability of transfer learning (TL) of automated tumor segmentation from cervical cancers toward a universal model for cervical and uterine malignancies in diffusion-weighted magnetic resonance imaging (DWI).

METHODS

In this retrospective multicenter study, we analyzed pelvic DWI data from 169 and 320 patients with cervical and uterine malignancies and divided them into the training (144 and 256) and testing (25 and 64) datasets, respectively. A pretrained model was established using DeepLab V3 + from the cervical cancer dataset, followed by TL experiments adjusting the training data sizes and fine-tuning layers. The model performance was evaluated using the dice similarity coefficient (DSC).

RESULTS

In predicting tumor segmentation for all cervical and uterine malignancies, TL models improved the DSCs from the pretrained cervical model (DSC 0.43) when adding 5, 13, 26, and 51 uterine cases for training (DSC improved from 0.57, 0.62, 0.68, 0.70, p < 0.001). Following the crossover at adding 128 cases (DSC 0.71), the model trained by combining data from adding all the 256 patients exhibited the highest DSCs for the combined cervical and uterine datasets (DSC 0.81) and cervical only dataset (DSC 0.91).

CONCLUSIONS

TL may improve the generalizability of automated tumor segmentation of DWI from a specific cancer type toward multiple types of uterine malignancies especially in limited case numbers.

Deep-Learning-Based Few-Angle Cardiac SPECT Reconstruction Using Transformer

Convolutional neural networks (CNNs) have been extremely successful in various medical imaging tasks. However, because the size of the convolutional kernel used in a CNN is much smaller than the image size, CNN has a strong spatial inductive bias and lacks a global understanding of the input images. Vision Transformer, a recently emerged network structure in computer vision, can potentially overcome the limitations of CNNs for image-reconstruction tasks. In this work, we proposed a slice-by-slice Transformer network (SSTrans-3D) to reconstruct cardiac SPECT images from 3D few-angle data. To be specific, the network reconstructs the whole 3D volume using a slice-by-slice scheme. By doing so, SSTrans-3D alleviates the memory burden required by 3D reconstructions using Transformer. The network can still obtain a global understanding of the image volume with the Transformer attention blocks. Lastly, already reconstructed slices are used as the input to the network so that SSTrans-3D can potentially obtain more informative features from these slices. Validated on porcine, phantom, and human studies acquired using a GE dedicated cardiac SPECT scanner, the proposed method produced images with clearer heart cavity, higher cardiac defect contrast, and more accurate quantitative measurements on the testing data as compared with a deep U-net.