3-D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning From a 2-D Trained Network

Fast and low-dose medical imaging generation empowered by hybrid deep-learning and iterative reconstruction

Fast and low-dose reconstructions of medical images are highly desired in clinical routines. We propose a hybrid deep-learning and iterative reconstruction (hybrid DL-IR) framework and apply it for fast magnetic resonance imaging (MRI), fast positron emission tomography (PET), and low-dose computed tomography (CT) image generation tasks. First, in a retrospective MRI study (6,066 cases), we demonstrate its capability of handling 3- to 10-fold under-sampled MR data, enabling organ-level coverage with only 10- to 100-s scan time; second, a low-dose CT study (142 cases) shows that our framework can successfully alleviate the noise and streak artifacts in scans performed with only 10% radiation dose (0.61 mGy); and last, a fast whole-body PET study (131 cases) allows us to faithfully reconstruct tumor-induced lesions, including small ones (<4 mm), from 2- to 4-fold-accelerated PET acquisition (30-60 s/bp). This study offers a promising avenue for accurate and high-quality image reconstruction with broad clinical value.

Self-trained deep convolutional neural network for noise reduction in CT

Purpose

Supervised deep convolutional neural network (CNN)-based methods have been actively used in clinical CT to reduce image noise. The networks of these methods are typically trained using paired high- and low-quality data from a massive number of patients and/or phantom images. This training process is tedious, and the network trained under a given condition may not be generalizable to patient images acquired and reconstructed under different conditions. We propose a self-trained deep CNN (ST_CNN) method for noise reduction in CT that does not rely on pre-existing training datasets.

Approach

The ST_CNN training was accomplished using extensive data augmentation in the projection domain, and the inference was applied to the data itself. Specifically, multiple independent noise insertions were applied to the original patient projection data to generate multiple realizations of low-quality projection data. Then, rotation augmentation was adopted for both the original and low-quality projection data by applying the rotation angle directly on the projection data so that images were rotated at arbitrary angles without introducing additional bias. A large number of paired low- and high-quality images from the same patient were reconstructed and paired for training the ST_CNN model.

Results

No significant difference was found between the ST_CNN and conventional CNN models in terms of the peak signal-to-noise ratio and structural similarity index measure. The ST_CNN model outperformed the conventional CNN model in terms of noise texture and homogeneity in liver parenchyma as well as better subjective visualization of liver lesions. The ST_CNN may sacrifice the sharpness of vessels slightly compared to the conventional CNN model but without affecting the visibility of peripheral vessels or diagnosis of vascular pathology.

Conclusions

The proposed ST_CNN method trained from the data itself may achieve similar image quality in comparison with conventional deep CNN denoising methods pre-trained on external datasets.

Multimodal-Boost: Multimodal Medical Image Super-Resolution Using Multi-Attention Network With Wavelet Transform

Multimodal medical images are widely used by clinicians and physicians to analyze and retrieve complementary information from high-resolution images in a non-invasive manner. Loss of corresponding image resolution adversely affects the overall performance of medical image interpretation. Deep learning-based single image super resolution (SISR) algorithms have revolutionized the overall diagnosis framework by continually improving the architectural components and training strategies associated with convolutional neural networks (CNN) on low-resolution images. However, existing work lacks in two ways: i) the SR output produced exhibits poor texture details, and often produce blurred edges, ii) most of the models have been developed for a single modality, hence, require modification to adapt to a new one. This work addresses (i) by proposing generative adversarial network (GAN) with deep multi-attention modules to learn high-frequency information from low-frequency data. Existing approaches based on the GAN have yielded good SR results; however, the texture details of their SR output have been experimentally confirmed to be deficient for medical images particularly. The integration of wavelet transform (WT) and GANs in our proposed SR model addresses the aforementioned limitation concerning textons. While the WT divides the LR image into multiple frequency bands, the transferred GAN uses multi-attention and upsample blocks to predict high-frequency components. Additionally, we present a learning method for training domain-specific classifiers as perceptual loss functions. Using a combination of multi-attention GAN loss and a perceptual loss function results in an efficient and reliable performance. Applying the same model for medical images from diverse modalities is challenging, our work addresses (ii) by training and performing on several modalities via transfer learning. Using two medical datasets, we validate our proposed SR network against existing state-of-the-art approaches and achieve promising results in terms of structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR).

Medical image super-resolution reconstruction algorithms based on deep learning: A survey

BACKGROUND AND OBJECTIVE

With the high-resolution (HR) requirements of medical images in clinical practice, super-resolution (SR) reconstruction algorithms based on low-resolution (LR) medical images have become a research hotspot. This type of method can significantly improve image SR without improving hardware equipment, so it is of great significance to review it.

METHODS

Aiming at the unique SR reconstruction algorithms in the field of medical images, based on subdivided medical fields such as magnetic resonance (MR) images, computed tomography (CT) images, and ultrasound images. Firstly, we deeply analyzed the research progress of SR reconstruction algorithms, and summarized and compared the different types of algorithms. Secondly, we introduced the evaluation indicators corresponding to the SR reconstruction algorithms. Finally, we prospected the development trend of SR reconstruction technology in the medical field.

RESULTS

The medical image SR reconstruction technology based on deep learning can provide more abundant lesion information, relieve the expert's diagnosis pressure, and improve the diagnosis efficiency and accuracy.

CONCLUSION

The medical image SR reconstruction technology based on deep learning helps to improve the quality of medicine, provides help for the diagnosis of experts, and lays a solid foundation for the subsequent analysis and identification tasks of the computer, which is of great significance for improving the diagnosis efficiency of experts and realizing intelligent medical care.

Segmentation-Free PVC for Cardiac SPECT Using a Densely-Connected Multi-Dimensional Dynamic Network

In nuclear imaging, limited resolution causes partial volume effects (PVEs) that affect image sharpness and quantitative accuracy. Partial volume correction (PVC) methods incorporating high-resolution anatomical information from CT or MRI have been demonstrated to be effective. However, such anatomical-guided methods typically require tedious image registration and segmentation steps. Accurately segmented organ templates are also hard to obtain, particularly in cardiac SPECT imaging, due to the lack of hybrid SPECT/CT scanners with high-end CT and associated motion artifacts. Slight mis-registration/mis-segmentation would result in severe degradation in image quality after PVC. In this work, we develop a deep-learning-based method for fast cardiac SPECT PVC without anatomical information and associated organ segmentation. The proposed network involves a densely-connected multi-dimensional dynamic mechanism, allowing the convolutional kernels to be adapted based on the input images, even after the network is fully trained. Intramyocardial blood volume (IMBV) is introduced as an additional clinical-relevant loss function for network optimization. The proposed network demonstrated promising performance on 28 canine studies acquired on a GE Discovery NM/CT 570c dedicated cardiac SPECT scanner with a 64-slice CT using Technetium-99m-labeled red blood cells. This work showed that the proposed network with densely-connected dynamic mechanism produced superior results compared with the same network without such mechanism. Results also showed that the proposed network without anatomical information could produce images with statistically comparable IMBV measurements to the images generated by anatomical-guided PVC methods, which could be helpful in clinical translation.

A self-supervised guided knowledge distillation framework for unpaired low-dose CT image denoising

Low-dose computed tomography (LDCT) can significantly reduce the damage of X-ray to the human body, but the reduction of CT dose will produce images with severe noise and artifacts, which will affect the diagnosis of doctors. Recently, deep learning has attracted more and more attention from researchers. However, most of the denoising networks applied to deep learning-based LDCT imaging are supervised methods, which require paired data for network training. In a realistic imaging scenario, obtaining well-aligned image pairs is challenging due to the error in the table re-positioning and the patient's physiological movement during data acquisition. In contrast, the unpaired learning method can overcome the drawbacks of supervised learning, making it more feasible to collect unpaired training data in most real-world imaging applications. In this study, we develop a novel unpaired learning framework, Self-Supervised Guided Knowledge Distillation (SGKD), which enables the guidance of supervised learning using the results generated by self-supervised learning. The proposed SGKD scheme contains two stages of network training. First, we can achieve the LDCT image quality improvement by the designed self-supervised cycle network. Meanwhile, it can also produce two complementary training datasets from the unpaired LDCT and NDCT images. Second, a knowledge distillation strategy with the above two datasets is exploited to further improve the LDCT image denoising performance. To evaluate the effectiveness and feasibility of the proposed method, extensive experiments were performed on the simulated AAPM challenging and real-world clinical LDCT datasets. The qualitative and quantitative results show that the proposed SGKD achieves better performance in terms of noise suppression and detail preservation compared with some state-of-the-art network models.

Automated nonlinear registration of coronary PET to CT angiography using pseudo-CT generated from PET with generative adversarial networks

BACKGROUND

Coronary 18F-sodium-fluoride (18F-NaF) positron emission tomography (PET) showed promise in imaging coronary artery disease activity. Currently image processing remains subjective due to the need for manual registration of PET and computed tomography (CT) angiography data. We aimed to develop a novel fully automated method to register coronary 18F-NaF PET to CT angiography using pseudo-CT generated by generative adversarial networks (GAN).

METHODS

A total of 169 patients, 139 in the training and 30 in the testing sets were considered for generation of pseudo-CT from non-attenuation corrected (NAC) PET using GAN. Non-rigid registration was used to register pseudo-CT to CT angiography and the resulting transformation was used to align PET with CT angiography. We compared translations, maximal standard uptake value (SUVmax) and target to background ratio (TBRmax) at the location of plaques, obtained after observer and automated alignment.

RESULTS

Automatic end-to-end registration was performed for 30 patients with 88 coronary vessels and took 27.5 seconds per patient. Difference in displacement motion vectors between GAN-based and observer-based registration in the x-, y-, and z-directions was 0.8 ± 3.0, 0.7 ± 3.0, and 1.7 ± 3.9 mm, respectively. TBRmax had a coefficient of repeatability (CR) of 0.31, mean bias of 0.03 and narrow limits of agreement (LOA) (95% LOA: - 0.29 to 0.33). SUVmax had CR of 0.26, mean bias of 0 and narrow LOA (95% LOA: - 0.26 to 0.26).

CONCLUSION

Pseudo-CT generated by GAN are perfectly registered to PET can be used to facilitate quick and fully automated registration of PET and CT angiography.

MLF-IOSC: Multi-Level Fusion Network With Independent Operation Search Cell for Low-Dose CT Denoising

Computed tomography (CT) is widely used in clinical medicine, and low-dose CT (LDCT) has become popular to reduce potential patient harm during CT acquisition. However, LDCT aggravates the problem of noise and artifacts in CT images, increasing diagnosis difficulty. Through deep learning, denoising CT images by artificial neural network has aroused great interest for medical imaging and has been hugely successful. We propose a framework to achieve excellent LDCT noise reduction using independent operation search cells, inspired by neural architecture search, and introduce the Laplacian to further improve image quality. Employing patch-based training, the proposed method can effectively eliminate CT image noise while retaining the original structures and details, hence significantly improving diagnosis efficiency and promoting LDCT clinical applications.

M3NAS: Multi-Scale and Multi-Level Memory-Efficient Neural Architecture Search for Low-Dose CT Denoising

Lowering the radiation dose in computed tomography (CT) can greatly reduce the potential risk to public health. However, the reconstructed images from dose-reduced CT or low-dose CT (LDCT) suffer from severe noise which compromises the subsequent diagnosis and analysis. Recently, convolutional neural networks have achieved promising results in removing noise from LDCT images. The network architectures that are used are either handcrafted or built on top of conventional networks such as ResNet and U-Net. Recent advances in neural network architecture search (NAS) have shown that the network architecture has a dramatic effect on the model performance. This indicates that current network architectures for LDCT may be suboptimal. Therefore, in this paper, we make the first attempt to apply NAS to LDCT and propose a multi-scale and multi-level memory-efficient NAS for LDCT denoising, termed M3NAS. On the one hand, the proposed M3NAS fuses features extracted by different scale cells to capture multi-scale image structural details. On the other hand, the proposed M3NAS can search a hybrid cell- and network-level structure for better performance. In addition, M3NAS can effectively reduce the number of model parameters and increase the speed of inference. Extensive experimental results on two different datasets demonstrate that the proposed M3NAS can achieve better performance and fewer parameters than several state-of-the-art methods. In addition, we also validate the effectiveness of the multi-scale and multi-level architecture for LDCT denoising, and present further analysis for different configurations of super-net.

AIGAN: Attention-encoding Integrated Generative Adversarial Network for the reconstruction of low-dose CT and low-dose PET images

X-ray computed tomography (CT) and positron emission tomography (PET) are two of the most commonly used medical imaging technologies for the evaluation of many diseases. Full-dose imaging for CT and PET ensures the image quality but usually raises concerns about the potential health risks of radiation exposure. The contradiction between reducing the radiation exposure and remaining diagnostic performance can be addressed effectively by reconstructing the low-dose CT (L-CT) and low-dose PET (L-PET) images to the same high-quality ones as full-dose (F-CT and F-PET). In this paper, we propose an Attention-encoding Integrated Generative Adversarial Network (AIGAN) to achieve efficient and universal full-dose reconstruction for L-CT and L-PET images. AIGAN consists of three modules: the cascade generator, the dual-scale discriminator and the multi-scale spatial fusion module (MSFM). A sequence of consecutive L-CT (L-PET) slices is first fed into the cascade generator that integrates with a generation-encoding-generation pipeline. The generator plays the zero-sum game with the dual-scale discriminator for two stages: the coarse and fine stages. In both stages, the generator generates the estimated F-CT (F-PET) images as like the original F-CT (F-PET) images as possible. After the fine stage, the estimated fine full-dose images are then fed into the MSFM, which fully explores the inter- and intra-slice structural information, to output the final generated full-dose images. Experimental results show that the proposed AIGAN achieves the state-of-the-art performances on commonly used metrics and satisfies the reconstruction needs for clinical standards.

Deep learning based unpaired image-to-image translation applications for medical physics: a systematic review

Purpose. There is a growing number of publications on the application of unpaired image-to-image (I2I) translation in medical imaging. However, a systematic review covering the current state of this topic for medical physicists is lacking. The aim of this article is to provide a comprehensive review of current challenges and opportunities for medical physicists and engineers to apply I2I translation in practice.Methods and materials. The PubMed electronic database was searched using terms referring to unpaired (unsupervised), I2I translation, and medical imaging. This review has been reported in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. From each full-text article, we extracted information extracted regarding technical and clinical applications of methods, Transparent Reporting for Individual Prognosis Or Diagnosis (TRIPOD) study type, performance of algorithm and accessibility of source code and pre-trained models.Results. Among 461 unique records, 55 full-text articles were included in the review. The major technical applications described in the selected literature are segmentation (26 studies), unpaired domain adaptation (18 studies), and denoising (8 studies). In terms of clinical applications, unpaired I2I translation has been used for automatic contouring of regions of interest in MRI, CT, x-ray and ultrasound images, fast MRI or low dose CT imaging, CT or MRI only based radiotherapy planning, etc Only 5 studies validated their models using an independent test set and none were externally validated by independent researchers. Finally, 12 articles published their source code and only one study published their pre-trained models.Conclusion. I2I translation of medical images offers a range of valuable applications for medical physicists. However, the scarcity of external validation studies of I2I models and the shortage of publicly available pre-trained models limits the immediate applicability of the proposed methods in practice.

GraphSKT: Graph-Guided Structured Knowledge Transfer for Domain Adaptive Lesion Detection

Adversarial-based adaptation has dominated the area of domain adaptive detection over the past few years. Despite their general efficacy for various tasks, the learned representations may not capture the intrinsic topological structures of the whole images and thus are vulnerable to distributional shifts especially in real-world applications, such as geometric distortions across imaging devices in medical images. In this case, forcefully matching data distributions across domains cannot ensure precise knowledge transfer and are prone to result in the negative transfer. In this paper, we explore the problem of domain adaptive lesion detection from the perspective of relational reasoning, and propose a Graph-Structured Knowledge Transfer (GraphSKT) framework to perform hierarchical reasoning by modeling both the intra- and inter-domain topological structures. To be specific, we utilize cross-domain correspondence to mine meaningful foreground regions for representing graph nodes and explicitly endow each node with contextual information. Then, the intra- and inter-domain graphs are built on the top of instance-level features to achieve a high-level understanding of the lesion and whole medical image, and transfer the structured knowledge from source to target domains. The contextual and semantic information is propagated through graph nodes methodically, enhancing the expressive power of learned features for the lesion detection tasks. Extensive experiments on two types of challenging datasets demonstrate that the proposed GraphSKT significantly outperforms the state-of-the-art approaches for detection of polyps in colonoscopy images and of mass in mammographic images.

Generalizable transfer learning of automated tumor segmentation from cervical cancers toward a universal model for uterine malignancies in diffusion-weighted MRI

PURPOSE

To investigate the generalizability of transfer learning (TL) of automated tumor segmentation from cervical cancers toward a universal model for cervical and uterine malignancies in diffusion-weighted magnetic resonance imaging (DWI).

METHODS

In this retrospective multicenter study, we analyzed pelvic DWI data from 169 and 320 patients with cervical and uterine malignancies and divided them into the training (144 and 256) and testing (25 and 64) datasets, respectively. A pretrained model was established using DeepLab V3 + from the cervical cancer dataset, followed by TL experiments adjusting the training data sizes and fine-tuning layers. The model performance was evaluated using the dice similarity coefficient (DSC).

RESULTS

In predicting tumor segmentation for all cervical and uterine malignancies, TL models improved the DSCs from the pretrained cervical model (DSC 0.43) when adding 5, 13, 26, and 51 uterine cases for training (DSC improved from 0.57, 0.62, 0.68, 0.70, p < 0.001). Following the crossover at adding 128 cases (DSC 0.71), the model trained by combining data from adding all the 256 patients exhibited the highest DSCs for the combined cervical and uterine datasets (DSC 0.81) and cervical only dataset (DSC 0.91).

CONCLUSIONS

TL may improve the generalizability of automated tumor segmentation of DWI from a specific cancer type toward multiple types of uterine malignancies especially in limited case numbers.

Deep-Learning-Based Few-Angle Cardiac SPECT Reconstruction Using Transformer

Convolutional neural networks (CNNs) have been extremely successful in various medical imaging tasks. However, because the size of the convolutional kernel used in a CNN is much smaller than the image size, CNN has a strong spatial inductive bias and lacks a global understanding of the input images. Vision Transformer, a recently emerged network structure in computer vision, can potentially overcome the limitations of CNNs for image-reconstruction tasks. In this work, we proposed a slice-by-slice Transformer network (SSTrans-3D) to reconstruct cardiac SPECT images from 3D few-angle data. To be specific, the network reconstructs the whole 3D volume using a slice-by-slice scheme. By doing so, SSTrans-3D alleviates the memory burden required by 3D reconstructions using Transformer. The network can still obtain a global understanding of the image volume with the Transformer attention blocks. Lastly, already reconstructed slices are used as the input to the network so that SSTrans-3D can potentially obtain more informative features from these slices. Validated on porcine, phantom, and human studies acquired using a GE dedicated cardiac SPECT scanner, the proposed method produced images with clearer heart cavity, higher cardiac defect contrast, and more accurate quantitative measurements on the testing data as compared with a deep U-net.

Domain-adaptive denoising network for low-dose CT via noise estimation and transfer learning

BACKGROUND

In recent years, low-dose computed tomography (LDCT) has played an important role in the diagnosis CT to reduce the potential adverse effects of X-ray radiation on patients, while maintaining the same diagnostic image quality.

PURPOSE

Deep learning (DL)-based methods have played an increasingly important role in the field of LDCT imaging. However, its performance is highly dependent on the consistency of feature distributions between training data and test data. Due to patient's breathing movements during data acquisition, the paired LDCT and normal dose CT images are difficult to obtain from realistic imaging scenarios. Moreover, LDCT images from simulation or clinical CT examination often have different feature distributions due to the pollution by different amounts and types of image noises. If a network model trained with a simulated dataset is used to directly test clinical patients' LDCT data, its denoising performance may be degraded. Based on this, we propose a novel domain-adaptive denoising network (DADN) via noise estimation and transfer learning to resolve the out-of-distribution problem in LDCT imaging.

METHODS

To overcome the previous adaptation issue, a novel network model consisting of a reconstruction network and a noise estimation network was designed. The noise estimation network based on a double branch structure is used for image noise extraction and adaptation. Meanwhile, the U-Net-based reconstruction network uses several spatially adaptive normalization modules to fuse multi-scale noise input. Moreover, to facilitate the adaptation of the proposed DADN network to new imaging scenarios, we set a two-stage network training plan. In the first stage, the public simulated dataset is used for training. In the second transfer training stage, we will continue to fine-tune the network model with a torso phantom dataset, while some parameters are frozen. The main reason using the two-stage training scheme is based on the fact that the feature distribution of image content from the public dataset is complex and diverse, whereas the feature distribution of noise pattern from the torso phantom dataset is closer to realistic imaging scenarios.

RESULTS

In an evaluation study, the trained DADN model is applied to both the public and clinical patient LDCT datasets. Through the comparison of visual inspection and quantitative results, it is shown that the proposed DADN network model can perform well in terms of noise and artifact suppression, while effectively preserving image contrast and details.

CONCLUSIONS

In this paper, we have proposed a new DL network to overcome the domain adaptation problem in LDCT image denoising. Moreover, the results demonstrate the feasibility and effectiveness of the application of our proposed DADN network model as a new DL-based LDCT image denoising method.

Compound feature attention network with edge enhancement for low-dose CT denoising

BACKGROUND

Low-dose CT (LDCT) images usually contain serious noise and artifacts, which weaken the readability of the image.

OBJECTIVE

To solve this problem, we propose a compound feature attention network with edge enhancement for LDCT denoising (CFAN-Net), which consists of an edge-enhanced module and a proposed compound feature attention block (CFAB).

METHODS

The edge enhancement module extracts edge details with the trainable Sobel convolution. CFAB consists of an interactive feature learning module (IFLM), a multi-scale feature fusion module (MFFM), and a joint attention module (JAB), which removes noise from LDCT images in a coarse-to-fine manner. First, in IFLM, the noise is initially removed by cross-latitude interactive judgment learning. Second, in MFFM, multi-scale and pixel attention are integrated to explore fine noise removal. Finally, in JAB, we focus on key information, extract useful features, and improve the efficiency of network learning. To construct a high-quality image, we repeat the above operation by cascading CFAB.

RESULTS

By applying CFAN-Net to process the 2016 NIH AAPM-Mayo LDCT challenge test dataset, experiments show that the peak signal-to-noise ratio value is 33.9692 and the structural similarity value is 0.9198.

CONCLUSIONS

Compared with several existing LDCT denoising algorithms, CFAN-Net effectively preserves the texture of CT images while removing noise and artifacts.

Multislice input for 2D and 3D residual convolutional neural network noise reduction in CT

Purpose

Deep convolutional neural network (CNN)-based methods are increasingly used for reducing image noise in computed tomography (CT). Current attempts at CNN denoising are based on 2D or 3D CNN models with a single- or multiple-slice input. Our work aims to investigate if the multiple-slice input improves the denoising performance compared with the single-slice input and if a 3D network architecture is better than a 2D version at utilizing the multislice input.

Approach

Two categories of network architectures can be used for the multislice input. First, multislice images can be stacked channel-wise as the multichannel input to a 2D CNN model. Second, multislice images can be employed as the 3D volumetric input to a 3D CNN model, in which the 3D convolution layers are adopted. We make performance comparisons among 2D CNN models with one, three, and seven input slices and two versions of 3D CNN models with seven input slices and one or three output slices. Evaluation was performed on liver CT images using three quantitative metrics with full-dose images as reference. Visual assessment was made by an experienced radiologist.

Results

When the input channels of the 2D CNN model increases from one to three to seven, a trend of improved performance was observed. Comparing the three models with the seven-slice input, the 3D CNN model with a one-slice output outperforms the other models in terms of noise texture and homogeneity in liver parenchyma as well as subjective visualization of vessels.

Conclusions

We conclude the that multislice input is an effective strategy for improving performance for 2D deep CNN denoising models. The pure 3D CNN model tends to have a better performance than the other models in terms of continuity across axial slices, but the difference was not significant compared with the 2D CNN model with the same number of slices as the input.

An Innovative Low-dose CT Inpainting Algorithm based on Limited-angle Imaging Inpainting Model

BACKGROUND

With the popularity of computed tomography (CT) technique, an increasing number of patients are receiving CT scans. Simultaneously, the public's attention to CT radiation dose is also increasing. How to obtain CT images suitable for clinical diagnosis while reducing the radiation dose has become the focus of researchers.

OBJECTIVE

To demonstrate that limited-angle CT imaging technique can be used to acquire lower dose CT images, we propose a generative adversarial network-based image inpainting model-Low-dose imaging and Limited-angle imaging inpainting Model (LDLAIM), this method can effectively restore low-dose CT images with limited-angle imaging, which verifies that limited-angle CT imaging technique can be used to acquire low-dose CT images.

METHODS

In this work, we used three datasets, including chest and abdomen dataset, head dataset and phantom dataset. They are used to synthesize low-dose and limited-angle CT images for network training. During training stage, we divide each dataset into training set, validation set and testing set according to the ratio of 8:1:1, and use the validation set to validate after finishing an epoch training, and use the testing set to test after finishing all the training. The proposed method is based on generative adversarial networks(GANs), which consists of a generator and a discriminator. The generator consists of residual blocks and encoder-decoder, and uses skip connection.

RESULTS

We use SSIM, PSNR and RMSE to evaluate the performance of the proposed method. In the chest and abdomen dataset, the mean SSIM, PSNR and RMSE of the testing set are 0.984, 35.385 and 0.017, respectively. In the head dataset, the mean SSIM, PSNR and RMSE of the testing set are 0.981, 38.664 and 0.011, respectively. In the phantom dataset, the mean SSIM, PSNR and RMSE of the testing set are 0.977, 33.468 and 0.022, respectively. By comparing the experimental results of other algorithms in these three datasets, it can be found that the proposed method is superior to other algorithms in these indicators. Meanwhile, the proposed method also achieved the highest score in the subjective quality score.

CONCLUSIONS

Experimental results show that the proposed method can effectively restore CT images when both low-dose CT imaging techniques and limited-angle CT imaging techniques are used simultaneously. This work proves that the limited-angle CT imaging technique can be used to reduce the CT radiation dose, and also provides a new idea for the research of low-dose CT imaging.

X-ray CT image denoising with MINF: A modularized iterative network framework for data from multiple dose levels

In clinical applications, multi-dose scan protocols will cause the noise levels of computed tomography (CT) images to fluctuate widely. The popular low-dose CT (LDCT) denoising network outputs denoised images through an end-to-end mapping between an LDCT image and its corresponding ground truth. The limitation of this method is that the reduced noise level of the image may not meet the diagnostic needs of doctors. To establish a denoising model adapted to the multi-noise levels robustness, we proposed a novel and efficient modularized iterative network framework (MINF) to learn the feature of the original LDCT and the outputs of the previous modules, which can be reused in each following module. The proposed network can achieve the goal of gradual denoising, outputting clinical images with different denoising levels, and providing the reviewing physicians with increased confidence in their diagnosis. Moreover, a multi-scale convolutional neural network (MCNN) module is designed to extract as much feature information as possible during the network's training. Extensive experiments on public and private clinical datasets were carried out, and comparisons with several state-of-the-art methods show that the proposed method can achieve satisfactory results for noise suppression of LDCT images. In further comparisons with modularized adaptive processing neural network (MAP-NN), the proposed network shows superior step-by-step or gradual denoising performance. Considering the high quality of gradual denoising results, the proposed method can obtain satisfactory performance in terms of image contrast and detail protection as the level of denoising increases, which shows its potential to be suitable for a multi-dose levels denoising task.

Deep learning-based noise reduction for coronary CT angiography: using four-dimensional noise-reduction images as the ground truth

Background

To assess low-contrast areas such as plaque and coronary artery stenosis, coronary computed tomography angiography (CCTA) needs to provide images with lower noise without increasing radiation doses.

Purpose

To develop a deep learning-based noise-reduction method for CCTA using four-dimensional noise reduction (4DNR) as the ground truth for supervised learning.

Material and Methods

\We retrospectively collected 100 retrospective ECG-gated CCTAs. We created 4DNR images using non-rigid registration and weighted averaging three timeline CCTA volumetric data with intervals of 50 ms in the mid-diastolic phase. Our method set the original reconstructed image as the input and the 4DNR as the target image and obtained the noise-reduced image via residual learning. We evaluated the objective image quality of the original and deep learning-based noise-reduction (DLNR) images based on the image noise of the aorta and the contrast-to-noise ratio (CNR) of the coronary arteries. Further, a board-certified radiologist evaluated the blurring of several heart structures using a 5-point Likert scale subjectively and assigned a coronary artery disease reporting and data system (CAD-RADS) category independently.

Results

DLNR CCTAs showed 64.5% lower image noise ( P < 0.001) and achieved a 2.9 times higher CNR of coronary arteries than that in original images, without significant blurring in subjective comparison ( P > 0.1). The intra-observer agreement of CAD-RADS in the DLNR image was excellent (0.87, 95% confidence interval = 0.77–0.99) with original CCTAs.

Conclusion

Our DLNR method supervised by 4DNR significantly reduced the image noise of CCTAs without affecting the assessment of coronary stenosis.

Noise Characteristics Modeled Unsupervised Network for Robust CT Image Reconstruction

Deep learning (DL)-based methods show great potential in computed tomography (CT) imaging field. The DL-based reconstruction methods are usually evaluated on the training and testing datasets which are obtained from the same distribution, i.e., the same CT scan protocol (i.e., the region setting, kVp, mAs, etc.). In this work, we focus on analyzing the robustness of the DL-based methods against protocol-specific distribution shifts (i.e., the training and testing datasets are from different region settings, different kVp settings, or different mAs settings, respectively). The results show that the DL-based reconstruction methods are sensitive to the protocol-specific perturbations which can be attributed to the noise distribution shift between the training and testing datasets. Based on these findings, we presented a low-dose CT reconstruction method using an unsupervised strategy with the consideration of noise distribution to address the issue of protocol-specific perturbations. Specifically, unpaired sinogram data is enrolled into the network training, which represents unique information for specific imaging protocol, and a Gaussian mixture model (GMM) is introduced to characterize the noise distribution in CT images. It can be termed as GMM based unsupervised CT reconstruction network (GMM-unNet) method. Moreover, an expectation-maximization algorithm is designed to optimize the presented GMM-unNet method. Extensive experiments are performed on three datasets from different scan protocols, which demonstrate that the presented GMM-unNet method outperforms the competing methods both qualitatively and quantitatively.

Denoise ultra-low-field 3D magnetic resonance images using a joint signal-image domain filter

Ultra-low-field magnetic resonance imaging (MRI) could suffer from heavy uncorrelated noise, and its removal could be a critical post-processing task. As a primary source of interference, Gaussian noise could corrupt the sampled MR signal (k-space data), especially at lower B₀ field strength. For this reason, we consider both signal and image domains by proposing a new joint filter characterized by a Kalman filter with linear prediction and a nonlocal mean filter with higher-order singular value decomposition (HOSVD) for denoising 3D MR data. The Kalman filter first attenuates the noise in k-space, and then its reconstruction images are used to guide HOSVD denoising process with exploring self-similarity among 3D structures. The clearer prefiltered images could also generate improved HOSVD learned bases used to transform the noise corrupted patch groups in the original MR data. The flexibility of proposed method is also demonstrated by integrating other k-space filters into the algorithm scheme. Experimental data includes simulated MR images with the varying noise level and real MR images obtained from our 50 mT MRI scanner. The results reveal that our method has a better noise-removal ability and introduces lesser unexpected artifacts than other related MRI denoising approaches.

ERA-WGAT: Edge-enhanced residual autoencoder with a window-based graph attention convolutional network for low-dose CT denoising

Computed tomography (CT) has become a powerful tool for medical diagnosis. However, minimizing X-ray radiation risk for the patient poses significant challenges to obtain suitable low dose CT images. Although various low-dose CT methods using deep learning techniques have produced impressive results, convolutional neural network based methods focus more on local information and hence are very limited for non-local information extraction. This paper proposes ERA-WGAT, a residual autoencoder incorporating an edge enhancement module that performs convolution with eight types of learnable operators providing rich edge information and a window-based graph attention convolutional network that combines static and dynamic attention modules to explore non-local self-similarity. We use the compound loss function that combines MSE loss and multi-scale perceptual loss to mitigate the over-smoothing problem. Compared with current low-dose CT denoising methods, ERA-WGAT confirmed superior noise suppression and perceived image quality.

Minimizing Image Quality Loss After Channel Count Reduction for Plane Wave Ultrasound via Deep Learning Inference

High-frame-rate ultrasound imaging uses unfocused transmissions to insonify an entire imaging view for each transmit event, thereby enabling frame rates over 1000 frames per second (fps). At these high frame rates, it is naturally challenging to realize real-time transfer of channel-domain raw data from the transducer to the system back end. Our work seeks to halve the total data transfer rate by uniformly decimating the receive channel count by 50% and, in turn, doubling the array pitch. We show that despite the reduced channel count and the inevitable use of a sparse array aperture, the resulting beamformed image quality can be maintained by designing a custom convolutional encoder-decoder neural network to infer the radio frequency (RF) data of the nullified channels. This deep learning framework was trained with in vivo human carotid data (5-MHz plane wave imaging, 128 channels, 31 steering angles over a 30° span, and 62 799 frames in total). After training, the network was tested on an in vitro point target scenario that was dissimilar to the training data, in addition to in vivo carotid validation datasets. In the point target phantom image beamformed from inferred channel data, spatial aliasing artifacts attributed to array pitch doubling were found to be reduced by up to 10 dB. For carotid imaging, our proposed approach yielded a lumen-to-tissue contrast that was on average within 3 dB compared to the full-aperture image, whereas without channel data inferencing, the carotid lumen was obscured. When implemented on an RTX-2080 GPU, the inference time to apply the trained network was 4 ms, which favors real-time imaging. Overall, our technique shows that with the help of deep learning, channel data transfer rates can be effectively halved with limited impact on the resulting image quality.

Wavelet subband-specific learning for low-dose computed tomography denoising

Deep neural networks have shown great improvements in low-dose computed tomography (CT) denoising. Early algorithms were primarily optimized to obtain an accurate image with low distortion between the denoised image and reference full-dose image at the cost of yielding an overly smoothed unrealistic CT image. Recent research has sought to preserve the fine details of denoised images with high perceptual quality, which has been accompanied by a decrease in objective quality due to a trade-off between perceptual quality and distortion. We pursue a network that can generate accurate and realistic CT images with high objective and perceptual quality within one network, achieving a better perception-distortion trade-off. To achieve this goal, we propose a stationary wavelet transform-assisted network employing the characteristics of high- and low-frequency domains of the wavelet transform and frequency subband-specific losses defined in the wavelet domain. We first introduce a stationary wavelet transform for the network training procedure. Then, we train the network using objective loss functions defined for high- and low-frequency domains to enhance the objective quality of the denoised CT image. With this network design, we train the network again after replacing the objective loss functions with perceptual loss functions in high- and low-frequency domains. As a result, we acquired denoised CT images with high perceptual quality using this strategy while minimizing the objective quality loss. We evaluated our algorithms on the phantom and clinical images, and the quantitative and qualitative results indicate that ours outperform the existing state-of-the-art algorithms in terms of objective and perceptual quality.

Low-Dose CT Image Denoising Based on Improved DD-Net and Local Filtered Mechanism

Low-dose CT (LDCT) images can reduce the radiation damage to the patients; however, the unavoidable information loss will influence the clinical diagnosis under low-dose conditions, such as noise, streak artifacts, and smooth details. LDCT image denoising is a significant topic in medical image processing to overcome the above deficits. This work proposes an improved DD-Net (DenseNet and deconvolution-based network) joint local filtered mechanism, the DD-Net is enhanced by introducing improved residual dense block to strengthen the feature representation ability, and the local filtered mechanism and gradient loss are also employed to effectively restore the subtle structures. First, the LDCT image is inputted into the network to obtain the denoised image. The original loss between the denoised image and normal-dose CT (NDCT) image is calculated, and the difference image between the NDCT image and the denoised image is obtained. Second, a mask image is generated by taking a threshold operation to the difference image, and the filtered LDCT and NDCT images are obtained by conducting an elementwise multiplication operation with LDCT and NDCT images using the mask image. Third, the filtered image is inputted into the network to obtain the filtered denoised image, and the correction loss is calculated. At last, the sum of original loss and correction loss of the improved DD-Net is used to optimize the network. Considering that it is insufficient to generate the edge information using the combination of mean square error (MSE) and multiscale structural similarity (MS-SSIM), we introduce the gradient loss that can calculate the loss of the high-frequency portion. The experimental results show that the proposed method can achieve better performance than conventional schemes and most neural networks. Our source code is made available at https://github.com/LHE-IT/Low-dose-CT-Image-Denoising/tree/main/Local Filtered Mechanism.

Deep learning versus iterative reconstruction on image quality and dose reduction in abdominal CT: a live animal study

Objective. While simulated low-dose CT images and phantom studies cannot fully approximate subjective and objective effects of deep learning (DL) denoising on image quality, live animal models may afford this assessment. This study is to investigate the potential of DL in CT dose reduction on image quality compared to iterative reconstruction (IR). Approach. The upper abdomen of a live 4 year old sheep was scanned on a CT scanner at different exposure levels. Images were reconstructed using FBP and ADMIRE with 5 strengths. A modularized DL network with 5 modules was used for image reconstruction via progressive denoising. Radiomic features were extracted from a region over the liver. Concordance correlation coefficient (CCC) was applied to quantify agreement between any two sets of radiomic features. Coefficient of variation was calculated to measure variation in a radiomic feature series. Structural similarity index (SSIM) was used to measure the similarity between any two images. Diagnostic quality, low-contrast detectability, and image texture were qualitatively evaluated by two radiologists. Pearson correlation coefficient was computed across all dose-reconstruction/denoising combinations. Results. A total of 66 image sets, with 405 radiomic features extracted from each, are analyzed. IR and DL can improve diagnostic quality and low-contrast detectability and similarly modulate image texture features. In terms of SSIM, DL has higher potential in preserving image structure. There is strong correlation between SSIM and radiologists’ evaluations for diagnostic quality (0.559) and low-contrast detectability (0.635) but moderate correlation for texture (0.313). There is moderate correlation between CCC of radiomic features and radiologists’ evaluation for diagnostic quality (0.397), low-contrast detectability (0.417), and texture (0.326), implying that improvement of image features may not relate to improvement of diagnostic quality. Conclusion. DL shows potential to further reduce radiation dose while preserving structural similarity, while IR is favored by radiologists and more predictably alters radiomic features.

Feasibility study of three-material decomposition in dual-energy cone-beam CT imaging with deep learning

Objective. In this work, a dedicated end-to-end deep convolutional neural network, named as Triple-CBCT, is proposed to demonstrate the feasibility of reconstructing three different material distribution volumes from the dual-energy CBCT projection data. Approach. In Triple-CBCT, the features of the sinogram and the CT image are independently extracted and cascaded via a customized domain transform network module. This Triple-CBCT network was trained by numerically synthesized dual-energy CBCT data, and was tested with experimental dual-energy CBCT data of the Iodine-CaCl2 solution and pig leg specimen scanned on an in-house benchtop system. Main results. Results show that the information stored in both the sinogram and CT image domains can be used together to improve the decomposition quality of multiple materials (water, iodine, CaCl2 or bone) from the dual-energy projections. In addition, both the numerical and experimental results demonstrate that the Triple-CBCT is able to generate high-fidelity dual-energy CBCT basis images. Significance. An innovative end-to-end network that joints the sinogram and CT image domain information is developed to facilitate high quality automatic decomposition from the dual-energy CBCT scans.

An encoder-decoder network for direct image reconstruction on sinograms of a long axial field of view PET

PURPOSE

Deep learning is an emerging reconstruction method for positron emission tomography (PET), which can tackle complex PET corrections in an integrated procedure. This paper optimizes the direct PET reconstruction from sinogram on a long axial field of view (LAFOV) PET.

METHODS

This paper proposes a novel deep learning architecture to reduce the biases during direct reconstruction from sinograms to images. This architecture is based on an encoder-decoder network, where the perceptual loss is used with pre-trained convolutional layers. It is trained and tested on data of 80 patients acquired from recent Siemens Biograph Vision Quadra long axial FOV (LAFOV) PET/CT. The patients are randomly split into a training dataset of 60 patients, a validation dataset of 10 patients, and a test dataset of 10 patients. The 3D sinograms are converted into 2D sinogram slices and used as input to the network. In addition, the vendor reconstructed images are considered as ground truths. Finally, the proposed method is compared with DeepPET, a benchmark deep learning method for PET reconstruction.

RESULTS

Compared with DeepPET, the proposed network significantly reduces the root-mean-squared error (NRMSE) from 0.63 to 0.6 (p < 0.01) and increases the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) from 0.93 to 0.95 (p < 0.01) and from 82.02 to 82.36 (p < 0.01), respectively. The reconstruction time is approximately 10 s per patient, which is shortened by 23 times compared with the conventional method. The errors of mean standardized uptake values (SUVmean) for lesions between ground truth and the predicted result are reduced from 33.5 to 18.7% (p = 0.03). In addition, the error of max SUV is reduced from 32.7 to 21.8% (p = 0.02).

CONCLUSION

The results demonstrate the feasibility of using deep learning to reconstruct images with acceptable image quality and short reconstruction time. It is shown that the proposed method can improve the quality of deep learning-based reconstructed images without additional CT images for attenuation and scattering corrections. This study demonstrated the feasibility of deep learning to rapidly reconstruct images without additional CT images for complex corrections from actual clinical measurements on LAFOV PET. Despite improving the current development, AI-based reconstruction does not work appropriately for untrained scenarios due to limited extrapolation capability and cannot completely replace conventional reconstruction currently.

Joint Spine Segmentation and Noise Removal From Ultrasound Volume Projection Images With Selective Feature Sharing

Volume Projection Imaging from ultrasound data is a promising technique to visualize spine features and diagnose Adolescent Idiopathic Scoliosis. In this paper, we present a novel multi-task framework to reduce the scan noise in volume projection images and to segment different spine features simultaneously, which provides an appealing alternative for intelligent scoliosis assessment in clinical applications. Our proposed framework consists of two streams: i) A noise removal stream based on generative adversarial networks, which aims to achieve effective scan noise removal in a weakly-supervised manner, i.e., without paired noisy-clean samples for learning; ii) A spine segmentation stream, which aims to predict accurate bone masks. To establish the interaction between these two tasks, we propose a selective feature-sharing strategy to transfer only the beneficial features, while filtering out the useless or harmful information. We evaluate our proposed framework on both scan noise removal and spine segmentation tasks. The experimental results demonstrate that our proposed method achieves promising performance on both tasks, which provides an appealing approach to facilitating clinical diagnosis.

A Comparative Study between Image- and Projection-Domain Self-Supervised Learning for Ultra Low-Dose CBCT

We compare image domain and projection domain denoising approaches with self-supervised learning for ultra low-dose cone-beam CT (CBCT), where number of detected x-ray photons is significantly low. For image-domain self-supervised denoising, we first reconstruct CBCT images with the standard filtered backprojection. For model training, we use blind-spot filtering to partially blind images and recover the blind spots. For projection-domain self-supervised denoising, we regard the post-log projections as training examples of convolutional neural network. From experimental results with various low-dose CBCT settings, the projection-domain denoiser outperforms the image-domain denoiser both in image quality and accuracy for ultra low-dose CBCT.

CCN-CL: A content-noise complementary network with contrastive learning for low-dose computed tomography denoising

In recent years, low-dose computed tomography (LDCT) has played an increasingly important role in the diagnosis CT to reduce the potential adverse effects of x-ray radiation on patients while maintaining the same diagnostic image quality. Current deep learning-based denoising methods applied to LDCT imaging only use normal dose CT (NDCT) images as positive examples to guide the denoising process. Recent studies on contrastive learning have proved that the original images as negative examples can also be helpful for network learning. Therefore, this paper proposes a novel content-noise complementary network with contrastive learning for an LDCT denoising task. First, to better train our proposed network, a contrastive learning loss, taking the NDCT image as a positive example and the original LDCT image as a negative example to guide the network learning is added. Furthermore, we also design a network structure that combines content-noise complementary learning strategy, attention mechanism, and deformable convolution for better network performance. In an evaluation study, we compare the performance of our designed network with some of the state-of-the-art methods in the 2016 NIH-AAPM-Mayo Clinic Low Dose CT Grand Challenge dataset. The quantitative and qualitative evaluation results demonstrate the feasibility and effectiveness of applying our proposed CCN-CL network model as a new deep learning-based LDCT denoising method.

Masked Joint Bilateral Filtering via Deep Image Prior for Digital X-ray Image Denoising

Medical image denoising faces great challenges. Although deep learning methods have shown great potential, their efficiency is severely affected by millions of trainable parameters. The non-linearity of neural networks also makes them difficult to be understood. Therefore, existing deep learning methods have been sparingly applied to clinical tasks. To this end, we integrate known filtering operators into deep learning and propose a novel Masked Joint Bilateral Filtering (MJBF) via deep image prior for digital X-ray image denoising. Specifically, MJBF consists of a deep image prior generator and an iterative filtering block. The deep image prior generator produces plentiful image priors by a multi-scale fusion network. The generated image priors serve as the guidance for the iterative filtering block, which is utilized for the actual edge-preserving denoising. The iterative filtering block contains three trainable Joint Bilateral Filters (JBFs), each with only 18 trainable parameters. Moreover, a masking strategy is introduced to reduce redundancy and improve the understanding of the proposed network. Experimental results on the ChestX-ray14 dataset and real data show that the proposed MJBF has achieved superior performance in terms of noise suppression and edge preservation. Tests on the portability of the proposed method demonstrate that this denoising modality is simple yet effective, and could have a clinical impact on medical imaging in the future.

MRCON-Net: Multiscale reweighted convolutional coding neural network for low-dose CT imaging

BACKGROUND AND OBJECTIVE

Low-dose computed tomography (LDCT) has become increasingly important for alleviating X-ray radiation damage. However, reducing the administered radiation dose may lead to degraded CT images with amplified mottle noise and nonstationary streak artifacts. Previous studies have confirmed that deep learning (DL) is promising for improving LDCT imaging. However, most DL-based frameworks are built intuitively, lack interpretability, and suffer from image detail information loss, which has become a general challenging issue.

METHODS

A multiscale reweighted convolutional coding neural network (MRCON-Net) is developed to address the above problems. MRCON-Net is compact and more explainable than other networks. First, inspired by the learning-based reweighted iterative soft thresholding algorithm (ISTA), we extend traditional convolutional sparse coding (CSC) to its reweighted convolutional learning form. Second, we use dilated convolution to extract multiscale image features, allowing our single model to capture the correlations between features of different scales. Finally, to automatically adjust the elements in the feature code to correct the obtained solution, a channel attention (CA) mechanism is utilized to learn appropriate weights.

RESULTS

The visual results obtained based on the American Association of Physicians in Medicine (AAPM) Challenge and United Image Healthcare (UIH) clinical datasets confirm that the proposed model significantly reduces serious artifact noise while retaining the desired structures. Quantitative results show that the average structural similarity index measurement (SSIM) and peak signal-to-noise ratio (PSNR) achieved on the AAPM Challenge dataset are 0.9491 and 40.66, respectively, and the SSIM and PSNR achieved on the UIH clinical dataset are 0.915 and 42.44, respectively; these are promising quantitative results.

CONCLUSION

Compared with recent state-of-the-art methods, the proposed model achieves subtle structure-enhanced LDCT imaging. In addition, through ablation studies, the components of the proposed model are validated to achieve performance improvements.

The use of deep learning methods in low-dose computed tomography image reconstruction: a systematic review

Conventional reconstruction techniques, such as filtered back projection (FBP) and iterative reconstruction (IR), which have been utilised widely in the image reconstruction process of computed tomography (CT) are not suitable in the case of low-dose CT applications, because of the unsatisfying quality of the reconstructed image and inefficient reconstruction time. Therefore, as the demand for CT radiation dose reduction continues to increase, the use of artificial intelligence (AI) in image reconstruction has become a trend that attracts more and more attention. This systematic review examined various deep learning methods to determine their characteristics, availability, intended use and expected outputs concerning low-dose CT image reconstruction. Utilising the methodology of Kitchenham and Charter, we performed a systematic search of the literature from 2016 to 2021 in Springer, Science Direct, arXiv, PubMed, ACM, IEEE, and Scopus. This review showed that algorithms using deep learning technology are superior to traditional IR methods in noise suppression, artifact reduction and structure preservation, in terms of improving the image quality of low-dose reconstructed images. In conclusion, we provided an overview of the use of deep learning approaches in low-dose CT image reconstruction together with their benefits, limitations, and opportunities for improvement.

Patch-based artifact reduction for three-dimensional volume projection data of sparse-view micro-computed tomography

Micro-computed tomography (micro-CT) enables the non-destructive acquisition of three-dimensional (3D) morphological structures at the micrometer scale. Although it is expected to be used in pathology and histology to analyze the 3D microstructure of tissues, micro-CT imaging of tissue specimens requires a long scan time. A high-speed imaging method, sparse-view CT, can reduce the total scan time and radiation dose; however, it causes severe streak artifacts on tomographic images reconstructed with analytical algorithms due to insufficient sampling. In this paper, we propose an artifact reduction method for 3D volume projection data from sparse-view micro-CT. Specifically, we developed a patch-based lightweight fully convolutional network to estimate full-view 3D volume projection data from sparse-view 3D volume projection data. We evaluated the effectiveness of the proposed method using physically acquired datasets. The qualitative and quantitative results showed that the proposed method achieved high estimation accuracy and suppressed streak artifacts in the reconstructed images. In addition, we confirmed that the proposed method requires both short training and prediction times. Our study demonstrates that the proposed method has great potential for artifact reduction for 3D volume projection data under sparse-view conditions.

The stability of oncologic MRI radiomic features and the potential role of deep learning: a review

The use of MRI radiomic models for the diagnosis, prognosis and treatment response prediction of tumors has been increasingly reported in literature. However, its widespread adoption in clinics is hampered by issues related to features stability. In the MRI radiomic workflow, the main factors that affect radiomic features computation can be found in the image acquisition and reconstruction phase, in the image pre-processing steps, and in the segmentation of the region of interest on which radiomic indices are extracted. Deep Neural Networks (DNNs), having shown their potentiality in the medical image processing and analysis field, can be seen as an attractive strategy to partially overcome the issues related to radiomic stability and mitigate their impact. In fact, DNN approaches can be prospectively integrated in the MRI radiomic workflow to improve image quality, obtain accurate and reproducible segmentations and generate standardized images. In this review, DNN methods that can be included in the image processing steps of the radiomic workflow are described and discussed, in the light of a detailed analysis of the literature in the context of MRI radiomic reliability.

Hierarchical Amortized GAN for 3D High Resolution Medical Image Synthesis

Generative Adversarial Networks (GAN) have many potential medical imaging applications, including data augmentation, domain adaptation, and model explanation. Due to the limited memory of Graphical Processing Units (GPUs), most current 3D GAN models are trained on low-resolution medical images, these models either cannot scale to high-resolution or are prone to patchy artifacts. In this work, we propose a novel end-to-end GAN architecture that can generate high-resolution 3D images. We achieve this goal by using different configurations between training and inference. During training, we adopt a hierarchical structure that simultaneously generates a low-resolution version of the image and a randomly selected sub-volume of the high-resolution image. The hierarchical design has two advantages: First, the memory demand for training on high-resolution images is amortized among sub-volumes. Furthermore, anchoring the high-resolution sub-volumes to a single low-resolution image ensures anatomical consistency between sub-volumes. During inference, our model can directly generate full high-resolution images. We also incorporate an encoder with a similar hierarchical structure into the model to extract features from the images. Experiments on 3D thorax CT and brain MRI demonstrate that our approach outperforms state of the art in image generation. We also demonstrate clinical applications of the proposed model in data augmentation and clinical-relevant feature extraction.

Generative models improve radiomics performance in different tasks and different datasets: An experimental study

PURPOSE

Radiomics is an active area of research focusing on high throughput feature extraction from medical images with a wide array of applications in clinical practice, such as clinical decision support in oncology. However, noise in low dose computed tomography (CT) scans can impair the accurate extraction of radiomic features. In this article, we investigate the possibility of using deep learning generative models to improve the performance of radiomics from low dose CTs.

METHODS

We used two datasets of low dose CT scans - NSCLC Radiogenomics and LIDC-IDRI - as test datasets for two tasks - pre-treatment survival prediction and lung cancer diagnosis. We used encoder-decoder networks and conditional generative adversarial networks (CGANs) trained in a previous study as generative models to transform low dose CT images into full dose CT images. Radiomic features extracted from the original and improved CT scans were used to build two classifiers - a support vector machine (SVM) and a deep attention based multiple instance learning model - for survival prediction and lung cancer diagnosis respectively. Finally, we compared the performance of the models derived from the original and improved CT scans.

RESULTS

Denoising with the encoder-decoder network and the CGAN improved the area under the curve (AUC) of survival prediction from 0.52 to 0.57 (p-value < 0.01). On the other hand, the encoder-decoder network and the CGAN improved the AUC of lung cancer diagnosis from 0.84 to 0.88 and 0.89 respectively (p-value < 0.01). Finally, there are no statistically significant improvements in AUC using encoder-decoder networks and CGAN (p-value = 0.34) when networks trained at 75 and 100 epochs.

CONCLUSION

Generative models can improve the performance of low dose CT-based radiomics in different tasks. Hence, denoising using generative models seems to be a necessary pre-processing step for calculating radiomic features from low dose CTs.

Limited parameter denoising for low-dose X-ray computed tomography using deep reinforcement learning

BACKGROUND

The use of deep learning has successfully solved several problems in the field of medical imaging. Deep learning has been applied to the CT denoising problem successfully. However, the use of deep learning requires large amounts of data to train deep convolutional networks (CNNs). Moreover, due to large parameter count, such deep CNNs may cause unexpected results.

PURPOSE

In this study, we introduce a novel CT denoising framework, which has interpretable behaviour, and provides useful results with limited data.

METHODS

We employ bilateral filtering in both the projection and volume domains to remove noise. To account for non-stationary noise, we tune the σ parameters of the volume for every projection view, and for every volume pixel. The tuning is carried out by two deep CNNs. Due to impracticality of labelling, the two deep CNNs are trained via a Deep-Q reinforcement learning task. The reward for the task is generated by using a custom reward function represented by a neural network. Our experiments were carried out on abdominal scans for the Mayo Clinic TCIA dataset, and the AAPM Low Dose CT Grand Challenge.

RESULTS

Our denoising framework has excellent denoising performance increasing the PSNR from 28.53 to 28.93, and increasing the SSIM from 0.8952 to 0.9204. We outperform several state-of-the-art deep CNNs, which have several orders of magnitude higher number of parameters (p-value (PSNR) = 0.000, p-value (SSIM) = 0.000). Our method does not introduce any blurring, which is introduced by MSE loss based methods, or any deep learning artifacts, which are introduced by WGAN based models. Our ablation studies show that parameter tuning and using our reward network results in the best possible results.

CONCLUSIONS

We present a novel CT denoising framework, which focuses on interpretability to deliver good denoising performance, especially with limited data. Our method outperforms state-of-the-art deep neural networks. Future work will be focused on accelerating our method, and generalizing to different geometries and body parts. This article is protected by copyright. All rights reserved.

Impact of the training loss in deep learning based CT reconstruction of bone microarchitecture

PURPOSE

Computed tomography (CT) is a technique of choice to image bone structure at different scales. Methods to enhance the quality of degraded reconstructions obtained from low-dose CT data have shown impressive results recently, especially in the realm of supervised deep learning. As the choice of the loss function affects the reconstruction quality, it is necessary to focus on the way neural networks evaluate the correspondence between predicted and target images during the training stage. This is even more true in the case of bone microarchitecture imaging at high spatial resolution where both the quantitative analysis of Bone Mineral Density (BMD) and bone microstructure are essential for assessing diseases such as osteoporosis. Our aim is thus to evaluate the quality of reconstruction on key metrics for diagnosis depending on the loss function that has been used for training the neural network.

METHODS

We compare and analyze volumes that are reconstructed with neural networks trained with pixelwise, structural and adversarial loss functions or with a combination of them. We perform realistic simulations of various low-dose acquisitions of bone microarchitecture. Our comparative study is performed with metrics that have an interest regarding the diagnosis of bone diseases. We therefore focus on bone-specific metrics such as BV/TV, resolution, connectivity assessed with the Euler number and quantitative analysis of BMD to evaluate the quality of reconstruction obtained with networks trained with the different loss functions.

RESULTS

We find that using L₁ norm as the pixelwise loss is the best choice compared to L₂ or no pixelwise loss since it improves resolution without deteriorating other metrics. VGG perceptual loss, especially when combined with an adversarial loss, allows to better retrieve topological and morphological parameters of bone microarchitecture compared to SSIM. This however leads to a decreased resolution performance. The adversarial loss enchances the reconstruction performance in terms of BMD distribution accuracy.

CONCLUSIONS

In order to retrieve the quantitative and structural characteristics of bone microarchitecture that are essential for post-reconstruction diagnosis, our results suggest to use L₁ norm as part of the loss function. Then, trade-offs should be made depending on the application: VGG perceptual loss improves accuracy in terms of connectivity at the cost of a deteriorated resolution, and adversarial losses help better retrieve BMD distribution while significantly increasing the training time. This article is protected by copyright. All rights reserved.

Content-Noise Complementary Learning for Medical Image Denoising

Medical imaging denoising faces great challenges, yet is in great demand. With its distinctive characteristics, medical imaging denoising in the image domain requires innovative deep learning strategies. In this study, we propose a simple yet effective strategy, the content-noise complementary learning (CNCL) strategy, in which two deep learning predictors are used to learn the respective content and noise of the image dataset complementarily. A medical image denoising pipeline based on the CNCL strategy is presented, and is implemented as a generative adversarial network, where various representative networks (including U-Net, DnCNN, and SRDenseNet) are investigated as the predictors. The performance of these implemented models has been validated on medical imaging datasets including CT, MR, and PET. The results show that this strategy outperforms state-of-the-art denoising algorithms in terms of visual quality and quantitative metrics, and the strategy demonstrates a robust generalization capability. These findings validate that this simple yet effective strategy demonstrates promising potential for medical image denoising tasks, which could exert a clinical impact in the future. Code is available at: https://github.com/gengmufeng/CNCL-denoising.