Learnable PM diffusion coefficients and reformative coordinate attention network for low dose CT denoising

Objective.Various deep learning methods have recently been used for low dose CT (LDCT) denoising. Aggressive denoising may destroy the edge and fine anatomical structures of CT images. Therefore a key issue in LDCT denoising tasks is the difficulty of balancing noise/artifact suppression and edge/structure preservation.Approach.We proposed an LDCT denoising network based on the encoder-decoder structure, namely the Learnable PM diffusion coefficient and efficient attention network (PMA-Net). First, using the powerful feature modeling capability of partial differential equations, we constructed a multiple learnable edge module to generate precise edge information, incorporating the anisotropic image processing idea of Perona-Malik (PM) model into the neural network. Second, a multiscale reformative coordinate attention module was designed to extract multiscale information. Non-overlapping dilated convolution capturing abundant contextual content was combined with coordinate attention which could embed the spatial location information of important features into the channel attention map. Finally, we imposed additional constraints on the edge information using edge-enhanced multiscale perceptual loss to avoid structure loss and over-smoothing.Main results.Experiments are conducted on simulated and real datasets. The quantitative and qualitative results show that the proposed method has better performance in suppressing noise/artifacts and preserving edges/structures.Significance.This work proposes a novel edge feature extraction method that unfolds partial differential equation into neural networks, which contributes to the interpretability and clinical application value of neural network.

StruNet: Perceptual and low-rank regularized transformer for medical image denoising

BACKGROUND

Various types of noise artifacts inevitably exist in some medical imaging modalities due to limitations of imaging techniques, which impair either clinical diagnosis or subsequent analysis. Recently, deep learning approaches have been rapidly developed and applied on medical images for noise removal or image quality enhancement. Nevertheless, due to complexity and diversity of noise distribution representations in different medical imaging modalities, most of the existing deep learning frameworks are incapable to flexibly remove noise artifacts while retaining detailed information. As a result, it remains challenging to design an effective and unified medical image denoising method that will work across a variety of noise artifacts for different imaging modalities without requiring specialized knowledge in performing the task.

PURPOSE

In this paper, we propose a novel encoder-decoder architecture called Swin transformer-based residual u-shape Network (StruNet), for medical image denoising.

METHODS

Our StruNet adopts a well-designed block as the backbone of the encoder-decoder architecture, which integrates Swin Transformer modules with residual block in parallel connection. Swin Transformer modules could effectively learn hierarchical representations of noise artifacts via self-attention mechanism in non-overlapping shifted windows and cross-window connection, while residual block is advantageous to compensate loss of detailed information via shortcut connection. Furthermore, perceptual loss and low-rank regularization are incorporated into loss function respectively in order to constrain the denoising results on feature-level consistency and low-rank characteristics.

RESULTS

To evaluate the performance of the proposed method, we have conducted experiments on three medical imaging modalities including computed tomography (CT), optical coherence tomography (OCT) and optical coherence tomography angiography (OCTA).

CONCLUSIONS

The results demonstrate that the proposed architecture yields a promising performance of suppressing multiform noise artifacts existing in different imaging modalities.

Multi-Domain Rapid Enhancement Networks for Underwater Images

Images captured during marine engineering operations suffer from color distortion and low contrast. Underwater image enhancement helps to alleviate these problems. Many deep learning models can infer multi-source data, where images with different perspectives exist from multiple sources. To this end, we propose a multichannel deep convolutional neural network (MDCNN) linked to a VGG that can target multi-source (multi-domain) underwater image enhancement. The designed MDCNN feeds data from different domains into separate channels and implements parameters by linking VGGs, which improves the domain adaptation of the model. In addition, to optimize performance, multi-domain image perception loss functions, multilabel soft edge loss for specific image enhancement tasks, pixel-level loss, and external monitoring loss for edge sharpness preprocessing are proposed. These loss functions are set to effectively enhance the structural and textural similarity of underwater images. A series of qualitative and quantitative experiments demonstrate that our model is superior to the state-of-the-art Shallow UWnet in terms of UIQM, and the performance evaluation conducted on different datasets increased by 0.11 on average.

Gram-GAN: Image Super-Resolution Based on Gram Matrix and Discriminator Perceptual Loss

The solution of a high-resolution (HR) image corresponding to a low-resolution (LR) image is not unique in most cases. However, single-LR-single-HR supervision is widely adopted in single-image super-resolution (SISR) tasks, which leads to inflexible inference logic of the model and poor generalization ability. To improve the flexibility of model inference, we constructed a novel form of supervision, except for the ground truth (GT). Specifically, considering the structural properties of natural images, we propose using extra supervision to focus on the textural similarity of the images. As textural similarity does not account for the position information of images, a Gram matrix was constructed to break the limitations of spatial position and focus on the textural information. Besides the use of traditional perceptual loss, we propose a discriminator perceptual loss based on the two-network architecture of generative adversarial networks (GAN). The difference between the discriminator features used in this loss and the traditional visual geometry group (VGG) features is that the discriminator features can describe the relevant information from the perspective of super-resolution. Quantitative and qualitative experiments were performed to demonstrate the effectiveness of the proposed method.

Generation and Evaluation of Synthetic Computed Tomography (CT) from Cone-Beam CT (CBCT) by Incorporating Feature-Driven Loss into Intensity-Based Loss Functions in Deep Convolutional Neural Network

Simple Summary

Despite numerous benefits of cone-beam computed tomography (CBCT), its applications to radiotherapy were limited mainly due to degraded image quality. Recently, enhancing the CBCT image quality by generating synthetic CT image by deep convolutional neural network (CNN) has become frequent. Most of the previous works, however, generated synthetic CT with simple, classical intensity-driven loss in network training, while not specifying a full-package of verifications. This work trained the network by combining feature- and intensity-driven losses and attempted to demonstrate clinical relevance of the synthetic CT images by assessing both image similarity and dose calculating accuracy throughout a commercial Monte-Carlo.

Abstract

Deep convolutional neural network (CNN) helped enhance image quality of cone-beam computed tomography (CBCT) by generating synthetic CT. Most of the previous works, however, trained network by intensity-based loss functions, possibly undermining to promote image feature similarity. The verifications were not sufficient to demonstrate clinical applicability, either. This work investigated the effect of variable loss functions combining feature- and intensity-driven losses in synthetic CT generation, followed by strengthening the verification of generated images in both image similarity and dosimetry accuracy. The proposed strategy highlighted the feature-driven quantification in (1) training the network by perceptual loss, besides L1 and structural similarity (SSIM) losses regarding anatomical similarity, and (2) evaluating image similarity by feature mapping ratio (FMR), besides conventional metrics. In addition, the synthetic CT images were assessed in terms of dose calculating accuracy by a commercial Monte-Carlo algorithm. The network was trained with 50 paired CBCT-CT scans acquired at the same CT simulator and treatment unit to constrain environmental factors any other than loss functions. For 10 independent cases, incorporating perceptual loss into L1 and SSIM losses outperformed the other combinations, which enhanced FMR of image similarity by 10%, and the dose calculating accuracy by 1–2% of gamma passing rate in 1%/1mm criterion.

Low-dose CT denoising via convolutional neural network with an observer loss function

PURPOSE

Convolutional neural network (CNN)-based denoising is an effective method for reducing complex computed tomography (CT) noise. However, the image blur induced by denoising processes is a major concern. The main source of image blur is the pixel-level loss (e.g., mean squared error [MSE] and mean absolute error [MAE]) used to train a CNN denoiser. To reduce the image blur, feature-level loss is utilized to train a CNN denoiser. A CNN denoiser trained using visual geometry group (VGG) loss can preserve the small structures, edges, and texture of the image.However, VGG loss, derived from an ImageNet-pretrained image classifier, is not optimal for training a CNN denoiser for CT images. ImageNet contains natural RGB images, so the features extracted by the ImageNet-pretrained model cannot represent the characteristics of CT images that are highly correlated with diagnosis. Furthermore, a CNN denoiser trained with VGG loss causes bias in CT number. Therefore, we propose to use a binary classification network trained using CT images as a feature extractor and newly define the feature-level loss as observer loss.

METHODS

As obtaining labeled CT images for training classification network is difficult, we create labels by inserting simulated lesions. We conduct two separate classification tasks, signal-known-exactly (SKE) and signal-known-statistically (SKS), and define the corresponding feature-level losses as SKE loss and SKS loss, respectively. We use SKE loss and SKS loss to train CNN denoiser.

RESULTS

Compared to pixel-level losses, a CNN denoiser trained using observer loss (i.e., SKE loss and SKS loss) is effective in preserving structure, edge, and texture. Observer loss also resolves the bias in CT number, which is a problem of VGG loss. Comparing observer losses using SKE and SKS tasks, SKS yields images having a more similar noise structure to reference images.

CONCLUSIONS

Using observer loss for training CNN denoiser is effective to preserve structure, edge, and texture in denoised images and prevent the CT number bias. In particular, when using SKS loss, denoised images having a similar noise structure to reference images are generated.

Rapid reconstruction of highly undersampled, non-Cartesian real-time cine k-space data using a perceptual complex neural network (PCNN)

Highly accelerated real-time cine MRI using compressed sensing (CS) is a promising approach to achieve high spatio-temporal resolution and clinically acceptable image quality in patients with arrhythmia and/or dyspnea. However, its lengthy image reconstruction time may hinder its clinical translation. The purpose of this study was to develop a neural network for reconstruction of non-Cartesian real-time cine MRI k-space data faster (<1 min per slice with 80 frames) than graphics processing unit (GPU)-accelerated CS reconstruction, without significant loss in image quality or accuracy in left ventricular (LV) functional parameters. We introduce a perceptual complex neural network (PCNN) that trains on complex-valued MRI signal and incorporates a perceptual loss term to suppress incoherent image details. This PCNN was trained and tested with multi-slice, multi-phase, cine images from 40 patients (20 for training, 20 for testing), where the zero-filled images were used as input and the corresponding CS reconstructed images were used as practical ground truth. The resulting images were compared using quantitative metrics (structural similarity index (SSIM) and normalized root mean square error (NRMSE)) and visual scores (conspicuity, temporal fidelity, artifacts, and noise scores), individually graded on a five-point scale (1, worst; 3, acceptable; 5, best), and LV ejection fraction (LVEF). The mean processing time per slice with 80 frames for PCNN was 23.7 ± 1.9 s for pre-processing (Step 1, same as CS) and 0.822 ± 0.004 s for dealiasing (Step 2, 166 times faster than CS). Our PCNN produced higher data fidelity metrics (SSIM = 0.88 ± 0.02, NRMSE = 0.014 ± 0.004) compared with CS. While all the visual scores were significantly different (P < 0.05), the median scores were all 4.0 or higher for both CS and PCNN. LVEFs measured from CS and PCNN were strongly correlated (R² = 0.92) and in good agreement (mean difference = -1.4% [2.3% of mean]; limit of agreement = 10.6% [17.6% of mean]). The proposed PCNN is capable of rapid reconstruction (25 s per slice with 80 frames) of non-Cartesian real-time cine MRI k-space data, without significant loss in image quality or accuracy in LV functional parameters.

Unsupervised learning for deformable registration of thoracic CT and cone-beam CT based on multiscale features matching with spatially adaptive weighting

PURPOSE

Cone-beam computed tomography (CBCT) is a common on-treatment imaging widely used in image-guided radiotherapy. Fast and accurate registration between the on-treatment CBCT and planning CT is significant for and precise adaptive radiotherapy treatment (ART). However, existing CT-CBCT registration methods, which are mostly affine or time-consuming intensity- based deformation registration, still need further study due to the considerable CT-CBCT intensity discrepancy and the artifacts in low-quality CBCT images. In this paper, we propose a deep learning-based CT-CBCT registration model to promote rapid and accurate CT-CBCT registration for radiotherapy.

METHODS

The proposed CT-CBCT registration model consists of a registration network and an innovative deep similarity metric network. The registration network is a novel fully convolution network adapted specially for patch-wise CT-CBCT registration. The metric network, going beyond intensity, automatically evaluates the high-dimensional attribute-based dissimilarity between the registered CT and CBCT images. In addition, considering the artifacts in low-quality CBCT images, we add spatial weighting (SW) block to adaptively attach more importance to those informative voxels while inhibit the interference of artifact regions. Such SW-based metric network is expected to extract the most meaningful and discriminative deep features, and form a more reliable CT-CBCT similarity measure to train the registration network.

RESULTS

We evaluate the proposed method on clinical thoracic CBCT and CT dataset, and compare the registration results with some other common image similarity metrics and some state-of-the-art registration algorithms. The proposed method provides the highest Structural Similarity index (86.17 ± 5.09), minimum Target Registration Error of landmarks (2.37 ± 0.32 mm), and the best DSC coefficient (78.71 ± 10.95) of tumor volumes. Moreover, our model also obtains comparable distance error of lung surfaces (1.75 ± 0.35 mm).

CONCLUSION

The proposed model shows both efficiency and efficacy for reliable thoracic CT-CBCT registration, and can generate the matched CT and CBCT images within few seconds, which is of great significance to clinical radiotherapy.

Personalized design technique for the dental occlusal surface based on conditional generative adversarial networks

The tooth defect is a frequently occurring disease within the field of dental clinic. However, the traditional manual restoration for the defective tooth needs an especially long treatment time, and dental computer aided design and manufacture (CAD/CAM) systems fail to restore the personalized anatomical features of natural teeth. Aiming to address the shortcomings of existed methods, this article proposes an intelligent network model for designing tooth crown surface based on conditional generative adversarial networks. Then, the data set for training the network model is constructed via generating depth maps of 3D tooth models scanned by the intraoral. Through adversarial training, the network model is able to generate tooth occlusal surface under the constraint of the space occlusal relationship, the perceptual loss, and occlusal groove filter loss. Finally, we carry out the assessment experiments for the quality of the occlusal surface and the occlusal relationship with the opposing tooth. The experimental results demonstrate that our method can automatically reconstruct the personalized anatomical features on occlusal surface and shorten the treatment time while restoring the full functionality of the defective tooth.

PET Image Denoising Using a Deep Neural Network Through Fine Tuning

Positron emission tomography (PET) is a functional imaging modality widely used in clinical diagnosis. In this work, we trained a deep convolutional neural network (CNN) to improve PET image quality. Perceptual loss based on features derived from a pre-trained VGG network, instead of the conventional mean squared error, was employed as the training loss function to preserve image details. As the number of real patient data set for training is limited, we propose to pre-train the network using simulation data and fine-tune the last few layers of the network using real data sets. Results from simulation, real brain and lung data sets show that the proposed method is more effective in removing noise than the traditional Gaussian filtering method.

Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss

The continuous development and extensive use of computed tomography (CT) in medical practice has raised a public concern over the associated radiation dose to the patient. Reducing the radiation dose may lead to increased noise and artifacts, which can adversely affect the radiologists' judgment and confidence. Hence, advanced image reconstruction from low-dose CT data is needed to improve the diagnostic performance, which is a challenging problem due to its ill-posed nature. Over the past years, various low-dose CT methods have produced impressive results. However, most of the algorithms developed for this application, including the recently popularized deep learning techniques, aim for minimizing the mean-squared error (MSE) between a denoised CT image and the ground truth under generic penalties. Although the peak signal-to-noise ratio is improved, MSE- or weighted-MSE-based methods can compromise the visibility of important structural details after aggressive denoising. This paper introduces a new CT image denoising method based on the generative adversarial network (GAN) with Wasserstein distance and perceptual similarity. The Wasserstein distance is a key concept of the optimal transport theory and promises to improve the performance of GAN. The perceptual loss suppresses noise by comparing the perceptual features of a denoised output against those of the ground truth in an established feature space, while the GAN focuses more on migrating the data noise distribution from strong to weak statistically. Therefore, our proposed method transfers our knowledge of visual perception to the image denoising task and is capable of not only reducing the image noise level but also trying to keep the critical information at the same time. Promising results have been obtained in our experiments with clinical CT images.