Generative adversarial network based regularized image reconstruction for PET

Single-Subject Deep-Learning Image Reconstruction With a Neural Optimization Transfer Algorithm for PET-Enabled Dual-Energy CT Imaging

Combining dual-energy computed tomography (DECT) with positron emission tomography (PET) offers many potential clinical applications but typically requires expensive hardware upgrades or increases radiation doses on PET/CT scanners due to an extra X-ray CT scan. The recent PET-enabled DECT method allows DECT imaging on PET/CT without requiring a second X-ray CT scan. It combines the already existing X-ray CT image with a 511 keV γ -ray CT (gCT) image reconstructed from time-of-flight PET emission data. A kernelized framework has been developed for reconstructing gCT image but this method has not fully exploited the potential of prior knowledge. Use of deep neural networks may explore the power of deep learning in this application. However, common approaches require a large database for training, which is impractical for a new imaging method like PET-enabled DECT. Here, we propose a single-subject method by using neural-network representation as a deep coefficient prior to improving gCT image reconstruction without population-based pre-training. The resulting optimization problem becomes the tomographic estimation of nonlinear neural-network parameters from gCT projection data. This complicated problem can be efficiently solved by utilizing the optimization transfer strategy with quadratic surrogates. Each iteration of the proposed neural optimization transfer algorithm includes: PET activity image update; gCT image update; and least-square neural-network learning in the gCT image domain. This algorithm is guaranteed to monotonically increase the data likelihood. Results from computer simulation, real phantom data and real patient data have demonstrated that the proposed method can significantly improve gCT image quality and consequent multi-material decomposition as compared to other methods.

Deep learning-based PET image denoising and reconstruction: a review

This review focuses on positron emission tomography (PET) imaging algorithms and traces the evolution of PET image reconstruction methods. First, we provide an overview of conventional PET image reconstruction methods from filtered backprojection through to recent iterative PET image reconstruction algorithms, and then review deep learning methods for PET data up to the latest innovations within three main categories. The first category involves post-processing methods for PET image denoising. The second category comprises direct image reconstruction methods that learn mappings from sinograms to the reconstructed images in an end-to-end manner. The third category comprises iterative reconstruction methods that combine conventional iterative image reconstruction with neural-network enhancement. We discuss future perspectives on PET imaging and deep learning technology.

An iterative reconstruction algorithm for unsupervised PET image

Objective.In recent years, convolutional neural networks (CNNs) have shown great potential in positron emission tomography (PET) image reconstruction. However, most of them rely on many low-quality and high-quality reference PET image pairs for training, which are not always feasible in clinical practice. On the other hand, many works improve the quality of PET image reconstruction by adding explicit regularization or optimizing the network structure, which may lead to complex optimization problems.Approach.In this paper, we develop a novel iterative reconstruction algorithm by integrating the deep image prior (DIP) framework, which only needs the prior information (e.g. MRI) and sinogram data of patients. To be specific, we construct the objective function as a constrained optimization problem and utilize the existing PET image reconstruction packages to streamline calculations. Moreover, to further improve both the reconstruction quality and speed, we introduce the Nesterov's acceleration part and the restart mechanism in each iteration.Main results.2D experiments on PET data sets based on computer simulations and real patients demonstrate that our proposed algorithm can outperform existing MLEM-GF, KEM and DIPRecon methods.Significance.Unlike traditional CNN methods, the proposed algorithm does not rely on large data sets, but only leverages inter-patient information. Furthermore, we enhance reconstruction performance by optimizing the iterative algorithm. Notably, the proposed method does not require much modification of the basic algorithm, allowing for easy integration into standard implementations.

A Review on Low-Dose Emission Tomography Post-Reconstruction Denoising with Neural Network Approaches

Low-dose emission tomography (ET) plays a crucial role in medical imaging, enabling the acquisition of functional information for various biological processes while minimizing the patient dose. However, the inherent randomness in the photon counting process is a source of noise which is amplified in low-dose ET. This review article provides an overview of existing post-processing techniques, with an emphasis on deep neural network (NN) approaches. Furthermore, we explore future directions in the field of NN-based low-dose ET. This comprehensive examination sheds light on the potential of deep learning in enhancing the quality and resolution of low-dose ET images, ultimately advancing the field of medical imaging.

Model-based deep CNN-regularized reconstruction for digital breast tomosynthesis with a task-based CNN image assessment approach

Objective. Digital breast tomosynthesis (DBT) is a quasi-three-dimensional breast imaging modality that improves breast cancer screening and diagnosis because it reduces fibroglandular tissue overlap compared with 2D mammography. However, DBT suffers from noise and blur problems that can lower the detectability of subtle signs of cancers such as microcalcifications (MCs). Our goal is to improve the image quality of DBT in terms of image noise and MC conspicuity.Approach. We proposed a model-based deep convolutional neural network (deep CNN or DCNN) regularized reconstruction (MDR) for DBT. It combined a model-based iterative reconstruction (MBIR) method that models the detector blur and correlated noise of the DBT system and the learning-based DCNN denoiser using the regularization-by-denoising framework. To facilitate the task-based image quality assessment, we also proposed two DCNN tools for image evaluation: a noise estimator (CNN-NE) trained to estimate the root-mean-square (RMS) noise of the images, and an MC classifier (CNN-MC) as a DCNN model observer to evaluate the detectability of clustered MCs in human subject DBTs.Main results. We demonstrated the efficacies of CNN-NE and CNN-MC on a set of physical phantom DBTs. The MDR method achieved low RMS noise and the highest detection area under the receiver operating characteristic curve (AUC) rankings evaluated by CNN-NE and CNN-MC among the reconstruction methods studied on an independent test set of human subject DBTs.Significance. The CNN-NE and CNN-MC may serve as a cost-effective surrogate for human observers to provide task-specific metrics for image quality comparisons. The proposed reconstruction method shows the promise of combining physics-based MBIR and learning-based DCNNs for DBT image reconstruction, which may potentially lead to lower dose and higher sensitivity and specificity for MC detection in breast cancer screening and diagnosis.

Medical applications of generative adversarial network: a visualization analysis

BACKGROUND

Deep learning (DL) is one of the latest approaches to artificial intelligence. As an unsupervised DL method, a generative adversarial network (GAN) can be used to synthesize new data.

PURPOSE

To explore GAN applications in medicine and point out the significance of its existence for clinical medical research, as well as to provide a visual bibliometric analysis of GAN applications in the medical field in combination with the scientometric software Citespace and statistical analysis methods.

MATERIAL AND METHODS

PubMed, MEDLINE, Web of Science, and Google Scholar were searched to identify studies of GAN in medical applications between 2017 and 2022. This study was performed and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Citespace was used to analyze the number of publications, authors, institutions, and keywords of articles related to GAN in medical applications.

RESULTS

The applications of GAN in medicine are not limited to medical image processing, but will also penetrate wider and more complex fields, or may be applied to clinical medicine. Eligibility criteria were the full texts of peer-reviewed journals reporting the application of GANs in medicine. Research selections included material published in English between 1 January 2017 and 1 December 2022.

CONCLUSION

GAN has been fully applied to the medical field and will be more deeply and widely used in clinical medicine, especially in the field of privacy protection and medical diagnosis. However, clinical applications of GAN require consideration of ethical and legal issues. GAN-based applications should be well validated by expert radiologists.

Neural KEM: A Kernel Method With Deep Coefficient Prior for PET Image Reconstruction

Image reconstruction of low-count positron emission tomography (PET) data is challenging. Kernel methods address the challenge by incorporating image prior information in the forward model of iterative PET image reconstruction. The kernelized expectation-maximization (KEM) algorithm has been developed and demonstrated to be effective and easy to implement. A common approach for a further improvement of the kernel method would be adding an explicit regularization, which however leads to a complex optimization problem. In this paper, we propose an implicit regularization for the kernel method by using a deep coefficient prior, which represents the kernel coefficient image in the PET forward model using a convolutional neural-network. To solve the maximum-likelihood neural network-based reconstruction problem, we apply the principle of optimization transfer to derive a neural KEM algorithm. Each iteration of the algorithm consists of two separate steps: a KEM step for image update from the projection data and a deep-learning step in the image domain for updating the kernel coefficient image using the neural network. This optimization algorithm is guaranteed to monotonically increase the data likelihood. The results from computer simulations and real patient data have demonstrated that the neural KEM can outperform existing KEM and deep image prior methods.

Phantom and clinical evaluation of the effect of a new Bayesian penalized likelihood reconstruction algorithm (HYPER Iterative) on 68Ga-DOTA-NOC PET/CT image quality

BACKGROUND

Bayesian penalized likelihood (BPL) algorithm is an effective way to suppress noise in the process of positron emission tomography (PET) image reconstruction by incorporating a smooth penalty. The strength of the smooth penalty is controlled by the penalization factor. The aim was to investigate the impact of different penalization factors and acquisition times in a new BPL algorithm, HYPER Iterative, on the quality of ⁶⁸Ga-DOTA-NOC PET/CT images. A phantom and 25 patients with neuroendocrine neoplasms who underwent ⁶⁸Ga-DOTA-NOC PET/CT were included. The PET data were acquired in a list-mode with a digital PET/CT scanner and reconstructed by ordered subset expectation maximization (OSEM) and the HYPER Iterative algorithm with seven penalization factors between 0.03 and 0.5 for acquisitions of 2 and 3 min per bed position (m/b), both including time-of-flight and point of spread function recovery. The contrast recovery (CR), background variability (BV) and radioactivity concentration ratio (RCR) of the phantom; The SUV_mean and coefficient of variation (CV) of the liver; and the SUV_max of the lesions were measured. Image quality was rated by two radiologists using a five-point Likert scale.

RESULTS

The CR, BV, and RCR decreased with increasing penalization factors for four "hot" spheres, and the HYPER Iterative 2 m/b groups with penalization factors of 0.07 to 0.2 had equivalent CR and superior BV performance compared to the OSEM 3 m/b group. The liver SUV_mean values were approximately equal in all reconstruction groups (range 5.95-5.97), and the liver CVs of the HYPER Iterative 2 m/b and 3 m/b groups with the penalization factors of 0.1 to 0.2 were equivalent to those of the OSEM 3 m/b group (p = 0.113-0.711 and p = 0.079-0.287, respectively), while the lesion SUV_max significantly increased by 19-22% and 25%, respectively (all p < 0.001). The highest qualitative score was attained at a penalization factor of 0.2 for the HYPER Iterative 2 m/b group (3.20 ± 0.52) and 3 m/b group (3.70 ± 0.36); those scores were comparable to or greater than that of the OSEM 3 m/b group (3.09 ± 0.36, p = 0.388 and p < 0.001, respectively).

CONCLUSIONS

The HYPER Iterative algorithm with a penalization factor of 0.2 resulted in higher lesion contrast and lower image noise than OSEM for ⁶⁸Ga-DOTA-NOC PET/CT, allowing the same image quality to be achieved with less injected radioactivity and a shorter acquisition time.

Deep Kernel Representation for Image Reconstruction in PET

Image reconstruction for positron emission tomography (PET) is challenging because of the ill-conditioned tomographic problem and low counting statistics. Kernel methods address this challenge by using kernel representation to incorporate image prior information in the forward model of iterative PET image reconstruction. Existing kernel methods construct the kernels commonly using an empirical process, which may lead to unsatisfactory performance. In this paper, we describe the equivalence between the kernel representation and a trainable neural network model. A deep kernel method is then proposed by exploiting a deep neural network to enable automated learning of an improved kernel model and is directly applicable to single subjects in dynamic PET. The training process utilizes available image prior data to form a set of robust kernels in an optimized way rather than empirically. The results from computer simulations and a real patient dataset demonstrate that the proposed deep kernel method can outperform the existing kernel method and neural network method for dynamic PET image reconstruction.

Unsupervised PET logan parametric image estimation using conditional deep image prior

Recently, deep learning-based denoising methods have been gradually used for PET images denoising and have shown great achievements. Among these methods, one interesting framework is conditional deep image prior (CDIP) which is an unsupervised method that does not need prior training or a large number of training pairs. In this work, we combined CDIP with Logan parametric image estimation to generate high-quality parametric images. In our method, the kinetic model is the Logan reference tissue model that can avoid arterial sampling. The neural network was utilized to represent the images of Logan slope and intercept. The patient's computed tomography (CT) image or magnetic resonance (MR) image was used as the network input to provide anatomical information. The optimization function was constructed and solved by the alternating direction method of multipliers (ADMM) algorithm. Both simulation and clinical patient datasets demonstrated that the proposed method could generate parametric images with more detailed structures. Quantification results showed that the proposed method results had higher contrast-to-noise (CNR) improvement ratios (PET/CT datasets: 62.25%±29.93%; striatum of brain PET datasets : 129.51%±32.13%, thalamus of brain PET datasets: 128.24%±31.18%) than Gaussian filtered results (PET/CT datasets: 23.33%±18.63%; striatum of brain PET datasets: 74.71%±8.71%, thalamus of brain PET datasets: 73.02%±9.34%) and nonlocal mean (NLM) denoised results (PET/CT datasets: 37.55%±26.56%; striatum of brain PET datasets: 100.89%±16.13%, thalamus of brain PET datasets: 103.59%±16.37%).

Deep Learning Based Joint PET Image Reconstruction and Motion Estimation

Respiratory motion is one of the main sources of motion artifacts in positron emission tomography (PET) imaging. The emission image and patient motion can be estimated simultaneously from respiratory gated data through a joint estimation framework. However, conventional motion estimation methods based on registration of a pair of images are sensitive to noise. The goal of this study is to develop a robust joint estimation method that incorporates a deep learning (DL)-based image registration approach for motion estimation. We propose a joint estimation framework by incorporating a learned image registration network into a regularized PET image reconstruction. The joint estimation was formulated as a constrained optimization problem with moving gated images related to a fixed image via the deep neural network. The constrained optimization problem is solved by the alternating direction method of multipliers (ADMM) algorithm. The effectiveness of the algorithm was demonstrated using simulated and real data. We compared the proposed DL-ADMM joint estimation algorithm with a monotonic iterative joint estimation. Motion compensated reconstructions using pre-calculated deformation fields by DL-based (DL-MC recon) and iterative (iterative-MC recon) image registration were also included for comparison. Our simulation study shows that the proposed DL-ADMM joint estimation method reduces bias compared to the ungated image without increasing noise and outperforms the competing methods. In the real data study, our proposed method also generated higher lesion contrast and sharper liver boundaries compared to the ungated image and had lower noise than the reference gated image.

Applications of Generative Adversarial Networks (GANs) in Positron Emission Tomography (PET) imaging: A review

PURPOSE

This paper reviews recent applications of Generative Adversarial Networks (GANs) in Positron Emission Tomography (PET) imaging. Recent advances in Deep Learning (DL) and GANs catalysed the research of their applications in medical imaging modalities. As a result, several unique GAN topologies have emerged and been assessed in an experimental environment over the last two years.

METHODS

The present work extensively describes GAN architectures and their applications in PET imaging. The identification of relevant publications was performed via approved publication indexing websites and repositories. Web of Science, Scopus, and Google Scholar were the major sources of information.

RESULTS

The research identified a hundred articles that address PET imaging applications such as attenuation correction, de-noising, scatter correction, removal of artefacts, image fusion, high-dose image estimation, super-resolution, segmentation, and cross-modality synthesis. These applications are presented and accompanied by the corresponding research works.

CONCLUSION

GANs are rapidly employed in PET imaging tasks. However, specific limitations must be eliminated to reach their full potential and gain the medical community's trust in everyday clinical practice.

Deep learning-based image reconstruction and post-processing methods in positron emission tomography for low-dose imaging and resolution enhancement

Image processing plays a crucial role in maximising diagnostic quality of positron emission tomography (PET) images. Recently, deep learning methods developed across many fields have shown tremendous potential when applied to medical image enhancement, resulting in a rich and rapidly advancing literature surrounding this subject. This review encapsulates methods for integrating deep learning into PET image reconstruction and post-processing for low-dose imaging and resolution enhancement. A brief introduction to conventional image processing techniques in PET is firstly presented. We then review methods which integrate deep learning into the image reconstruction framework as either deep learning-based regularisation or as a fully data-driven mapping from measured signal to images. Deep learning-based post-processing methods for low-dose imaging, temporal resolution enhancement and spatial resolution enhancement are also reviewed. Finally, the challenges associated with applying deep learning to enhance PET images in the clinical setting are discussed and future research directions to address these challenges are presented.

Radiomics in PET Imaging:: A Practical Guide for Newcomers

Radiomics has undergone considerable development in recent years. In PET imaging, very promising results concerning the ability of handcrafted features to predict the biological characteristics of lesions and to assess patient prognosis or response to treatment have been reported in the literature. This article presents a checklist for designing a reliable radiomic study, gives an overview of the steps of the pipeline, and outlines approaches for data harmonization. Tips are provided for critical reading of the content of articles. The advantages and limitations of handcrafted radiomics compared with deep-learning approaches for the characterization of PET images are also discussed.

The Evolution of Image Reconstruction in PET: From Filtered Back-Projection to Artificial Intelligence

PET can provide functional images revealing physiologic processes in vivo. Although PET has many applications, there are still some limitations that compromise its precision: the absorption of photons in the body causes signal attenuation; the dead-time limit of system components leads to the loss of the count rate; the scattered and random events received by the detector introduce additional noise; the characteristics of the detector limit the spatial resolution; and the low signal-to-noise ratio caused by the scan-time limit (eg, dynamic scans) and dose concern. The early PET reconstruction methods are analytical approaches based on an idealized mathematical model.

Anatomically aided PET image reconstruction using deep neural networks

PURPOSE

The developments of PET/CT and PET/MR scanners provide opportunities for improving PET image quality by using anatomical information. In this paper, we propose a novel co-learning three-dimensional (3D) convolutional neural network (CNN) to extract modality-specific features from PET/CT image pairs and integrate complementary features into an iterative reconstruction framework to improve PET image reconstruction.

METHODS

We used a pretrained deep neural network to represent PET images. The network was trained using low-count PET and CT image pairs as inputs and high-count PET images as labels. This network was then incorporated into a constrained maximum likelihood framework to regularize PET image reconstruction. Two different network structures were investigated for the integration of anatomical information from CT images. One was a multichannel CNN, which treated PET and CT volumes as separate channels of the input. The other one was multibranch CNN, which implemented separate encoders for PET and CT images to extract latent features and fed the combined latent features into a decoder. Using computer-based Monte Carlo simulations and two real patient datasets, the proposed method has been compared with existing methods, including the maximum likelihood expectation maximization (MLEM) reconstruction, a kernel-based reconstruction and a CNN-based deep penalty method with and without anatomical guidance.

RESULTS

Reconstructed images showed that the proposed constrained ML reconstruction approach produced higher quality images than the competing methods. The tumors in the lung region have higher contrast in the proposed constrained ML reconstruction than in the CNN-based deep penalty reconstruction. The image quality was further improved by incorporating the anatomical information. Moreover, the liver standard deviation was lower in the proposed approach than all the competing methods at a matched lesion contrast.

CONCLUSIONS

The supervised co-learning strategy can improve the performance of constrained maximum likelihood reconstruction. Compared with existing techniques, the proposed method produced a better lesion contrast versus background standard deviation trade-off curve, which can potentially improve lesion detection.

Applications of artificial intelligence in nuclear medicine image generation

Recently, the application of artificial intelligence (AI) in medical imaging (including nuclear medicine imaging) has rapidly developed. Most AI applications in nuclear medicine imaging have focused on the diagnosis, treatment monitoring, and correlation analyses with pathology or specific gene mutation. It can also be used for image generation to shorten the time of image acquisition, reduce the dose of injected tracer, and enhance image quality. This work provides an overview of the application of AI in image generation for single-photon emission computed tomography (SPECT) and positron emission tomography (PET) either without or with anatomical information [CT or magnetic resonance imaging (MRI)]. This review focused on four aspects, including imaging physics, image reconstruction, image postprocessing, and internal dosimetry. AI application in generating attenuation map, estimating scatter events, boosting image quality, and predicting internal dose map is summarized and discussed.

Improved low-dose positron emission tomography image reconstruction using deep learned prior

Positron emission tomography (PET) is a promising medical imaging technology that provides non-invasive and quantitative measurement of biochemical process in the human bodies. PET image reconstruction is challenging due to the ill-poseness of the inverse problem. With lower statistics caused by the limited detected photons, low-dose PET imaging leads to noisy reconstructed images with much quality degradation. Recently, deep neural networks (DNN) have been widely used in computer vision tasks and attracted growing interests in medical imaging. In this paper, we proposed a maximuma posteriori(MAP) reconstruction algorithm incorporating a convolutional neural network (CNN) representation in the formation of the prior. Rather than using the CNN in post-processing, we embedded the neural network in the reconstruction framework for image representation. Using the simulated data, we first quantitatively evaluated our proposed method in terms of the noise-bias tradeoff, and compared with the filtered maximum likelihood (ML), the conventional MAP, and the CNN post-processing methods. In addition to the simulation experiments, the proposed method was further quantitatively validated on the acquired patient brain and body data with the tradeoff between noise and contrast. The results demonstrated that the proposed CNN-MAP method improved noise-bias tradeoff compared with the filtered ML, the conventional MAP, and the CNN post-processing methods in the simulation study. For the patient study, the CNN-MAP method achieved better noise-contrast tradeoff over the other three methods. The quantitative enhancements indicate the potential value of the proposed CNN-MAP method in low-dose PET imaging.

3D/4D Reconstruction and Quantitative Total Body Imaging

Total-body PET image reconstruction follows a similar procedure to the image reconstruction process for standard whole-body PET scanners. One unique aspect of total-body imaging is simultaneous coverage of the entire human body, which makes it convenient to perform total-body dynamic PET scans. Therefore, four-dimensional dynamic PET reconstruction and parametric imaging are of great interest in total-body imaging. This article covers some basics of PET image reconstruction and then focuses on three- and four-dimensional PET reconstruction for total-body imaging. Methods for image formation from raw measurements in total-body PET are described. Challenges and opportunities in total-body PET image reconstruction are discussed.

Advanced Machine Learning-Based Analytics on COVID-19 Data Using Generative Adversarial Networks

The domain of medical diagnosis and predictive analytics is one of the key domains of research with enormous dimensions whereby the diseases of different types can be predicted. Nowadays, there is a huge panic of impact and rapid mutation of the COVID-19 virus impression. The world is getting affected by this virus to a huge extent and there is no vaccine developed so far. India is also having more than 10,000 patients with than 300 deceased. The global human community is having around 20 lacs of Coronavirus patients. The Generative Adversarial Network (GAN) is the contemporary high-performance approach in which the use of advanced neural networks is done for the cavernous analytics of the images and multimedia data. In this research work, the analytics of key points from medical images of the COVID-19 dataset is to be presented using which the diagnosis and predictions can be done for the patients. The GANs are used for the generation, transformation as well as presentation of the dataset and key points using advanced deep learning models which can analyze the patterns in the medical images including X-Ray, CT Scan, and many others. Using such approaches with the integration of GANs, the overall predictive analytics can be made high performance aware as compared to the classical neural networks with multiple layers. In this research manuscript, the inscription of work is projected on the benchmark datasets with the advanced scripting so that the predictive mining and knowledge discovery can be done effectively with more accuracy.