Joint correction of attenuation and scatter in image space using deep convolutional neural networks for dedicated brain 18F-FDG PET

Contrast-enhanced to non-contrast-enhanced image translation to exploit a clinical data warehouse of T1-weighted brain MRI

BACKGROUND

Clinical data warehouses provide access to massive amounts of medical images, but these images are often heterogeneous. They can for instance include images acquired both with or without the injection of a gadolinium-based contrast agent. Harmonizing such data sets is thus fundamental to guarantee unbiased results, for example when performing differential diagnosis. Furthermore, classical neuroimaging software tools for feature extraction are typically applied only to images without gadolinium. The objective of this work is to evaluate how image translation can be useful to exploit a highly heterogeneous data set containing both contrast-enhanced and non-contrast-enhanced images from a clinical data warehouse.

METHODS

We propose and compare different 3D U-Net and conditional GAN models to convert contrast-enhanced T1-weighted (T1ce) into non-contrast-enhanced (T1nce) brain MRI. These models were trained using 230 image pairs and tested on 77 image pairs from the clinical data warehouse of the Greater Paris area.

RESULTS

Validation using standard image similarity measures demonstrated that the similarity between real and synthetic T1nce images was higher than between real T1nce and T1ce images for all the models compared. The best performing models were further validated on a segmentation task. We showed that tissue volumes extracted from synthetic T1nce images were closer to those of real T1nce images than volumes extracted from T1ce images.

CONCLUSION

We showed that deep learning models initially developed with research quality data could synthesize T1nce from T1ce images of clinical quality and that reliable features could be extracted from the synthetic images, thus demonstrating the ability of such methods to help exploit a data set coming from a clinical data warehouse.

Attenuation correction and truncation completion for breast PET/MR imaging using deep learning

Objective. Simultaneous PET/MR scanners combine the high sensitivity of MR imaging with the functional imaging of PET. However, attenuation correction of breast PET/MR imaging is technically challenging. The purpose of this study is to establish a robust attenuation correction algorithm for breast PET/MR images that relies on deep learning (DL) to recreate the missing portions of the patient's anatomy (truncation completion), as well as to provide bone information for attenuation correction from only the PET data.Approach. Data acquired from 23 female subjects with invasive breast cancer scanned with18F-fluorodeoxyglucose PET/CT and PET/MR localized to the breast region were used for this study. Three DL models, U-Net with mean absolute error loss (DLMAE) model, U-Net with mean squared error loss (DLMSE) model, and U-Net with perceptual loss (DLPerceptual) model, were trained to predict synthetic CT images (sCT) for PET attenuation correction (AC) given non-attenuation corrected (NAC) PETPET/MRimages as inputs. The DL and Dixon-based sCT reconstructed PET images were compared against those reconstructed from CT images by calculating the percent error of the standardized uptake value (SUV) and conducting Wilcoxon signed rank statistical tests.Main results. sCT images from the DLMAEmodel, the DLMSEmodel, and the DLPerceptualmodel were similar in mean absolute error (MAE), peak-signal-to-noise ratio, and normalized cross-correlation. No significant difference in SUV was found between the PET images reconstructed using the DLMSEand DLPerceptualsCTs compared to the reference CT for AC in all tissue regions. All DL methods performed better than the Dixon-based method according to SUV analysis.Significance. A 3D U-Net with MSE or perceptual loss model can be implemented into a reconstruction workflow, and the derived sCT images allow successful truncation completion and attenuation correction for breast PET/MR images.

Strategies for deep learning-based attenuation and scatter correction of brain 18 F-FDG PET images in the image domain

BACKGROUND

Attenuation and scatter correction is crucial for quantitative positron emission tomography (PET) imaging. Direct attenuation correction (AC) in the image domain using deep learning approaches has been recently proposed for combined PET/MR and standalone PET modalities lacking transmission scanning devices or anatomical imaging.

PURPOSE

In this study, different input settings were considered in the model training to investigate deep learning-based AC in the image space.

METHODS

Three different deep learning methods were developed for direct AC in the image space: (i) use of non-attenuation-corrected PET images as input (NonAC-PET), (ii) use of attenuation-corrected PET images with a simple two-class AC map (composed of soft-tissue and background air) obtained from NonAC-PET images (PET segmentation-based AC [SegAC-PET]), and (iii) use of both NonAC-PET and SegAC-PET images in a Double-Channel fashion to predict ground truth attenuation corrected PET images with Computed Tomography images (CTAC-PET). Since a simple two-class AC map (generated from NonAC-PET images) can easily be generated, this work assessed the added value of incorporating SegAC-PET images into direct AC in the image space. A 4-fold cross-validation scheme was adopted to train and evaluate the different models based using 80 brain 18 F-Fluorodeoxyglucose PET/CT images. The voxel-wise and region-wise accuracy of the models were examined via measuring the standardized uptake value (SUV) quantification bias in different regions of the brain.

RESULTS

The overall root mean square error (RMSE) for the Double-Channel setting was 0.157 ± 0.08 SUV in the whole brain region, while RMSEs of 0.214 ± 0.07 and 0.189 ± 0.14 SUV were observed in NonAC-PET and SegAC-PET models, respectively. A mean SUV bias of 0.01 ± 0.26% was achieved by the Double-Channel model regarding the activity concentration in cerebellum region, as opposed to 0.08 ± 0.28% and 0.05 ± 0.28% SUV biases for the network that uniquely used NonAC-PET or SegAC-PET as input, respectively. SegAC-PET images with an SUV bias of -1.15 ± 0.54%, served as a benchmark for clinically accepted errors. In general, the Double-Channel network, relying on both SegAC-PET and NonAC-PET images, outperformed the other AC models.

CONCLUSION

Since the generation of two-class AC maps from non-AC PET images is straightforward, the current study investigated the potential added value of incorporating SegAC-PET images into a deep learning-based direct AC approach. Altogether, compared with models that use only NonAC-PET and SegAC-PET images, the Double-Channel deep learning network exhibited superior attenuation correction accuracy.

Applications of machine learning and deep learning in SPECT and PET imaging: General overview, challenges and future prospects

The integration of positron emission tomography (PET) and single-photon emission computed tomography (SPECT) imaging techniques with machine learning (ML) algorithms, including deep learning (DL) models, is a promising approach. This integration enhances the precision and efficiency of current diagnostic and treatment strategies while offering invaluable insights into disease mechanisms. In this comprehensive review, we delve into the transformative impact of ML and DL in this domain. Firstly, a brief analysis is provided of how these algorithms have evolved and which are the most widely applied in this domain. Their different potential applications in nuclear imaging are then discussed, such as optimization of image adquisition or reconstruction, biomarkers identification, multimodal fusion and the development of diagnostic, prognostic, and disease progression evaluation systems. This is because they are able to analyse complex patterns and relationships within imaging data, as well as extracting quantitative and objective measures. Furthermore, we discuss the challenges in implementation, such as data standardization and limited sample sizes, and explore the clinical opportunities and future horizons, including data augmentation and explainable AI. Together, these factors are propelling the continuous advancement of more robust, transparent, and reliable systems.

Deep-learning-based methods of attenuation correction for SPECT and PET

Attenuation correction (AC) is essential for quantitative analysis and clinical diagnosis of single-photon emission computed tomography (SPECT) and positron emission tomography (PET). In clinical practice, computed tomography (CT) is utilized to generate attenuation maps (μ-maps) for AC of hybrid SPECT/CT and PET/CT scanners. However, CT-based AC methods frequently produce artifacts due to CT artifacts and misregistration of SPECT-CT and PET-CT scans. Segmentation-based AC methods using magnetic resonance imaging (MRI) for PET/MRI scanners are inaccurate and complicated since MRI does not contain direct information of photon attenuation. Computational AC methods for SPECT and PET estimate attenuation coefficients directly from raw emission data, but suffer from low accuracy, cross-talk artifacts, high computational complexity, and high noise level. The recently evolving deep-learning-based methods have shown promising results in AC of SPECT and PET, which can be generally divided into two categories: indirect and direct strategies. Indirect AC strategies apply neural networks to transform emission, transmission, or MR images into synthetic μ-maps or CT images which are then incorporated into AC reconstruction. Direct AC strategies skip the intermediate steps of generating μ-maps or CT images and predict AC SPECT or PET images from non-attenuation-correction (NAC) SPECT or PET images directly. These deep-learning-based AC methods show comparable and even superior performance to non-deep-learning methods. In this article, we first discussed the principles and limitations of non-deep-learning AC methods, and then reviewed the status and prospects of deep-learning-based methods for AC of SPECT and PET.

Objective Image Quality Comparison Between Brain-Dedicated PET and PET/CT Scanners

As part of a clinical validation of a new brain-dedicated PET system (CMB), image quality of this scanner has been compared to that of a whole-body PET/CT scanner. To that goal, Hoffman phantom and patient data were obtined with both devices. Since CMB does not use a CT for attenuation correction (AC) which is crucial for PET images quality, this study includes the evaluation of CMB PET images using emission-based or CT-based attenuation maps. PET images were compared using 34 image quality metrics. Moreover, a neural network was used to evaluate the degree of agreement between both devices on the patients diagnosis prediction. Overall, results showed that CMB images have higher contrast and recovery coefficient but higher noise than PET/CT images. Although SUVr values presented statistically significant differences in many brain regions, relative differences were low. An asymmetry between left and right hemispheres, however, was identified. Even so, the variations between the two devices were minor. Finally, there is a greater similarity between PET/CT and CMB CT-based AC PET images than between PET/CT and the CMB emission-based AC PET images.

Quantitative Assessment of Myocardial Ischemia With Positron Emission Tomography

Recent advances in positron emission tomography (PET) technology and reconstruction techniques have now made quantitative assessment using cardiac PET readily available in most cardiac PET imaging centers. Multiple PET myocardial perfusion imaging (MPI) radiopharmaceuticals are available for quantitative examination of myocardial ischemia, with each having distinct convenience and accuracy profile. Important properties of these radiopharmaceuticals ( 15 O-water, 13 N-ammonia, 82 Rb, 11 C-acetate, and 18 F-flurpiridaz) including radionuclide half-life, mean positron range in tissue, and the relationship between kinetic parameters and myocardial blood flow (MBF) are presented. Absolute quantification of MBF requires PET MPI to be performed with protocols that allow the generation of dynamic multiframes of reconstructed data. Using a tissue compartment model, the rate constant that governs the rate of PET MPI radiopharmaceutical extraction from the blood plasma to myocardial tissue is calculated. Then, this rate constant ( K1 ) is converted to MBF using an established extraction formula for each radiopharmaceutical. As most of the modern PET scanners acquire the data only in list mode, techniques of processing the list-mode data into dynamic multiframes are also reviewed. Finally, the impact of modern PET technologies such as PET/CT, PET/MR, total-body PET, machine learning/deep learning on comprehensive and quantitative assessment of myocardial ischemia is briefly described in this review.

Synthetic Attenuation Correction Maps for SPECT Imaging Using Deep Learning: A Study on Myocardial Perfusion Imaging

(1) Background: The CT-based attenuation correction of SPECT images is essential for obtaining accurate quantitative images in cardiovascular imaging. However, there are still many SPECT cameras without associated CT scanners throughout the world, especially in developing countries. Performing additional CT scans implies troublesome planning logistics and larger radiation doses for patients, making it a suboptimal solution. Deep learning (DL) offers a revolutionary way to generate complementary images for individual patients at a large scale. Hence, we aimed to generate linear attenuation coefficient maps from SPECT emission images reconstructed without attenuation correction using deep learning. (2) Methods: A total of 384 SPECT myocardial perfusion studies that used ^99mTc-sestamibi were included. A DL model based on a 2D U-Net architecture was trained using information from 312 patients. The quality of the generated synthetic attenuation correction maps (ACMs) and reconstructed emission values were evaluated using three metrics and compared to standard-of-care data using Bland-Altman plots. Finally, a quantitative evaluation of myocardial uptake was performed, followed by a semi-quantitative evaluation of myocardial perfusion. (3) Results: In a test set of 66 test patients, the ACM quality metrics were MSSIM = 0.97 ± 0.001 and NMAE = 3.08 ± 1.26 (%), and the reconstructed emission quality metrics were MSSIM = 0.99 ± 0.003 and NMAE = 0.23 ± 0.13 (%). The 95% limits of agreement (LoAs) at the voxel level for reconstructed SPECT images were: [-9.04; 9.00]%, and for the segment level, they were [-11; 10]%. The 95% LoAs for the Summed Stress Score values between the images reconstructed were [-2.8, 3.0]. When global perfusion scores were assessed, only 2 out of 66 patients showed changes in perfusion categories. (4) Conclusion: Deep learning can generate accurate attenuation correction maps from non-attenuation-corrected cardiac SPECT images. These high-quality attenuation maps are suitable for attenuation correction in myocardial perfusion SPECT imaging and could obviate the need for additional imaging in standalone SPECT scanners.

PET scatter estimation using deep learning U-Net architecture

Objective.Positron emission tomography (PET) image reconstruction needs to be corrected for scatter in order to produce quantitatively accurate images. Scatter correction is traditionally achieved by incorporating an estimated scatter sinogram into the forward model during image reconstruction. Existing scatter estimated methods compromise between accuracy and computing time. Nowadays scatter estimation is routinely performed using single scatter simulation (SSS), which does not accurately model multiple scatter and scatter from outside the field-of-view, leading to reduced qualitative and quantitative PET reconstructed image accuracy. On the other side, Monte-Carlo (MC) methods provide a high precision, but are computationally expensive and time-consuming, even with recent progress in MC acceleration.Approach.In this work we explore the potential of deep learning (DL) for accurate scatter correction in PET imaging, accounting for all scatter coincidences. We propose a network based on a U-Net convolutional neural network architecture with 5 convolutional layers. The network takes as input the emission and computed tomography (CT)-derived attenuation factor (AF) sinograms and returns the estimated scatter sinogram. The network training was performed using MC simulated PET datasets. Multiple anthropomorphic extended cardiac-torso phantoms of two different regions (lung and pelvis) were created, considering three different body sizes and different levels of statistics. In addition, two patient datasets were used to assess the performance of the method in clinical practice.Main results.Our experiments showed that the accuracy of our method, namely DL-based scatter estimation (DLSE), was independent of the anatomical region (lungs or pelvis). They also showed that the DLSE-corrected images were similar to that reconstructed from scatter-free data and more accurate than SSS-corrected images.Significance.The proposed method is able to estimate scatter sinograms from emission and attenuation data. It has shown a better accuracy than the SSS, while being faster than MC scatter estimation methods.

The performance of machine learning approaches for attenuation correction of PET in neuroimaging: A meta-analysis

PURPOSE

This systematic review provides a consensus on the clinical feasibility of machine learning (ML) methods for brain PET attenuation correction (AC). Performance of ML-AC were compared to clinical standards.

METHODS

Two hundred and eighty studies were identified through electronic searches of brain PET studies published between January 1, 2008, and August 1, 2022. Reported outcomes for image quality, tissue classification performance, regional and global bias were extracted to evaluate ML-AC performance. Methodological quality of included studies and the quality of evidence of analysed outcomes were assessed using QUADAS-2 and GRADE, respectively.

RESULTS

A total of 19 studies (2371 participants) met the inclusion criteria. Overall, the global bias of ML methods was 0.76 ± 1.2%. For image quality, the relative mean square error (RMSE) was 0.20 ± 0.4 while for tissues classification, the Dice similarity coefficient (DSC) for bone/soft tissue/air were 0.82 ± 0.1 / 0.95 ± 0.03 / 0.85 ± 0.14.

CONCLUSIONS

In general, ML-AC performance is within acceptable limits for clinical PET imaging. The sparse information on ML-AC robustness and its limited qualitative clinical evaluation may hinder clinical implementation in neuroimaging, especially for PET/MRI or emerging brain PET systems where standard AC approaches are not readily available.

DeTransUnet: attenuation correction of gated cardiac images without structural information

Objective. Myocardial perfusion imaging (MPI) with positron emission tomography (PET) is a non-invasive imaging method, and it is of great significance to the diagnosis and prognosis of coronary heart disease. Attenuation correction (AC) for PET images is a necessary step for further quantitative analysis. In order not to use magnetic resonance (MR) or computed tomography (CT) images for AC, this work proposes DeTransUnet to obtain AC PET images directly from no-attenuation corrected (NAC) PET images. Approach. The proposed DeTransUnet is a 3D structure which combines the multi-scale deformable transformer layers and the 3D convolutional neural network (CNN). And it integrates the advantages of transformer with long-range dependence and CNN suitable for image calculation. The AC images using CT images for AC and scatter correction (SC) and are considered as training labels, while the NAC images are reconstructed without AC and SC. Standard uptake value (SUV) values are calculated for both NAC and AC images to exclude the influence of weight and injection dose. With NAC SUV images as the inputs of the DeTransUnet, the outputs of DeTransUnet are AC SUV images. Main results. The proposed DeTransUnet was performed on an MPI gated-PET dataset, and the results were compared with Unet2D and Unet2.5D. The metrics of the whole image and the left ventricular myocardium show that the proposed method has advantages over other deep learning methods. Significance. The proposed DeTransUnet is a novel AC framework that does not require CT or MR images. It can be used as an independent AC method on PET/MR instrument. In addition, when CT images contain defects or cannot be registered with PET images on PET/CT instrument, DeTransUnet is able to repair the defects and keep consistent with the NAC images.

Application of artificial intelligence in nuclear medicine and molecular imaging: a review of current status and future perspectives for clinical translation

Artificial intelligence (AI) will change the face of nuclear medicine and molecular imaging as it will in everyday life. In this review, we focus on the potential applications of AI in the field, both from a physical (radiomics, underlying statistics, image reconstruction and data analysis) and a clinical (neurology, cardiology, oncology) perspective. Challenges for transferability from research to clinical practice are being discussed as is the concept of explainable AI. Finally, we focus on the fields where challenges should be set out to introduce AI in the field of nuclear medicine and molecular imaging in a reliable manner.

Attenuation Coefficient Estimation for PET/MRI With Bayesian Deep Learning Pseudo-CT and Maximum-Likelihood Estimation of Activity and Attenuation

A major remaining challenge for magnetic resonance-based attenuation correction methods (MRAC) is their susceptibility to sources of magnetic resonance imaging (MRI) artifacts (e.g., implants and motion) and uncertainties due to the limitations of MRI contrast (e.g., accurate bone delineation and density, and separation of air/bone). We propose using a Bayesian deep convolutional neural network that in addition to generating an initial pseudo-CT from MR data, it also produces uncertainty estimates of the pseudo-CT to quantify the limitations of the MR data. These outputs are combined with the maximum-likelihood estimation of activity and attenuation (MLAA) reconstruction that uses the PET emission data to improve the attenuation maps. With the proposed approach uncertainty estimation and pseudo-CT prior for robust MLAA (UpCT-MLAA), we demonstrate accurate estimation of PET uptake in pelvic lesions and show recovery of metal implants. In patients without implants, UpCT-MLAA had acceptable but slightly higher root-mean-squared-error (RMSE) than Zero-echotime and Dixon Deep pseudo-CT when compared to CTAC. In patients with metal implants, MLAA recovered the metal implant; however, anatomy outside the implant region was obscured by noise and crosstalk artifacts. Attenuation coefficients from the pseudo-CT from Dixon MRI were accurate in normal anatomy; however, the metal implant region was estimated to have attenuation coefficients of air. UpCT-MLAA estimated attenuation coefficients of metal implants alongside accurate anatomic depiction outside of implant regions.

Robust-Deep: A Method for Increasing Brain Imaging Datasets to Improve Deep Learning Models' Performance and Robustness

A small dataset commonly affects generalization, robustness, and overall performance of deep neural networks (DNNs) in medical imaging research. Since gathering large clinical databases is always difficult, we proposed an analytical method for producing a large realistic/diverse dataset. Clinical brain PET/CT/MR images including full-dose (FD), low-dose (LD) corresponding to only 5 % of events acquired in the FD scan, non-attenuated correction (NAC) and CT-based measured attenuation correction (MAC) PET images, CT images and T1 and T2 MR sequences of 35 patients were included. All images were registered to the Montreal Neurological Institute (MNI) template. Laplacian blending was used to make a natural presentation using information in the frequency domain of images from two separate patients, as well as the blending mask. This classical technique from the computer vision and image processing communities is still widely used and unlike modern DNNs, does not require the availability of training data. A modified ResNet DNN was implemented to evaluate four image-to-image translation tasks, including LD to FD, LD+MR to FD, NAC to MAC, and MRI to CT, with and without using the synthesized images. Quantitative analysis using established metrics, including the peak signal-to-noise ratio (PSNR), structural similarity index metric (SSIM), and joint histogram analysis was performed for quantitative evaluation. The quantitative comparison between the registered small dataset containing 35 patients and the large dataset containing 350 synthesized plus 35 real dataset demonstrated improvement of the RMSE and SSIM by 29% and 8% for LD to FD, 40% and 7% for LD+MRI to FD, 16% and 8% for NAC to MAC, and 24% and 11% for MRI to CT mapping task, respectively. The qualitative/quantitative analysis demonstrated that the proposed model improved the performance of all four DNN models through producing images of higher quality and lower quantitative bias and variance compared to reference images.

A review on AI in PET imaging

Artificial intelligence (AI) has been applied to various medical imaging tasks, such as computer-aided diagnosis. Specifically, deep learning techniques such as convolutional neural network (CNN) and generative adversarial network (GAN) have been extensively used for medical image generation. Image generation with deep learning has been investigated in studies using positron emission tomography (PET). This article reviews studies that applied deep learning techniques for image generation on PET. We categorized the studies for PET image generation with deep learning into three themes as follows: (1) recovering full PET data from noisy data by denoising with deep learning, (2) PET image reconstruction and attenuation correction with deep learning and (3) PET image translation and synthesis with deep learning. We introduce recent studies based on these three categories. Finally, we mention the limitations of applying deep learning techniques to PET image generation and future prospects for PET image generation.

Artificial intelligence with deep learning in nuclear medicine and radiology

The use of deep learning in medical imaging has increased rapidly over the past few years, finding applications throughout the entire radiology pipeline, from improved scanner performance to automatic disease detection and diagnosis. These advancements have resulted in a wide variety of deep learning approaches being developed, solving unique challenges for various imaging modalities. This paper provides a review on these developments from a technical point of view, categorizing the different methodologies and summarizing their implementation. We provide an introduction to the design of neural networks and their training procedure, after which we take an extended look at their uses in medical imaging. We cover the different sections of the radiology pipeline, highlighting some influential works and discussing the merits and limitations of deep learning approaches compared to other traditional methods. As such, this review is intended to provide a broad yet concise overview for the interested reader, facilitating adoption and interdisciplinary research of deep learning in the field of medical imaging.

Comparison of deep learning-based emission-only attenuation correction methods for positron emission tomography

PURPOSE

This study aims to compare two approaches using only emission PET data and a convolution neural network (CNN) to correct the attenuation (μ) of the annihilation photons in PET.

METHODS

One of the approaches uses a CNN to generate μ-maps from the non-attenuation-corrected (NAC) PET images (μ-CNN_NAC). In the other method, CNN is used to improve the accuracy of μ-maps generated using maximum likelihood estimation of activity and attenuation (MLAA) reconstruction (μ-CNN_MLAA). We investigated the improvement in the CNN performance by combining the two methods (μ-CNN_MLAA+NAC) and the suitability of μ-CNN_NAC for providing the scatter distribution required for MLAA reconstruction. Image data from ¹⁸F-FDG (n = 100) or ⁶⁸ Ga-DOTATOC (n = 50) PET/CT scans were used for neural network training and testing.

RESULTS

The error of the attenuation correction factors estimated using μ-CT and μ-CNN_NAC was over 7%, but that of scatter estimates was only 2.5%, indicating the validity of the scatter estimation from μ-CNN_NAC. However, CNN_NAC provided less accurate bone structures in the μ-maps, while the best results in recovering the fine bone structures were obtained by applying CNN_MLAA+NAC. Additionally, the μ-values in the lungs were overestimated by CNN_NAC. Activity images (λ) corrected for attenuation using μ-CNN_MLAA and μ-CNN_MLAA+NAC were superior to those corrected using μ-CNN_NAC, in terms of their similarity to λ-CT. However, the improvement in the similarity with λ-CT by combining the CNN_NAC and CNN_MLAA approaches was insignificant (percent error for lung cancer lesions, λ-CNN_NAC = 5.45% ± 7.88%; λ-CNN_MLAA = 1.21% ± 5.74%; λ-CNN_MLAA+NAC = 1.91% ± 4.78%; percent error for bone cancer lesions, λ-CNN_NAC = 1.37% ± 5.16%; λ-CNN_MLAA = 0.23% ± 3.81%; λ-CNN_MLAA+NAC = 0.05% ± 3.49%).

CONCLUSION

The use of CNN_NAC was feasible for scatter estimation to address the chicken-egg dilemma in MLAA reconstruction, but CNN_MLAA outperformed CNN_NAC.

Artificial Intelligence and Cardiac PET/Computed Tomography Imaging

Artificial intelligence is an important technology, with rapidly expanding applications for cardiac PET. We review the common terminology, including methods for training and testing, which are fundamental to understanding artificial intelligence. Next, we highlight applications to improve image acquisition, reconstruction, and segmentation. Computed tomographic imaging is commonly acquired in conjunction with PET and various artificial intelligence methods have been applied, including methods to automatically extract anatomic information or generate synthetic attenuation images. Last, we describe methods to automate disease diagnosis or risk stratification. This summary highlights the current and future clinical applications of artificial intelligence to cardiovascular PET imaging.

DeepTOFSino: A deep learning model for synthesizing full-dose time-of-flight bin sinograms from their corresponding low-dose sinograms

PURPOSE

Reducing the injected activity and/or the scanning time is a desirable goal to minimize radiation exposure and maximize patients' comfort. To achieve this goal, we developed a deep neural network (DNN) model for synthesizing full-dose (FD) time-of-flight (TOF) bin sinograms from their corresponding fast/low-dose (LD) TOF bin sinograms.

METHODS

Clinical brain PET/CT raw data of 140 normal and abnormal patients were employed to create LD and FD TOF bin sinograms. The LD TOF sinograms were created through 5% undersampling of FD list-mode PET data. The TOF sinograms were split into seven time bins (0, ±1, ±2, ±3). Residual network (ResNet) algorithms were trained separately to generate FD bins from LD bins. An extra ResNet model was trained to synthesize FD images from LD images to compare the performance of DNN in sinogram space (SS) vs implementation in image space (IS). Comprehensive quantitative and statistical analysis was performed to assess the performance of the proposed model using established quantitative metrics, including the peak signal-to-noise ratio (PSNR), structural similarity index metric (SSIM) region-wise standardized uptake value (SUV) bias and statistical analysis for 83 brain regions.

RESULTS

SSIM and PSNR values of 0.97 ± 0.01, 0.98 ± 0.01 and 33.70 ± 0.32, 39.36 ± 0.21 were obtained for IS and SS, respectively, compared to 0.86 ± 0.02and 31.12 ± 0.22 for reference LD images. The absolute average SUV bias was 0.96 ± 0.95% and 1.40 ± 0.72% for SS and IS implementations, respectively. The joint histogram analysis revealed the lowest mean square error (MSE) and highest correlation (R² = 0.99, MSE = 0.019) was achieved by SS compared to IS (R² = 0.97, MSE= 0.028). The Bland & Altman analysis showed that the lowest SUV bias (-0.4%) and minimum variance (95% CI: -2.6%, +1.9%) were achieved by SS images. The voxel-wise t-test analysis revealed the presence of voxels with statistically significantly lower values in LD, IS, and SS images compared to FD images respectively.

CONCLUSION

The results demonstrated that images reconstructed from the predicted TOF FD sinograms using the SS approach led to higher image quality and lower bias compared to images predicted from LD images.

Artificial Intelligence-Based Data Corrections for Attenuation and Scatter in Position Emission Tomography and Single-Photon Emission Computed Tomography

Recent developments in artificial intelligence (AI) technology have enabled new developments that can improve attenuation and scatter correction in PET and single-photon emission computed tomography (SPECT). These technologies will enable the use of accurate and quantitative imaging without the need to acquire a computed tomography image, greatly expanding the capability of PET/MR imaging, PET-only, and SPECT-only scanners. The use of AI to aid in scatter correction will lead to improvements in image reconstruction speed, and improve patient throughput. This article outlines the use of these new tools, surveys contemporary implementation, and discusses their limitations.

Convolutional neural network based attenuation correction for123I-FP-CIT SPECT with focused striatum imaging

SPECT imaging with¹²³I-FP-CIT is used for diagnosis of neurodegenerative disorders like Parkinson's disease. Attenuation correction (AC) can be useful for quantitative analysis of¹²³I-FP-CIT SPECT. Ideally, AC would be performed based on attenuation maps (μ-maps) derived from perfectly registered CT scans. Suchμ-maps, however, are most times not available and possible errors in image registration can induce quantitative inaccuracies in AC corrected SPECT images. Earlier, we showed that a convolutional neural network (CNN) based approach allows to estimate SPECT-alignedμ-maps for full brain perfusion imaging using only emission data. Here we investigate the feasibility of similar CNN methods for axially focused¹²³I-FP-CIT scans. We tested our approach on a high-resolution multi-pinhole prototype clinical SPECT system in a Monte Carlo simulation study. Three CNNs that estimateμ-maps in a voxel-wise, patch-wise and image-wise manner were investigated. As the added value of AC on clinical¹²³I-FP-CIT scans is still debatable, the impact of AC was also reported to check in which cases CNN based AC could be beneficial. AC using the ground truthμ-maps (GT-AC) and CNN estimatedμ-maps (CNN-AC) were compared with the case when no AC was done (No-AC). Results show that the effect of using GT-AC versus CNN-AC or No-AC on striatal shape and symmetry is minimal. Specific binding ratios (SBRs) from localized regions show a deviation from GT-AC≤2.5% for all three CNN-ACs while No-AC systematically underestimates SBRs by 13.1%. A strong correlation (r≥0.99) was obtained between GT-AC based SBRs and SBRs from CNN-ACs and No-AC. Absolute quantification (in kBq ml^-1) shows a deviation from GT-AC within 2.2% for all three CNN-ACs and of 71.7% for No-AC. To conclude, all three CNNs show comparable performance in accurateμ-map estimation and¹²³I-FP-CIT quantification. CNN-estimatedμ-map can be a promising substitute for CT-basedμ-map.

MRI-guided attenuation correction in torso PET/MRI: Assessment of segmentation-, atlas-, and deep learning-based approaches in the presence of outliers

Purpose

We compare the performance of three commonly used MRI‐guided attenuation correction approaches in torso PET/MRI, namely segmentation‐, atlas‐, and deep learning‐based algorithms.

Methods

Twenty‐five co‐registered torso ¹⁸F‐FDG PET/CT and PET/MR images were enrolled. PET attenuation maps were generated from in‐phase Dixon MRI using a three‐tissue class segmentation‐based approach (soft‐tissue, lung, and background air), voxel‐wise weighting atlas‐based approach, and a residual convolutional neural network. The bias in standardized uptake value (SUV) was calculated for each approach considering CT‐based attenuation corrected PET images as reference. In addition to the overall performance assessment of these approaches, the primary focus of this work was on recognizing the origins of potential outliers, notably body truncation, metal‐artifacts, abnormal anatomy, and small malignant lesions in the lungs.

Results

The deep learning approach outperformed both atlas‐ and segmentation‐based methods resulting in less than 4% SUV bias across 25 patients compared to the segmentation‐based method with up to 20% SUV bias in bony structures and the atlas‐based method with 9% bias in the lung. The deep learning‐based method exhibited superior performance. Yet, in case of sever truncation and metallic‐artifacts in the input MRI, this approach was outperformed by the atlas‐based method, exhibiting suboptimal performance in the affected regions. Conversely, for abnormal anatomies, such as a patient presenting with one lung or small malignant lesion in the lung, the deep learning algorithm exhibited promising performance compared to other methods.

Conclusion

The deep learning‐based method provides promising outcome for synthetic CT generation from MRI. However, metal‐artifact and body truncation should be specifically addressed.

Feasibility of Deep Learning-Guided Attenuation and Scatter Correction of Whole-Body 68Ga-PSMA PET Studies in the Image Domain

OBJECTIVE

This study evaluates the feasibility of direct scatter and attenuation correction of whole-body 68Ga-PSMA PET images in the image domain using deep learning.

METHODS

Whole-body 68Ga-PSMA PET images of 399 subjects were used to train a residual deep learning model, taking PET non-attenuation-corrected images (PET-nonAC) as input and CT-based attenuation-corrected PET images (PET-CTAC) as target (reference). Forty-six whole-body 68Ga-PSMA PET images were used as an independent validation dataset. For validation, synthetic deep learning-based attenuation-corrected PET images were assessed considering the corresponding PET-CTAC images as reference. The evaluation metrics included the mean absolute error (MAE) of the SUV, peak signal-to-noise ratio, and structural similarity index (SSIM) in the whole body, as well as in different regions of the body, namely, head and neck, chest, and abdomen and pelvis.

RESULTS

The deep learning-guided direct attenuation and scatter correction produced images of comparable visual quality to PET-CTAC images. It achieved an MAE, relative error (RE%), SSIM, and peak signal-to-noise ratio of 0.91 ± 0.29 (SUV), -2.46% ± 10.10%, 0.973 ± 0.034, and 48.171 ± 2.964, respectively, within whole-body images of the independent external validation dataset. The largest RE% was observed in the head and neck region (-5.62% ± 11.73%), although this region exhibited the highest value of SSIM metric (0.982 ± 0.024). The MAE (SUV) and RE% within the different regions of the body were less than 2.0% and 6%, respectively, indicating acceptable performance of the deep learning model.

CONCLUSIONS

This work demonstrated the feasibility of direct attenuation and scatter correction of whole-body 68Ga-PSMA PET images in the image domain using deep learning with clinically tolerable errors. The technique has the potential of performing attenuation correction on stand-alone PET or PET/MRI systems.

Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks

Pathological examination is the gold standard for the diagnosis of cancer. Common pathological examinations include hematoxylin-eosin (H&E) staining and immunohistochemistry (IHC). In some cases, it is hard to make accurate diagnoses of cancer by referring only to H&E staining images. Whereas, the IHC examination can further provide enough evidence for the diagnosis process. Hence, the generation of virtual IHC images from H&E-stained images will be a good solution for current IHC examination hard accessibility issue, especially for some low-resource regions. However, existing approaches have limitations in microscopic structural preservation and the consistency of pathology properties. In addition, pixel-level paired data is hard available. In our work, we propose a novel adversarial learning method for effective Ki-67-stained image generation from corresponding H&E-stained image. Our method takes fully advantage of structural similarity constraint and skip connection to improve structural details preservation; and pathology consistency constraint and pathological representation network are first proposed to enforce the generated and source images hold the same pathological properties in different staining domains. We empirically demonstrate the effectiveness of our approach on two different unpaired histopathological datasets. Extensive experiments indicate the superior performance of our method that surpasses the state-of-the-art approaches by a significant margin. In addition, our approach also achieves a stable and good performance on unbalanced datasets, which shows our method has strong robustness. We believe that our method has significant potential in clinical virtual staining and advance the progress of computer-aided multi-staining histology image analysis.

A Path to Qualification of PET/MR Scanners for Multicenter Brain Imaging Studies: Evaluation of MR-based Attenuation Correction Methods Using a Patient Phantom

PET/MRI scanners cannot be qualified in the manner adopted for hybrid PET/CT devices. The main hurdle with qualification in PET/MRI is that attenuation correction (AC) cannot be adequately measured in conventional PET phantoms because of the difficulty in converting the MR images of the physical structures (e.g., plastic) into electron density maps. Over the last decade, a plethora of novel MRI-based algorithms has been developed to more accurately derive the attenuation properties of the human head, including the skull. Although promising, none of these techniques has yet emerged as an optimal and universally adopted strategy for AC in PET/MRI. In this work, we propose a path for PET/MRI qualification for multicenter brain imaging studies. Specifically, our solution is to separate the head AC from the other factors that affect PET data quantification and use a patient as a phantom to assess the former. The emission data collected on the integrated PET/MRI scanner to be qualified should be reconstructed using both MRI- and CT-based AC methods, and whole-brain qualitative and quantitative (both voxelwise and regional) analyses should be performed. The MRI-based approach will be considered satisfactory if the PET quantification bias is within the acceptance criteria specified here. We have implemented this approach successfully across 2 PET/MRI scanner manufacturers at 2 sites.

Assessment of deep learning-based PET attenuation correction frameworks in the sinogram domain

This study set out to investigate various deep learning frameworks for PET attenuation correction in the sinogram domain. Different models for both time-of-flight (TOF) and non-TOF PET emission data were implemented, including direct estimation of the attenuation corrected (AC) emission sinograms from the nonAC sinograms, estimation of the attenuation correction factors (ACFs) from PET emission data, correction of scattered photons prior to training of the models, and separate training of the models for each segment of the emission sinograms. A segmentation-based 2-class AC map was included as a bottom-line technique for comparison of the different models considering PET/CT AC as reference. Fifty clinical TOF PET/CT brain scans were employed for training whereas 20 were used for evaluation of the models. Quantitative analysis of the resulting PET images was carried out through region-wise standardized uptake value (SUV) bias calculation. The models relying on TOF information significantly outperformed the nonTOF models as well as the segmentation-based AC map resulting in maximum SUV bias of 6.5%, 9.5%, and 14.0%, respectively. Estimation of ACFs from either TOF or nonTOF PET emission data was very sensitive to prior scatter correction. However, direct estimation of AC sinograms from nonAC sinograms revealed no sensitivity to scatter correction, thus obviating the need for prior scatter estimation. For TOF PET data, though direct prediction of the AC sinograms does not require prior estimation of scattered photons, it requires input/output channels equal to the number of TOF bins which might be computationally or memory-wise expensive. Prediction of the ACF matrices from TOF emission data is less demanding in terms of memory as it requires only a single channel for output. AC in the sinogram domain of TOF PET data exhibited superior performance compared to both nonTOF and segmentation-based methods. However, such models require multiple input/output channels.

Development of attenuation correction methods using deep learning in brain-perfusion single-photon emission computed tomography

PURPOSE

Computed tomography (CT)-based attenuation correction (CTAC) in single-photon emission computed tomography (SPECT) is highly accurate, but it requires hybrid SPECT/CT instruments and additional radiation exposure. To obtain attenuation correction (AC) without the need for additional CT images, a deep learning method was used to generate pseudo-CT images has previously been reported, but it is limited because of cross-modality transformation, resulting in misalignment and modality-specific artifacts. This study aimed to develop a deep learning-based approach using non-attenuation-corrected (NAC) images and CTAC-based images for training to yield AC images in brain-perfusion SPECT. This study also investigated whether the proposed approach is superior to conventional Chang's AC (ChangAC).

METHODS

In total, 236 patients who underwent brain-perfusion SPECT were randomly divided into two groups: the training group (189 patients; 80%) and the test group (47 patients; 20%). Two models were constructed using Autoencoder (AutoencoderAC) and U-Net (U-NetAC), respectively. ChangAC, AutoencoderAC, and U-NetAC approaches were compared with CTAC using qualitative analysis (visual evaluation) and quantitative analysis (normalized mean squared error [NMSE] and the percentage error in each brain region). Statistical analyses were performed using the Wilcoxon signed-rank sum test and Bland-Altman analysis.

RESULTS

U-NetAC had the highest visual evaluation score. The NMSE results for the U-NetAC were the lowest, followed by AutoencoderAC and ChangAC (P < 0.001). Bland-Altman analysis showed a fixed bias for ChangAC and AutoencoderAC and a proportional bias for ChangAC. ChangAC underestimated counts by 30-40% in all brain regions. AutoencoderAC and U-NetAC produced mean errors of <1% and maximum errors of 3%, respectively.

CONCLUSION

New deep learning-based AC methods for AutoencoderAC and U-NetAC were developed. Their accuracy was higher than that obtained by ChangAC. U-NetAC exhibited higher qualitative and quantitative accuracy than AutoencoderAC. We generated highly accurate AC images directly from NAC images without the need for intermediate pseudo-CT images. To verify our models' generalizability, external validation is required.

Synthesizing PET/MR (T1-weighted) images from non-attenuation-corrected PET images

Positron emission tomography (PET) imaging can be used for early detection, diagnosis and postoperative patient monitoring of many diseases. Traditional PET imaging requires not only additional computed tomography (CT) imaging or magnetic resonance imaging (MR) to provide anatomical information but also attenuation correction (AC) map calculation based on CT images or MR images for accurate quantitative estimation. During a patient's treatment, PET/CT or PET/MR scans are inevitably repeated many times, leading to additional doses of ionizing radiation (CT scans) and additional economic and time costs (MR scans). To reduce adverse effects while obtaining high-quality PET/MR images in the course of a patient's treatment, especially in the stage of evaluating the effect of postoperative treatment, in this work, we propose a new method based on deep learning, which can directly obtain synthetic attenuation-corrected PET (sAC PET) and synthetic T1-weighted MR (sMR) images based only on non-attenuation-corrected PET (NAC PET) images. Our model, based on the Wasserstein generative adversarial network, first removes noise and artifacts from the NAC PET images to generate sAC PET images and then generates sMR images from the obtained sAC PET images. To evaluate the performance of this generative model, we evaluated it on paired PET/MR images from a total of eighty clinical patients. Based on qualitative and quantitative analysis, the generated sAC PET and sMR images showed a high degree of similarity to the real AC PET and real MR images. These results indicated that our proposed method can reduce the frequency of additional anatomical imaging scans during PET imaging and has great potential in improving doctors' clinical diagnosis efficiency, saving patients' economic expenditure and reducing the radiation risk brought by CT scanning.

Motion estimation and correction in SPECT, PET and CT

Patient motion impacts single photon emission computed tomography (SPECT), positron emission tomography (PET) and X-ray computed tomography (CT) by giving rise to projection data inconsistencies that can manifest as reconstruction artifacts, thereby degrading image quality and compromising accurate image interpretation and quantification. Methods to estimate and correct for patient motion in SPECT, PET and CT have attracted considerable research effort over several decades. The aims of this effort have been two-fold: to estimate relevant motion fields characterizing the various forms of voluntary and involuntary motion; and to apply these motion fields within a modified reconstruction framework to obtain motion-corrected images. The aims of this review are to outline the motion problem in medical imaging and to critically review published methods for estimating and correcting for the relevant motion fields in clinical and preclinical SPECT, PET and CT. Despite many similarities in how motion is handled between these modalities, utility and applications vary based on differences in temporal and spatial resolution. Technical feasibility has been demonstrated in each modality for both rigid and non-rigid motion, but clinical feasibility remains an important target. There is considerable scope for further developments in motion estimation and correction, and particularly in data-driven methods that will aid clinical utility. State-of-the-art machine learning methods may have a unique role to play in this context.

Applications of artificial intelligence in nuclear medicine image generation

Recently, the application of artificial intelligence (AI) in medical imaging (including nuclear medicine imaging) has rapidly developed. Most AI applications in nuclear medicine imaging have focused on the diagnosis, treatment monitoring, and correlation analyses with pathology or specific gene mutation. It can also be used for image generation to shorten the time of image acquisition, reduce the dose of injected tracer, and enhance image quality. This work provides an overview of the application of AI in image generation for single-photon emission computed tomography (SPECT) and positron emission tomography (PET) either without or with anatomical information [CT or magnetic resonance imaging (MRI)]. This review focused on four aspects, including imaging physics, image reconstruction, image postprocessing, and internal dosimetry. AI application in generating attenuation map, estimating scatter events, boosting image quality, and predicting internal dose map is summarized and discussed.

Deep learning-based attenuation correction for brain PET with various radiotracers

OBJECTIVES

Attenuation correction (AC) is crucial for ensuring the quantitative accuracy of positron emission tomography (PET) imaging. However, obtaining accurate μ-maps from brain-dedicated PET scanners without AC acquisition mechanism is challenging. Therefore, to overcome these problems, we developed a deep learning-based PET AC (deep AC) framework to synthesize transmission computed tomography (TCT) images from non-AC (NAC) PET images using a convolutional neural network (CNN) with a huge dataset of various radiotracers for brain PET imaging.

METHODS

The proposed framework is comprised of three steps: (1) NAC PET image generation, (2) synthetic TCT generation using CNN, and (3) PET image reconstruction. We trained the CNN by combining the mixed image dataset of six radiotracers to avoid overfitting, including [¹⁸F]FDG, [¹⁸F]BCPP-EF, [¹¹C]Racropride, [¹¹C]PIB, [¹¹C]DPA-713, and [¹¹C]PBB3. We used 1261 brain NAC PET and TCT images (1091 for training and 70 for testing). We did not include [¹¹C]Methionine subjects in the training dataset, but included them in the testing dataset.

RESULTS

The image quality of the synthetic TCT images obtained using the CNN trained on the mixed dataset of six radiotracers was superior to those obtained using the CNN trained on the split dataset generated from each radiotracer. In the [¹⁸F]FDG study, the mean relative PET biases of the emission-segmented AC (ESAC) and deep AC were 8.46 ± 5.24 and - 5.69 ± 4.97, respectively. The deep AC PET and TCT AC PET images exhibited excellent correlation for all seven radiotracers (R² = 0.912-0.982).

CONCLUSION

These results indicate that our proposed deep AC framework can be leveraged to provide quantitatively superior PET images when using the CNN trained on the mixed dataset of PET tracers than when using the CNN trained on the split dataset which means specific for each tracer.

Quantitative Molecular Positron Emission Tomography Imaging Using Advanced Deep Learning Techniques

The widespread availability of high-performance computing and the popularity of artificial intelligence (AI) with machine learning and deep learning (ML/DL) algorithms at the helm have stimulated the development of many applications involving the use of AI-based techniques in molecular imaging research. Applications reported in the literature encompass various areas, including innovative design concepts in positron emission tomography (PET) instrumentation, quantitative image reconstruction and analysis techniques, computer-aided detection and diagnosis, as well as modeling and prediction of outcomes. This review reflects the tremendous interest in quantitative molecular imaging using ML/DL techniques during the past decade, ranging from the basic principles of ML/DL techniques to the various steps required for obtaining quantitatively accurate PET data, including algorithms used to denoise or correct for physical degrading factors as well as to quantify tracer uptake and metabolic tumor volume for treatment monitoring or radiation therapy treatment planning and response prediction.This review also addresses future opportunities and current challenges facing the adoption of ML/DL approaches and their role in multimodality imaging.

Quantitative PET in the 2020s: a roadmap

Positron emission tomography (PET) plays an increasingly important role in research and clinical applications, catalysed by remarkable technical advances and a growing appreciation of the need for reliable, sensitive biomarkers of human function in health and disease. Over the last 30 years, a large amount of the physics and engineering effort in PET has been motivated by the dominant clinical application during that period, oncology. This has led to important developments such as PET/CT, whole-body PET, 3D PET, accelerated statistical image reconstruction, and time-of-flight PET. Despite impressive improvements in image quality as a result of these advances, the emphasis on static, semi-quantitative 'hot spot' imaging for oncologic applications has meant that the capability of PET to quantify biologically relevant parameters based on tracer kinetics has not been fully exploited. More recent advances, such as PET/MR and total-body PET, have opened up the ability to address a vast range of new research questions, from which a future expansion of applications and radiotracers appears highly likely. Many of these new applications and tracers will, at least initially, require quantitative analyses that more fully exploit the exquisite sensitivity of PET and the tracer principle on which it is based. It is also expected that they will require more sophisticated quantitative analysis methods than those that are currently available. At the same time, artificial intelligence is revolutionizing data analysis and impacting the relationship between the statistical quality of the acquired data and the information we can extract from the data. In this roadmap, leaders of the key sub-disciplines of the field identify the challenges and opportunities to be addressed over the next ten years that will enable PET to realise its full quantitative potential, initially in research laboratories and, ultimately, in clinical practice.

Automatic attenuation map estimation from SPECT data only for brain perfusion scans using convolutional neural networks

In clinical brain SPECT, correction for photon attenuation in the patient is essential to obtain images which provide quantitative information on the regional activity concentration per unit volume (kBq.[Formula: see text]). This correction generally requires an attenuation map ([Formula: see text] map) denoting the attenuation coefficient at each voxel which is often derived from a CT or MRI scan. However, such an additional scan is not always available and the method may suffer from registration errors. Therefore, we propose a SPECT-only-based strategy for [Formula: see text] map estimation that we apply to a stationary multi-pinhole clinical SPECT system (G-SPECT-I) for ^99mTc-HMPAO brain perfusion imaging. The method is based on the use of a convolutional neural network (CNN) and was validated with Monte Carlo simulated scans. Data acquired in list mode was used to employ the energy information of both primary and scattered photons to obtain information about the tissue attenuation as much as possible. Multiple SPECT reconstructions were performed from different energy windows over a large energy range. Locally extracted 4D SPECT patches (three spatial plus one energy dimension) were used as input for the CNN which was trained to predict the attenuation coefficient of the corresponding central voxel of the patch. Results show that Attenuation Correction using the Ground Truth [Formula: see text] maps (GT-AC) or using the CNN estimated [Formula: see text] maps (CNN-AC) achieve comparable accuracy. This was confirmed by a visual assessment as well as a quantitative comparison; the mean deviation from the GT-AC when using the CNN-AC is within 1.8% for the standardized uptake values in all brain regions. Therefore, our results indicate that a CNN-based method can be an automatic and accurate tool for SPECT attenuation correction that is independent of attenuation data from other imaging modalities or human interpretations about head contours.

CT-less Direct Correction of Attenuation and Scatter in the Image Space Using Deep Learning for Whole-Body FDG PET: Potential Benefits and Pitfalls

PURPOSE

To demonstrate the feasibility of CT-less attenuation and scatter correction (ASC) in the image space using deep learning for whole-body PET, with a focus on the potential benefits and pitfalls.

MATERIALS AND METHODS

In this retrospective study, 110 whole-body fluorodeoxyglucose (FDG) PET/CT studies acquired in 107 patients (mean age ± standard deviation, 58 years ± 18; age range, 11-92 years; 72 females) from February 2016 through January 2018 were randomly collected. A total of 37.3% (41 of 110) of the studies showed metastases, with diverse FDG PET findings throughout the whole body. A U-Net-based network was developed for directly transforming noncorrected PET (PET_NC) into attenuation- and scatter-corrected PET (PET_ASC). Deep learning-corrected PET (PET_DL) images were quantitatively evaluated by using the standardized uptake value (SUV) of the normalized root mean square error, the peak signal-to-noise ratio, and the structural similarity index, in addition to a joint histogram for statistical analysis. Qualitative reviews by radiologists revealed the potential benefits and pitfalls of this correction method.

RESULTS

The normalized root mean square error (0.21 ± 0.05 [mean SUV ± standard deviation]), mean peak signal-to-noise ratio (36.3 ± 3.0), mean structural similarity index (0.98 ± 0.01), and voxelwise correlation (97.62%) of PET_DL demonstrated quantitatively high similarity with PET_ASC. Radiologist reviews revealed the overall quality of PET_DL. The potential benefits of PET_DL include a radiation dose reduction on follow-up scans and artifact removal in the regions with attenuation correction- and scatter correction-based artifacts. The pitfalls involve potential false-negative results due to blurring or missing lesions or false-positive results due to pseudo-low-uptake patterns.

CONCLUSION

Deep learning-based direct ASC at whole-body PET is feasible and potentially can be used to overcome the current limitations of CT-based approaches, benefiting patients who are sensitive to radiation from CT.Supplemental material is available for this article.© RSNA, 2020.

MR-based Attenuation Correction for Brain PET Using 3D Cycle-Consistent Adversarial Network

Attenuation correction (AC) is important for the quantitative merits of positron emission tomography (PET). However, attenuation coefficients cannot be derived from magnetic resonance (MR) images directly for PET/MR systems. In this work, we aimed to derive continuous AC maps from Dixon MR images without the requirement of MR and computed tomography (CT) image registration. To achieve this, a 3D generative adversarial network with both discriminative and cycle-consistency loss (Cycle-GAN) was developed. The modified 3D U-net was employed as the structure of the generative networks to generate the pseudo CT/MR images. The 3D patch-based discriminative networks were used to distinguish the generated pseudo CT/MR images from the true CT/MR images. To evaluate its performance, datasets from 32 patients were used in the experiment. The Dixon segmentation and atlas methods provided by the vendor and the convolutional neural network (CNN) method which utilized registered MR and CT images were employed as the reference methods. Dice coefficients of the pseudo-CT image and the regional quantification in the reconstructed PET images were compared. Results show that the Cycle-GAN framework can generate better AC compared to the Dixon segmentation and atlas methods, and shows comparable performance compared to the CNN method.

Direct Image-Based Attenuation Correction using Conditional Generative Adversarial Network for SPECT Myocardial Perfusion Imaging

Attenuation correction (AC) is important for an accurate interpretation and quantitative analysis of SPECT myocardial perfusion imaging. Dedicated cardiac SPECT systems have invaluable efficacy in the evaluation and risk stratification of patients with known or suspected cardiovascular disease. However, most dedicated cardiac SPECT systems are standalone, not combined with a transmission imaging capability such as computed tomography (CT) for generating attenuation maps for AC. To address this problem, we propose to apply a conditional generative adversarial network (cGAN) for generating attenuation-corrected SPECT images (SPECT_GAN ) directly from non-corrected SPECT images (SPECT_NC ) in image domain as a one-step process without requiring additional intermediate step. The proposed network was trained and tested for 100 cardiac SPECT/CT data from a GE Discovery NM 570c SPECT/CT, collected retrospectively at Yale New Haven Hospital.The generated images were evaluated quantitatively through the normalized root mean square error (NRMSE), peak signal to noise ratio (PSNR), and structural similarity index (SSIM) and statistically through joint histogram and error maps. In comparison to the reference CT-based correction (SPECT_CTAC ), NRMSEs were 0.2258±0.0777 and 0.1410±0.0768 (37.5% reduction of errors); PSNRs 31.7712±2.9965 and 36.3823±3.7424 (14.5% improvement in signal to noise ratio); SSIMs 0.9877±0.0075 and 0.9949±0.0043 (0.7% improvement in structural similarity) for SPECT_NC and SPECT_GAN , respectively. This work demonstrates that the conditional adversarial training can achieve accurate CT-less attenuation correction for SPECT MPI, that is quantitatively comparable to CTAC. Standalone dedicated cardiac SPECT scanners can benefit from the proposed GAN to reduce attenuation artifacts efficiently.

A review on medical imaging synthesis using deep learning and its clinical applications

This paper reviewed the deep learning-based studies for medical imaging synthesis and its clinical application. Specifically, we summarized the recent developments of deep learning-based methods in inter- and intra-modality image synthesis by listing and highlighting the proposed methods, study designs, and reported performances with related clinical applications on representative studies. The challenges among the reviewed studies were then summarized with discussion.

Deep learning for brain disorders: from data processing to disease treatment

In order to reach precision medicine and improve patients' quality of life, machine learning is increasingly used in medicine. Brain disorders are often complex and heterogeneous, and several modalities such as demographic, clinical, imaging, genetics and environmental data have been studied to improve their understanding. Deep learning, a subpart of machine learning, provides complex algorithms that can learn from such various data. It has become state of the art in numerous fields, including computer vision and natural language processing, and is also growingly applied in medicine. In this article, we review the use of deep learning for brain disorders. More specifically, we identify the main applications, the concerned disorders and the types of architectures and data used. Finally, we provide guidelines to bridge the gap between research studies and clinical routine.

Attenuation correction for human PET/MRI studies

Attenuation correction has been one of the main methodological challenges in the integrated positron emission tomography and magnetic resonance imaging (PET/MRI) field. As standard transmission or computed tomography approaches are not available in integrated PET/MRI scanners, MR-based attenuation correction approaches had to be developed. Aspects that have to be considered for implementing accurate methods include the need to account for attenuation in bone tissue, normal and pathological lung and the MR hardware present in the PET field-of-view, to reduce the impact of subject motion, to minimize truncation and susceptibility artifacts, and to address issues related to the data acquisition and processing both on the PET and MRI sides. The standard MR-based attenuation correction techniques implemented by the PET/MRI equipment manufacturers and their impact on clinical and research PET data interpretation and quantification are first discussed. Next, the more advanced methods, including the latest generation deep learning-based approaches that have been proposed for further minimizing the attenuation correction related bias are described. Finally, a future perspective focused on the needed developments in the field is given.

Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation

Multimodal fusion in neuroimaging combines data from multiple imaging modalities to overcome the fundamental limitations of individual modalities. Neuroimaging fusion can achieve higher temporal and spatial resolution, enhance contrast, correct imaging distortions, and bridge physiological and cognitive information. In this study, we analyzed over 450 references from PubMed, Google Scholar, IEEE, ScienceDirect, Web of Science, and various sources published from 1978 to 2020. We provide a review that encompasses (1) an overview of current challenges in multimodal fusion (2) the current medical applications of fusion for specific neurological diseases, (3) strengths and limitations of available imaging modalities, (4) fundamental fusion rules, (5) fusion quality assessment methods, and (6) the applications of fusion for atlas-based segmentation and quantification. Overall, multimodal fusion shows significant benefits in clinical diagnosis and neuroscience research. Widespread education and further research amongst engineers, researchers and clinicians will benefit the field of multimodal neuroimaging.

A deep neural network for fast and accurate scatter estimation in quantitative SPECT/CT under challenging scatter conditions

PURPOSE

A major challenge for accurate quantitative SPECT imaging of some radionuclides is the inadequacy of simple energy window-based scatter estimation methods, widely available on clinic systems. A deep learning approach for SPECT/CT scatter estimation is investigated as an alternative to computationally expensive Monte Carlo (MC) methods for challenging SPECT radionuclides, such as 90Y.

METHODS

A deep convolutional neural network (DCNN) was trained to separately estimate each scatter projection from the measured 90Y bremsstrahlung SPECT emission projection and CT attenuation projection that form the network inputs. The 13-layer deep architecture consisted of separate paths for the emission and attenuation projection that are concatenated before the final convolution steps. The training label consisted of MC-generated "true" scatter projections in phantoms (MC is needed only for training) with the mean square difference relative to the model output serving as the loss function. The test data set included a simulated sphere phantom with a lung insert, measurements of a liver phantom, and patients after 90Y radioembolization. OS-EM SPECT reconstruction without scatter correction (NO-SC), with the true scatter (TRUE-SC) (available for simulated data only), with the DCNN estimated scatter (DCNN-SC), and with a previously developed MC scatter model (MC-SC) were compared, including with 90Y PET when available.

RESULTS

The contrast recovery (CR) vs. noise and lung insert residual error vs. noise curves for images reconstructed with DCNN-SC and MC-SC estimates were similar. At the same noise level of 10% (across multiple realizations), the average sphere CR was 24%, 52%, 55%, and 67% for NO-SC, MC-SC, DCNN-SC, and TRUE-SC, respectively. For the liver phantom, the average CR for liver inserts were 32%, 73%, and 65% for NO-SC, MC-SC, and DCNN-SC, respectively while the corresponding values for average contrast-to-noise ratio (visibility index) in low-concentration extra-hepatic inserts were 2, 19, and 61, respectively. In patients, there was high concordance between lesion-to-liver uptake ratios for SPECT reconstruction with DCNN-SC (median 4.8, range 0.02-13.8) compared with MC-SC (median 4.0, range 0.13-12.1; CCC = 0.98) and with 90Y PET (median 4.9, range 0.02-11.2; CCC = 0.96) while the concordance with NO-SC was poor (median 2.8, range 0.3-7.2; CCC = 0.59). The trained DCNN took ~ 40 s (using a single i5 processor on a desktop computer) to generate the scatter estimates for all 128 views in a patient scan, compared to ~ 80 min for the MC scatter model using 12 processors.

CONCLUSIONS

For diverse 90Y test data that included patient studies, we demonstrated comparable performance between images reconstructed with deep learning and MC-based scatter estimates using metrics relevant for dosimetry and for safety. This approach that can be generalized to other radionuclides by changing the training data is well suited for real-time clinical use because of the high speed, orders of magnitude faster than MC, while maintaining high accuracy.

Applications of artificial intelligence and deep learning in molecular imaging and radiotherapy

This brief review summarizes the major applications of artificial intelligence (AI), in particular deep learning approaches, in molecular imaging and radiation therapy research. To this end, the applications of artificial intelligence in five generic fields of molecular imaging and radiation therapy, including PET instrumentation design, PET image reconstruction quantification and segmentation, image denoising (low-dose imaging), radiation dosimetry and computer-aided diagnosis, and outcome prediction are discussed. This review sets out to cover briefly the fundamental concepts of AI and deep learning followed by a presentation of seminal achievements and the challenges facing their adoption in clinical setting.

Deep learning-guided joint attenuation and scatter correction in multitracer neuroimaging studies

PET attenuation correction (AC) on systems lacking CT/transmission scanning, such as dedicated brain PET scanners and hybrid PET/MRI, is challenging. Direct AC in image-space, wherein PET images corrected for attenuation and scatter are synthesized from nonattenuation corrected PET (PET-nonAC) images in an end-to-end fashion using deep learning approaches (DLAC) is evaluated for various radiotracers used in molecular neuroimaging studies. One hundred eighty brain PET scans acquired using 18 F-FDG, 18 F-DOPA, 18 F-Flortaucipir (targeting tau pathology), and 18 F-Flutemetamol (targeting amyloid pathology) radiotracers (40 + 5, training/validation + external test, subjects for each radiotracer) were included. The PET data were reconstructed using CT-based AC (CTAC) to generate reference PET-CTAC and without AC to produce PET-nonAC images. A deep convolutional neural network was trained to generate PET attenuation corrected images (PET-DLAC) from PET-nonAC. The quantitative accuracy of this approach was investigated separately for each radiotracer considering the values obtained from PET-CTAC images as reference. A segmented AC map (PET-SegAC) containing soft-tissue and background air was also included in the evaluation. Quantitative analysis of PET images demonstrated superior performance of the DLAC approach compared to SegAC technique for all tracers. Despite the relatively low quantitative bias observed when using the DLAC approach, this approach appears vulnerable to outliers, resulting in noticeable local pseudo uptake and false cold regions. Direct AC in image-space using deep learning demonstrated quantitatively acceptable performance with less than 9% absolute SUV bias for the four different investigated neuroimaging radiotracers. However, this approach is vulnerable to outliers which result in large local quantitative bias.

AI-driven attenuation correction for brain PET/MRI: Clinical evaluation of a dementia cohort and importance of the training group size

INTRODUCTION

Robust and reliable attenuation correction (AC) is a prerequisite for accurate quantification of activity concentration. In combined PET/MRI, AC is challenged by the lack of bone signal in the MRI from which the AC maps has to be derived. Deep learning-based image-to-image translation networks present itself as an optimal solution for MRI-derived AC (MR-AC). High robustness and generalizability of these networks are expected to be achieved through large training cohorts. In this study, we implemented an MR-AC method based on deep learning, and investigated how training cohort size, transfer learning, and MR input affected robustness, and subsequently evaluated the method in a clinical setup, with the overall aim to explore if this method could be implemented in clinical routine for PET/MRI examinations.

METHODS

A total cohort of 1037 adult subjects from the Siemens Biograph mMR with two different software versions (VB20P and VE11P) was used. The software upgrade included updates to all MRI sequences. The impact of training group size was investigated by training a convolutional neural network (CNN) on an increasing training group size from 10 to 403. The ability to adapt to changes in the input images between software versions were evaluated using transfer learning from a large cohort to a smaller cohort, by varying training group size from 5 to 91 subjects. The impact of MRI sequence was evaluated by training three networks based on the Dixon VIBE sequence (DeepDixon), T1-weighted MPRAGE (DeepT1), and ultra-short echo time (UTE) sequence (DeepUTE). Blinded clinical evaluation relative to the reference low-dose CT (CT-AC) was performed for DeepDixon in 104 independent 2-[¹⁸F]fluoro-2-deoxy-d-glucose ([¹⁸F]FDG) PET patient studies performed for suspected neurodegenerative disorder using statistical surface projections.

RESULTS

Robustness increased with group size in the training data set: 100 subjects were required to reduce the number of outliers compared to a state-of-the-art segmentation-based method, and a cohort >400 subjects further increased robustness in terms of reduced variation and number of outliers. When using transfer learning to adapt to changes in the MRI input, as few as five subjects were sufficient to minimize outliers. Full robustness was achieved at 20 subjects. Comparable robust and accurate results were obtained using all three types of MRI input with a bias below 1% relative to CT-AC in any brain region. The clinical PET evaluation using DeepDixon showed no clinically relevant differences compared to CT-AC.

CONCLUSION

Deep learning based AC requires a large training cohort to achieve accurate and robust performance. Using transfer learning, only five subjects were needed to fine-tune the method to large changes to the input images. No clinically relevant differences were found compared to CT-AC, indicating that clinical implementation of our deep learning-based MR-AC method will be feasible across MRI system types using transfer learning and a limited number of subjects.

Machine learning in quantitative PET: A review of attenuation correction and low-count image reconstruction methods

The rapid expansion of machine learning is offering a new wave of opportunities for nuclear medicine. This paper reviews applications of machine learning for the study of attenuation correction (AC) and low-count image reconstruction in quantitative positron emission tomography (PET). Specifically, we present the developments of machine learning methodology, ranging from random forest and dictionary learning to the latest convolutional neural network-based architectures. For application in PET attenuation correction, two general strategies are reviewed: 1) generating synthetic CT from MR or non-AC PET for the purposes of PET AC, and 2) direct conversion from non-AC PET to AC PET. For low-count PET reconstruction, recent deep learning-based studies and the potential advantages over conventional machine learning-based methods are presented and discussed. In each application, the proposed methods, study designs and performance of published studies are listed and compared with a brief discussion. Finally, the overall contributions and remaining challenges are summarized.

Improving PET Imaging Acquisition and Analysis With Machine Learning: A Narrative Review With Focus on Alzheimer's Disease and Oncology

Machine learning (ML) algorithms have found increasing utility in the medical imaging field and numerous applications in the analysis of digital biomarkers within positron emission tomography (PET) imaging have emerged. Interest in the use of artificial intelligence in PET imaging for the study of neurodegenerative diseases and oncology stems from the potential for such techniques to streamline decision support for physicians providing early and accurate diagnosis and allowing personalized treatment regimens. In this review, the use of ML to improve PET image acquisition and reconstruction is presented, along with an overview of its applications in the analysis of PET images for the study of Alzheimer's disease and oncology.