Synthesis of Patient-Specific Transmission Data for PET Attenuation Correction for PET/MRI Neuroimaging Using a Convolutional Neural Network

Artificial intelligence-based joint attenuation and scatter correction strategies for multi-tracer total-body PET

BACKGROUND

Low-dose ungated CT is commonly used for total-body PET attenuation and scatter correction (ASC). However, CT-based ASC (CT-ASC) is limited by radiation dose risks of CT examinations, propagation of CT-based artifacts and potential mismatches between PET and CT. We demonstrate the feasibility of direct ASC for multi-tracer total-body PET in the image domain.

METHODS

Clinical uEXPLORER total-body PET/CT datasets of [18F]FDG (N = 52), [18F]FAPI (N = 46) and [68Ga]FAPI (N = 60) were retrospectively enrolled in this study. We developed an improved 3D conditional generative adversarial network (cGAN) to directly estimate attenuation and scatter-corrected PET images from non-attenuation and scatter-corrected (NASC) PET images. The feasibility of the proposed 3D cGAN-based ASC was validated using four training strategies: (1) Paired 3D NASC and CT-ASC PET images from three tracers were pooled into one centralized server (CZ-ASC). (2) Paired 3D NASC and CT-ASC PET images from each tracer were individually used (DL-ASC). (3) Paired NASC and CT-ASC PET images from one tracer ([18F]FDG) were used to train the networks, while the other two tracers were used for testing without fine-tuning (NFT-ASC). (4) The pre-trained networks of (3) were fine-tuned with two other tracers individually (FT-ASC). We trained all networks in fivefold cross-validation. The performance of all ASC methods was evaluated by qualitative and quantitative metrics using CT-ASC as the reference.

RESULTS

CZ-ASC, DL-ASC and FT-ASC showed comparable visual quality with CT-ASC for all tracers. CZ-ASC and DL-ASC resulted in a normalized mean absolute error (NMAE) of 8.51 ± 7.32% versus 7.36 ± 6.77% (p < 0.05), outperforming NASC (p < 0.0001) in [18F]FDG dataset. CZ-ASC, FT-ASC and DL-ASC led to NMAE of 6.44 ± 7.02%, 6.55 ± 5.89%, and 7.25 ± 6.33% in [18F]FAPI dataset, and NMAE of 5.53 ± 3.99%, 5.60 ± 4.02%, and 5.68 ± 4.12% in [68Ga]FAPI dataset, respectively. CZ-ASC, FT-ASC and DL-ASC were superior to NASC (p < 0.0001) and NFT-ASC (p < 0.0001) in terms of NMAE results.

CONCLUSIONS

CZ-ASC, DL-ASC and FT-ASC demonstrated the feasibility of providing accurate and robust ASC for multi-tracer total-body PET, thereby reducing the radiation hazards to patients from redundant CT examinations. CZ-ASC and FT-ASC could outperform DL-ASC for cross-tracer total-body PET AC.

A systematic literature review: deep learning techniques for synthetic medical image generation and their applications in radiotherapy

The aim of this systematic review is to determine whether Deep Learning (DL) algorithms can provide a clinically feasible alternative to classic algorithms for synthetic Computer Tomography (sCT). The following categories are presented in this study: ∙ MR-based treatment planning and synthetic CT generation techniques. ∙ Generation of synthetic CT images based on Cone Beam CT images. ∙ Low-dose CT to High-dose CT generation. ∙ Attenuation correction for PET images. To perform appropriate database searches, we reviewed journal articles published between January 2018 and June 2023. Current methodology, study strategies, and results with relevant clinical applications were analyzed as we outlined the state-of-the-art of deep learning based approaches to inter-modality and intra-modality image synthesis. This was accomplished by contrasting the provided methodologies with traditional research approaches. The key contributions of each category were highlighted, specific challenges were identified, and accomplishments were summarized. As a final step, the statistics of all the cited works from various aspects were analyzed, which revealed that DL-based sCTs have achieved considerable popularity, while also showing the potential of this technology. In order to assess the clinical readiness of the presented methods, we examined the current status of DL-based sCT generation.

Contrast-enhanced to non-contrast-enhanced image translation to exploit a clinical data warehouse of T1-weighted brain MRI

BACKGROUND

Clinical data warehouses provide access to massive amounts of medical images, but these images are often heterogeneous. They can for instance include images acquired both with or without the injection of a gadolinium-based contrast agent. Harmonizing such data sets is thus fundamental to guarantee unbiased results, for example when performing differential diagnosis. Furthermore, classical neuroimaging software tools for feature extraction are typically applied only to images without gadolinium. The objective of this work is to evaluate how image translation can be useful to exploit a highly heterogeneous data set containing both contrast-enhanced and non-contrast-enhanced images from a clinical data warehouse.

METHODS

We propose and compare different 3D U-Net and conditional GAN models to convert contrast-enhanced T1-weighted (T1ce) into non-contrast-enhanced (T1nce) brain MRI. These models were trained using 230 image pairs and tested on 77 image pairs from the clinical data warehouse of the Greater Paris area.

RESULTS

Validation using standard image similarity measures demonstrated that the similarity between real and synthetic T1nce images was higher than between real T1nce and T1ce images for all the models compared. The best performing models were further validated on a segmentation task. We showed that tissue volumes extracted from synthetic T1nce images were closer to those of real T1nce images than volumes extracted from T1ce images.

CONCLUSION

We showed that deep learning models initially developed with research quality data could synthesize T1nce from T1ce images of clinical quality and that reliable features could be extracted from the synthetic images, thus demonstrating the ability of such methods to help exploit a data set coming from a clinical data warehouse.

Attenuation correction and truncation completion for breast PET/MR imaging using deep learning

Objective. Simultaneous PET/MR scanners combine the high sensitivity of MR imaging with the functional imaging of PET. However, attenuation correction of breast PET/MR imaging is technically challenging. The purpose of this study is to establish a robust attenuation correction algorithm for breast PET/MR images that relies on deep learning (DL) to recreate the missing portions of the patient's anatomy (truncation completion), as well as to provide bone information for attenuation correction from only the PET data.Approach. Data acquired from 23 female subjects with invasive breast cancer scanned with18F-fluorodeoxyglucose PET/CT and PET/MR localized to the breast region were used for this study. Three DL models, U-Net with mean absolute error loss (DLMAE) model, U-Net with mean squared error loss (DLMSE) model, and U-Net with perceptual loss (DLPerceptual) model, were trained to predict synthetic CT images (sCT) for PET attenuation correction (AC) given non-attenuation corrected (NAC) PETPET/MRimages as inputs. The DL and Dixon-based sCT reconstructed PET images were compared against those reconstructed from CT images by calculating the percent error of the standardized uptake value (SUV) and conducting Wilcoxon signed rank statistical tests.Main results. sCT images from the DLMAEmodel, the DLMSEmodel, and the DLPerceptualmodel were similar in mean absolute error (MAE), peak-signal-to-noise ratio, and normalized cross-correlation. No significant difference in SUV was found between the PET images reconstructed using the DLMSEand DLPerceptualsCTs compared to the reference CT for AC in all tissue regions. All DL methods performed better than the Dixon-based method according to SUV analysis.Significance. A 3D U-Net with MSE or perceptual loss model can be implemented into a reconstruction workflow, and the derived sCT images allow successful truncation completion and attenuation correction for breast PET/MR images.

Machine learning methods for tracer kinetic modelling

Tracer kinetic modelling based on dynamic PET is an important field of Nuclear Medicine for quantitative functional imaging. Yet, its implementation in clinical routine has been constrained by its complexity and computational costs. Machine learning poses an opportunity to improve modelling processes in terms of arterial input function prediction, the prediction of kinetic modelling parameters and model selection in both clinical and preclinical studies while reducing processing time. Moreover, it can help improving kinetic modelling data used in downstream tasks such as tumor detection. In this review, we introduce the basics of tracer kinetic modelling and present a literature review of original works and conference papers using machine learning methods in this field.

Transfer learning-based attenuation correction for static and dynamic cardiac PET using a generative adversarial network

PURPOSE

The goal of this work is to demonstrate the feasibility of directly generating attenuation-corrected PET images from non-attenuation-corrected (NAC) PET images for both rest and stress-state static or dynamic [13N]ammonia MP PET based on a generative adversarial network.

METHODS

We recruited 60 subjects for rest-only scans and 14 subjects for rest-stress scans, all of whom underwent [13N]ammonia cardiac PET/CT examinations to acquire static and dynamic frames with both 3D NAC and CT-based AC (CTAC) PET images. We developed a 3D pix2pix deep learning AC (DLAC) framework via a U-net + ResNet-based generator and a convolutional neural network-based discriminator. Paired static or dynamic NAC and CTAC PET images from 60 rest-only subjects were used as network inputs and labels for static (S-DLAC) and dynamic (D-DLAC) training, respectively. The pre-trained S-DLAC network was then fine-tuned by paired dynamic NAC and CTAC PET frames of 60 rest-only subjects to derive an improved D-DLAC-FT for dynamic PET images. The 14 rest-stress subjects were used as an internal testing dataset and separately tested on different network models without training. The proposed methods were evaluated using visual quality and quantitative metrics.

RESULTS

The proposed S-DLAC, D-DLAC, and D-DLAC-FT methods were consistent with clinical CTAC in terms of various images and quantitative metrics. The S-DLAC (slope = 0.9423, R2 = 0.947) showed a higher correlation with the reference static CTAC as compared to static NAC (slope = 0.0992, R2 = 0.654). D-DLAC-FT yielded lower myocardial blood flow (MBF) errors in the whole left ventricular myocardium than D-DLAC, but with no significant difference, both for the 60 rest-state subjects (6.63 ± 5.05% vs. 7.00 ± 6.84%, p = 0.7593) and the 14 stress-state subjects (1.97 ± 2.28% vs. 3.21 ± 3.89%, p = 0.8595).

CONCLUSION

The proposed S-DLAC, D-DLAC, and D-DLAC-FT methods achieve comparable performance with clinical CTAC. Transfer learning shows promising potential for dynamic MP PET.

A review of PET attenuation correction methods for PET-MR

Despite being thirteen years since the installation of the first PET-MR system, the scanners constitute a very small proportion of the total hybrid PET systems installed. This is in stark contrast to the rapid expansion of the PET-CT scanner, which quickly established its importance in patient diagnosis within a similar timeframe. One of the main hurdles is the development of an accurate, reproducible and easy-to-use method for attenuation correction. Quantitative discrepancies in PET images between the manufacturer-provided MR methods and the more established CT- or transmission-based attenuation correction methods have led the scientific community in a continuous effort to develop a robust and accurate alternative. These can be divided into four broad categories: (i) MR-based, (ii) emission-based, (iii) atlas-based and the (iv) machine learning-based attenuation correction, which is rapidly gaining momentum. The first is based on segmenting the MR images in various tissues and allocating a predefined attenuation coefficient for each tissue. Emission-based attenuation correction methods aim in utilising the PET emission data by simultaneously reconstructing the radioactivity distribution and the attenuation image. Atlas-based attenuation correction methods aim to predict a CT or transmission image given an MR image of a new patient, by using databases containing CT or transmission images from the general population. Finally, in machine learning methods, a model that could predict the required image given the acquired MR or non-attenuation-corrected PET image is developed by exploiting the underlying features of the images. Deep learning methods are the dominant approach in this category. Compared to the more traditional machine learning, which uses structured data for building a model, deep learning makes direct use of the acquired images to identify underlying features. This up-to-date review goes through the literature of attenuation correction approaches in PET-MR after categorising them. The various approaches in each category are described and discussed. After exploring each category separately, a general overview is given of the current status and potential future approaches along with a comparison of the four outlined categories.

Objective Image Quality Comparison Between Brain-Dedicated PET and PET/CT Scanners

As part of a clinical validation of a new brain-dedicated PET system (CMB), image quality of this scanner has been compared to that of a whole-body PET/CT scanner. To that goal, Hoffman phantom and patient data were obtined with both devices. Since CMB does not use a CT for attenuation correction (AC) which is crucial for PET images quality, this study includes the evaluation of CMB PET images using emission-based or CT-based attenuation maps. PET images were compared using 34 image quality metrics. Moreover, a neural network was used to evaluate the degree of agreement between both devices on the patients diagnosis prediction. Overall, results showed that CMB images have higher contrast and recovery coefficient but higher noise than PET/CT images. Although SUVr values presented statistically significant differences in many brain regions, relative differences were low. An asymmetry between left and right hemispheres, however, was identified. Even so, the variations between the two devices were minor. Finally, there is a greater similarity between PET/CT and CMB CT-based AC PET images than between PET/CT and the CMB emission-based AC PET images.

The performance of machine learning approaches for attenuation correction of PET in neuroimaging: A meta-analysis

PURPOSE

This systematic review provides a consensus on the clinical feasibility of machine learning (ML) methods for brain PET attenuation correction (AC). Performance of ML-AC were compared to clinical standards.

METHODS

Two hundred and eighty studies were identified through electronic searches of brain PET studies published between January 1, 2008, and August 1, 2022. Reported outcomes for image quality, tissue classification performance, regional and global bias were extracted to evaluate ML-AC performance. Methodological quality of included studies and the quality of evidence of analysed outcomes were assessed using QUADAS-2 and GRADE, respectively.

RESULTS

A total of 19 studies (2371 participants) met the inclusion criteria. Overall, the global bias of ML methods was 0.76 ± 1.2%. For image quality, the relative mean square error (RMSE) was 0.20 ± 0.4 while for tissues classification, the Dice similarity coefficient (DSC) for bone/soft tissue/air were 0.82 ± 0.1 / 0.95 ± 0.03 / 0.85 ± 0.14.

CONCLUSIONS

In general, ML-AC performance is within acceptable limits for clinical PET imaging. The sparse information on ML-AC robustness and its limited qualitative clinical evaluation may hinder clinical implementation in neuroimaging, especially for PET/MRI or emerging brain PET systems where standard AC approaches are not readily available.

Deep learning-based attenuation map generation with simultaneously reconstructed PET activity and attenuation and low-dose application

Objective. In PET/CT imaging, CT is used for positron emission tomography (PET) attenuation correction (AC). CT artifacts or misalignment between PET and CT can cause AC artifacts and quantification errors in PET. Simultaneous reconstruction (MLAA) of PET activity (λ-MLAA) and attenuation (μ-MLAA) maps was proposed to solve those issues using the time-of-flight PET raw data only. However,λ-MLAA still suffers from quantification error as compared to reconstruction using the gold-standard CT-based attenuation map (μ-CT). Recently, a deep learning (DL)-based framework was proposed to improve MLAA by predictingμ-DL fromλ-MLAA andμ-MLAA using an image domain loss function (IM-loss). However, IM-loss does not directly measure the AC errors according to the PET attenuation physics. Our preliminary studies showed that an additional physics-based loss function can lead to more accurate PET AC. The main objective of this study is to optimize the attenuation map generation framework for clinical full-dose¹⁸F-FDG studies. We also investigate the effectiveness of the optimized network on predicting attenuation maps for synthetic low-dose oncological PET studies.Approach. We optimized the proposed DL framework by applying different preprocessing steps and hyperparameter optimization, including patch size, weights of the loss terms and number of angles in the projection-domain loss term. The optimization was performed based on 100 skull-to-toe¹⁸F-FDG PET/CT scans with minimal misalignment. The optimized framework was further evaluated on 85 clinical full-dose neck-to-thigh¹⁸F-FDG cancer datasets as well as synthetic low-dose studies with only 10% of the full-dose raw data.Main results. Clinical evaluation of tumor quantification as well as physics-based figure-of-merit metric evaluation validated the promising performance of our proposed method. For both full-dose and low-dose studies, the proposed framework achieved <1% error in tumor standardized uptake value measures.Significance. It is of great clinical interest to achieve CT-less PET reconstruction, especially for low-dose PET studies.

Attenuation Coefficient Estimation for PET/MRI With Bayesian Deep Learning Pseudo-CT and Maximum-Likelihood Estimation of Activity and Attenuation

A major remaining challenge for magnetic resonance-based attenuation correction methods (MRAC) is their susceptibility to sources of magnetic resonance imaging (MRI) artifacts (e.g., implants and motion) and uncertainties due to the limitations of MRI contrast (e.g., accurate bone delineation and density, and separation of air/bone). We propose using a Bayesian deep convolutional neural network that in addition to generating an initial pseudo-CT from MR data, it also produces uncertainty estimates of the pseudo-CT to quantify the limitations of the MR data. These outputs are combined with the maximum-likelihood estimation of activity and attenuation (MLAA) reconstruction that uses the PET emission data to improve the attenuation maps. With the proposed approach uncertainty estimation and pseudo-CT prior for robust MLAA (UpCT-MLAA), we demonstrate accurate estimation of PET uptake in pelvic lesions and show recovery of metal implants. In patients without implants, UpCT-MLAA had acceptable but slightly higher root-mean-squared-error (RMSE) than Zero-echotime and Dixon Deep pseudo-CT when compared to CTAC. In patients with metal implants, MLAA recovered the metal implant; however, anatomy outside the implant region was obscured by noise and crosstalk artifacts. Attenuation coefficients from the pseudo-CT from Dixon MRI were accurate in normal anatomy; however, the metal implant region was estimated to have attenuation coefficients of air. UpCT-MLAA estimated attenuation coefficients of metal implants alongside accurate anatomic depiction outside of implant regions.

Evaluation of a 2D UNet-Based Attenuation Correction Methodology for PET/MR Brain Studies

Deep learning (DL) strategies applied to magnetic resonance (MR) images in positron emission tomography (PET)/MR can provide synthetic attenuation correction (AC) maps, and consequently PET images, more accurate than segmentation or atlas-registration strategies. As first objective, we aim to investigate the best MR image to be used and the best point of the AC pipeline to insert the synthetic map in. Sixteen patients underwent a 18F-fluorodeoxyglucose (FDG) PET/computed tomography (CT) and a PET/MR brain study in the same day. PET/CT images were reconstructed with attenuation maps obtained: (1) from CT (reference), (2) from MR with an atlas-based and a segmentation-based method and (3) with a 2D UNet trained on MR image/attenuation map pairs. As for MR, T1-weighted and Zero Time Echo (ZTE) images were considered; as for attenuation maps, CTs and 511 keV low-resolution attenuation maps were assessed. As second objective, we assessed the ability of DL strategies to provide proper AC maps in presence of cranial anatomy alterations due to surgery. Three 11C-methionine (METH) PET/MR studies were considered. PET images were reconstructed with attenuation maps obtained: (1) from diagnostic coregistered CT (reference), (2) from MR with an atlas-based and a segmentation-based method and (3) with 2D UNets trained on the sixteen FDG anatomically normal patients. Only UNets taking ZTE images in input were considered. FDG and METH PET images were quantitatively evaluated. As for anatomically normal FDG patients, UNet AC models generally provide an uptake estimate with lower bias than atlas-based or segmentation-based methods. The intersubject average bias on images corrected with UNet AC maps is always smaller than 1.5%, except for AC maps generated on too coarse grids. The intersubject bias variability is the lowest (always lower than 2%) for UNet AC maps coming from ZTE images, larger for other methods. UNet models working on MR ZTE images and generating synthetic CT or 511 keV low-resolution attenuation maps therefore provide the best results in terms of both accuracy and variability. As for METH anatomically altered patients, DL properly reconstructs anatomical alterations. Quantitative results on PET images confirm those found on anatomically normal FDG patients.

BPGAN: Brain PET synthesis from MRI using generative adversarial network for multi-modal Alzheimer's disease diagnosis

BACKGROUND AND OBJECTIVE

Multi-modal medical images, such as magnetic resonance imaging (MRI) and positron emission tomography (PET), have been widely used for the diagnosis of brain disorder diseases like Alzheimer's disease (AD) since they can provide various information. PET scans can detect cellular changes in organs and tissues earlier than MRI. Unlike MRI, PET data is difficult to acquire due to cost, radiation, or other limitations. Moreover, PET data is missing for many subjects in the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. To solve this problem, a 3D end-to-end generative adversarial network (named BPGAN) is proposed to synthesize brain PET from MRI scans, which can be used as a potential data completion scheme for multi-modal medical image research.

METHODS

We propose BPGAN, which learns an end-to-end mapping function to transform the input MRI scans to their underlying PET scans. First, we design a 3D multiple convolution U-Net (MCU) generator architecture to improve the visual quality of synthetic results while preserving the diverse brain structures of different subjects. By further employing a 3D gradient profile (GP) loss and structural similarity index measure (SSIM) loss, the synthetic PET scans have higher-similarity to the ground truth. In this study, we explore alternative data partitioning ways to study their impact on the performance of the proposed method in different medical scenarios.

RESULTS

We conduct experiments on a publicly available ADNI database. The proposed BPGAN is evaluated by mean absolute error (MAE), peak-signal-to-noise-ratio (PSNR) and SSIM, superior to other compared models in these quantitative evaluation metrics. Qualitative evaluations also validate the effectiveness of our approach. Additionally, combined with MRI and our synthetic PET scans, the accuracies of multi-class AD diagnosis on dataset-A and dataset-B are 85.00% and 56.47%, which have been improved by about 1% and 1%, respectively, compared to the stand-alone MRI.

CONCLUSIONS

The experimental results of quantitative measures, qualitative displays, and classification evaluation demonstrate that the synthetic PET images by BPGAN are reasonable and high-quality, which provide complementary information to improve the performance of AD diagnosis. This work provides a valuable reference for multi-modal medical image analysis.

Composite attenuation correction method using a 68Ge-transmission multi-atlas for quantitative brain PET/MR

In positron emission tomography (PET), ⁶⁸Ge-transmission scanning is considered the gold standard in attenuation correction (AC) though not available in current dual imaging systems. In this experimental study we evaluated a novel AC method for PET/magnetic resonance (MR) imaging which is essentially based on a composite database of multiple ⁶⁸Ge-transmission maps and T1-weighted (T1w) MR image-pairs (composite transmission, CTR-AC). This proof-of-concept study used retrospectively a database with 125 pairs of co-registered ⁶⁸Ge-AC maps and T1w MR images from anatomical normal subjects and a validation dataset comprising dynamic [¹¹C]PE2I PET data from nine patients with Parkinsonism. CTR-AC maps were generated by non-rigid image registration of all database T1w MRI to each subject's T1w, applying the same transformation to every ⁶⁸Ge-AC map, and averaging the resulting ⁶⁸Ge-AC maps. [¹¹C]PE2I PET images were reconstructed using CTR-AC and a patient-specific ⁶⁸Ge-AC map as the reference standard. Standardized uptake values (SUV) and quantitative parameters of kinetic analysis were compared, i.e., relative delivery (R₁) and non-displaceable binding potential (BP_ND). CTR-AC showed high accuracy for whole-brain SUV (mean %bias ± SD: 0.5 ± 3.5%), whole-brain R₁ (-0.1 ± 3.2%), and putamen BP_ND (3.7 ± 8.1%). SUV and R₁ precision (SD of %bias) were modest and lowest in the anterior cortex, with an R₁ %bias of -1.1 ± 6.4%). The prototype CTR-AC is capable of providing accurate MRAC-maps with continuous linear attenuation coefficients though still experimental. The method's accuracy is comparable to the best MRAC methods published so far, both in SUV and as found for ZTE-AC in quantitative parameters of kinetic modelling.

Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesis

Fusing multi-modality medical images, such as magnetic resonance (MR) imaging and positron emission tomography (PET), can provide various anatomical and functional information about the human body. However, PET data is not always available for several reasons, such as high cost, radiation hazard, and other limitations. This paper proposes a 3D end-to-end synthesis network called Bidirectional Mapping Generative Adversarial Networks (BMGAN). Image contexts and latent vectors are effectively used for brain MR-to-PET synthesis. Specifically, a bidirectional mapping mechanism is designed to embed the semantic information of PET images into the high-dimensional latent space. Moreover, the 3D Dense-UNet generator architecture and the hybrid loss functions are further constructed to improve the visual quality of cross-modality synthetic images. The most appealing part is that the proposed method can synthesize perceptually realistic PET images while preserving the diverse brain structures of different subjects. Experimental results demonstrate that the performance of the proposed method outperforms other competitive methods in terms of quantitative measures, qualitative displays, and evaluation metrics for classification.

Deep learning based synthetic-CT generation in radiotherapy and PET: A review

Recently,deep learning (DL)-based methods for the generation of synthetic computed tomography (sCT) have received significant research attention as an alternative to classical ones. We present here a systematic review of these methods by grouping them into three categories, according to their clinical applications: (i) to replace computed tomography in magnetic resonance (MR) based treatment planning, (ii) facilitate cone-beam computed tomography based image-guided adaptive radiotherapy, and (iii) derive attenuation maps for the correction of positron emission tomography. Appropriate database searching was performed on journal articles published between January 2014 and December 2020. The DL methods' key characteristics were extracted from each eligible study, and a comprehensive comparison among network architectures and metrics was reported. A detailed review of each category was given, highlighting essential contributions, identifying specific challenges, and summarizing the achievements. Lastly, the statistics of all the cited works from various aspects were analyzed, revealing the popularity and future trends and the potential of DL-based sCT generation. The current status of DL-based sCT generation was evaluated, assessing the clinical readiness of the presented methods.

Data-driven respiratory phase-matched PET attenuation correction without CT

We propose a deep learning-based data-driven respiratory phase-matched gated-PET attenuation correction (AC) method that does not need a gated-CT. The proposed method is a multi-step process that consists of data-driven respiratory gating, gated attenuation map estimation using maximum-likelihood reconstruction of attenuation and activity (MLAA) algorithm, and enhancement of the gated attenuation maps using convolutional neural network (CNN). The gated MLAA attenuation maps enhanced by the CNN allowed for the phase-matched AC of gated-PET images. We conducted a non-rigid registration of the gated-PET images to generate motion-free PET images. We trained the CNN by conducting a 3D patch-based learning with 80 oncologic whole-body¹⁸F-fluorodeoxyglucose (¹⁸F-FDG) PET/CT scan data and applied it to seven regional PET/CT scans that cover the lower lung and upper liver. We investigated the impact of the proposed respiratory phase-matched AC of PET without utilizing CT on tumor size and standard uptake value (SUV) assessment, and PET image quality (%STD). The attenuation corrected gated and motion-free PET images generated using the proposed method yielded sharper organ boundaries and better noise characteristics than conventional gated and ungated PET images. A banana artifact observed in a phase-mismatched CT-based AC was not observed in the proposed approach. By employing the proposed method, the size of tumor was reduced by 12.3% and SUV_90%was increased by 13.3% in tumors with larger movements than 5 mm. %STD of liver uptake was reduced by 11.1%. The deep learning-based data-driven respiratory phase-matched AC method improved the PET image quality and reduced the motion artifacts.

Attenuation correction using deep Learning and integrated UTE/multi-echo Dixon sequence: evaluation in amyloid and tau PET imaging

PURPOSE

PET measures of amyloid and tau pathologies are powerful biomarkers for the diagnosis and monitoring of Alzheimer's disease (AD). Because cortical regions are close to bone, quantitation accuracy of amyloid and tau PET imaging can be significantly influenced by errors of attenuation correction (AC). This work presents an MR-based AC method that combines deep learning with a novel ultrashort time-to-echo (UTE)/multi-echo Dixon (mUTE) sequence for amyloid and tau imaging.

METHODS

Thirty-five subjects that underwent both 11C-PiB and 18F-MK6240 scans were included in this study. The proposed method was compared with Dixon-based atlas method as well as magnetization-prepared rapid acquisition with gradient echo (MPRAGE)- or Dixon-based deep learning methods. The Dice coefficient and validation loss of the generated pseudo-CT images were used for comparison. PET error images regarding standardized uptake value ratio (SUVR) were quantified through regional and surface analysis to evaluate the final AC accuracy.

RESULTS

The Dice coefficients of the deep learning methods based on MPRAGE, Dixon, and mUTE images were 0.84 (0.91), 0.84 (0.92), and 0.87 (0.94) for the whole-brain (above-eye) bone regions, respectively, higher than the atlas method of 0.52 (0.64). The regional SUVR error for the atlas method was around 6%, higher than the regional SUV error. The regional SUV and SUVR errors for all deep learning methods were below 2%, with mUTE-based deep learning method performing the best. As for the surface analysis, the atlas method showed the largest error (> 10%) near vertices inside superior frontal, lateral occipital, superior parietal, and inferior temporal cortices. The mUTE-based deep learning method resulted in the least number of regions with error higher than 1%, with the largest error (> 5%) showing up near the inferior temporal and medial orbitofrontal cortices.

CONCLUSION

Deep learning with mUTE can generate accurate AC for amyloid and tau imaging in PET/MR.

MR-based Attenuation Correction for Brain PET Using 3D Cycle-Consistent Adversarial Network

Attenuation correction (AC) is important for the quantitative merits of positron emission tomography (PET). However, attenuation coefficients cannot be derived from magnetic resonance (MR) images directly for PET/MR systems. In this work, we aimed to derive continuous AC maps from Dixon MR images without the requirement of MR and computed tomography (CT) image registration. To achieve this, a 3D generative adversarial network with both discriminative and cycle-consistency loss (Cycle-GAN) was developed. The modified 3D U-net was employed as the structure of the generative networks to generate the pseudo CT/MR images. The 3D patch-based discriminative networks were used to distinguish the generated pseudo CT/MR images from the true CT/MR images. To evaluate its performance, datasets from 32 patients were used in the experiment. The Dixon segmentation and atlas methods provided by the vendor and the convolutional neural network (CNN) method which utilized registered MR and CT images were employed as the reference methods. Dice coefficients of the pseudo-CT image and the regional quantification in the reconstructed PET images were compared. Results show that the Cycle-GAN framework can generate better AC compared to the Dixon segmentation and atlas methods, and shows comparable performance compared to the CNN method.

Application and Construction of Deep Learning Networks in Medical Imaging

Deep learning (DL) approaches are part of the machine learning (ML) subfield concerned with the development of computational models to train artificial intelligence systems. DL models are characterized by automatically extracting high-level features from the input data to learn the relationship between matching datasets. Thus, its implementation offers an advantage over common ML methods that often require the practitioner to have some domain knowledge of the input data to select the best latent representation. As a result of this advantage, DL has been successfully applied within the medical imaging field to address problems, such as disease classification and tumor segmentation for which it is difficult or impossible to determine which image features are relevant. Therefore, taking into consideration the positive impact of DL on the medical imaging field, this article reviews the key concepts associated with its evolution and implementation. The sections of this review summarize the milestones related to the development of the DL field, followed by a description of the elements of deep neural network and an overview of its application within the medical imaging field. Subsequently, the key steps necessary to implement a supervised DL application are defined, and associated limitations are discussed.

A review on medical imaging synthesis using deep learning and its clinical applications

This paper reviewed the deep learning-based studies for medical imaging synthesis and its clinical application. Specifically, we summarized the recent developments of deep learning-based methods in inter- and intra-modality image synthesis by listing and highlighting the proposed methods, study designs, and reported performances with related clinical applications on representative studies. The challenges among the reviewed studies were then summarized with discussion.

Accuracy and precision of zero-echo-time, single- and multi-atlas attenuation correction for dynamic [11C]PE2I PET-MR brain imaging

BACKGROUND

A valid photon attenuation correction (AC) method is instrumental for obtaining quantitatively correct PET images. Integrated PET/MR systems provide no direct information on attenuation, and novel methods for MR-based AC (MRAC) are still under investigation. Evaluations of various AC methods have mainly focused on static brain PET acquisitions. In this study, we determined the validity of three MRAC methods in a dynamic PET/MR study of the brain.

METHODS

Nine participants underwent dynamic brain PET/MR scanning using the dopamine transporter radioligand [11C]PE2I. Three MRAC methods were evaluated: single-atlas (Atlas), multi-atlas (MaxProb) and zero-echo-time (ZTE). The 68Ge-transmission data from a previous stand-alone PET scan was used as reference method. Parametric relative delivery (R1) images and binding potential (BPND) maps were generated using cerebellar grey matter as reference region. Evaluation was based on bias in MRAC maps, accuracy and precision of [11C]PE2I BPND and R1 estimates, and [11C]PE2I time-activity curves. BPND was examined for striatal regions and R1 in clusters of regions across the brain.

RESULTS

For BPND, ZTE-MRAC showed the highest accuracy (bias < 2%) in striatal regions. Atlas-MRAC exhibited a significant bias in caudate nucleus (- 12%) while MaxProb-MRAC revealed a substantial, non-significant bias in the putamen (9%). R1 estimates had a marginal bias for all MRAC methods (- 1.0-3.2%). MaxProb-MRAC showed the largest intersubject variability for both R1 and BPND. Standardized uptake values (SUV) of striatal regions displayed the strongest average bias for ZTE-MRAC (~ 10%), although constant over time and with the smallest intersubject variability. Atlas-MRAC had highest variation in bias over time (+10 to - 10%), followed by MaxProb-MRAC (+5 to - 5%), but MaxProb showed the lowest mean bias. For the cerebellum, MaxProb-MRAC showed the highest variability while bias was constant over time for Atlas- and ZTE-MRAC.

CONCLUSIONS

Both Maxprob- and ZTE-MRAC performed better than Atlas-MRAC when using a 68Ge transmission scan as reference method. Overall, ZTE-MRAC showed the highest precision and accuracy in outcome parameters of dynamic [11C]PE2I PET analysis with use of kinetic modelling.

Comparison of deep learning synthesis of synthetic CTs using clinical MRI inputs

There has been substantial interest in developing techniques for synthesizing CT-like images from MRI inputs, with important applications in simultaneous PET/MR and radiotherapy planning. Deep learning has recently shown great potential for solving this problem. The goal of this research was to investigate the capability of four common clinical MRI sequences (T1-weighted gradient-echo [T1], T2-weighted fat-suppressed fast spin-echo [T2-FatSat], post-contrast T1-weighted gradient-echo [T1-Post], and fast spin-echo T2-weighted fluid-attenuated inversion recovery [CUBE-FLAIR]) as inputs into a deep CT synthesis pipeline. Data were obtained retrospectively in 92 subjects who had undergone an MRI and CT scan on the same day. The patient's MR and CT scans were registered to one another using affine registration. The deep learning model was a convolutional neural network encoder-decoder with skip connections similar to the U-net architecture and Inception V3 inspired blocks instead of sequential convolution blocks. After training with 150 epochs and a batch size of 6, the model was evaluated using structural similarity index (SSIM), peak SNR (PSNR), mean absolute error (MAE), and dice coefficient. We found that feasible results were attainable for each image type, and no single image type was superior for all analyses. The MAE (in HU) of the resulting synthesized CT in the whole brain was 51.236 ± 4.504 for CUBE-FLAIR, 45.432 ± 8.517 for T1, 44.558 ± 7.478 for T1-Post, and 45.721 ± 8.7767 for T2, showing not only feasible, but also very compelling results on clinical images. Deep learning-based synthesis of CT images from MRI is possible with a wide range of inputs, suggesting that viable images can be created from a wide range of clinical input types.

Deep learning for brain disorders: from data processing to disease treatment

In order to reach precision medicine and improve patients' quality of life, machine learning is increasingly used in medicine. Brain disorders are often complex and heterogeneous, and several modalities such as demographic, clinical, imaging, genetics and environmental data have been studied to improve their understanding. Deep learning, a subpart of machine learning, provides complex algorithms that can learn from such various data. It has become state of the art in numerous fields, including computer vision and natural language processing, and is also growingly applied in medicine. In this article, we review the use of deep learning for brain disorders. More specifically, we identify the main applications, the concerned disorders and the types of architectures and data used. Finally, we provide guidelines to bridge the gap between research studies and clinical routine.

A novel supervised learning method to generate CT images for attenuation correction in delayed pet scans

BACKGROUND AND OBJECTIVES

Attenuation correction is important for PET image reconstruction. In clinical PET/CT scans, the attenuation information is usually obtained by CT. However, additional CT scans for delayed PET imaging may increase the risk of cancer. In this paper, we propose a novel CT generation method for attenuation correction in delayed PET imaging that requires no additional CT scans.

METHODS

As only PET raw data is available for the delayed PET scan, routine image registration methods are difficult to use directly. To solve this problem, a reconstruction network is developed to produce pseudo PET images from raw data first. Then a second network is used to generate the CT image through mapping PET/CT images from the first scan to the delayed scan. The inputs of the second network are the two pseudo PET images from the first and delayed scans, and the CT image from the first scan. The labels are taken from the ground truth CT image in the delayed scan. The loss function contains an image similarity term and a regularization term, which reflect the anatomy matching accuracy and the smoothness of the non-rigid deformation field, respectively.

RESULTS

We evaluated the proposed method with simulated and clinical PET/CT datasets. Standard Uptake Value was computed and compared with the gold standard (with coregistered CT for attenuation correction). The results show that the proposed supervised learning method can generate PET images with high quality and quantitative accuracy. For the test cases in our study, the average MAE and RMSE of the proposed supervised learning method were 4.61 and 22.75 respectively, and the average PSNR between the reconstructed PET image and the ground truth PET image was 62.13 dB.

CONCLUSIONS

The proposed method is able to generate accurate CT images for attenuation correction in delayed PET scans. Experiments indicate that the proposed method outperforms traditional methods with respect to quantitative PET image accuracy.

Potentials and caveats of AI in hybrid imaging

State-of-the-art patient management frequently mandates the investigation of both anatomy and physiology of the patients. Hybrid imaging modalities such as the PET/MRI, PET/CT and SPECT/CT have the ability to provide both structural and functional information of the investigated tissues in a single examination. With the introduction of such advanced hardware fusion, new problems arise such as the exceedingly large amount of multi-modality data that requires novel approaches of how to extract a maximum of clinical information from large sets of multi-dimensional imaging data. Artificial intelligence (AI) has emerged as one of the leading technologies that has shown promise in facilitating highly integrative analysis of multi-parametric data. Specifically, the usefulness of AI algorithms in the medical imaging field has been heavily investigated in the realms of (1) image acquisition and reconstruction, (2) post-processing and (3) data mining and modelling. Here, we aim to provide an overview of the challenges encountered in hybrid imaging and discuss how AI algorithms can facilitate potential solutions. In addition, we highlight the pitfalls and challenges in using advanced AI algorithms in the context of hybrid imaging and provide suggestions for building robust AI solutions that enable reproducible and transparent research.

Machine Learning in Nuclear Medicine: Part 2-Neural Networks and Clinical Aspects

This article is the second part in our machine learning series. Part 1 provided a general overview of machine learning in nuclear medicine. Part 2 focuses on neural networks. We start with an example illustrating how neural networks work and a discussion of potential applications. Recognizing that there is a spectrum of applications, we focus on recent publications in the areas of image reconstruction, low-dose PET, disease detection, and models used for diagnosis and outcome prediction. Finally, since the way machine learning algorithms are reported in the literature is extremely variable, we conclude with a call to arms regarding the need for standardized reporting of design and outcome metrics and we propose a basic checklist our community might follow going forward.

Deep-Learning Generation of Synthetic Intermediate Projections Improves 177Lu SPECT Images Reconstructed with Sparsely Acquired Projections

The aims of this study were to decrease the ¹⁷⁷Lu-SPECT acquisition time by reducing the number of projections and to circumvent image degradation by adding deep-learning-generated synthesized projections. Methods: We constructed a deep convolutional U-net-shaped neural network for generation of synthetic intermediate projections (CUSIPs). The number of SPECT investigations was 352 for training, 37 for validation, and 15 for testing. The input was every fourth projection of 120 acquired SPECT projections, that is, 30 projections. The output was 30 synthetic intermediate projections (SIPs) per CUSIP. SPECT images were reconstructed with 120 or 30 projections, or with 120 projections when 90 SIPs were generated from 30 projections (30-120SIPs), using 3 CUSIPs. The reconstructions were performed with 2 ordered-subset expectation maximization (OSEM) algorithms: attenuation-corrected (AC) OSEM, and attenuation, scatter, and collimator response-corrected (ASCC) OSEM. The quality of the SIPs and SPECT images was quantitatively evaluated with root-mean-square error, peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) index metrics. From a Jaszczak SPECT phantom, the recovery and signal-to-noise ratio (SNR) were determined. In addition, an experienced observer qualitatively assessed the SPECT image quality of the test set. Kidney activity concentrations, as determined from the different SPECT images, were compared. Results: The generated SIPs had a mean SSIM value of 0.926 (SD, 0.061). For AC-OSEM, the reconstruction with 30-120SIPs had higher SSIM (0.993 vs. 0.989, P < 0.001) and PSNR (49.5 vs. 47.2, P < 0.001) values than the reconstruction with 30 projections. ASCC-OSEM had higher SSIM and PSNR values than AC-OSEM (P < 0.001). There was a minor loss in recovery for 30-120SIPs, but SNR was clearly improved compared with 30 projections. The observer assessed 27 of 30 images reconstructed with 30 projections as having unacceptable noise levels, whereas the corresponding values were 2 of 60 for 30-120SIPs and 120 projections. Image quality did not differ significantly between 30-120SIPs and 120 projections. The kidney activity concentration was similar between the different projection sets, excepting a minor reduction of 2.5% for ASCC-OSEM 30-120SIPs. Conclusion: Adopting SIPs for sparsely acquired projections considerably recovers image quality and could allow a reduced SPECT acquisition time in clinical dosimetry protocols.

PET/MRI of atherosclerosis

Myocardial infarction and stroke are the most prevalent global causes of death. Each year 15 million people worldwide die due to myocardial infarction or stroke. Rupture of a vulnerable atherosclerotic plaque is the main underlying cause of stroke and myocardial infarction. Key features of a vulnerable plaque are inflammation, a large lipid-rich necrotic core (LRNC) with a thin or ruptured overlying fibrous cap, and intraplaque hemorrhage (IPH). Noninvasive imaging of these features could have a role in risk stratification of myocardial infarction and stroke and can potentially be utilized for treatment guidance and monitoring. The recent development of hybrid PET/MRI combining the superior soft tissue contrast of MRI with the opportunity to visualize specific plaque features using various radioactive tracers, paves the way for comprehensive plaque imaging. In this review, the use of hybrid PET/MRI for atherosclerotic plaque imaging in carotid and coronary arteries is discussed. The pros and cons of different hybrid PET/MRI systems are reviewed. The challenges in the development of PET/MRI and potential solutions are described. An overview of PET and MRI acquisition techniques for imaging of atherosclerosis including motion correction is provided, followed by a summary of vessel wall imaging PET/MRI studies in patients with carotid and coronary artery disease. Finally, the future of imaging of atherosclerosis with PET/MRI is discussed.

Machine learning in quantitative PET: A review of attenuation correction and low-count image reconstruction methods

The rapid expansion of machine learning is offering a new wave of opportunities for nuclear medicine. This paper reviews applications of machine learning for the study of attenuation correction (AC) and low-count image reconstruction in quantitative positron emission tomography (PET). Specifically, we present the developments of machine learning methodology, ranging from random forest and dictionary learning to the latest convolutional neural network-based architectures. For application in PET attenuation correction, two general strategies are reviewed: 1) generating synthetic CT from MR or non-AC PET for the purposes of PET AC, and 2) direct conversion from non-AC PET to AC PET. For low-count PET reconstruction, recent deep learning-based studies and the potential advantages over conventional machine learning-based methods are presented and discussed. In each application, the proposed methods, study designs and performance of published studies are listed and compared with a brief discussion. Finally, the overall contributions and remaining challenges are summarized.

MR-based PET attenuation correction using a combined ultrashort echo time/multi-echo Dixon acquisition

PURPOSE

To develop a magnetic resonance (MR)-based method for estimation of continuous linear attenuation coefficients (LACs) in positron emission tomography (PET) using a physical compartmental model and ultrashort echo time (UTE)/multi-echo Dixon (mUTE) acquisitions.

METHODS

We propose a three-dimensional (3D) mUTE sequence to acquire signals from water, fat, and short T₂ components (e.g., bones) simultaneously in a single acquisition. The proposed mUTE sequence integrates 3D UTE with multi-echo Dixon acquisitions and uses sparse radial trajectories to accelerate imaging speed. Errors in the radial k-space trajectories are measured using a special k-space trajectory mapping sequence and corrected for image reconstruction. A physical compartmental model is used to fit the measured multi-echo MR signals to obtain fractions of water, fat, and bone components for each voxel, which are then used to estimate the continuous LAC map for PET attenuation correction.

RESULTS

The performance of the proposed method was evaluated via phantom and in vivo human studies, using LACs from computed tomography (CT) as reference. Compared to Dixon- and atlas-based MRAC methods, the proposed method yielded PET images with higher correlation and similarity in relation to the reference. The relative absolute errors of PET activity values reconstructed by the proposed method were below 5% in all of the four lobes (frontal, temporal, parietal, and occipital), cerebellum, whole white matter, and gray matter regions across all subjects (n = 6).

CONCLUSIONS

The proposed mUTE method can generate subject-specific, continuous LAC map for PET attenuation correction in PET/MR.

Deep learning-based attenuation correction in the absence of structural information for whole-body positron emission tomography imaging

Deriving accurate structural maps for attenuation correction (AC) of whole-body positron emission tomography (PET) remains challenging. Common problems include truncation, inter-scan motion, and erroneous transformation of structural voxel-intensities to PET µ-map values (e.g. modality artifacts, implanted devices, or contrast agents). This work presents a deep learning-based attenuation correction (DL-AC) method to generate attenuation corrected PET (AC PET) from non-attenuation corrected PET (NAC PET) images for whole-body PET imaging, without the use of structural information. 3D patch-based cycle-consistent generative adversarial networks (CycleGAN) is introduced to include NAC-PET-to-AC-PET mapping and inverse mapping from AC PET to NAC PET, which constrains NAC-PET-to-AC-PET mapping to be closer to one-to-one mapping. Since NAC PET images share similar anatomical structures to the AC PET image but lack contrast information, residual blocks, which aim to learn the differences between NAC PET and AC PET, are used to construct generators of CycleGAN. After training, patches from NAC PET images were fed into NAC-PET-to-AC-PET mapping to generate DL-AC PET patches. DL-AC PET image was then reconstructed through patch fusion. We conducted a retrospective study on 55 datasets of whole-body PET/CT scans to evaluate the proposed method. In comparing DL-AC PET with original AC PET, average mean error (ME) and normalized mean square error (NMSE) of the whole-body were 0.62%  ±  1.26% and 0.72%  ±  0.34%. The average intensity changes measured on sequential PET images with AC and DL-AC on both normal tissues and lesions differ less than 3%. There was no significant difference of the intensity changes between AC and DL-AC PET, which demonstrate DL-AC PET images generated by the proposed DL-AC method can reach a same level to that of original AC PET images. The method demonstrates excellent quantification accuracy and reliability and is applicable to PET data collected on a single PET scanner or hybrid platform with computed tomography (PET/CT) or magnetic resonance imaging (PET/MRI).

Applying Deep Learning to Predicting Dementia and Mild Cognitive Impairment

Dementia has a large negative impact on the global healthcare and society. Diagnosis is rather challenging as there is no standardised test. The purpose of this paper is to conduct an analysis on ADNI data and determine its effectiveness for building classification models to differentiate the categories Cognitively Normal (CN), Mild Cognitive Impairment (MCI), and Dementia (DEM), based on tuning three Deep Learning models: two Multi-Layer Perceptron (MLP1 and MLP2) models and a Convolutional Bidirectional Long Short-Term Memory (ConvBLSTM) model. The results show that the MLP1 and MLP2 models accurately distinguish the DEM, MCI and CN classes, with accuracies as high as 0.86 (SD 0.01). The ConvBLSTM model was slightly less accurate but was explored in view of comparisons with the MLP models, and for future extensions of this work that will take advantage of time-related information. Although the performance of ConvBLSTM model was negatively impacted by a lack of visit code data, opportunities were identified for improvement, particularly in terms of pre-processing.

Machine Learning in PET: from Photon Detection to Quantitative Image Reconstruction

Machine learning has found unique applications in nuclear medicine from photon detection to quantitative image reconstruction. While there have been impressive strides in detector development for time-of-flight positron emission tomography, most detectors still make use of simple signal processing methods to extract the time and position information from the detector signals. Now with the availability of fast waveform digitizers, machine learning techniques have been applied to estimate the position and arrival time of high-energy photons. In quantitative image reconstruction, machine learning has been used to estimate various corrections factors, including scattered events and attenuation images, as well as to reduce statistical noise in reconstructed images. Here machine learning either provides a faster alternative to an existing time-consuming computation, such as in the case of scatter estimation, or creates a data-driven approach to map an implicitly defined function, such as in the case of estimating the attenuation map for PET/MR scans. In this article, we will review the abovementioned applications of machine learning in nuclear medicine.

Deep neural network for automatic characterization of lesions on 68Ga-PSMA-11 PET/CT

PURPOSE

This study proposes an automated prostate cancer (PC) lesion characterization method based on the deep neural network to determine tumor burden on ⁶⁸Ga-PSMA-11 PET/CT to potentially facilitate the optimization of PSMA-directed radionuclide therapy.

METHODS

We collected ⁶⁸Ga-PSMA-11 PET/CT images from 193 patients with metastatic PC at three medical centers. For proof-of-concept, we focused on the detection of pelvis bone and lymph node lesions. A deep neural network (triple-combining 2.5D U-Net) was developed for the automated characterization of these lesions. The proposed method simultaneously extracts features from axial, coronal, and sagittal planes, which mimics the workflow of physicians and reduces computational and memory requirements.

RESULTS

Among all the labeled lesions, the network achieved 99% precision, 99% recall, and an F1 score of 99% on bone lesion detection and 94%, precision 89% recall, and an F1 score of 92% on lymph node lesion detection. The segmentation accuracy is lower than the detection. The performance of the network was correlated with the amount of training data.

CONCLUSION

We developed a deep neural network to characterize automatically the PC lesions on ⁶⁸Ga-PSMA-11 PET/CT. The preliminary test within the pelvic area confirms the potential of deep learning methods. Increasing the amount of training data should further enhance the performance of the proposed method and may ultimately allow whole-body assessments.

Synthetic CT generation from non-attenuation corrected PET images for whole-body PET imaging

Attenuation correction (AC) of PET/MRI faces challenges including inter-scan motion, image artifacts such as truncation and distortion, and erroneous transformation of structural voxel-intensities to PET mu-map values. We propose a deep-learning-based method to derive synthetic CT (sCT) images from non-attenuation corrected PET (NAC PET) images for AC on whole-body PET/MRI imaging. A 3D cycle-consistent generative adversarial networks (CycleGAN) framework was employed to synthesize CT images from NAC PET. The method learns a transformation that minimizes the difference between sCT, generated from NAC PET, and true CT. It also learns an inverse transformation such that cycle NAC PET image generated from the sCT is close to true NAC PET image. A self-attention strategy was also utilized to identify the most informative component and mitigate the disturbance of noise. We conducted a retrospective study on a total of 119 sets of whole-body PET/CT, with 80 sets for training and 39 sets for testing and evaluation. The whole-body sCT images generated with proposed method demonstrate great resemblance to true CT images, and show good contrast on soft tissue, lung and bony tissues. The mean absolute error (MAE) of sCT over true CT is less than 110 HU. Using sCT for whole-body PET AC, the mean error of PET quantification is less than 1% and normalized mean square error (NMSE) is less than 1.4%. Average normalized cross correlation on whole body is close to one, and PSNR is larger than 42 dB. We proposed a deep learning-based approach to generate sCT from whole-body NAC PET for PET AC. sCT generated with proposed method shows great similarity to true CT images both qualitatively and quantitatively, and demonstrates great potential for whole-body PET AC in the absence of structural information.

Attenuation correction using 3D deep convolutional neural network for brain 18F-FDG PET/MR: Comparison with Atlas, ZTE and CT based attenuation correction

One of the main technical challenges of PET/MRI is to achieve an accurate PET attenuation correction (AC) estimation. In current systems, AC is accomplished by generating an MRI-based surrogate computed tomography (CT) from which AC-maps are derived. Nevertheless, all techniques currently implemented in clinical routine suffer from bias. We present here a convolutional neural network (CNN) that generated AC-maps from Zero Echo Time (ZTE) MR images. Seventy patients referred to our institution for ¹⁸FDG-PET/MR exam (SIGNA PET/MR, GE Healthcare) as part of the investigation of suspected dementia, were included. 23 patients were added to the training set of the manufacturer and 47 were used for validation. Brain computed tomography (CT) scan, two-point LAVA-flex MRI (for atlas-based AC) and ZTE-MRI were available in all patients. Three AC methods were evaluated and compared to CT-based AC (CTAC): one based on a single head-atlas, one based on ZTE-segmentation and one CNN with a 3D U-net architecture to generate AC maps from ZTE MR images. Impact on brain metabolism was evaluated combining voxel and regions-of-interest based analyses with CTAC set as reference. The U-net AC method yielded the lowest bias, the lowest inter-individual and inter-regional variability compared to PET images reconstructed with ZTE and Atlas methods. The impact on brain metabolism was negligible with average errors of -0.2% in most cortical regions. These results suggest that the U-net AC is more reliable for correcting photon attenuation in brain FDG-PET/MR than atlas-AC and ZTE-AC methods.

Impact of non-uniform attenuation correction in a dynamic [18F]-FDOPA brain PET/MRI study

Background

PET (positron emission tomography) biokinetic modelling relies on accurate quantitative data. One of the main corrections required in PET imaging to obtain high quantitative accuracy is tissue attenuation correction (AC). Incorrect non-uniform PET-AC may result in local bias in the emission images, and thus in relative activity distributions and time activity curves for different regions. MRI (magnetic resonance imaging)-based AC is an active area of research in PET/MRI neuroimaging, where several groups developed in the last few years different methods to calculate accurate attenuation (μ-)maps. Some AC methods have been evaluated for different PET radioisotopes and pathologies. However, AC in PET/MRI has scantly been investigated in dynamic PET studies where the aim is to get quantitative kinetic parameters, rather than semi-quantitative parameters from static PET studies. In this work, we investigated the impact of AC accuracy in PET image absolute quantification and, more importantly, in the slope of the Patlak analysis based on the simplified reference tissue model, from a dynamic [¹⁸F]-fluorodopa (FDOPA) PET/MRI study. In the study, we considered the two AC methods provided by the vendor and an in-house AC method based on the dual ultrashort time echo MRI sequence, using as reference a multi-atlas-based AC method based on a T1-weighted MRI sequence.

Results

Non-uniform bias in absolute PET quantification across the brain, from − 20% near the skull to − 10% in the central region, was observed using the two vendor’s μ-maps. The AC method developed in-house showed a − 5% and 1% bias, respectively. Our study resulted in a 5–9% overestimation of the PET kinetic parameters with the vendor-provided μ-maps, while our in-house-developed AC method showed < 2% overestimation compared to the atlas-based AC method, using the cerebellar cortex as reference region. The overestimation obtained using the occipital pole as reference region resulted in a 7–10% with the vendor-provided μ-maps, while our in-house-developed AC method showed < 6% overestimation.

Conclusions

PET kinetic analyses based on a reference region are especially sensitive to the non-uniform bias in PET quantification from AC inaccuracies in brain PET/MRI. Depending on the position of the reference region and the bias with respect to the analysed region, kinetic analyses suffer different levels of bias. Considering bone in the μ-map can potentially result in larger errors, compared to the absence of bone, when non-uniformities in PET quantification are introduced.

Direct attenuation correction of brain PET images using only emission data via a deep convolutional encoder-decoder (Deep-DAC)

OBJECTIVE

To obtain attenuation-corrected PET images directly from non-attenuation-corrected images using a convolutional encoder-decoder network.

METHODS

Brain PET images from 129 patients were evaluated. The network was designed to map non-attenuation-corrected (NAC) images to pixel-wise continuously valued measured attenuation-corrected (MAC) PET images via an encoder-decoder architecture. Image quality was evaluated using various evaluation metrics. Image quantification was assessed for 19 radiomic features in 83 brain regions as delineated using the Hammersmith atlas (n30r83). Reliability of measurements was determined using pixel-wise relative errors (RE; %) for radiomic feature values in reference MAC PET images.

RESULTS

Peak signal-to-noise ratio (PSNR) and structural similarity index metric (SSIM) values were 39.2 ± 3.65 and 0.989 ± 0.006 for the external validation set, respectively. RE (%) of SUV_mean was - 0.10 ± 2.14 for all regions, and only 3 of 83 regions depicted significant differences. However, the mean RE (%) of this region was 0.02 (range, - 0.83 to 1.18). SUV_max had mean RE (%) of - 3.87 ± 2.84 for all brain regions, and 17 regions in the brain depicted significant differences with respect to MAC images with a mean RE of - 3.99 ± 2.11 (range, - 8.46 to 0.76). Homogeneity amongst Haralick-based radiomic features had the highest number (20) of regions with significant differences with a mean RE (%) of 7.22 ± 2.99.

CONCLUSIONS

Direct AC of PET images using deep convolutional encoder-decoder networks is a promising technique for brain PET images. The proposed deep learning method shows significant potential for emission-based AC in PET images with applications in PET/MRI and dedicated brain PET scanners.

KEY POINTS

• We demonstrate direct emission-based attenuation correction of PET images without using anatomical information. • We performed radiomics analysis of 83 brain regions to show robustness of direct attenuation correction of PET images. • Deep learning methods have significant promise for emission-based attenuation correction in PET images with potential applications in PET/MRI and dedicated brain PET scanners.