Feasibility of Deep Learning-Based PET/MR Attenuation Correction in the Pelvis Using Only Diagnostic MR Images

A systematic literature review: deep learning techniques for synthetic medical image generation and their applications in radiotherapy

The aim of this systematic review is to determine whether Deep Learning (DL) algorithms can provide a clinically feasible alternative to classic algorithms for synthetic Computer Tomography (sCT). The following categories are presented in this study: ∙ MR-based treatment planning and synthetic CT generation techniques. ∙ Generation of synthetic CT images based on Cone Beam CT images. ∙ Low-dose CT to High-dose CT generation. ∙ Attenuation correction for PET images. To perform appropriate database searches, we reviewed journal articles published between January 2018 and June 2023. Current methodology, study strategies, and results with relevant clinical applications were analyzed as we outlined the state-of-the-art of deep learning based approaches to inter-modality and intra-modality image synthesis. This was accomplished by contrasting the provided methodologies with traditional research approaches. The key contributions of each category were highlighted, specific challenges were identified, and accomplishments were summarized. As a final step, the statistics of all the cited works from various aspects were analyzed, which revealed that DL-based sCTs have achieved considerable popularity, while also showing the potential of this technology. In order to assess the clinical readiness of the presented methods, we examined the current status of DL-based sCT generation.

Attenuation correction and truncation completion for breast PET/MR imaging using deep learning

Objective. Simultaneous PET/MR scanners combine the high sensitivity of MR imaging with the functional imaging of PET. However, attenuation correction of breast PET/MR imaging is technically challenging. The purpose of this study is to establish a robust attenuation correction algorithm for breast PET/MR images that relies on deep learning (DL) to recreate the missing portions of the patient's anatomy (truncation completion), as well as to provide bone information for attenuation correction from only the PET data.Approach. Data acquired from 23 female subjects with invasive breast cancer scanned with18F-fluorodeoxyglucose PET/CT and PET/MR localized to the breast region were used for this study. Three DL models, U-Net with mean absolute error loss (DLMAE) model, U-Net with mean squared error loss (DLMSE) model, and U-Net with perceptual loss (DLPerceptual) model, were trained to predict synthetic CT images (sCT) for PET attenuation correction (AC) given non-attenuation corrected (NAC) PETPET/MRimages as inputs. The DL and Dixon-based sCT reconstructed PET images were compared against those reconstructed from CT images by calculating the percent error of the standardized uptake value (SUV) and conducting Wilcoxon signed rank statistical tests.Main results. sCT images from the DLMAEmodel, the DLMSEmodel, and the DLPerceptualmodel were similar in mean absolute error (MAE), peak-signal-to-noise ratio, and normalized cross-correlation. No significant difference in SUV was found between the PET images reconstructed using the DLMSEand DLPerceptualsCTs compared to the reference CT for AC in all tissue regions. All DL methods performed better than the Dixon-based method according to SUV analysis.Significance. A 3D U-Net with MSE or perceptual loss model can be implemented into a reconstruction workflow, and the derived sCT images allow successful truncation completion and attenuation correction for breast PET/MR images.

Evaluating a radiotherapy deep learning synthetic CT algorithm for PET-MR attenuation correction in the pelvis

BACKGROUND

Positron emission tomography-magnetic resonance (PET-MR) attenuation correction is challenging because the MR signal does not represent tissue density and conventional MR sequences cannot image bone. A novel zero echo time (ZTE) MR sequence has been previously developed which generates signal from cortical bone with images acquired in 65 s. This has been combined with a deep learning model to generate a synthetic computed tomography (sCT) for MR-only radiotherapy. This study aimed to evaluate this algorithm for PET-MR attenuation correction in the pelvis.

METHODS

Ten patients being treated with ano-rectal radiotherapy received a [Formula: see text]F-FDG-PET-MR in the radiotherapy position. Attenuation maps were generated from ZTE-based sCT (sCTAC) and the standard vendor-supplied MRAC. The radiotherapy planning CT scan was rigidly registered and cropped to generate a gold standard attenuation map (CTAC). PET images were reconstructed using each attenuation map and compared for standard uptake value (SUV) measurement, automatic thresholded gross tumour volume (GTV) delineation and GTV metabolic parameter measurement. The last was assessed for clinical equivalence to CTAC using two one-sided paired t tests with a significance level corrected for multiple testing of [Formula: see text]. Equivalence margins of [Formula: see text] were used.

RESULTS

Mean whole-image SUV differences were -0.02% (sCTAC) compared to -3.0% (MRAC), with larger differences in the bone regions (-0.5% to -16.3%). There was no difference in thresholded GTVs, with Dice similarity coefficients [Formula: see text]. However, there were larger differences in GTV metabolic parameters. Mean differences to CTAC in [Formula: see text] were [Formula: see text] (± standard error, sCTAC) and [Formula: see text] (MRAC), and [Formula: see text] (sCTAC) and [Formula: see text] (MRAC) in [Formula: see text]. The sCTAC was statistically equivalent to CTAC within a [Formula: see text] equivalence margin for [Formula: see text] and [Formula: see text] ([Formula: see text] and [Formula: see text]), whereas the MRAC was not ([Formula: see text] and [Formula: see text]).

CONCLUSION

Attenuation correction using this radiotherapy ZTE-based sCT algorithm was substantially more accurate than current MRAC methods with only a 40 s increase in MR acquisition time. This did not impact tumour delineation but did significantly improve the accuracy of whole-image and tumour SUV measurements, which were clinically equivalent to CTAC. This suggests PET images reconstructed with sCTAC would enable accurate quantitative PET images to be acquired on a PET-MR scanner.

Systematic evaluation of human soft tissue attenuation correction in whole-body PET/MR: Implications from PET/CT for optimization of MR-based AC in patients with normal lung tissue

BACKGROUND

Attenuation correction (AC) is an important methodical step in positron emission tomography/magnetic resonance imaging (PET/MRI) to correct for attenuated and scattered PET photons.

PURPOSE

The overall quality of magnetic resonance (MR)-based AC in whole-body PET/MRI was evaluated in direct comparison to computed tomography (CT)-based AC serving as reference. The quantitative impact of isolated tissue classes in the MR-AC was systematically investigated to identify potential optimization needs and strategies.

METHODS

Data of n = 60 whole-body PET/CT patients with normal lung tissue and without metal implants/prostheses were used to generate six different AC-models based on the CT data for each patient, simulating variations of MR-AC. The original continuous CT-AC (CT-org) is referred to as reference. A pseudo MR-AC (CT-mrac), generated from CT data, with four tissue classes and a bone atlas represents the MR-AC. Relative difference in linear attenuation coefficients (LAC) and standardized uptake values were calculated. From the results two improvements regarding soft tissue AC and lung AC were proposed and evaluated.

RESULTS

The overall performance of MR-AC is in good agreement compared to CT-AC. Lungs, heart, and bone tissue were identified as the regions with most deviation to the CT-AC (myocardium -15%, bone tissue -14%, and lungs ±20%). Using single-valued LACs for AC in the lung only provides limited accuracy. For improved soft tissue AC, splitting the combined soft tissue class into muscles and organs each with adapted LAC could reduce the deviations to the CT-AC to < ±1%. For improved lung AC, applying a gradient LAC in the lungs could remarkably reduce over- or undercorrections in PET signal compared to CT-AC (±5%).

CONCLUSIONS

The AC is important to ensure best PET image quality and accurate PET quantification for diagnostics and radiotherapy planning. The optimized segment-based AC proposed in this study, which was evaluated on PET/CT data, inherently reduces quantification bias in normal lung tissue and soft tissue compared to the CT-AC reference.

Attenuation Correction of Long Axial Field-of-View Positron Emission Tomography Using Synthetic Computed Tomography Derived from the Emission Data: Application to Low-Count Studies and Multiple Tracers

Recent advancements in PET/CT, including the emergence of long axial field-of-view (LAFOV) PET/CT scanners, have increased PET sensitivity substantially. Consequently, there has been a significant reduction in the required tracer activity, shifting the primary source of patient radiation dose exposure to the attenuation correction (AC) CT scan during PET imaging. This study proposes a parameter-transferred conditional generative adversarial network (PT-cGAN) architecture to generate synthetic CT (sCT) images from non-attenuation corrected (NAC) PET images, with separate networks for [18F]FDG and [15O]H2O tracers. The study includes a total of 1018 subjects (n = 972 [18F]FDG, n = 46 [15O]H2O). Testing was performed on the LAFOV scanner for both datasets. Qualitative analysis found no differences in image quality in 30 out of 36 cases in FDG patients, with minor insignificant differences in the remaining 6 cases. Reduced artifacts due to motion between NAC PET and CT were found. For the selected organs, a mean average error of 0.45% was found for the FDG cohort, and that of 3.12% was found for the H2O cohort. Simulated low-count images were included in testing, which demonstrated good performance down to 45 s scans. These findings show that the AC of total-body PET is feasible across tracers and in low-count studies and might reduce the artifacts due to motion and metal implants.

Deep-learning-based methods of attenuation correction for SPECT and PET

Attenuation correction (AC) is essential for quantitative analysis and clinical diagnosis of single-photon emission computed tomography (SPECT) and positron emission tomography (PET). In clinical practice, computed tomography (CT) is utilized to generate attenuation maps (μ-maps) for AC of hybrid SPECT/CT and PET/CT scanners. However, CT-based AC methods frequently produce artifacts due to CT artifacts and misregistration of SPECT-CT and PET-CT scans. Segmentation-based AC methods using magnetic resonance imaging (MRI) for PET/MRI scanners are inaccurate and complicated since MRI does not contain direct information of photon attenuation. Computational AC methods for SPECT and PET estimate attenuation coefficients directly from raw emission data, but suffer from low accuracy, cross-talk artifacts, high computational complexity, and high noise level. The recently evolving deep-learning-based methods have shown promising results in AC of SPECT and PET, which can be generally divided into two categories: indirect and direct strategies. Indirect AC strategies apply neural networks to transform emission, transmission, or MR images into synthetic μ-maps or CT images which are then incorporated into AC reconstruction. Direct AC strategies skip the intermediate steps of generating μ-maps or CT images and predict AC SPECT or PET images from non-attenuation-correction (NAC) SPECT or PET images directly. These deep-learning-based AC methods show comparable and even superior performance to non-deep-learning methods. In this article, we first discussed the principles and limitations of non-deep-learning AC methods, and then reviewed the status and prospects of deep-learning-based methods for AC of SPECT and PET.

A review of PET attenuation correction methods for PET-MR

Despite being thirteen years since the installation of the first PET-MR system, the scanners constitute a very small proportion of the total hybrid PET systems installed. This is in stark contrast to the rapid expansion of the PET-CT scanner, which quickly established its importance in patient diagnosis within a similar timeframe. One of the main hurdles is the development of an accurate, reproducible and easy-to-use method for attenuation correction. Quantitative discrepancies in PET images between the manufacturer-provided MR methods and the more established CT- or transmission-based attenuation correction methods have led the scientific community in a continuous effort to develop a robust and accurate alternative. These can be divided into four broad categories: (i) MR-based, (ii) emission-based, (iii) atlas-based and the (iv) machine learning-based attenuation correction, which is rapidly gaining momentum. The first is based on segmenting the MR images in various tissues and allocating a predefined attenuation coefficient for each tissue. Emission-based attenuation correction methods aim in utilising the PET emission data by simultaneously reconstructing the radioactivity distribution and the attenuation image. Atlas-based attenuation correction methods aim to predict a CT or transmission image given an MR image of a new patient, by using databases containing CT or transmission images from the general population. Finally, in machine learning methods, a model that could predict the required image given the acquired MR or non-attenuation-corrected PET image is developed by exploiting the underlying features of the images. Deep learning methods are the dominant approach in this category. Compared to the more traditional machine learning, which uses structured data for building a model, deep learning makes direct use of the acquired images to identify underlying features. This up-to-date review goes through the literature of attenuation correction approaches in PET-MR after categorising them. The various approaches in each category are described and discussed. After exploring each category separately, a general overview is given of the current status and potential future approaches along with a comparison of the four outlined categories.

Pelvic PET/MR attenuation correction in the image space using deep learning

Introduction

The five-class Dixon-based PET/MR attenuation correction (AC) model, which adds bone information to the four-class model by registering major bones from a bone atlas, has been shown to be error-prone. In this study, we introduce a novel method of accounting for bone in pelvic PET/MR AC by directly predicting the errors in the PET image space caused by the lack of bone in four-class Dixon-based attenuation correction.

Methods

A convolutional neural network was trained to predict the four-class AC error map relative to CT-based attenuation correction. Dixon MR images and the four-class attenuation correction µ-map were used as input to the models. CT and PET/MR examinations for 22 patients ([18F]FDG) were used for training and validation, and 17 patients were used for testing (6 [18F]PSMA-1007 and 11 [68Ga]Ga-PSMA-11). A quantitative analysis of PSMA uptake using voxel- and lesion-based error metrics was used to assess performance.

Results

In the voxel-based analysis, the proposed model reduced the median root mean squared percentage error from 12.1% and 8.6% for the four- and five-class Dixon-based AC methods, respectively, to 6.2%. The median absolute percentage error in the maximum standardized uptake value (SUVmax) in bone lesions improved from 20.0% and 7.0% for four- and five-class Dixon-based AC methods to 3.8%.

Conclusion

The proposed method reduces the voxel-based error and SUVmax errors in bone lesions when compared to the four- and five-class Dixon-based AC models.

Deep learning-based attenuation map generation with simultaneously reconstructed PET activity and attenuation and low-dose application

Objective. In PET/CT imaging, CT is used for positron emission tomography (PET) attenuation correction (AC). CT artifacts or misalignment between PET and CT can cause AC artifacts and quantification errors in PET. Simultaneous reconstruction (MLAA) of PET activity (λ-MLAA) and attenuation (μ-MLAA) maps was proposed to solve those issues using the time-of-flight PET raw data only. However,λ-MLAA still suffers from quantification error as compared to reconstruction using the gold-standard CT-based attenuation map (μ-CT). Recently, a deep learning (DL)-based framework was proposed to improve MLAA by predictingμ-DL fromλ-MLAA andμ-MLAA using an image domain loss function (IM-loss). However, IM-loss does not directly measure the AC errors according to the PET attenuation physics. Our preliminary studies showed that an additional physics-based loss function can lead to more accurate PET AC. The main objective of this study is to optimize the attenuation map generation framework for clinical full-dose¹⁸F-FDG studies. We also investigate the effectiveness of the optimized network on predicting attenuation maps for synthetic low-dose oncological PET studies.Approach. We optimized the proposed DL framework by applying different preprocessing steps and hyperparameter optimization, including patch size, weights of the loss terms and number of angles in the projection-domain loss term. The optimization was performed based on 100 skull-to-toe¹⁸F-FDG PET/CT scans with minimal misalignment. The optimized framework was further evaluated on 85 clinical full-dose neck-to-thigh¹⁸F-FDG cancer datasets as well as synthetic low-dose studies with only 10% of the full-dose raw data.Main results. Clinical evaluation of tumor quantification as well as physics-based figure-of-merit metric evaluation validated the promising performance of our proposed method. For both full-dose and low-dose studies, the proposed framework achieved <1% error in tumor standardized uptake value measures.Significance. It is of great clinical interest to achieve CT-less PET reconstruction, especially for low-dose PET studies.

Data Management and Network Architecture Effect on Performance Variability in Direct Attenuation Correction via Deep Learning for Cardiac SPECT: A Feasibility Study

Attenuation correction (AC) is important for accurate interpretation of SPECT myocardial perfusion imaging (MPI). However, it is challenging to perform AC in dedicated cardiac systems not equipped with a transmission imaging capability. Previously, we demonstrated the feasibility of generating attenuation-corrected SPECT images using a deep learning technique (SPECTDL) directly from non-corrected images (SPECTNC). However, we observed performance variability across patients which is an important factor for clinical translation of the technique. In this study, we investigate the feasibility of overcoming the performance variability across patients for the direct AC in SPECT MPI by proposing to develop an advanced network and a data management strategy. To investigate, we compared the accuracy of the SPECTDL for the conventional U-Net and Wasserstein cycle GAN (WCycleGAN) networks. To manage the training data, clustering was applied to a representation of data in the lower-dimensional space, and the training data were chosen based on the similarity of data in this space. Quantitative analysis demonstrated that DL model with an advanced network improves the global performance for the AC task with the limited data. However, the regional results were not improved. The proposed data management strategy demonstrated that the clustered training has potential benefit for effective training.

A deep learning-based whole-body solution for PET/MRI attenuation correction

BACKGROUND

Deep convolutional neural networks have demonstrated robust and reliable PET attenuation correction (AC) as an alternative to conventional AC methods in integrated PET/MRI systems. However, its whole-body implementation is still challenging due to anatomical variations and the limited MRI field of view. The aim of this study is to investigate a deep learning (DL) method to generate voxel-based synthetic CT (sCT) from Dixon MRI and use it as a whole-body solution for PET AC in a PET/MRI system.

MATERIALS AND METHODS

Fifteen patients underwent PET/CT followed by PET/MRI with whole-body coverage from skull to feet. We performed MRI truncation correction and employed co-registered MRI and CT images for training and leave-one-out cross-validation. The network was pretrained with region-specific images. The accuracy of the AC maps and reconstructed PET images were assessed by performing a voxel-wise analysis and calculating the quantification error in SUV obtained using DL-based sCT (PET_sCT) and a vendor-provided atlas-based method (PET_Atlas), with the CT-based reconstruction (PET_CT) serving as the reference. In addition, region-specific analysis was performed to compare the performances of the methods in brain, lung, liver, spine, pelvic bone, and aorta.

RESULTS

Our DL-based method resulted in better estimates of AC maps with a mean absolute error of 62 HU, compared to 109 HU for the atlas-based method. We found an excellent voxel-by-voxel correlation between PET_CT and PET_sCT (R² = 0.98). The absolute percentage difference in PET quantification for the entire image was 6.1% for PET_sCT and 11.2% for PET_Atlas. The regional analysis showed that the average errors and the variability for PET_sCT were lower than PET_Atlas in all regions. The largest errors were observed in the lung, while the smallest biases were observed in the brain and liver.

CONCLUSIONS

Experimental results demonstrated that a DL approach for whole-body PET AC in PET/MRI is feasible and allows for more accurate results compared with conventional methods. Further evaluation using a larger training cohort is required for more accurate and robust performance.

Quantitative evaluation of a deep learning-based framework to generate whole-body attenuation maps using LSO background radiation in long axial FOV PET scanners

Purpose

Attenuation correction is a critically important step in data correction in positron emission tomography (PET) image formation. The current standard method involves conversion of Hounsfield units from a computed tomography (CT) image to construct attenuation maps (µ-maps) at 511 keV. In this work, the increased sensitivity of long axial field-of-view (LAFOV) PET scanners was exploited to develop and evaluate a deep learning (DL) and joint reconstruction-based method to generate µ-maps utilizing background radiation from lutetium-based (LSO) scintillators.

Methods

Data from 18 subjects were used to train convolutional neural networks to enhance initial µ-maps generated using joint activity and attenuation reconstruction algorithm (MLACF) with transmission data from LSO background radiation acquired before and after the administration of ¹⁸F-fluorodeoxyglucose (¹⁸F-FDG) (µ-map_MLACF-PRE and µ-map_MLACF-POST respectively). The deep learning-enhanced µ-maps (µ-map_DL-MLACF-PRE and µ-map_{DL-MLACF-POST}) were compared against MLACF-derived and CT-based maps (µ-map_CT). The performance of the method was also evaluated by assessing PET images reconstructed using each µ-map and computing volume-of-interest based standard uptake value measurements and percentage relative mean error (rME) and relative mean absolute error (rMAE) relative to CT-based method.

Results

No statistically significant difference was observed in rME values for µ-map_DL-MLACF-PRE and µ-map_{DL-MLACF-POST} both in fat-based and water-based soft tissue as well as bones, suggesting that presence of the radiopharmaceutical activity in the body had negligible effects on the resulting µ-maps. The rMAE values µ-map_{DL-MLACF-POST} were reduced by a factor of 3.3 in average compared to the rMAE of µ-map_MLACF-POST. Similarly, the average rMAE values of PET images reconstructed using µ-map_{DL-MLACF-POST} (PET_{DL-MLACF-POST}) were 2.6 times smaller than the average rMAE values of PET images reconstructed using µ-map_MLACF-POST. The mean absolute errors in SUV values of PET_{DL-MLACF-POST} compared to PET_CT were less than 5% in healthy organs, less than 7% in brain grey matter and 4.3% for all tumours combined.

Conclusion

We describe a deep learning-based method to accurately generate µ-maps from PET emission data and LSO background radiation, enabling CT-free attenuation and scatter correction in LAFOV PET scanners.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00259-022-05909-3.

Deep learning-based attenuation correction for whole-body PET - a multi-tracer study with 18F-FDG, 68 Ga-DOTATATE, and 18F-Fluciclovine

A novel deep learning (DL)-based attenuation correction (AC) framework was applied to clinical whole-body oncology studies using 18F-FDG, 68 Ga-DOTATATE, and 18F-Fluciclovine. The framework used activity (λ-MLAA) and attenuation (µ-MLAA) maps estimated by the maximum likelihood reconstruction of activity and attenuation (MLAA) algorithm as inputs to a modified U-net neural network with a novel imaging physics-based loss function to learn a CT-derived attenuation map (µ-CT).

METHODS

Clinical whole-body PET/CT datasets of 18F-FDG (N = 113), 68 Ga-DOTATATE (N = 76), and 18F-Fluciclovine (N = 90) were used to train and test tracer-specific neural networks. For each tracer, forty subjects were used to train the neural network to predict attenuation maps (µ-DL). µ-DL and µ-MLAA were compared to the gold-standard µ-CT. PET images reconstructed using the OSEM algorithm with µ-DL (OSEMDL) and µ-MLAA (OSEMMLAA) were compared to the CT-based reconstruction (OSEMCT). Tumor regions of interest were segmented by two radiologists and tumor SUV and volume measures were reported, as well as evaluation using conventional image analysis metrics.

RESULTS

µ-DL yielded high resolution and fine detail recovery of the attenuation map, which was superior in quality as compared to µ-MLAA in all metrics for all tracers. Using OSEMCT as the gold-standard, OSEMDL provided more accurate tumor quantification than OSEMMLAA for all three tracers, e.g., error in SUVmax for OSEMMLAA vs. OSEMDL: - 3.6 ± 4.4% vs. - 1.7 ± 4.5% for 18F-FDG (N = 152), - 4.3 ± 5.1% vs. 0.4 ± 2.8% for 68 Ga-DOTATATE (N = 70), and - 7.3 ± 2.9% vs. - 2.8 ± 2.3% for 18F-Fluciclovine (N = 44). OSEMDL also yielded more accurate tumor volume measures than OSEMMLAA, i.e., - 8.4 ± 14.5% (OSEMMLAA) vs. - 3.0 ± 15.0% for 18F-FDG, - 14.1 ± 19.7% vs. 1.8 ± 11.6% for 68 Ga-DOTATATE, and - 15.9 ± 9.1% vs. - 6.4 ± 6.4% for 18F-Fluciclovine.

CONCLUSIONS

The proposed framework provides accurate and robust attenuation correction for whole-body 18F-FDG, 68 Ga-DOTATATE and 18F-Fluciclovine in tumor SUV measures as well as tumor volume estimation. The proposed method provides clinically equivalent quality as compared to CT in attenuation correction for the three tracers.

Unsupervised-learning-based method for chest MRI-CT transformation using structure constrained unsupervised generative attention networks

The integrated positron emission tomography/magnetic resonance imaging (PET/MRI) scanner simultaneously acquires metabolic information via PET and morphological information using MRI. However, attenuation correction, which is necessary for quantitative PET evaluation, is difficult as it requires the generation of attenuation-correction maps from MRI, which has no direct relationship with the gamma-ray attenuation information. MRI-based bone tissue segmentation is potentially available for attenuation correction in relatively rigid and fixed organs such as the head and pelvis regions. However, this is challenging for the chest region because of respiratory and cardiac motions in the chest, its anatomically complicated structure, and the thin bone cortex. We propose a new method using unsupervised generative attentional networks with adaptive layer-instance normalisation for image-to-image translation (U-GAT-IT), which specialised in unpaired image transformation based on attention maps for image transformation. We added the modality-independent neighbourhood descriptor (MIND) to the loss of U-GAT-IT to guarantee anatomical consistency in the image transformation between different domains. Our proposed method obtained a synthesised computed tomography of the chest. Experimental results showed that our method outperforms current approaches. The study findings suggest the possibility of synthesising clinically acceptable computed tomography images from chest MRI with minimal changes in anatomical structures without human annotation.

Radiomics and artificial intelligence in prostate cancer: new tools for molecular hybrid imaging and theragnostics

In prostate cancer (PCa), the use of new radiopharmaceuticals has improved the accuracy of diagnosis and staging, refined surveillance strategies, and introduced specific and personalized radioreceptor therapies. Nuclear medicine, therefore, holds great promise for improving the quality of life of PCa patients, through managing and processing a vast amount of molecular imaging data and beyond, using a multi-omics approach and improving patients’ risk-stratification for tailored medicine. Artificial intelligence (AI) and radiomics may allow clinicians to improve the overall efficiency and accuracy of using these “big data” in both the diagnostic and theragnostic field: from technical aspects (such as semi-automatization of tumor segmentation, image reconstruction, and interpretation) to clinical outcomes, improving a deeper understanding of the molecular environment of PCa, refining personalized treatment strategies, and increasing the ability to predict the outcome. This systematic review aims to describe the current literature on AI and radiomics applied to molecular imaging of prostate cancer.

2D-to-3D image translation of complex nanoporous volumes using generative networks

Image-based characterization offers a powerful approach to studying geological porous media at the nanoscale and images are critical to understanding reactive transport mechanisms in reservoirs relevant to energy and sustainability technologies such as carbon sequestration, subsurface hydrogen storage, and natural gas recovery. Nanoimaging presents a trade off, however, between higher-contrast sample-destructive and lower-contrast sample-preserving imaging modalities. Furthermore, high-contrast imaging modalities often acquire only 2D images, while 3D volumes are needed to characterize fully a source rock sample. In this work, we present deep learning image translation models to predict high-contrast focused ion beam-scanning electron microscopy (FIB-SEM) image volumes from transmission X-ray microscopy (TXM) images when only 2D paired training data is available. We introduce a regularization method for improving 3D volume generation from 2D-to-2D deep learning image models and apply this approach to translate 3D TXM volumes to FIB-SEM fidelity. We then segment a predicted FIB-SEM volume into a flow simulation domain and calculate the sample apparent permeability using a lattice Boltzmann method (LBM) technique. Results show that our image translation approach produces simulation domains suitable for flow visualization and allows for accurate characterization of petrophysical properties from non-destructive imaging data.

Comparison of different deep learning architectures for synthetic CT generation from MR images

PURPOSE

Among the different available methods for synthetic CT generation from MR images for the task of MR-guided radiation planning, the deep learning algorithms have and do outperform their conventional counterparts. In this study, we investigated the performance of some most popular deep learning architectures including eCNN, U-Net, GAN, V-Net, and Res-Net for the task of sCT generation. As a baseline, an atlas-based method is implemented to which the results of the deep learning-based model are compared.

METHODS

A dataset consisting of 20 co-registered MR-CT pairs of the male pelvis is applied to assess the different sCT production methods' performance. The mean error (ME), mean absolute error (MAE), Pearson correlation coefficient (PCC), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR) metrics were computed between the estimated sCT and the ground truth (reference) CT images.

RESULTS

The visual inspection revealed that the sCTs produced by eCNN, V-Net, and ResNet, unlike the other methods, were less noisy and greatly resemble the ground truth CT image. In the whole pelvis region, the eCNN yielded the lowest MAE (26.03 ± 8.85 HU) and ME (0.82 ± 7.06 HU), and the highest PCC metrics were yielded by the eCNN (0.93 ± 0.05) and ResNet (0.91 ± 0.02) methods. The ResNet model had the highest PSNR of 29.38 ± 1.75 among all models. In terms of the Dice similarity coefficient, the eCNN method revealed superior performance in major tissue identification (air, bone, and soft tissue).

CONCLUSIONS

All in all, the eCNN and ResNet deep learning methods revealed acceptable performance with clinically tolerable quantification errors.

Deep learning based synthetic-CT generation in radiotherapy and PET: A review

Recently,deep learning (DL)-based methods for the generation of synthetic computed tomography (sCT) have received significant research attention as an alternative to classical ones. We present here a systematic review of these methods by grouping them into three categories, according to their clinical applications: (i) to replace computed tomography in magnetic resonance (MR) based treatment planning, (ii) facilitate cone-beam computed tomography based image-guided adaptive radiotherapy, and (iii) derive attenuation maps for the correction of positron emission tomography. Appropriate database searching was performed on journal articles published between January 2014 and December 2020. The DL methods' key characteristics were extracted from each eligible study, and a comprehensive comparison among network architectures and metrics was reported. A detailed review of each category was given, highlighting essential contributions, identifying specific challenges, and summarizing the achievements. Lastly, the statistics of all the cited works from various aspects were analyzed, revealing the popularity and future trends and the potential of DL-based sCT generation. The current status of DL-based sCT generation was evaluated, assessing the clinical readiness of the presented methods.

Toward PET/MRI as one-stop shop for radiotherapy planning in cervical cancer patients

BACKGROUND

Radiotherapy (RT) planning for cervical cancer patients entails the acquisition of both Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). Further, molecular imaging by Positron Emission Tomography (PET) could contribute to target volume delineation as well as treatment response monitoring. The objective of this study was to investigate the feasibility of a PET/MRI-only RT planning workflow of patients with cervical cancer. This includes attenuation correction (AC) of MRI hardware and dedicated positioning equipment as well as evaluating MRI-derived synthetic CT (sCT) of the pelvic region for positioning verification and dose calculation to enable a PET/MRI-only setup.

MATERIAL AND METHODS

16 patients underwent PET/MRI using a dedicated RT setup after the routine CT (or PET/CT), including eight pilot patients and eight cervical cancer patients who were subsequently referred for RT. Data from 18 patients with gynecological cancer were added for training a deep convolutional neural network to generate sCT from Dixon MRI. The mean absolute difference between the dose distributions calculated on sCT and a reference CT was measured in the RT target volume and organs at risk. PET AC by sCT and a reference CT were compared in the tumor volume.

RESULTS

All patients completed the examination. sCT was inferred for each patient in less than 5 s. The dosimetric analysis of the sCT-based dose planning showed a mean absolute error (MAE) of 0.17 ± 0.12 Gy inside the planning target volumes (PTV). PET images reconstructed with sCT and CT had no significant difference in quantification for all patients.

CONCLUSIONS

These results suggest that multiparametric PET/MRI can be successfully integrated as a one-stop-shop in the RT workflow of patients with cervical cancer.

Evaluation of Deep Learning-based Approaches to Segment Bowel Air Pockets and Generate Pelvis Attenuation Maps from CAIPIRINHA-accelerated Dixon MR Images

Attenuation correction (AC) remains a challenge in pelvis PET/MR imaging. In addition to the segmentation/model-based approaches, deep learning methods have shown promise in synthesizing accurate pelvis attenuation maps (μ-maps). However, these methods often misclassify air pockets in the digestive tract, which can introduce bias in the reconstructed PET images. The aims of this work were to develop deep learning-based methods to automatically segment air pockets and generate pseudo-CT images from CAIPIRINHA-accelerated MR Dixon images. Methods: A convolutional neural network (CNN) was trained to segment air pockets using 3D CAIPIRINHA-accelerated MR Dixon datasets from 35 subjects and was evaluated against semi-automated segmentations. A separate CNN was trained to synthesize pseudo-CT μ-maps from the Dixon images. Its accuracy was evaluated by comparing the deep learning-, model- and CT-based μ-maps using data from 30 of the subjects. Finally, the impact of different μ-maps and air pocket segmentation methods on the PET quantification was investigated. Results: Air pockets segmented using the CNN agreed well with semi-automated segmentations, with a mean Dice similarity coefficient of 0.75. Volumetric similarity score between two segmentations was 0.85 ± 0.14. The mean absolute relative change (RCs) with respect to the CT-based μ-maps were 2.6% and 5.1% in the whole pelvis for the deep learning and model-based μ-maps, respectively. The average RC between PET images reconstructed with deep learning and CT-based μ-maps was 2.6%. Conclusion: We presented a deep learning-based method to automatically segment air pockets from CAIPIRINHA-accelerated Dixon images with comparable accuracy to semi-automatic segmentations. We also showed that the μ-maps synthesized using a deep learning-based method from CAIPIRINHA-accelerated Dixon images are more accurate than those generated with the model-based approach available on integrated PET/MRI scanner.

Applications of artificial intelligence in nuclear medicine image generation

Recently, the application of artificial intelligence (AI) in medical imaging (including nuclear medicine imaging) has rapidly developed. Most AI applications in nuclear medicine imaging have focused on the diagnosis, treatment monitoring, and correlation analyses with pathology or specific gene mutation. It can also be used for image generation to shorten the time of image acquisition, reduce the dose of injected tracer, and enhance image quality. This work provides an overview of the application of AI in image generation for single-photon emission computed tomography (SPECT) and positron emission tomography (PET) either without or with anatomical information [CT or magnetic resonance imaging (MRI)]. This review focused on four aspects, including imaging physics, image reconstruction, image postprocessing, and internal dosimetry. AI application in generating attenuation map, estimating scatter events, boosting image quality, and predicting internal dose map is summarized and discussed.

Data-driven respiratory phase-matched PET attenuation correction without CT

We propose a deep learning-based data-driven respiratory phase-matched gated-PET attenuation correction (AC) method that does not need a gated-CT. The proposed method is a multi-step process that consists of data-driven respiratory gating, gated attenuation map estimation using maximum-likelihood reconstruction of attenuation and activity (MLAA) algorithm, and enhancement of the gated attenuation maps using convolutional neural network (CNN). The gated MLAA attenuation maps enhanced by the CNN allowed for the phase-matched AC of gated-PET images. We conducted a non-rigid registration of the gated-PET images to generate motion-free PET images. We trained the CNN by conducting a 3D patch-based learning with 80 oncologic whole-body¹⁸F-fluorodeoxyglucose (¹⁸F-FDG) PET/CT scan data and applied it to seven regional PET/CT scans that cover the lower lung and upper liver. We investigated the impact of the proposed respiratory phase-matched AC of PET without utilizing CT on tumor size and standard uptake value (SUV) assessment, and PET image quality (%STD). The attenuation corrected gated and motion-free PET images generated using the proposed method yielded sharper organ boundaries and better noise characteristics than conventional gated and ungated PET images. A banana artifact observed in a phase-mismatched CT-based AC was not observed in the proposed approach. By employing the proposed method, the size of tumor was reduced by 12.3% and SUV_90%was increased by 13.3% in tumors with larger movements than 5 mm. %STD of liver uptake was reduced by 11.1%. The deep learning-based data-driven respiratory phase-matched AC method improved the PET image quality and reduced the motion artifacts.

Attenuation correction for human PET/MRI studies

Attenuation correction has been one of the main methodological challenges in the integrated positron emission tomography and magnetic resonance imaging (PET/MRI) field. As standard transmission or computed tomography approaches are not available in integrated PET/MRI scanners, MR-based attenuation correction approaches had to be developed. Aspects that have to be considered for implementing accurate methods include the need to account for attenuation in bone tissue, normal and pathological lung and the MR hardware present in the PET field-of-view, to reduce the impact of subject motion, to minimize truncation and susceptibility artifacts, and to address issues related to the data acquisition and processing both on the PET and MRI sides. The standard MR-based attenuation correction techniques implemented by the PET/MRI equipment manufacturers and their impact on clinical and research PET data interpretation and quantification are first discussed. Next, the more advanced methods, including the latest generation deep learning-based approaches that have been proposed for further minimizing the attenuation correction related bias are described. Finally, a future perspective focused on the needed developments in the field is given.

Improved PET/MRI attenuation correction in the pelvic region using a statistical decomposition method on T2-weighted images

Background

Attenuation correction of PET/MRI is a remaining problem for whole-body PET/MRI. The statistical decomposition algorithm (SDA) is a probabilistic atlas-based method that calculates synthetic CTs from T2-weighted MRI scans. In this study, we evaluated the application of SDA for attenuation correction of PET images in the pelvic region.

Materials and method

Twelve patients were retrospectively selected from an ongoing prostate cancer research study. The patients had same-day scans of [11C]acetate PET/MRI and CT. The CT images were non-rigidly registered to the PET/MRI geometry, and PET images were reconstructed with attenuation correction employing CT, SDA-generated CT, and the built-in Dixon sequence-based method of the scanner. The PET images reconstructed using CT-based attenuation correction were used as ground truth.

Results

The mean whole-image PET uptake error was reduced from − 5.4% for Dixon-PET to − 0.9% for SDA-PET. The prostate standardized uptake value (SUV) quantification error was significantly reduced from − 5.6% for Dixon-PET to − 2.3% for SDA-PET.

Conclusion

Attenuation correction with SDA improves quantification of PET/MR images in the pelvic region compared to the Dixon-based method.

MR Image-Based Attenuation Correction of Brain PET Imaging: Review of Literature on Machine Learning Approaches for Segmentation

Recent emerging hybrid technology of positron emission tomography/magnetic resonance (PET/MR) imaging has generated a great need for an accurate MR image-based PET attenuation correction. MR image segmentation, as a robust and simple method for PET attenuation correction, has been clinically adopted in commercial PET/MR scanners. The general approach in this method is to segment the MR image into different tissue types, each assigned an attenuation constant as in an X-ray CT image. Machine learning techniques such as clustering, classification and deep networks are extensively used for brain MR image segmentation. However, only limited work has been reported on using deep learning in brain PET attenuation correction. In addition, there is a lack of clinical evaluation of machine learning methods in this application. The aim of this review is to study the use of machine learning methods for MR image segmentation and its application in attenuation correction for PET brain imaging. Furthermore, challenges and future opportunities in MR image-based PET attenuation correction are discussed.

[Artificial intelligence in hybrid imaging]

CLINICAL ISSUE

Hybrid imaging enables the precise visualization of cellular metabolism by combining anatomical and metabolic information. Advances in artificial intelligence (AI) offer new methods for processing and evaluating this data.

METHODOLOGICAL INNOVATIONS

This review summarizes current developments and applications of AI methods in hybrid imaging. Applications in image processing as well as methods for disease-related evaluation are presented and discussed.

MATERIALS AND METHODS

This article is based on a selective literature search with the search engines PubMed and arXiv.

ASSESSMENT

Currently, there are only a few AI applications using hybrid imaging data and no applications are established in clinical routine yet. Although the first promising approaches are emerging, they still need to be evaluated prospectively. In the future, AI applications will support radiologists and nuclear medicine radiologists in diagnosis and therapy.

Improving PET Imaging Acquisition and Analysis With Machine Learning: A Narrative Review With Focus on Alzheimer's Disease and Oncology

Machine learning (ML) algorithms have found increasing utility in the medical imaging field and numerous applications in the analysis of digital biomarkers within positron emission tomography (PET) imaging have emerged. Interest in the use of artificial intelligence in PET imaging for the study of neurodegenerative diseases and oncology stems from the potential for such techniques to streamline decision support for physicians providing early and accurate diagnosis and allowing personalized treatment regimens. In this review, the use of ML to improve PET image acquisition and reconstruction is presented, along with an overview of its applications in the analysis of PET images for the study of Alzheimer's disease and oncology.

Independent attenuation correction of whole body [18F]FDG-PET using a deep learning approach with Generative Adversarial Networks

BACKGROUND

Attenuation correction (AC) of PET data is usually performed using a second imaging for the generation of attenuation maps. In certain situations however-when CT- or MR-derived attenuation maps are corrupted or CT acquisition solely for the purpose of AC shall be avoided-it would be of value to have the possibility of obtaining attenuation maps only based on PET information. The purpose of this study was to thus develop, implement, and evaluate a deep learning-based method for whole body [18F]FDG-PET AC which is independent of other imaging modalities for acquiring the attenuation map.

METHODS

The proposed method is investigated on whole body [18F]FDG-PET data using a Generative Adversarial Networks (GAN) deep learning framework. It is trained to generate pseudo CT images (CTGAN) based on paired training data of non-attenuation corrected PET data (PETNAC) and corresponding CT data. Generated pseudo CTs are then used for subsequent PET AC. One hundred data sets of whole body PETNAC and corresponding CT were used for training. Twenty-five PET/CT examinations were used as test data sets (not included in training). On these test data sets, AC of PET was performed using the acquired CT as well as CTGAN resulting in the corresponding PET data sets PETAC and PETGAN. CTGAN and PETGAN were evaluated qualitatively by visual inspection and by visual analysis of color-coded difference maps. Quantitative analysis was performed by comparison of organ and lesion SUVs between PETAC and PETGAN.

RESULTS

Qualitative analysis revealed no major SUV deviations on PETGAN for most anatomic regions; visually detectable deviations were mainly observed along the diaphragm and the lung border. Quantitative analysis revealed mean percent deviations of SUVs on PETGAN of - 0.8 ± 8.6% over all organs (range [- 30.7%, + 27.1%]). Mean lesion SUVs showed a mean deviation of 0.9 ± 9.2% (range [- 19.6%, + 29.2%]).

CONCLUSION

Independent AC of whole body [18F]FDG-PET is feasible using the proposed deep learning approach yielding satisfactory PET quantification accuracy. Further clinical validation is necessary prior to implementation in clinical routine applications.

Next generation research applications for hybrid PET/MR and PET/CT imaging using deep learning

INTRODUCTION

Recently there have been significant advances in the field of machine learning and artificial intelligence (AI) centered around imaging-based applications such as computer vision. In particular, the tremendous power of deep learning algorithms, primarily based on convolutional neural network strategies, is becoming increasingly apparent and has already had direct impact on the fields of radiology and nuclear medicine. While most early applications of computer vision to radiological imaging have focused on classification of images into disease categories, it is also possible to use these methods to improve image quality. Hybrid imaging approaches, such as PET/MRI and PET/CT, are ideal for applying these methods.

METHODS

This review will give an overview of the application of AI to improve image quality for PET imaging directly and how the additional use of anatomic information from CT and MRI can lead to further benefits. For PET, these performance gains can be used to shorten imaging scan times, with improvement in patient comfort and motion artifacts, or to push towards lower radiotracer doses. It also opens the possibilities for dual tracer studies, more frequent follow-up examinations, and new imaging indications. How to assess quality and the potential effects of bias in training and testing sets will be discussed.

CONCLUSION

Harnessing the power of these new technologies to extract maximal information from hybrid PET imaging will open up new vistas for both research and clinical applications with associated benefits in patient care.

Summary of the First ISMRM-SNMMI Workshop on PET/MRI: Applications and Limitations

Since the introduction of simultaneous PET/MRI in 2011, there have been significant advancements. In this review, we highlight several technical advancements that have been made primarily in attenuation and motion correction and discuss the status of multiple clinical applications using PET/MRI. This review is based on the experience at the first PET/MRI conference cosponsored by the International Society for Magnetic Resonance in Medicine and the Society of Nuclear Medicine and Molecular Imaging.