Application of Deep Learning to Predict Standardized Uptake Value Ratio and Amyloid Status on 18F-Florbetapir PET Using ADNI Data

Multi-instance learning attention model for amyloid quantification of brain sub regions in longitudinal cognitive decline

Amyloid PET scans help in identifying the beta-amyloid deposition in different brain regions. The purpose of this study is to develop a deep learning model that can automate the task of finding amyloid deposition in different regions of the brain only by using PET scan and without the corresponding MRI scan. 2647 18F-Florbetapir PET scans are collected from Alzheimer's Disease Neuroimaging Initiative (ADNI) from multiple centres taken over a period. A deep learning model based on multi-instance learning and attention is proposed which is trained and validated using 80% of the scans and the remaining 20% of the scans are used for testing the model. The performance of the model is validated using Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). The proposed model is further tested upon an external dataset consisting of 1413 18F-Florbetapir PET scans from the Anti-Amyloid Treatment in Asymptomatic Alzheimer's (A4) study. The proposed model achieves MAE of 0.0243 and RMSE of 0.0320 for summary Standardized Uptake Value Ratio (SUVR) based on composite reference region for ADNI test set. When tested on the A4-study dataset, the proposed model achieves MAE of 0.038 and RMSE of 0.0495 for summary SUVR based on the composite region. The results show that the proposed model provides less MAE and RMSE when compared with existing models. A graphical user interface is developed based on the proposed model where the predictions are made by selecting the files of 18F-Florbetapir PET scans.

Bridging Imaging and Clinical Scores in Parkinson's Progression via Multimodal Self-Supervised Deep Learning

Neurodegenerative diseases pose a formidable challenge to medical research, demanding a nuanced understanding of their progressive nature. In this regard, latent generative models can effectively be used in a data-driven modeling of different dimensions of neurodegeneration, framed within the context of the manifold hypothesis. This paper proposes a joint framework for a multi-modal, common latent generative model to address the need for a more comprehensive understanding of the neurodegenerative landscape in the context of Parkinson's disease (PD). The proposed architecture uses coupled variational autoencoders (VAEs) to joint model a common latent space to both neuroimaging and clinical data from the Parkinson's Progression Markers Initiative (PPMI). Alternative loss functions, different normalization procedures, and the interpretability and explainability of latent generative models are addressed, leading to a model that was able to predict clinical symptomatology in the test set, as measured by the unified Parkinson's disease rating scale (UPDRS), with R2 up to 0.86 for same-modality and 0.441 cross-modality (using solely neuroimaging). The findings provide a foundation for further advancements in the field of clinical research and practice, with potential applications in decision-making processes for PD. The study also highlights the limitations and capabilities of the proposed model, emphasizing its direct interpretability and potential impact on understanding and interpreting neuroimaging patterns associated with PD symptomatology.

Generative AI unlocks PET insights: brain amyloid dynamics and quantification

Introduction

Studying the spatiotemporal patterns of amyloid accumulation in the brain over time is crucial in understanding Alzheimer's disease (AD). Positron Emission Tomography (PET) imaging plays a pivotal role because it allows for the visualization and quantification of abnormal amyloid beta (Aβ) load in the living brain, providing a powerful tool for tracking disease progression and evaluating the efficacy of anti-amyloid therapies. Generative artificial intelligence (AI) can learn complex data distributions and generate realistic synthetic images. In this study, we demonstrate for the first time the potential of Generative Adversarial Networks (GANs) to build a low-dimensional representation space that effectively describes brain amyloid load and its dynamics.

Methods

Using a cohort of 1,259 subjects with AV45 PET images from the Alzheimer's Disease Neuroimaging Initiative (ADNI), we develop a 3D GAN model to project images into a latent representation space and generate back synthetic images. Then, we build a progression model on the representation space based on non-parametric ordinary differential equations to study brain amyloid evolution.

Results

We found that global SUVR can be accurately predicted with a linear regression model only from the latent representation space (RMSE = 0.08 ± 0.01). We generated synthetic PET trajectories and illustrated predicted Aβ change in four years compared with actual progression.

Discussion

Generative AI can generate rich representations for statistical prediction and progression modeling and simulate evolution in synthetic patients, providing an invaluable tool for understanding AD, assisting in diagnosis, and designing clinical trials. The aim of this study was to illustrate the huge potential that generative AI has in brain amyloid imaging and to encourage its advancement by providing use cases and ideas for future research tracks.

AmyloidPETNet: Classification of Amyloid Positivity in Brain PET Imaging Using End-to-End Deep Learning

Background Visual assessment of amyloid PET scans relies on the availability of radiologist expertise, whereas quantification of amyloid burden typically involves MRI for processing and analysis, which can be computationally expensive. Purpose To develop a deep learning model to classify minimally processed brain PET scans as amyloid positive or negative, evaluate its performance on independent data sets and different tracers, and compare it with human visual reads. Materials and Methods This retrospective study used 8476 PET scans (6722 patients) obtained from late 2004 to early 2023 that were analyzed across five different data sets. A deep learning model, AmyloidPETNet, was trained on 1538 scans from 766 patients, validated on 205 scans from 95 patients, and internally tested on 184 scans from 95 patients in the Alzheimer's Disease Neuroimaging Initiative (ADNI) fluorine 18 (18F) florbetapir (FBP) data set. It was tested on ADNI scans using different tracers and scans from independent data sets. Scan amyloid positivity was based on mean cortical standardized uptake value ratio cutoffs. To compare with model performance, each scan from both the Centiloid Project and a subset of the Anti-Amyloid Treatment in Asymptomatic Alzheimer's Disease (A4) study were visually interpreted with a confidence level (low, intermediate, high) of amyloid positivity/negativity. The area under the receiver operating characteristic curve (AUC) and other performance metrics were calculated, and Cohen κ was used to measure physician-model agreement. Results The model achieved an AUC of 0.97 (95% CI: 0.95, 0.99) on test ADNI 18F-FBP scans, which generalized well to 18F-FBP scans from the Open Access Series of Imaging Studies (AUC, 0.95; 95% CI: 0.93, 0.97) and the A4 study (AUC, 0.98; 95% CI: 0.98, 0.98). Model performance was high when applied to data sets with different tracers (AUC ≥ 0.97). Other performance metrics provided converging evidence. Physician-model agreement ranged from fair (Cohen κ = 0.39; 95% CI: 0.16, 0.60) on a sample of mostly equivocal cases from the A4 study to almost perfect (Cohen κ = 0.93; 95% CI: 0.86, 1.0) on the Centiloid Project. Conclusion The developed model was capable of automatically and accurately classifying brain PET scans as amyloid positive or negative without relying on experienced readers or requiring structural MRI. Clinical trial registration no. NCT00106899 © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Bryan and Forghani in this issue.

ArcheD, a residual neural network for prediction of cerebrospinal fluid amyloid-beta from amyloid PET images

Detection and measurement of amyloid-beta (Aβ) in the brain is a key factor for early identification and diagnosis of Alzheimer's disease (AD). We aimed to develop a deep learning model to predict Aβ cerebrospinal fluid (CSF) concentration directly from amyloid PET images, independent of tracers, brain reference regions or preselected regions of interest. We used 1870 Aβ PET images and CSF measurements to train and validate a convolutional neural network ("ArcheD"). We evaluated the ArcheD performance in relation to episodic memory and the standardized uptake value ratio (SUVR) of cortical Aβ. We also compared the brain region's relevance for the model's CSF prediction within clinical-based and biological-based classifications. ArcheD-predicted Aβ CSF values correlated with measured Aβ CSF values (r = 0.92; q < 0.01), SUVR (rAV45 = -0.64, rFBB = -0.69; q < 0.01) and episodic memory measures (0.33 < r < 0.44; q < 0.01). For both classifications, cerebral white matter significantly contributed to CSF prediction (q < 0.01), specifically in non-symptomatic and early stages of AD. However, in late-stage disease, the brain stem, subcortical areas, cortical lobes, limbic lobe and basal forebrain made more significant contributions (q < 0.01). Considering cortical grey matter separately, the parietal lobe was the strongest predictor of CSF amyloid levels in those with prodromal or early AD, while the temporal lobe played a more crucial role for those with AD. In summary, ArcheD reliably predicted Aβ CSF concentration from Aβ PET scans, offering potential clinical utility for Aβ level determination.

Predicting meningioma grades and pathologic marker expression via deep learning

OBJECTIVES

To establish a deep learning (DL) model for predicting tumor grades and expression of pathologic markers of meningioma.

METHODS

A total of 1192 meningioma patients from two centers who underwent surgical resection between September 2018 and December 2021 were retrospectively included. The pathological data and post-contrast T1-weight images for each patient were collected. The patients from institute I were subdivided into training, validation, and testing sets, while the patients from institute II served as the external testing cohort. The fine-tuned ResNet50 model based on transfer learning was adopted to classify WHO grade in the whole cohort and predict Ki-67 index, H3K27me3, and progesterone receptor (PR) status of grade 1 meningiomas. The predictive performance was evaluated by the accuracy and loss curve, confusion matrix, receiver operating characteristic curve (ROC), and area under curve (AUC).

RESULTS

The DL prediction model for each label achieved high predictive performance in two cohorts. For WHO grade prediction, the area under the curve (AUC) was 0.966 (95%CI 0.957-0.975) in the internal testing set and 0.669 (95%CI 0.643-0.695) in the external validation cohort. The AUC in predicting Ki-67 index, H3K27me3, and PR status were 0.905 (95%CI 0.895-0.915), 0.773 (95%CI 0.760-0.786), and 0.771 (95%CI 0.750-0.792) in the internal testing set and 0.591 (95%CI 0.562-0.620), 0.658 (95%CI 0.648-0.668), and 0.703 (95%CI 0.674-0.732) in the external validation cohort, respectively.

CONCLUSION

DL models can preoperatively predict meningioma grades and pathologic marker expression with favorable predictive performance.

CLINICAL RELEVANCE STATEMENT

Our DL model could predict meningioma grades and expression of pathologic markers and identify high-risk patients with WHO grade 1 meningioma, which would suggest a more aggressive operative intervention preoperatively and a more frequent follow-up schedule postoperatively.

KEY POINTS

WHO grades and some pathologic markers of meningioma were associated with therapeutic strategies and clinical outcomes. A deep learning-based approach was employed to develop a model for predicting meningioma grades and the expression of pathologic markers. Preoperative prediction of meningioma grades and the expression of pathologic markers was beneficial for clinical decision-making.

Deep Learning-Driven Estimation of Centiloid Scales from Amyloid PET Images with 11C-PiB and 18F-Labeled Tracers in Alzheimer's Disease

BACKGROUND

Standard methods for deriving Centiloid scales from amyloid PET images are time-consuming and require considerable expert knowledge. We aimed to develop a deep learning method of automating Centiloid scale calculations from amyloid PET images with 11C-Pittsburgh Compound-B (PiB) tracer and assess its applicability to 18F-labeled tracers without retraining.

METHODS

We trained models on 231 11C-PiB amyloid PET images using a 50-layer 3D ResNet architecture. The models predicted the Centiloid scale, and accuracy was assessed using mean absolute error (MAE), linear regression analysis, and Bland-Altman plots.

RESULTS

The MAEs for Alzheimer's disease (AD) and young controls (YC) were 8.54 and 2.61, respectively, using 11C-PiB, and 8.66 and 3.56, respectively, using 18F-NAV4694. The MAEs for AD and YC were higher with 18F-florbetaben (39.8 and 7.13, respectively) and 18F-florbetapir (40.5 and 12.4, respectively), and the error rate was moderate for 18F-flutemetamol (21.3 and 4.03, respectively). Linear regression yielded a slope of 1.00, intercept of 1.26, and R2 of 0.956, with a mean bias of -1.31 in the Centiloid scale prediction.

CONCLUSIONS

We propose a deep learning means of directly predicting the Centiloid scale from amyloid PET images in a native space. Transferring the model trained on 11C-PiB directly to 18F-NAV4694 without retraining was feasible.

Machine-learning-assisted and real-time-feedback-controlled growth of InAs/GaAs quantum dots

The applications of self-assembled InAs/GaAs quantum dots (QDs) for lasers and single photon sources strongly rely on their density and quality. Establishing the process parameters in molecular beam epitaxy (MBE) for a specific density of QDs is a multidimensional optimization challenge, usually addressed through time-consuming and iterative trial-and-error. Here, we report a real-time feedback control method to realize the growth of QDs with arbitrary density, which is fully automated and intelligent. We develop a machine learning (ML) model named 3D ResNet 50 trained using reflection high-energy electron diffraction (RHEED) videos as input instead of static images and providing real-time feedback on surface morphologies for process control. As a result, we demonstrate that ML from previous growth could predict the post-growth density of QDs, by successfully tuning the QD densities in near-real time from 1.5 × 1010 cm-2 down to 3.8 × 108 cm-2 or up to 1.4 × 1011 cm-2. Compared to traditional methods, our approach can dramatically expedite the optimization process and improve the reproducibility of MBE. The concepts and methodologies proved feasible in this work are promising to be applied to a variety of material growth processes, which will revolutionize semiconductor manufacturing for optoelectronic and microelectronic industries.

Automated detection of Alzheimer's disease: a multi-modal approach with 3D MRI and amyloid PET

Recent advances in deep learning and imaging technologies have revolutionized automated medical image analysis, especially in diagnosing Alzheimer's disease through neuroimaging. Despite the availability of various imaging modalities for the same patient, the development of multi-modal models leveraging these modalities remains underexplored. This paper addresses this gap by proposing and evaluating classification models using 2D and 3D MRI images and amyloid PET scans in uni-modal and multi-modal frameworks. Our findings demonstrate that models using volumetric data learn more effective representations than those using only 2D images. Furthermore, integrating multiple modalities enhances model performance over single-modality approaches significantly. We achieved state-of-the-art performance on the OASIS-3 cohort. Additionally, explainability analyses with Grad-CAM indicate that our model focuses on crucial AD-related regions for its predictions, underscoring its potential to aid in understanding the disease's causes.

ArcheD, a residual neural network for prediction of cerebrospinal fluid amyloid-beta from amyloid PET images

Detection and measurement of amyloid-beta (Aβ) aggregation in the brain is a key factor for early identification and diagnosis of Alzheimer's disease (AD). We aimed to develop a deep learning model to predict Aβ cerebrospinal fluid (CSF) concentration directly from amyloid PET images, independent of tracers, brain reference regions or preselected regions of interest. We used 1870 Aβ PET images and CSF measurements to train and validate a convolutional neural network ("ArcheD"). We evaluated the ArcheD performance in relation to episodic memory and the standardized uptake value ratio (SUVR) of cortical Aβ. We also compared the brain region's relevance for the model's CSF prediction within clinical-based and biological-based classifications. ArcheD-predicted Aβ CSF values correlated strongly with measured Aβ CSF values ( r =0.81; p <0.001) and showed correlations with SUVR and episodic memory measures in all participants except in those with AD. For both clinical and biological classifications, cerebral white matter significantly contributed to CSF prediction ( q <0.01), specifically in non-symptomatic and early stages of AD. However, in late-stage disease, brain stem, subcortical areas, cortical lobes, limbic lobe, and basal forebrain made more significant contributions (q<0.01). Considering cortical gray matter separately, the parietal lobe was the strongest predictor of CSF amyloid levels in those with prodromal or early AD, while the temporal lobe played a more crucial role for those with AD. In summary, ArcheD reliably predicted Aβ CSF concentration from Aβ PET scans, offering potential clinical utility for Aβ level determination and early AD detection.

Machine learning in the positron emission tomography imaging of Alzheimer's disease

The utilization of machine learning techniques in medicine has exponentially increased over the last decades due to innovations in computer processing, algorithm development, and access to big data. Applications of machine learning techniques to neuroimaging specifically have unveiled various hidden interactions, structures, and mechanisms related to various neurological disorders. One application of interest is the imaging of Alzheimer's disease, the most common cause of progressive dementia. The diagnoses of Alzheimer's disease, mild cognitive impairment, and preclinical Alzheimer's disease have been difficult. Molecular imaging, particularly via PET scans, holds tremendous value in the imaging of Alzheimer's disease. To date, many novel algorithms have been developed with great success that leverage machine learning in the context of Alzheimer's disease. This review article provides an overview of the diverse applications of machine learning to PET imaging of Alzheimer's disease.

Estimation of brain amyloid accumulation using deep learning in clinical [11C]PiB PET imaging

INTRODUCTION

Estimation of brain amyloid accumulation is valuable for evaluation of patients with cognitive impairment in both research and clinical routine. The development of high throughput and accurate strategies for the determination of amyloid status could be an important tool in patient selection for clinical trials and amyloid directed treatment. Here, we propose the use of deep learning to quantify amyloid accumulation using standardized uptake value ratio (SUVR) and classify amyloid status based on their PET images.

METHODS

A total of 1309 patients with cognitive impairment scanned with [¹¹C]PIB PET/CT or PET/MRI were included. Two convolutional neural networks (CNNs) for reading-based amyloid status and SUVR prediction were trained using 75% of the PET/CT data. The remaining PET/CT (n = 300) and all PET/MRI (n = 100) data was used for evaluation.

RESULTS

The prevalence of amyloid positive patients was 61%. The amyloid status classification model reproduced the expert reader's classification with 99% accuracy. There was a high correlation between reference and predicted SUVR (R² = 0.96). Both reference and predicted SUVR had an accuracy of 97% compared to expert classification when applying a predetermined SUVR threshold of 1.35 for binary classification of amyloid status.

CONCLUSION

The proposed CNN models reproduced both the expert classification and quantitative measure of amyloid accumulation in a large local dataset. This method has the potential to replace or simplify existing clinical routines and can facilitate fast and accurate classification well-suited for a high throughput pipeline.

DeepAD: A deep learning application for predicting amyloid standardized uptake value ratio through PET for Alzheimer's prognosis

Introduction

Amyloid deposition is a vital biomarker in the process of Alzheimer's diagnosis. ¹⁸F-florbetapir PET scans can provide valuable imaging data to determine cortical amyloid quantities. However, the process is labor and doctor intensive, requiring extremely specialized education and resources that may not be accessible to everyone, making the amyloid calculation process inefficient. Deep learning is a rising tool in Alzheimer's research which could be used to determine amyloid deposition.

Materials and methods

Using data from the Alzheimer's Disease Neuroimaging Initiative, we identified 2,980 patients with PET imaging, clinical, and genetic data. We tested various ResNet, EfficientNet, and RegNet convolutional neural networks and later combined the best performing model with Gradient Boosting Decision Tree algorithms to predict standardized uptake value ratio (SUVR) of amyloid in each patient session. We tried several configurations to find the best model tuning for regression-to-SUVR.

Results

We found that the RegNet X064 architecture combined with a grid search-tuned Gradient Boosting Decision Tree with 3 axial input slices and clinical and genetic data achieved the lowest loss. Using the mean-absolute-error metric, the loss converged to an MAE of 0.0441, equating to 96.4% accuracy across the 596-patient test set.

Discussion

We showed that this method is more consistent and accessible in comparison to human readers from previous studies, with lower margins of error and substantially faster calculation times. We implemented our deep learning model on to a web application named DeepAD which allows our diagnostic tool to be accessible. DeepAD could be used in hospitals and clinics with resource limitations for amyloid deposition and shows promise for more imaging tasks as well.

Application of artificial intelligence in brain molecular imaging

Initial development of artificial Intelligence (AI) and machine learning (ML) dates back to the mid-twentieth century. A growing awareness of the potential for AI, as well as increases in computational resources, research, and investment are rapidly advancing AI applications to medical imaging and, specifically, brain molecular imaging. AI/ML can improve imaging operations and decision making, and potentially perform tasks that are not readily possible by physicians, such as predicting disease prognosis, and identifying latent relationships from multi-modal clinical information. The number of applications of image-based AI algorithms, such as convolutional neural network (CNN), is increasing rapidly. The applications for brain molecular imaging (MI) include image denoising, PET and PET/MRI attenuation correction, image segmentation and lesion detection, parametric image formation, and the detection/diagnosis of Alzheimer's disease and other brain disorders. When effectively used, AI will likely improve the quality of patient care, instead of replacing radiologists. A regulatory framework is being developed to facilitate AI adaptation for medical imaging.

Predicting future amyloid biomarkers in dementia patients with machine learning to improve clinical trial patient selection

INTRODUCTION

In Alzheimer's disease, asymptomatic patients may have amyloid deposition, but predicting their progression rate remains a substantial challenge with implications for clinical trial enrollment. Here, we demonstrate an artificial intelligence approach to use baseline clinical information and images to predict changes in quantitative biomarkers of brain pathology on future images.

METHODS

Patients from the Alzheimer's Disease Neuroimaging Initiative (ADNI) who underwent positron emission tomography (PET) with the amyloid radiotracer 18F-AV45 (florbetapir) were included. We identified important baseline PET image features using a deep convolutional neural network based on ResNet. These were combined with eight clinical, demographic, and genetic markers using a gradient-boosted decision tree (GBDT) algorithm to predict future quantitative standardized uptake value ratio (SUVR), an established biomarker of brain amyloid deposition. We used this model to better identify individuals with the highest positive change in amyloid deposition on future images and compared this to typical inclusion criteria for clinical trials. We also compared the model's performance to other methods such as multivariate linear regression and GBDT without imaging features.

FINDINGS

Using 2577 PET scans from 1224 unique individuals, we showed that the GBDT with deep image features was significantly more accurate than the other approaches, reaching a root mean squared error of 0.0339 ± 0.0027 for future SUVR prediction. Using this approach, we could identify individuals with the highest 10% SUVR accumulation at rates 2- to 4-fold higher than by random pick or existing inclusion criteria.

DISCUSSION

Predicting quantitative biomarkers on future images using machine learning methods consisting of deep image features combined with clinical data may allow better targeting of treatments or enrollment in clinical trials.

Multiparametric magnetic resonance imaging and positron emission tomography findings in neurodegenerative diseases: Current status and future directions

Neurodegenerative diseases (NDDs) are characterized by progressive neuronal loss, leading to dementia and movement disorders. NDDs broadly include Alzheimer's disease, frontotemporal lobar degeneration, parkinsonian syndromes, and prion diseases. There is an ever-increasing prevalence of mild cognitive impairment and dementia, with an accompanying immense economic impact, prompting efforts aimed at early identification and effective interventions. Neuroimaging is an essential tool for the early diagnosis of NDDs in both clinical and research settings. Structural, functional, and metabolic imaging modalities, including magnetic resonance imaging (MRI) and positron emission tomography (PET), are widely available. They show encouraging results for diagnosis, monitoring, and treatment response evaluation. The current review focuses on the complementary role of various imaging modalities in relation to NDDs, the qualitative and quantitative utility of newer MRI techniques, novel radiopharmaceuticals, and integrated PET/MRI in the setting of NDDs.

Deep-learning-based cardiac amyloidosis classification from early acquired pet images

The objective of the present work was to evaluate the potential of deep learning tools for characterizing the presence of cardiac amyloidosis from early acquired PET images, i.e. 15 min after [18F]-Florbetaben tracer injection. 47 subjects were included in the study: 13 patients with transthyretin-related amyloidosis cardiac amyloidosis (ATTR-CA), 15 patients with immunoglobulin light-chain amyloidosis (AL-CA), and 19 control-patients (CTRL). [18F]-Florbetaben PET/CT images were acquired in list mode and data was sorted into a sinogram, covering a time interval of 5 min starting 15 min after the injection. The resulting sinogram was reconstructed using OSEM iterative algorithm. A deep convolutional neural network (CAclassNet) was designed and implemented, consisting of five 2D convolutional layers, three fully connected layers and a final classifier returning AL, ATTR and CTRL scores. A total of 1107 2D images (375 from AL-subtype patients, 312 from ATTR-subtype, and 420 from Controls) have been considered in the study and used to train, validate and test the proposed network. CAclassNet cross-validation resulted with train error mean ± sd of 2.001% ± 0.96%, validation error of 4.5% ± 2.26%, and net accuracy of 95.49% ± 2.26%. Network test error resulted in a mean ± sd values of 10.73% ± 0.76%. Sensitivity, specificity, and accuracy evaluated on the test dataset were respectively for AL-CA sub-type: 1, 0.912, 0.936; for ATTR-CA: 0.935, 0.897, 0.972; for control subjects: 0.809, 0.971, 0.909. In conclusion, the proposed CAclassNet model seems very promising as an aid for the clinician in the diagnosis of CA from cardiac [18F]-Florbetaben PET images acquired a few minutes after the injection.

Artificial Intelligence for Optimization and Interpretation of PET/CT and PET/MR Images

Artificial intelligence (AI) has recently attracted much attention for its potential use in healthcare applications. The use of AI to improve and extract more information out of medical images, given their parallels with natural images and the immense progress in the area of computer vision, has been at the forefront of these advances. This is due to a convergence of factors, including the increasing numbers of scans performed, the availability of open source AI tools, and decreases in the costs of hardware required to implement these technologies. In this article, we review the progress in the use of AI toward optimizing PET/CT and PET/MRI studies. These two methods, which combine molecular information with structural and (in the case of MRI) functional imaging, are extremely valuable for a wide range of clinical indications. They are also tremendously data-rich modalities and as such are highly amenable to data-driven technologies such as AI. The first half of the article will focus on methods to improve PET reconstruction and image quality, which has multiple benefits including faster image acquisition, image reconstruction, and lower or even "zero" radiation dose imaging. It will also address the value of AI-driven methods to perform MR-based attenuation correction. The second half will address how some of these advances can be used to perform to optimize diagnosis from the acquired images, with examples given for whole-body oncology, cardiology, and neurology indications. Overall, it is likely that the use of AI will markedly improve both the quality and safety of PET/CT and PET/MRI as well as enhance our ability to interpret the scans and follow lesions over time. This will hopefully lead to expanded clinical use cases for these valuable technologies leading to better patient care.