Nuclear Medicine and Artificial Intelligence: Best Practices for Evaluation (the RELAINCE Guidelines)

Technological Advances in SPECT and SPECT/CT Imaging

Single photon emission tomography/computed tomography (SPECT/CT) is a mature imaging technology with a dynamic role in the diagnosis and monitoring of a wide array of diseases. This paper reviews the technological advances, clinical impact, and future directions of SPECT and SPECT/CT imaging. The focus of this review is on signal amplifier devices, detector materials, camera head and collimator designs, image reconstruction techniques, and quantitative methods. Bulky photomultiplier tubes (PMTs) are being replaced by position-sensitive PMTs (PSPMTs), avalanche photodiodes (APDs), and silicon PMs to achieve higher detection efficiency and improved energy resolution and spatial resolution. Most recently, new SPECT cameras have been designed for cardiac imaging. The new design involves using specialised collimators in conjunction with conventional sodium iodide detectors (NaI(Tl)) or an L-shaped camera head, which utilises semiconductor detector materials such as CdZnTe (CZT: cadmium-zinc-telluride). The clinical benefits of the new design include shorter scanning times, improved image quality, enhanced patient comfort, reduced claustrophobic effects, and decreased overall size, particularly in specialised clinical centres. These noticeable improvements are also attributed to the implementation of resolution-recovery iterative reconstructions. Immense efforts have been made to establish SPECT and SPECT/CT imaging as quantitative tools by incorporating camera-specific modelling. Moreover, this review includes clinical examples in oncology, neurology, cardiology, musculoskeletal, and infection, demonstrating the impact of these advancements on clinical practice in radiology and molecular imaging departments.

Is Automatic Tumor Segmentation on Whole-Body 18F-FDG PET Images a Clinical Reality?

The integration of automated whole-body tumor segmentation using 18F-FDG PET/CT images represents a pivotal shift in oncologic diagnostics, enhancing the precision and efficiency of tumor burden assessment. This editorial examines the transition toward automation, propelled by advancements in artificial intelligence, notably through deep learning techniques. We highlight the current availability of commercial tools and the academic efforts that have set the stage for these developments. Further, we comment on the challenges of data diversity, validation needs, and regulatory barriers. The role of metabolic tumor volume and total lesion glycolysis as vital metrics in cancer management underscores the significance of this evaluation. Despite promising progress, we call for increased collaboration across academia, clinical users, and industry to better realize the clinical benefits of automated segmentation, thus helping to streamline workflows and improve patient outcomes in oncology.

Inspiring a convergent engineering approach to measure and model the tissue microenvironment

Understanding the molecular and physical complexity of the tissue microenvironment (TiME) in the context of its spatiotemporal organization has remained an enduring challenge. Recent advances in engineering and data science are now promising the ability to study the structure, functions, and dynamics of the TiME in unprecedented detail; however, many advances still occur in silos that rarely integrate information to study the TiME in its full detail. This review provides an integrative overview of the engineering principles underlying chemical, optical, electrical, mechanical, and computational science to probe, sense, model, and fabricate the TiME. In individual sections, we first summarize the underlying principles, capabilities, and scope of emerging technologies, the breakthrough discoveries enabled by each technology and recent, promising innovations. We provide perspectives on the potential of these advances in answering critical questions about the TiME and its role in various disease and developmental processes. Finally, we present an integrative view that appreciates the major scientific and educational aspects in the study of the TiME.

Automated multiclass segmentation, quantification, and visualization of the diseased aorta on hybrid PET/CT-SEQUOIA

BACKGROUND

Cardiovascular disease is the most common cause of death worldwide, including infection and inflammation related conditions. Multiple studies have demonstrated potential advantages of hybrid positron emission tomography combined with computed tomography (PET/CT) as an adjunct to current clinical inflammatory and infectious biochemical markers. To quantitatively analyze vascular diseases at PET/CT, robust segmentation of the aorta is necessary. However, manual segmentation is extremely time-consuming and labor-intensive.

PURPOSE

To investigate the feasibility and accuracy of an automated tool to segment and quantify multiple parts of the diseased aorta on unenhanced low-dose computed tomography (LDCT) as an anatomical reference for PET-assessed vascular disease.

METHODS

A software pipeline was developed including automated segmentation using a 3D U-Net, calcium scoring, PET uptake quantification, background measurement, radiomics feature extraction, and 2D surface visualization of vessel wall calcium and tracer uptake distribution. To train the 3D U-Net, 352 non-contrast LDCTs from (2-[18F]FDG and Na[18F]F) PET/CTs performed in patients with various vascular pathologies with manual segmentation of the ascending aorta, aortic arch, descending aorta, and abdominal aorta were used. The last 22 consecutive scans were used as a hold-out internal test set. The remaining dataset was randomly split into training (n = 264; 80%) and validation (n = 66; 20%) sets. Further evaluation was performed on an external test set of 49 PET/CTs. The dice similarity coefficient (DSC) and Hausdorff distance (HD) were used to assess segmentation performance. Automatically obtained calcium scores and uptake values were compared with manual scoring obtained using clinical softwares (syngo.via and Affinity Viewer) in six patient images. intraclass correlation coefficients (ICC) were calculated to validate calcium and uptake values.

RESULTS

Fully automated segmentation of the aorta using a 3D U-Net was feasible in LDCT obtained from PET/CT scans. The external test set yielded a DSC of 0.867 ± 0.030 and HD of 1.0 [0.6-1.4] mm, similar to an open-source model with a DSC of 0.864 ± 0.023 and HD of 1.4 [1.0-1.8] mm. Quantification of calcium and uptake values were in excellent agreement with clinical software (ICC: 1.00 [1.00-1.00] and 0.99 [0.93-1.00] for calcium and uptake values, respectively).

CONCLUSIONS

We present an automated pipeline to segment the ascending aorta, aortic arch, descending aorta, and abdominal aorta on LDCT from PET/CT and to accurately provide uptake values, calcium scores, background measurement, radiomics features, and a 2D visualization. We call this algorithm SEQUOIA (SEgmentation, QUantification, and visualizatiOn of the dIseased Aorta) and is available at https://github.com/UMCG-CVI/SEQUOIA. This model could augment the utility of aortic evaluation at PET/CT studies tremendously, irrespective of the tracer, and potentially provide fast and reliable quantification of cardiovascular diseases in clinical practice, both for primary diagnosis and disease monitoring.

Deep learning generation of preclinical positron emission tomography (PET) images from low-count PET with task-based performance assessment

BACKGROUND

Preclinical low-count positron emission tomography (LC-PET) imaging offers numerous advantages such as facilitating imaging logistics, enabling longitudinal studies of long- and short-lived isotopes as well as increasing scanner throughput. However, LC-PET is characterized by reduced photon-count levels resulting in low signal-to-noise ratio (SNR), segmentation difficulties, and quantification uncertainties.

PURPOSE

We developed and evaluated a novel deep-learning (DL) architecture-Attention based Residual-Dilated Net (ARD-Net)-to generate standard-count PET (SC-PET) images from LC-PET images. The performance of the ARD-Net framework was evaluated for numerous low count realizations using fidelity-based qualitative metrics, task-based segmentation, and quantitative metrics.

METHOD

Patient Derived tumor Xenograft (PDX) with tumors implanted in the mammary fat-pad were subjected to preclinical [18F]-Fluorodeoxyglucose (FDG)-PET/CT imaging. SC-PET images were derived from a 10 min static FDG-PET acquisition, 50 min post administration of FDG, and were resampled to generate four distinct LC-PET realizations corresponding to 10%, 5%, 1.6%, and 0.8% of SC-PET count-level. ARD-Net was trained and optimized using 48 preclinical FDG-PET datasets, while 16 datasets were utilized to assess performance. Further, the performance of ARD-Net was benchmarked against two leading DL-based methods (Residual UNet, RU-Net; and Dilated Network, D-Net) and non-DL methods (Non-Local Means, NLM; and Block Matching 3D Filtering, BM3D). The performance of the framework was evaluated using traditional fidelity-based image quality metrics such as Structural Similarity Index Metric (SSIM) and Normalized Root Mean Square Error (NRMSE), as well as human observer-based tumor segmentation performance (Dice Score and volume bias) and quantitative analysis of Standardized Uptake Value (SUV) measurements. Additionally, radiomics-derived features were utilized as a measure of quality assurance (QA) in comparison to true SC-PET. Finally, a performance ensemble score (EPS) was developed by integrating fidelity-based and task-based metrics. Concordance Correlation Coefficient (CCC) was utilized to determine concordance between measures. The non-parametric Friedman Test with Bonferroni correction was used to compare the performance of ARD-Net against benchmarked methods with significance at adjusted p-value ≤0.01.

RESULTS

ARD-Net-generated SC-PET images exhibited significantly better (p ≤ 0.01 post Bonferroni correction) overall image fidelity scores in terms of SSIM and NRMSE at majority of photon-count levels compared to benchmarked DL and non-DL methods. In terms of task-based quantitative accuracy evaluated by SUVMean and SUVPeak, ARD-Net exhibited less than 5% median absolute bias for SUVMean compared to true SC-PET and lower degree of variability compared to benchmarked DL and non-DL based methods in generating SC-PET. Additionally, ARD-Net-generated SC-PET images displayed higher degree of concordance to SC-PET images in terms of radiomics features compared to non-DL and other DL approaches. Finally, the ensemble score suggested that ARD-Net exhibited significantly superior performance compared to benchmarked algorithms (p ≤ 0.01 post Bonferroni correction).

CONCLUSION

ARD-Net provides a robust framework to generate SC-PET from LC-PET images. ARD-Net generated SC-PET images exhibited superior performance compared other DL and non-DL approaches in terms of image-fidelity based metrics, task-based segmentation metrics, and minimal bias in terms of task-based quantification performance for preclinical PET imaging.

TMTV-Net: fully automated total metabolic tumor volume segmentation in lymphoma PET/CT images - a multi-center generalizability analysis

PURPOSE

Total metabolic tumor volume (TMTV) segmentation has significant value enabling quantitative imaging biomarkers for lymphoma management. In this work, we tackle the challenging task of automated tumor delineation in lymphoma from PET/CT scans using a cascaded approach.

METHODS

Our study included 1418 2-[18F]FDG PET/CT scans from four different centers. The dataset was divided into 900 scans for development/validation/testing phases and 518 for multi-center external testing. The former consisted of 450 lymphoma, lung cancer, and melanoma scans, along with 450 negative scans, while the latter consisted of lymphoma patients from different centers with diffuse large B cell, primary mediastinal large B cell, and classic Hodgkin lymphoma cases. Our approach involves resampling PET/CT images into different voxel sizes in the first step, followed by training multi-resolution 3D U-Nets on each resampled dataset using a fivefold cross-validation scheme. The models trained on different data splits were ensemble. After applying soft voting to the predicted masks, in the second step, we input the probability-averaged predictions, along with the input imaging data, into another 3D U-Net. Models were trained with semi-supervised loss. We additionally considered the effectiveness of using test time augmentation (TTA) to improve the segmentation performance after training. In addition to quantitative analysis including Dice score (DSC) and TMTV comparisons, the qualitative evaluation was also conducted by nuclear medicine physicians.

RESULTS

Our cascaded soft-voting guided approach resulted in performance with an average DSC of 0.68 ± 0.12 for the internal test data from developmental dataset, and an average DSC of 0.66 ± 0.18 on the multi-site external data (n = 518), significantly outperforming (p < 0.001) state-of-the-art (SOTA) approaches including nnU-Net and SWIN UNETR. While TTA yielded enhanced performance gains for some of the comparator methods, its impact on our cascaded approach was found to be negligible (DSC: 0.66 ± 0.16). Our approach reliably quantified TMTV, with a correlation of 0.89 with the ground truth (p < 0.001). Furthermore, in terms of visual assessment, concordance between quantitative evaluations and clinician feedback was observed in the majority of cases. The average relative error (ARE) and the absolute error (AE) in TMTV prediction on external multi-centric dataset were ARE = 0.43 ± 0.54 and AE = 157.32 ± 378.12 (mL) for all the external test data (n = 518), and ARE = 0.30 ± 0.22 and AE = 82.05 ± 99.78 (mL) when the 10% outliers (n = 53) were excluded.

CONCLUSION

TMTV-Net demonstrates strong performance and generalizability in TMTV segmentation across multi-site external datasets, encompassing various lymphoma subtypes. A negligible reduction of 2% in overall performance during testing on external data highlights robust model generalizability across different centers and cancer types, likely attributable to its training with resampled inputs. Our model is publicly available, allowing easy multi-site evaluation and generalizability analysis on datasets from different institutions.

A quality assurance framework for routine monitoring of deep learning cardiac substructure computed tomography segmentation models in radiotherapy

BACKGROUND

For autosegmentation models, the data used to train the model (e.g., public datasets and/or vendor-collected data) and the data on which the model is deployed in the clinic are typically not the same, potentially impacting the performance of these models by a process called domain shift. Tools to routinely monitor and predict segmentation performance are needed for quality assurance. Here, we develop an approach to perform such monitoring and performance prediction for cardiac substructure segmentation.

PURPOSE

To develop a quality assurance (QA) framework for routine or continuous monitoring of domain shift and the performance of cardiac substructure autosegmentation algorithms.

METHODS

A benchmark dataset consisting of computed tomography (CT) images along with manual cardiac substructure delineations of 241 breast cancer radiotherapy patients were collected, including one "normal" image domain of clean images and five "abnormal" domains containing images with artifact (metal, contrast), pathology, or quality variations due to scanner protocol differences (field of view, noise, reconstruction kernel, and slice thickness). The QA framework consisted of an image domain shift detector which operated on the input CT images and a shape quality detector on the output of an autosegmentation model, and a regression model for predicting autosegmentation model performance. The image domain shift detector was composed of a trained denoising autoencoder (DAE) and two hand-engineered image quality features to detect normal versus abnormal domains in the input CT images. The shape quality detector was a variational autoencoder (VAE) trained to estimate the shape quality of the auto-segmentation results. The output from the image domain shift and shape quality detectors was used to train a regression model to predict the per-patient segmentation accuracy, measured by Dice coefficient similarity (DSC) to physician contours. Different regression techniques were investigated including linear regression, Bagging, Gaussian process regression, random forest, and gradient boost regression. Of the 241 patients, 60 were used to train the autosegmentation models, 120 for training the QA framework, and the remaining 61 for testing the QA framework. A total of 19 autosegmentation models were used to evaluate QA framework performance, including 18 convolutional neural network (CNN)-based and one transformer-based model.

RESULTS

When tested on the benchmark dataset, all abnormal domains resulted in a significant DSC decrease relative to the normal domain for CNN models (p < 0.001 $p < 0.001$ ), but only for some domains for the transformer model. No significant relationship was found between the performance of an autosegmentation model and scanner protocol parameters (p = 0.42 $p = 0.42$ ) except noise (p = 0.01 $p = 0.01$ ). CNN-based autosegmentation models demonstrated a decreased DSC ranging from 0.07 to 0.41 with added noise, while the transformer-based model was not significantly affected (ANOVA,p = 0.99 $p=0.99$ ). For the QA framework, linear regression models with bootstrap aggregation resulted in the highest mean absolute error (MAE) of0.041 ± 0.002 $0.041 \pm 0.002$ , in predicted DSC (relative to true DSC between autosegmentation and physician). MAE was lowest when combining both input (image) detectors and output (shape) detectors compared to output detectors alone.

CONCLUSIONS

A QA framework was able to predict cardiac substructure autosegmentation model performance for clinically anticipated "abnormal" domain shifts.

DEMIST: A Deep-Learning-Based Detection-Task-Specific Denoising Approach for Myocardial Perfusion SPECT

There is an important need for methods to process myocardial perfusion imaging (MPI) single-photon emission computed tomography (SPECT) images acquired at lower-radiation dose and/or acquisition time such that the processed images improve observer performance on the clinical task of detecting perfusion defects compared to low-dose images. To address this need, we build upon concepts from model-observer theory and our understanding of the human visual system to propose a detection task-specific deep-learning-based approach for denoising MPI SPECT images (DEMIST). The approach, while performing denoising, is designed to preserve features that influence observer performance on detection tasks. We objectively evaluated DEMIST on the task of detecting perfusion defects using a retrospective study with anonymized clinical data in patients who underwent MPI studies across two scanners (N = 338). The evaluation was performed at low-dose levels of 6.25%, 12.5%, and 25% and using an anthropomorphic channelized Hotelling observer. Performance was quantified using area under the receiver operating characteristics curve (AUC). Images denoised with DEMIST yielded significantly higher AUC compared to corresponding low-dose images and images denoised with a commonly used task-agnostic deep learning-based denoising method. Similar results were observed with stratified analysis based on patient sex and defect type. Additionally, DEMIST improved visual fidelity of the low-dose images as quantified using root mean squared error and structural similarity index metric. A mathematical analysis revealed that DEMIST preserved features that assist in detection tasks while improving the noise properties, resulting in improved observer performance. The results provide strong evidence for further clinical evaluation of DEMIST to denoise low-count images in MPI SPECT.

Semi-supervised learning towards automated segmentation of PET images with limited annotations: application to lymphoma patients

Manual segmentation poses a time-consuming challenge for disease quantification, therapy evaluation, treatment planning, and outcome prediction. Convolutional neural networks (CNNs) hold promise in accurately identifying tumor locations and boundaries in PET scans. However, a major hurdle is the extensive amount of supervised and annotated data necessary for training. To overcome this limitation, this study explores semi-supervised approaches utilizing unlabeled data, specifically focusing on PET images of diffuse large B-cell lymphoma (DLBCL) and primary mediastinal large B-cell lymphoma (PMBCL) obtained from two centers. We considered 2-[18F]FDG PET images of 292 patients PMBCL (n = 104) and DLBCL (n = 188) (n = 232 for training and validation, and n = 60 for external testing). We harnessed classical wisdom embedded in traditional segmentation methods, such as the fuzzy clustering loss function (FCM), to tailor the training strategy for a 3D U-Net model, incorporating both supervised and unsupervised learning approaches. Various supervision levels were explored, including fully supervised methods with labeled FCM and unified focal/Dice loss, unsupervised methods with robust FCM (RFCM) and Mumford-Shah (MS) loss, and semi-supervised methods combining FCM with supervised Dice loss (MS + Dice) or labeled FCM (RFCM + FCM). The unified loss function yielded higher Dice scores (0.73 ± 0.11; 95% CI 0.67-0.8) than Dice loss (p value < 0.01). Among the semi-supervised approaches, RFCM + αFCM (α = 0.3) showed the best performance, with Dice score of 0.68 ± 0.10 (95% CI 0.45-0.77), outperforming MS + αDice for any supervision level (any α) (p < 0.01). Another semi-supervised approach with MS + αDice (α = 0.2) achieved Dice score of 0.59 ± 0.09 (95% CI 0.44-0.76) surpassing other supervision levels (p < 0.01). Given the time-consuming nature of manual delineations and the inconsistencies they may introduce, semi-supervised approaches hold promise for automating medical imaging segmentation workflows.

Synthetic PET from CT improves diagnosis and prognosis for lung cancer: Proof of concept

[18F]Fluorodeoxyglucose positron emission tomography (FDG-PET) and computed tomography (CT) are indispensable components in modern medicine. Although PET can provide additional diagnostic value, it is costly and not universally accessible, particularly in low-income countries. To bridge this gap, we have developed a conditional generative adversarial network pipeline that can produce FDG-PET from diagnostic CT scans based on multi-center multi-modal lung cancer datasets (n = 1,478). Synthetic PET images are validated across imaging, biological, and clinical aspects. Radiologists confirm comparable imaging quality and tumor contrast between synthetic and actual PET scans. Radiogenomics analysis further proves that the dysregulated cancer hallmark pathways of synthetic PET are consistent with actual PET. We also demonstrate the clinical values of synthetic PET in improving lung cancer diagnosis, staging, risk prediction, and prognosis. Taken together, this proof-of-concept study testifies to the feasibility of applying deep learning to obtain high-fidelity PET translated from CT.

Need for Objective Task-Based Evaluation of Image Segmentation Algorithms for Quantitative PET: A Study with ACRIN 6668/RTOG 0235 Multicenter Clinical Trial Data

Reliable performance of PET segmentation algorithms on clinically relevant tasks is required for their clinical translation. However, these algorithms are typically evaluated using figures of merit (FoMs) that are not explicitly designed to correlate with clinical task performance. Such FoMs include the Dice similarity coefficient (DSC), the Jaccard similarity coefficient (JSC), and the Hausdorff distance (HD). The objective of this study was to investigate whether evaluating PET segmentation algorithms using these task-agnostic FoMs yields interpretations consistent with evaluation on clinically relevant quantitative tasks. Methods: We conducted a retrospective study to assess the concordance in the evaluation of segmentation algorithms using the DSC, JSC, and HD and on the tasks of estimating the metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of primary tumors from PET images of patients with non-small cell lung cancer. The PET images were collected from the American College of Radiology Imaging Network 6668/Radiation Therapy Oncology Group 0235 multicenter clinical trial data. The study was conducted in 2 contexts: (1) evaluating conventional segmentation algorithms, namely those based on thresholding (SUVmax40% and SUVmax50%), boundary detection (Snakes), and stochastic modeling (Markov random field-Gaussian mixture model); (2) evaluating the impact of network depth and loss function on the performance of a state-of-the-art U-net-based segmentation algorithm. Results: Evaluation of conventional segmentation algorithms based on the DSC, JSC, and HD showed that SUVmax40% significantly outperformed SUVmax50%. However, SUVmax40% yielded lower accuracy on the tasks of estimating MTV and TLG, with a 51% and 54% increase, respectively, in the ensemble normalized bias. Similarly, the Markov random field-Gaussian mixture model significantly outperformed Snakes on the basis of the task-agnostic FoMs but yielded a 24% increased bias in estimated MTV. For the U-net-based algorithm, our evaluation showed that although the network depth did not significantly alter the DSC, JSC, and HD values, a deeper network yielded substantially higher accuracy in the estimated MTV and TLG, with a decreased bias of 91% and 87%, respectively. Additionally, whereas there was no significant difference in the DSC, JSC, and HD values for different loss functions, up to a 73% and 58% difference in the bias of the estimated MTV and TLG, respectively, existed. Conclusion: Evaluation of PET segmentation algorithms using task-agnostic FoMs could yield findings discordant with evaluation on clinically relevant quantitative tasks. This study emphasizes the need for objective task-based evaluation of image segmentation algorithms for quantitative PET.

Artificial intelligence in the risk prediction models of cardiovascular disease and development of an independent validation screening tool: a systematic review

BACKGROUND

A comprehensive overview of artificial intelligence (AI) for cardiovascular disease (CVD) prediction and a screening tool of AI models (AI-Ms) for independent external validation are lacking. This systematic review aims to identify, describe, and appraise AI-Ms of CVD prediction in the general and special populations and develop a new independent validation score (IVS) for AI-Ms replicability evaluation.

METHODS

PubMed, Web of Science, Embase, and IEEE library were searched up to July 2021. Data extraction and analysis were performed for the populations, distribution, predictors, algorithms, etc. The risk of bias was evaluated with the prediction risk of bias assessment tool (PROBAST). Subsequently, we designed IVS for model replicability evaluation with five steps in five items, including transparency of algorithms, performance of models, feasibility of reproduction, risk of reproduction, and clinical implication, respectively. The review is registered in PROSPERO (No. CRD42021271789).

RESULTS

In 20,887 screened references, 79 articles (82.5% in 2017-2021) were included, which contained 114 datasets (67 in Europe and North America, but 0 in Africa). We identified 486 AI-Ms, of which the majority were in development (n = 380), but none of them had undergone independent external validation. A total of 66 idiographic algorithms were found; however, 36.4% were used only once and only 39.4% over three times. A large number of different predictors (range 5-52,000, median 21) and large-span sample size (range 80-3,660,000, median 4466) were observed. All models were at high risk of bias according to PROBAST, primarily due to the incorrect use of statistical methods. IVS analysis confirmed only 10 models as "recommended"; however, 281 and 187 were "not recommended" and "warning," respectively.

CONCLUSION

AI has led the digital revolution in the field of CVD prediction, but is still in the early stage of development as the defects of research design, report, and evaluation systems. The IVS we developed may contribute to independent external validation and the development of this field.

METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII

PURPOSE

To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies.

METHODS

We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated.

RESULT

In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community.

CONCLUSION

In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers.

CRITICAL RELEVANCE STATEMENT

A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning.

KEY POINTS

• A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score ( https://metricsscore.github.io/metrics/METRICS.html ) and a repository created to collect feedback from the radiomics community ( https://github.com/metricsscore/metrics ).

Radiomics and Artificial Intelligence in Radiotheranostics: A Review of Applications for Radioligands Targeting Somatostatin Receptors and Prostate-Specific Membrane Antigens

Radiotheranostics refers to the pairing of radioactive imaging biomarkers with radioactive therapeutic compounds that deliver ionizing radiation. Given the introduction of very promising radiopharmaceuticals, the radiotheranostics approach is creating a novel paradigm in personalized, targeted radionuclide therapies (TRTs), also known as radiopharmaceuticals (RPTs). Radiotherapeutic pairs targeting somatostatin receptors (SSTR) and prostate-specific membrane antigens (PSMA) are increasingly being used to diagnose and treat patients with metastatic neuroendocrine tumors (NETs) and prostate cancer. In parallel, radiomics and artificial intelligence (AI), as important areas in quantitative image analysis, are paving the way for significantly enhanced workflows in diagnostic and theranostic fields, from data and image processing to clinical decision support, improving patient selection, personalized treatment strategies, response prediction, and prognostication. Furthermore, AI has the potential for tremendous effectiveness in patient dosimetry which copes with complex and time-consuming tasks in the RPT workflow. The present work provides a comprehensive overview of radiomics and AI application in radiotheranostics, focusing on pairs of SSTR- or PSMA-targeting radioligands, describing the fundamental concepts and specific imaging/treatment features. Our review includes ligands radiolabeled by 68Ga, 18F, 177Lu, 64Cu, 90Y, and 225Ac. Specifically, contributions via radiomics and AI towards improved image acquisition, reconstruction, treatment response, segmentation, restaging, lesion classification, dose prediction, and estimation as well as ongoing developments and future directions are discussed.

An Artificial Intelligence System for Optimizing Radioactive Iodine Therapy Dosimetry

Thyroid cancer, specifically differentiated thyroid carcinoma (DTC), is one of the most prevalent endocrine malignancies worldwide. Radioactive iodine therapy (RAIT) using I-131 has been a standard-of-care approach for DTC due to its ability to ablate remnant thyroid disease following surgery, thus reducing the risk of recurrence. It is also used for the treatment of iodine-avid metastases. RAIT dosimetry can be employed to determine the optimal treatment dose of I-131 to effectively treat cancer cells while safeguarding against undesirable radiation effects such as bone marrow toxicity or radiation pneumonitis. Conventional dosimetry protocols for RAIT, however, are complex and time-consuming, involving multiple days of imaging and blood sampling. This study explores the use of Artificial Intelligence (AI) in simplifying and optimizing RAIT. A retrospective analysis was conducted on 83 adult patients with DTC who underwent RAIT dosimetry at our institution between 1996 and 2023. The conventional MIRD-based dosimetry protocol involved imaging and blood sampling at 4, 24, 48, 72, and 96 h post-administration of a tracer activity of I-131. An AI system based on a deep-learning neural network was developed to predict the maximum permissible activity (MPA) for RAIT using only the data obtained from the initial 4, 24, and 48 h time points. The AI system predicted the MPA values with high accuracy, showing no significant difference compared to the results obtained from conventional MIRD-based analysis utilizing a paired t-test (p = 0.351, 95% CI). The developed AI system offers the potential to streamline the dosimetry process, reducing the number of imaging and blood sampling sessions while also optimizing resource allocation. Additionally, the AI approach can uncover underlying relationships in data that were previously unknown. Our findings suggest that AI-based dosimetry may be a promising method for patient-specific treatment planning in differentiated thyroid carcinoma, representing a step towards applying precision medicine for thyroid cancer. Further validation and implementation studies are warranted to assess the clinical applicability of the AI system.

Observer studies of image quality of denoising reduced-count cardiac single photon emission computed tomography myocardial perfusion imaging by three-dimensional Gaussian post-reconstruction filtering and deep learning

BACKGROUND

The aim of this research was to asses perfusion-defect detection-accuracy by human observers as a function of reduced-counts for 3D Gaussian post-reconstruction filtering vs deep learning (DL) denoising to determine if there was improved performance with DL.

METHODS

SPECT projection data of 156 normally interpreted patients were used for these studies. Half were altered to include hybrid perfusion defects with defect presence and location known. Ordered-subset expectation-maximization (OSEM) reconstruction was employed with the optional correction of attenuation (AC) and scatter (SC) in addition to distance-dependent resolution (RC). Count levels varied from full-counts (100%) to 6.25% of full-counts. The denoising strategies were previously optimized for defect detection using total perfusion deficit (TPD). Four medical physicist (PhD) and six physician (MD) observers rated the slices using a graphical user interface. Observer ratings were analyzed using the LABMRMC multi-reader, multi-case receiver-operating-characteristic (ROC) software to calculate and compare statistically the area-under-the-ROC-curves (AUCs).

RESULTS

For the same count-level no statistically significant increase in AUCs for DL over Gaussian denoising was determined when counts were reduced to either the 25% or 12.5% of full-counts. The average AUC for full-count OSEM with solely RC and Gaussian filtering was lower than for the strategies with AC and SC, except for a reduction to 6.25% of full-counts, thus verifying the utility of employing AC and SC with RC.

CONCLUSION

We did not find any indication that at the dose levels investigated and with the DL network employed, that DL denoising was superior in AUC to optimized 3D post-reconstruction Gaussian filtering.

Ethical Considerations for Artificial Intelligence in Medical Imaging: Data Collection, Development, and Evaluation

The development of artificial intelligence (AI) within nuclear imaging involves several ethically fraught components at different stages of the machine learning pipeline, including during data collection, model training and validation, and clinical use. Drawing on the traditional principles of medical and research ethics, and highlighting the need to ensure health justice, the AI task force of the Society of Nuclear Medicine and Molecular Imaging has identified 4 major ethical risks: privacy of data subjects, data quality and model efficacy, fairness toward marginalized populations, and transparency of clinical performance. We provide preliminary recommendations to developers of AI-driven medical devices for mitigating the impact of these risks on patients and populations.

The Past, Present, and Future Role of Artificial Intelligence in Ventilation/Perfusion Scintigraphy: A Systematic Review

Ventilation-perfusion (V/Q) lung scans constitute one of the oldest nuclear medicine procedures, remain one of the few studies performed in the acute setting, and are amongst the few performed in the emergency setting. V/Q studies have witnessed a long fluctuation in adoption rates in parallel to continuous advances in image processing and computer vision techniques. This review provides an overview on the status of artificial intelligence (AI) in V/Q scintigraphy. To clearly assess the past, current, and future role of AI in V/Q scans, we conducted a systematic Ovid MEDLINE(R) literature search from 1946 to August 5, 2022 in addition to a manual search. The literature was reviewed and summarized in terms of methodologies and results for the various applications of AI to V/Q scans. The PRISMA guidelines were followed. Thirty-one publications fulfilled our search criteria and were grouped into two distinct categories: (1) disease diagnosis/detection (N = 22, 71.0%) and (2) cross-modality image translation into V/Q images (N = 9, 29.0%). Studies on disease diagnosis and detection relied heavily on shallow artificial neural networks for acute pulmonary embolism (PE) diagnosis and were primarily published between the mid-1990s and early 2000s. Recent applications almost exclusively regard image translation tasks from CT to ventilation or perfusion images with modern algorithms, such as convolutional neural networks, and were published between 2019 and 2022. AI research in V/Q scintigraphy for acute PE diagnosis in the mid-90s to early 2000s yielded promising results but has since been largely neglected and thus have yet to benefit from today's state-of-the art machine-learning techniques, such as deep neural networks. Recently, the main application of AI for V/Q has shifted towards generating synthetic ventilation and perfusion images from CT. There is therefore considerable potential to expand and modernize the use of real V/Q studies with state-of-the-art deep learning approaches, especially for workflow optimization and PE detection at both acute and chronic stages. We discuss future challenges and potential directions to compensate for the lag in this domain and enhance the value of this traditional nuclear medicine scan.

Ethical Considerations for Artificial Intelligence in Medical Imaging: Deployment and Governance

The deployment of artificial intelligence (AI) has the potential to make nuclear medicine and medical imaging faster, cheaper, and both more effective and more accessible. This is possible, however, only if clinicians and patients feel that these AI medical devices (AIMDs) are trustworthy. Highlighting the need to ensure health justice by fairly distributing benefits and burdens while respecting individual patients' rights, the AI Task Force of the Society of Nuclear Medicine and Molecular Imaging has identified 4 major ethical risks that arise during the deployment of AIMD: autonomy of patients and clinicians, transparency of clinical performance and limitations, fairness toward marginalized populations, and accountability of physicians and developers. We provide preliminary recommendations for governing these ethical risks to realize the promise of AIMD for patients and populations.

Sensitivity of an AI method for [18F]FDG PET/CT outcome prediction of diffuse large B-cell lymphoma patients to image reconstruction protocols

BACKGROUND

Convolutional neural networks (CNNs), applied to baseline [18F]-FDG PET/CT maximum intensity projections (MIPs), show potential for treatment outcome prediction in diffuse large B-cell lymphoma (DLBCL). The aim of this study is to investigate the robustness of CNN predictions to different image reconstruction protocols. Baseline [18F]FDG PET/CT scans were collected from 20 DLBCL patients. EARL1, EARL2 and high-resolution (HR) protocols were applied per scan, generating three images with different image qualities. Image-based transformation was applied by blurring EARL2 and HR images to generate EARL1 compliant images using a Gaussian filter of 5 and 7 mm, respectively. MIPs were generated for each of the reconstructions, before and after image transformation. An in-house developed CNN predicted the probability of tumor progression within 2 years for each MIP. The difference in probabilities per patient was then calculated between both EARL2 and HR with respect to EARL1 (delta probabilities or ΔP). We compared these to the probabilities obtained after aligning the data with ComBat using the difference in median and interquartile range (IQR).

RESULTS

CNN probabilities were found to be sensitive to different reconstruction protocols (EARL2 ΔP: median = 0.09, interquartile range (IQR) = [0.06, 0.10] and HR ΔP: median = 0.1, IQR = [0.08, 0.16]). Moreover, higher resolution images (EARL2 and HR) led to higher probability values. After image-based and ComBat transformation, an improved agreement of CNN probabilities among reconstructions was found for all patients. This agreement was slightly better after image-based transformation (transformed EARL2 ΔP: median = 0.022, IQR = [0.01, 0.02] and transformed HR ΔP: median = 0.029, IQR = [0.01, 0.03]).

CONCLUSION

Our CNN-based outcome predictions are affected by the applied reconstruction protocols, yet in a predictable manner. Image-based harmonization is a suitable approach to harmonize CNN predictions across image reconstruction protocols.

Quantitative PET-based biomarkers in lymphoma: getting ready for primetime

The use of functional quantitative biomarkers extracted from routine PET-CT scans to characterize clinical responses in patients with lymphoma is gaining increased attention, and these biomarkers can outperform established clinical risk factors. Total metabolic tumour volume enables individualized estimation of survival outcomes in patients with lymphoma and has shown the potential to predict response to therapy suitable for  risk-adapted treatment approaches in clinical trials. The deployment of machine learning tools in molecular imaging research can assist in recognizing complex patterns and, with image classification, in tumour identification and segmentation of data from PET-CT scans. Initial studies using fully automated approaches to calculate metabolic tumour volume and other PET-based biomarkers have demonstrated appropriate correlation with calculations from experts, warranting further testing in large-scale studies. The extraction of computer-based quantitative tumour characterization through radiomics can provide a comprehensive view of phenotypic heterogeneity that better captures the molecular and functional features of the disease. Additionally, radiomics can be integrated with genomic data to provide more accurate prognostic information. Further improvements in PET-based biomarkers are imminent, although their incorporation into clinical decision-making currently has methodological shortcomings that need to be addressed with confirmatory prospective validation in selected patient populations. In this Review, we discuss the current knowledge, challenges and opportunities in the integration of quantitative PET-based biomarkers in clinical trials and the routine management of patients with lymphoma.

Machine learning in the positron emission tomography imaging of Alzheimer's disease

The utilization of machine learning techniques in medicine has exponentially increased over the last decades due to innovations in computer processing, algorithm development, and access to big data. Applications of machine learning techniques to neuroimaging specifically have unveiled various hidden interactions, structures, and mechanisms related to various neurological disorders. One application of interest is the imaging of Alzheimer's disease, the most common cause of progressive dementia. The diagnoses of Alzheimer's disease, mild cognitive impairment, and preclinical Alzheimer's disease have been difficult. Molecular imaging, particularly via PET scans, holds tremendous value in the imaging of Alzheimer's disease. To date, many novel algorithms have been developed with great success that leverage machine learning in the context of Alzheimer's disease. This review article provides an overview of the diverse applications of machine learning to PET imaging of Alzheimer's disease.

An artificial intelligence method using FDG PET to predict treatment outcome in diffuse large B cell lymphoma patients

Convolutional neural networks (CNNs) may improve response prediction in diffuse large B-cell lymphoma (DLBCL). The aim of this study was to investigate the feasibility of a CNN using maximum intensity projection (MIP) images from 18F-fluorodeoxyglucose (18F-FDG) positron emission tomography (PET) baseline scans to predict the probability of time-to-progression (TTP) within 2 years and compare it with the International Prognostic Index (IPI), i.e. a clinically used score. 296 DLBCL 18F-FDG PET/CT baseline scans collected from a prospective clinical trial (HOVON-84) were analysed. Cross-validation was performed using coronal and sagittal MIPs. An external dataset (340 DLBCL patients) was used to validate the model. Association between the probabilities, metabolic tumour volume and Dmaxbulk was assessed. Probabilities for PET scans with synthetically removed tumors were also assessed. The CNN provided a 2-year TTP prediction with an area under the curve (AUC) of 0.74, outperforming the IPI-based model (AUC = 0.68). Furthermore, high probabilities (> 0.6) of the original MIPs were considerably decreased after removing the tumours (< 0.4, generally). These findings suggest that MIP-based CNNs are able to predict treatment outcome in DLBCL.

A Guide to Cross-Validation for Artificial Intelligence in Medical Imaging

Artificial intelligence (AI) is being increasingly used to automate and improve technologies within the field of medical imaging. A critical step in the development of an AI algorithm is estimating its prediction error through cross-validation (CV). The use of CV can help prevent overoptimism in AI algorithms and can mitigate certain biases associated with hyperparameter tuning and algorithm selection. This article introduces the principles of CV and provides a practical guide on the use of CV for AI algorithm development in medical imaging. Different CV techniques are described, as well as their advantages and disadvantages under different scenarios. Common pitfalls in prediction error estimation and guidance on how to avoid them are also discussed. Keywords: Education, Research Design, Technical Aspects, Statistics, Supervised Learning, Convolutional Neural Network (CNN) Supplemental material is available for this article. © RSNA, 2023.

Need for objective task-based evaluation of deep learning-based denoising methods: A study in the context of myocardial perfusion SPECT

BACKGROUND

Artificial intelligence-based methods have generated substantial interest in nuclear medicine. An area of significant interest has been the use of deep-learning (DL)-based approaches for denoising images acquired with lower doses, shorter acquisition times, or both. Objective evaluation of these approaches is essential for clinical application.

PURPOSE

DL-based approaches for denoising nuclear-medicine images have typically been evaluated using fidelity-based figures of merit (FoMs) such as root mean squared error (RMSE) and structural similarity index measure (SSIM). However, these images are acquired for clinical tasks and thus should be evaluated based on their performance in these tasks. Our objectives were to: (1) investigate whether evaluation with these FoMs is consistent with objective clinical-task-based evaluation; (2) provide a theoretical analysis for determining the impact of denoising on signal-detection tasks; and (3) demonstrate the utility of virtual imaging trials (VITs) to evaluate DL-based methods.

METHODS

A VIT to evaluate a DL-based method for denoising myocardial perfusion SPECT (MPS) images was conducted. To conduct this evaluation study, we followed the recently published best practices for the evaluation of AI algorithms for nuclear medicine (the RELAINCE guidelines). An anthropomorphic patient population modeling clinically relevant variability was simulated. Projection data for this patient population at normal and low-dose count levels (20%, 15%, 10%, 5%) were generated using well-validated Monte Carlo-based simulations. The images were reconstructed using a 3-D ordered-subsets expectation maximization-based approach. Next, the low-dose images were denoised using a commonly used convolutional neural network-based approach. The impact of DL-based denoising was evaluated using both fidelity-based FoMs and area under the receiver operating characteristic curve (AUC), which quantified performance on the clinical task of detecting perfusion defects in MPS images as obtained using a model observer with anthropomorphic channels. We then provide a mathematical treatment to probe the impact of post-processing operations on signal-detection tasks and use this treatment to analyze the findings of this study.

RESULTS

Based on fidelity-based FoMs, denoising using the considered DL-based method led to significantly superior performance. However, based on ROC analysis, denoising did not improve, and in fact, often degraded detection-task performance. This discordance between fidelity-based FoMs and task-based evaluation was observed at all the low-dose levels and for different cardiac-defect types. Our theoretical analysis revealed that the major reason for this degraded performance was that the denoising method reduced the difference in the means of the reconstructed images and of the channel operator-extracted feature vectors between the defect-absent and defect-present cases.

CONCLUSIONS

The results show the discrepancy between the evaluation of DL-based methods with fidelity-based metrics versus the evaluation on clinical tasks. This motivates the need for objective task-based evaluation of DL-based denoising approaches. Further, this study shows how VITs provide a mechanism to conduct such evaluations computationally, in a time and resource-efficient setting, and avoid risks such as radiation dose to the patient. Finally, our theoretical treatment reveals insights into the reasons for the limited performance of the denoising approach and may be used to probe the effect of other post-processing operations on signal-detection tasks.

An Investigation of Lesion Detection Accuracy for Artificial Intelligence-Based Denoising of Low-Dose 64Cu-DOTATATE PET Imaging in Patients with Neuroendocrine Neoplasms

Frequent somatostatin receptor PET, for example, 64Cu-DOTATATE PET, is part of the diagnostic work-up of patients with neuroendocrine neoplasms (NENs), resulting in high accumulated radiation doses. Scan-related radiation exposure should be minimized in accordance with the as-low-as-reasonably achievable principle, for example, by reducing injected radiotracer activity. Previous investigations found that reducing 64Cu-DOTATATE activity to below 50 MBq results in inadequate image quality and lesion detection. We therefore investigated whether image quality and lesion detection of less than 50 MBq of 64Cu-DOTATATE PET could be restored using artificial intelligence (AI). Methods: We implemented a parameter-transferred Wasserstein generative adversarial network for patients with NENs on simulated low-dose 64Cu-DOTATATE PET images corresponding to 25% (PET25%), or about 48 MBq, of the injected activity of the reference full dose (PET100%), or about 191 MBq, to generate denoised PET images (PETAI). We included 38 patients in the training sets for network optimization. We analyzed PET intensity correlation, peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and mean-square error (MSE) of PETAI/PET100% versus PET25%/PET100% Two readers assessed Likert scale-defined image quality (1, very poor; 2, poor; 3, moderate; 4, good; 5, excellent) and identified lesion-suspicious foci on PETAI and PET100% in a subset of the patients with no more than 20 lesions per organ (n = 33) to allow comparison of all foci on a 1:1 basis. Detected foci were scored (C1, definite lesion; C0, lesion-suspicious focus) and matched with PET100% as the reference. True-positive (TP), false-positive (FP), and false-negative (FN) lesions were assessed. Results: For PETAI/PET100% versus PET25%/PET100%, PET intensity correlation had a goodness-of-fit value of 0.94 versus 0.81, PSNR was 58.1 versus 53.0, SSIM was 0.908 versus 0.899, and MSE was 2.6 versus 4.7. Likert scale-defined image quality was rated good or excellent in 33 of 33 and 32 of 33 patients on PET100% and PETAI, respectively. Total number of detected lesions was 118 on PET100% and 115 on PETAI Only 78 PETAI lesions were TP, 40 were FN, and 37 were FP, yielding detection sensitivity (TP/(TP+FN)) and a false discovery rate (FP/(TP+FP)) of 66% (78/118) and 32% (37/115), respectively. In 62% (23/37) of cases, the FP lesion was scored C1, suggesting a definite lesion. Conclusion: PETAI improved visual similarity with PET100% compared with PET25%, and PETAI and PET100% had similar Likert scale-defined image quality. However, lesion detection analysis performed by physicians showed high proportions of FP and FN lesions on PETAI, highlighting the need for clinical validation of AI algorithms.

Carotid atherosclerotic plaque segmentation in multi-weighted MRI using a two-stage neural network: advantages of training with high-resolution imaging and histology

Introduction

A reliable and automated method to segment and classify carotid artery atherosclerotic plaque components is needed to efficiently analyze multi-weighted magnetic resonance (MR) images to allow their integration into patient risk assessment for ischemic stroke. Certain plaque components such as lipid-rich necrotic core (LRNC) with hemorrhage suggest a greater likelihood of plaque rupture and stroke event. Assessment for presence and extent of LRNC could assist in directing treatment with impact upon patient outcomes.

Methods

To address the need to accurately determine the presence and extent of plaque components on carotid plaque MRI, we proposed a two-staged deep-learning-based approach that consists of a convolutional neural network (CNN), followed by a Bayesian neural network (BNN). The rationale for the two-stage network approach is to account for the class imbalance of vessel wall and background by providing an attention mask to the BNN. A unique feature of the network training was to use ground truth defined by both high-resolution ex vivo MRI data and histopathology. More specifically, standard resolution 1.5 T in vivo MR image sets with corresponding high resolution 3.0 T ex vivo MR image sets and histopathology image sets were used to define ground-truth segmentations. Of these, data from 7 patients was used for training and from the remaining two was used for testing the proposed method. Next, to evaluate the generalizability of the method, we tested the method with an additional standard resolution 3.0 T in vivo data set of 23 patients obtained from a different scanner.

Results

Our results show that the proposed method yielded accurate segmentation of carotid atherosclerotic plaque and outperforms not only manual segmentation by trained readers, who did not have access to the ex vivo or histopathology data, but also three state-of-the-art deep-learning-based segmentation methods. Further, the proposed approach outperformed a strategy where the ground truth was generated without access to the high resolution ex vivo MRI and histopathology. The accurate performance of this method was also observed in the additional 23-patient dataset from a different scanner.

Conclusion

In conclusion, the proposed method provides a mechanism to perform accurate segmentation of the carotid atherosclerotic plaque in multi-weighted MRI. Further, our study shows the advantages of using high-resolution imaging and histology to define ground truth for training deep-learning-based segmentation methods.

CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII

Even though radiomics can hold great potential for supporting clinical decision-making, its current use is mostly limited to academic research, without applications in routine clinical practice. The workflow of radiomics is complex due to several methodological steps and nuances, which often leads to inadequate reporting and evaluation, and poor reproducibility. Available reporting guidelines and checklists for artificial intelligence and predictive modeling include relevant good practices, but they are not tailored to radiomic research. There is a clear need for a complete radiomics checklist for study planning, manuscript writing, and evaluation during the review process to facilitate the repeatability and reproducibility of studies. We here present a documentation standard for radiomic research that can guide authors and reviewers. Our motivation is to improve the quality and reliability and, in turn, the reproducibility of radiomic research. We name the checklist CLEAR (CheckList for EvaluAtion of Radiomics research), to convey the idea of being more transparent. With its 58 items, the CLEAR checklist should be considered a standardization tool providing the minimum requirements for presenting clinical radiomics research. In addition to a dynamic online version of the checklist, a public repository has also been set up to allow the radiomics community to comment on the checklist items and adapt the checklist for future versions. Prepared and revised by an international group of experts using a modified Delphi method, we hope the CLEAR checklist will serve well as a single and complete scientific documentation tool for authors and reviewers to improve the radiomics literature.

Critical Appraisal of Artificial Intelligence-Enabled Imaging Tools Using the Levels of Evidence System

Clinical adoption of an artificial intelligence-enabled imaging tool requires critical appraisal of its life cycle from development to implementation by using a systematic, standardized, and objective approach that can verify both its technical and clinical efficacy. Toward this concerted effort, the ASFNR/ASNR Artificial Intelligence Workshop Technology Working Group is proposing a hierarchal evaluation system based on the quality, type, and amount of scientific evidence that the artificial intelligence-enabled tool can demonstrate for each component of its life cycle. The current proposal is modeled after the levels of evidence in medicine, with the uppermost level of the hierarchy showing the strongest evidence for potential impact on patient care and health care outcomes. The intended goal of establishing an evidence-based evaluation system is to encourage transparency, foster an understanding of the creation of artificial intelligence tools and the artificial intelligence decision-making process, and to report the relevant data on the efficacy of artificial intelligence tools that are developed. The proposed system is an essential step in working toward a more formalized, clinically validated, and regulated framework for the safe and effective deployment of artificial intelligence imaging applications that will be used in clinical practice.

Observer-study-based approaches to quantitatively evaluate the realism of synthetic medical images

Objective.Synthetic images generated by simulation studies have a well-recognized role in developing and evaluating imaging systems and methods. However, for clinically relevant development and evaluation, the synthetic images must be clinically realistic and, ideally, have the same distribution as that of clinical images. Thus, mechanisms that can quantitatively evaluate this clinical realism and, ideally, the similarity in distributions of the real and synthetic images, are much needed.Approach.We investigated two observer-study-based approaches to quantitatively evaluate the clinical realism of synthetic images. In the first approach, we presented a theoretical formalism for the use of an ideal-observer study to quantitatively evaluate the similarity in distributions between the real and synthetic images. This theoretical formalism provides a direct relationship between the area under the receiver operating characteristic curve, AUC, for an ideal observer and the distributions of real and synthetic images. The second approach is based on the use of expert-human-observer studies to quantitatively evaluate the realism of synthetic images. In this approach, we developed a web-based software to conduct two-alternative forced-choice (2-AFC) experiments with expert human observers. The usability of this software was evaluated by conducting a system usability scale (SUS) survey with seven expert human readers and five observer-study designers. Further, we demonstrated the application of this software to evaluate a stochastic and physics-based image-synthesis technique for oncologic positron emission tomography (PET). In this evaluation, the 2-AFC study with our software was performed by six expert human readers, who were highly experienced in reading PET scans, with years of expertise ranging from 7 to 40 years (median: 12 years, average: 20.4 years).Main results.In the ideal-observer-study-based approach, we theoretically demonstrated that the AUC for an ideal observer can be expressed, to an excellent approximation, by the Bhattacharyya distance between the distributions of the real and synthetic images. This relationship shows that a decrease in the ideal-observer AUC indicates a decrease in the distance between the two image distributions. Moreover, a lower bound of ideal-observer AUC = 0.5 implies that the distributions of synthetic and real images exactly match. For the expert-human-observer-study-based approach, our software for performing the 2-AFC experiments is available athttps://apps.mir.wustl.edu/twoafc. Results from the SUS survey demonstrate that the web application is very user friendly and accessible. As a secondary finding, evaluation of a stochastic and physics-based PET image-synthesis technique using our software showed that expert human readers had limited ability to distinguish the real images from the synthetic images.Significance.This work addresses the important need for mechanisms to quantitatively evaluate the clinical realism of synthetic images. The mathematical treatment in this paper shows that quantifying the similarity in the distribution of real and synthetic images is theoretically possible by using an ideal-observer-study-based approach. Our developed software provides a platform for designing and performing 2-AFC experiments with human observers in a highly accessible, efficient, and secure manner. Additionally, our results on the evaluation of the stochastic and physics-based image-synthesis technique motivate the application of this technique to develop and evaluate a wide array of PET imaging methods.

Development and task-based evaluation of a scatter-window projection and deep learning-based transmission-less attenuation compensation method for myocardial perfusion SPECT

Attenuation compensation (AC) is beneficial for visual interpretation tasks in single-photon emission computed tomography (SPECT) myocardial perfusion imaging (MPI). However, traditional AC methods require the availability of a transmission scan, most often a CT scan. This approach has the disadvantages of increased radiation dose, increased scanner cost, and the possibility of inaccurate diagnosis in cases of misregistration between the SPECT and CT images. Further, many SPECT systems do not include a CT component. To address these issues, we developed a Scatter-window projection and deep Learning-based AC (SLAC) method to perform AC without a separate transmission scan. To investigate the clinical efficacy of this method, we then objectively evaluated the performance of this method on the clinical task of detecting perfusion defects on MPI in a retrospective study with anonymized clinical SPECT/CT stress MPI images. The proposed method was compared with CT-based AC (CTAC) and no-AC (NAC) methods. Our results showed that the SLAC method yielded an almost overlapping receiver operating characteristic (ROC) plot and a similar area under the ROC (AUC) to the CTAC method on this task. These results demonstrate the capability of the SLAC method for transmission-less AC in SPECT and motivate further clinical evaluation.

Development and task-based evaluation of a scatter-window projection and deep learning-based transmission-less attenuation compensation method for myocardial perfusion SPECT

Attenuation compensation (AC) is beneficial for visual interpretation tasks in single-photon emission computed tomography (SPECT) myocardial perfusion imaging (MPI). However, traditional AC methods require the availability of a transmission scan, most often a CT scan. This approach has the disadvantage of increased radiation dose, increased scanner costs, and the possibility of inaccurate diagnosis in cases of misregistration between the SPECT and CT images. Further, many SPECT systems do not include a CT component. To address these issues, we developed a Scatter-window projection and deep Learning-based AC (SLAC) method to perform AC without a separate transmission scan. To investigate the clinical efficacy of this method, we then objectively evaluated the performance of this method on the clinical task of detecting perfusion defects on MPI in a retrospective study with anonymized clinical SPECT/CT stress MPI images. The proposed method was compared with CT-based AC (CTAC) and no-AC (NAC) methods. Our results showed that the SLAC method yielded an almost overlapping receiver operating characteristic (ROC) plot and a similar area under the ROC (AUC) to the CTAC method on this task. These results demonstrate the capability of the SLAC method for transmission-less AC in SPECT and motivate further clinical evaluation.

A task-specific deep-learning-based denoising approach for myocardial perfusion SPECT

Deep-learning (DL)-based methods have shown significant promise in denoising myocardial perfusion SPECT images acquired at low dose. For clinical application of these methods, evaluation on clinical tasks is crucial. Typically, these methods are designed to minimize some fidelity-based criterion between the predicted denoised image and some reference normal-dose image. However, while promising, studies have shown that these methods may have limited impact on the performance of clinical tasks in SPECT. To address this issue, we use concepts from the literature on model observers and our understanding of the human visual system to propose a DL-based denoising approach designed to preserve observer-related information for detection tasks. The proposed method was objectively evaluated on the task of detecting perfusion defect in myocardial perfusion SPECT images using a retrospective study with anonymized clinical data. Our results demonstrate that the proposed method yields improved performance on this detection task compared to using low-dose images. The results show that by preserving task-specific information, DL may provide a mechanism to improve observer performance in low-dose myocardial perfusion SPECT.

Artificial Intelligence in Nuclear Medicine: Opportunities, Challenges, and Responsibilities Toward a Trustworthy Ecosystem

Trustworthiness is a core tenet of medicine. The patient-physician relationship is evolving from a dyad to a broader ecosystem of health care. With the emergence of artificial intelligence (AI) in medicine, the elements of trust must be revisited. We envision a road map for the establishment of trustworthy AI ecosystems in nuclear medicine. In this report, AI is contextualized in the history of technologic revolutions. Opportunities for AI applications in nuclear medicine related to diagnosis, therapy, and workflow efficiency, as well as emerging challenges and critical responsibilities, are discussed. Establishing and maintaining leadership in AI require a concerted effort to promote the rational and safe deployment of this innovative technology by engaging patients, nuclear medicine physicians, scientists, technologists, and referring providers, among other stakeholders, while protecting our patients and society. This strategic plan was prepared by the AI task force of the Society of Nuclear Medicine and Molecular Imaging.

Need for objective task-based evaluation of AI-based segmentation methods for quantitative PET

Artificial intelligence (AI)-based methods are showing substantial promise in segmenting oncologic positron emission tomography (PET) images. For clinical translation of these methods, assessing their performance on clinically relevant tasks is important. However, these methods are typically evaluated using metrics that may not correlate with the task performance. One such widely used metric is the Dice score, a figure of merit that measures the spatial overlap between the estimated segmentation and a reference standard (e.g., manual segmentation). In this work, we investigated whether evaluating AI-based segmentation methods using Dice scores yields a similar interpretation as evaluation on the clinical tasks of quantifying metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of primary tumor from PET images of patients with non-small cell lung cancer. The investigation was conducted via a retrospective analysis with the ECOG-ACRIN 6668/RTOG 0235 multi-center clinical trial data. Specifically, we evaluated different structures of a commonly used AI-based segmentation method using both Dice scores and the accuracy in quantifying MTV/TLG. Our results show that evaluation using Dice scores can lead to findings that are inconsistent with evaluation using the task-based figure of merit. Thus, our study motivates the need for objective task-based evaluation of AI-based segmentation methods for quantitative PET.

Joint EANM/SNMMI guideline on radiomics in nuclear medicine : Jointly supported by the EANM Physics Committee and the SNMMI Physics, Instrumentation and Data Sciences Council

PURPOSE

The purpose of this guideline is to provide comprehensive information on best practices for robust radiomics analyses for both hand-crafted and deep learning-based approaches.

METHODS

In a cooperative effort between the EANM and SNMMI, we agreed upon current best practices and recommendations for relevant aspects of radiomics analyses, including study design, quality assurance, data collection, impact of acquisition and reconstruction, detection and segmentation, feature standardization and implementation, as well as appropriate modelling schemes, model evaluation, and interpretation. We also offer an outlook for future perspectives.

CONCLUSION

Radiomics is a very quickly evolving field of research. The present guideline focused on established findings as well as recommendations based on the state of the art. Though this guideline recognizes both hand-crafted and deep learning-based radiomics approaches, it primarily focuses on the former as this field is more mature. This guideline will be updated once more studies and results have contributed to improved consensus regarding the application of deep learning methods for radiomics. Although methodological recommendations in the present document are valid for most medical image modalities, we focus here on nuclear medicine, and specific recommendations when necessary are made for PET/CT, PET/MR, and quantitative SPECT.

Monitoring of Current Cancer Therapy by Positron Emission Tomography and Possible Role of Radiomics Assessment

Evaluation of cancer therapy with imaging is crucial as a surrogate marker of effectiveness and survival. The unique response patterns to therapy with immune-checkpoint inhibitors have facilitated the revision of response evaluation criteria using FDG-PET, because the immune response recalls reactive cells such as activated T-cells and macrophages, which show increased glucose metabolism and apparent progression on morphological imaging. Cellular metabolism and function are critical determinants of the viability of active cells in the tumor microenvironment, which would be novel targets of therapies, such as tumor immunity, metabolism, and genetic mutation. Considering tumor heterogeneity and variation in therapy response specific to the mechanisms of therapy, appropriate response evaluation is required. Radiomics approaches, which combine objective image features with a machine learning algorithm as well as pathologic and genetic data, have remarkably progressed over the past decade, and PET radiomics has increased quality and reliability based on the prosperous publications and standardization initiatives. PET and multimodal imaging will play a definitive role in personalized therapeutic strategies by the precise monitoring in future cancer therapy.

A tissue‐fraction estimation‐based segmentation method for quantitative dopamine transporter SPECT

BACKGROUND

Quantitative measures of dopamine transporter (DaT) uptake in caudate, putamen, and globus pallidus (GP) derived from dopamine transporter-single-photon emission computed tomography (DaT-SPECT) images have potential as biomarkers for measuring the severity of Parkinson's disease. Reliable quantification of this uptake requires accurate segmentation of the considered regions. However, segmentation of these regions from DaT-SPECT images is challenging, a major reason being partial-volume effects (PVEs) in SPECT. The PVEs arise from two sources, namely the limited system resolution and reconstruction of images over finite-sized voxel grids. The limited system resolution results in blurred boundaries of the different regions. The finite voxel size leads to TFEs, that is, voxels contain a mixture of regions. Thus, there is an important need for methods that can account for the PVEs, including the TFEs, and accurately segment the caudate, putamen, and GP, from DaT-SPECT images.

PURPOSE

Design and objectively evaluate a fully automated tissue-fraction estimation-based segmentation method that segments the caudate, putamen, and GP from DaT-SPECT images.

METHODS

The proposed method estimates the posterior mean of the fractional volumes occupied by the caudate, putamen, and GP within each voxel of a three-dimensional DaT-SPECT image. The estimate is obtained by minimizing a cost function based on the binary cross-entropy loss between the true and estimated fractional volumes over a population of SPECT images, where the distribution of true fractional volumes is obtained from existing populations of clinical magnetic resonance images. The method is implemented using a supervised deep-learning-based approach.

RESULTS

Evaluations using clinically guided highly realistic simulation studies show that the proposed method accurately segmented the caudate, putamen, and GP with high mean Dice similarity coefficients of ∼ 0.80 and significantly outperformed ( p < 0.01 $p &lt; 0.01$ ) all other considered segmentation methods. Further, an objective evaluation of the proposed method on the task of quantifying regional uptake shows that the method yielded reliable quantification with low ensemble normalized root mean square error (NRMSE) < 20% for all the considered regions. In particular, the method yielded an even lower ensemble NRMSE of ∼ 10% for the caudate and putamen.

CONCLUSIONS

The proposed tissue-fraction estimation-based segmentation method for DaT-SPECT images demonstrated the ability to accurately segment the caudate, putamen, and GP, and reliably quantify the uptake within these regions. The results motivate further evaluation of the method with physical-phantom and patient studies.

Investigating the limited performance of a deep-learning-based SPECT denoising approach: An observer-study-based characterization

Multiple objective assessment of image-quality (OAIQ)-based studies have reported that several deep-learning (DL)-based denoising methods show limited performance on signal-detection tasks. Our goal was to investigate the reasons for this limited performance. To achieve this goal, we conducted a task-based characterization of a DL-based denoising approach for individual signal properties. We conducted this study in the context of evaluating a DL-based approach for denoising single photon-emission computed tomography (SPECT) images. The training data consisted of signals of different sizes and shapes within a clustered-lumpy background, imaged with a 2D parallel-hole-collimator SPECT system. The projections were generated at normal and 20% low-count level, both of which were reconstructed using an ordered-subset-expectation-maximization (OSEM) algorithm. A convolutional neural network (CNN)-based denoiser was trained to process the low-count images. The performance of this CNN was characterized for five different signal sizes and four different signal-to-background ratio (SBRs) by designing each evaluation as a signal-known-exactly/background-known-statistically (SKE/BKS) signal-detection task. Performance on this task was evaluated using an anthropomorphic channelized Hotelling observer (CHO). As in previous studies, we observed that the DL-based denoising method did not improve performance on signal-detection tasks. Evaluation using the idea of observer-study-based characterization demonstrated that the DL-based denoising approach did not improve performance on the signal-detection task for any of the signal types. Overall, these results provide new insights on the performance of the DL-based denoising approach as a function of signal size and contrast. More generally, the observer study-based characterization provides a mechanism to evaluate the sensitivity of the method to specific object properties, and may be explored as analogous to characterizations such as modulation transfer function for linear systems. Finally, this work underscores the need for objective task-based evaluation of DL-based denoising approaches.