Automated liver lesion detection in 68Ga DOTATATE PET/CT using a deep fully convolutional neural network

Models using comprehensive, lesion-level, longitudinal [68Ga]Ga-DOTA-TATE PET-derived features lead to superior outcome prediction in neuroendocrine tumor patients treated with [177Lu]Lu-DOTA-TATE

PURPOSE

Somatostatin receptor (SSTR) imaging features are predictive of treatment outcome for neuroendocrine tumor (NET) patients receiving peptide receptor radionuclide therapy (PRRT). However, comprehensive (all metastatic lesions), longitudinal (temporal variation), and lesion-level measured features have never been explored. Such features allow for capturing the heterogeneity in disease response to treatment. Furthermore, models combining these features are lacking. In this work we evaluated the predictive power of comprehensive, longitudinal, lesion-level 68GA-SSTR-PET features combined with a multivariate linear regression (MLR) model.

METHODS

This retrospective study enrolled NET patients treated with [177Lu]Lu-DOTA-TATE and imaged with [68Ga]Ga-DOTA-TATE at baseline and post-therapy. All lesions were segmented, anatomically labeled, and longitudinally matched. Lesion-level uptake and variation in uptake were measured. Patient-level features were engineered and selected for modeling of progression-free survival (PFS). The model was validated via concordance index, patient classification (ROC analysis), and survival analysis (Kaplan-Meier and Cox proportional hazards). The MLR was benchmarked against single feature predictions.

RESULTS

Thirty-six NET patients were enrolled and stratified into poor and good responders (PFS ≥ 25 months). Four patient-level features were selected, the MLR concordance index was 0.826, and the AUC was 0.88 (0.85 specificity, 0.81 sensitivity). Survival analysis led to significant patient stratification (p<.001) and hazard ratio (3⨯10-5). Lastly, in a benchmark study, the MLR modeling approach outperformed all the single feature predictors.

CONCLUSION

Comprehensive, lesion-level, longitudinal 68GA-SSTR-PET analysis, combined with MLR modeling, leads to excellent predictions of PRRT outcome in NET patients, outperforming non-comprehensive, patient-level, and single time-point feature predictions.

MESSAGE

Neuroendocrine tumor, peptide receptor radionuclide therapy, Somatostatin Receptor Imaging, Outcome Prediction, Treatment Response Assessment.

Improving Generalizability of PET DL Algorithms: List-Mode Reconstructions Improve DOTATATE PET Hepatic Lesion Detection Performance

Deep learning (DL) algorithms used for DOTATATE PET lesion detection typically require large, well-annotated training datasets. These are difficult to obtain due to low incidence of gastroenteropancreatic neuroendocrine tumors (GEP-NETs) and the high cost of manual annotation. Furthermore, networks trained and tested with data acquired from site specific PET/CT instrumentation, acquisition and processing protocols have reduced performance when tested with offsite data. This lack of generalizability requires even larger, more diverse training datasets. The objective of this study is to investigate the feasibility of improving DL algorithm performance by better matching the background noise in training datasets to higher noise, out-of-domain testing datasets. 68Ga-DOTATATE PET/CT datasets were obtained from two scanners: Scanner1, a state-of-the-art digital PET/CT (GE DMI PET/CT; n = 83 subjects), and Scanner2, an older-generation analog PET/CT (GE STE; n = 123 subjects). Set1, the data set from Scanner1, was reconstructed with standard clinical parameters (5 min; Q.Clear) and list-mode reconstructions (VPFXS 2, 3, 4, and 5-min). Set2, data from Scanner2 representing out-of-domain clinical scans, used standard iterative reconstruction (5 min; OSEM). A deep neural network was trained with each dataset: Network1 for Scanner1 and Network2 for Scanner2. DL performance (Network1) was tested with out-of-domain test data (Set2). To evaluate the effect of training sample size, we tested DL model performance using a fraction (25%, 50% and 75%) of Set1 for training. Scanner1, list-mode 2-min reconstructed data demonstrated the most similar noise level compared that of Set2, resulting in the best performance (F1 = 0.713). This was not significantly different compared to the highest performance, upper-bound limit using in-domain training for Network2 (F1 = 0.755; p-value = 0.103). Regarding sample size, the F1 score significantly increased from 25% training data (F1 = 0.478) to 100% training data (F1 = 0.713; p < 0.001). List-mode data from modern PET scanners can be reconstructed to better match the noise properties of older scanners. Using existing data and their associated annotations dramatically reduces the cost and effort in generating these datasets and significantly improves the performance of existing DL algorithms. List-mode reconstructions can provide an efficient, low-cost method to improve DL algorithm generalizability.

Learning Without Real Data Annotations to Detect Hepatic Lesions in PET Images

OBJECTIVE

Deep neural networks have been recently applied to lesion identification in fluorodeoxyglucose (FDG) positron emission tomography (PET) images, but they typically rely on a large amount of well-annotated data for model training. This is extremely difficult to achieve for neuroendocrine tumors (NETs), because of low incidence of NETs and expensive lesion annotation in PET images. The objective of this study is to design a novel, adaptable deep learning method, which uses no real lesion annotations but instead low-cost, list mode-simulated data, for hepatic lesion detection in real-world clinical NET PET images.

METHODS

We first propose a region-guided generative adversarial network (RG-GAN) for lesion-preserved image-to-image translation. Then, we design a specific data augmentation module for our list-mode simulated data and incorporate this module into the RG-GAN to improve model training. Finally, we combine the RG-GAN, the data augmentation module and a lesion detection neural network into a unified framework for joint-task learning to adaptatively identify lesions in real-world PET data.

RESULTS

The proposed method outperforms recent state-of-the-art lesion detection methods in real clinical 68Ga-DOTATATE PET images, and produces very competitive performance with the target model that is trained with real lesion annotations.

CONCLUSION

With RG-GAN modeling and specific data augmentation, we can obtain good lesion detection performance without using any real data annotations.

SIGNIFICANCE

This study introduces an adaptable deep learning method for hepatic lesion identification in NETs, which can significantly reduce human effort for data annotation and improve model generalizability for lesion detection with PET imaging.

Radiomics and Artificial Intelligence in Radiotheranostics: A Review of Applications for Radioligands Targeting Somatostatin Receptors and Prostate-Specific Membrane Antigens

Radiotheranostics refers to the pairing of radioactive imaging biomarkers with radioactive therapeutic compounds that deliver ionizing radiation. Given the introduction of very promising radiopharmaceuticals, the radiotheranostics approach is creating a novel paradigm in personalized, targeted radionuclide therapies (TRTs), also known as radiopharmaceuticals (RPTs). Radiotherapeutic pairs targeting somatostatin receptors (SSTR) and prostate-specific membrane antigens (PSMA) are increasingly being used to diagnose and treat patients with metastatic neuroendocrine tumors (NETs) and prostate cancer. In parallel, radiomics and artificial intelligence (AI), as important areas in quantitative image analysis, are paving the way for significantly enhanced workflows in diagnostic and theranostic fields, from data and image processing to clinical decision support, improving patient selection, personalized treatment strategies, response prediction, and prognostication. Furthermore, AI has the potential for tremendous effectiveness in patient dosimetry which copes with complex and time-consuming tasks in the RPT workflow. The present work provides a comprehensive overview of radiomics and AI application in radiotheranostics, focusing on pairs of SSTR- or PSMA-targeting radioligands, describing the fundamental concepts and specific imaging/treatment features. Our review includes ligands radiolabeled by 68Ga, 18F, 177Lu, 64Cu, 90Y, and 225Ac. Specifically, contributions via radiomics and AI towards improved image acquisition, reconstruction, treatment response, segmentation, restaging, lesion classification, dose prediction, and estimation as well as ongoing developments and future directions are discussed.

Application of Artificial Intelligence in Oncologic Molecular PET-Imaging: A Narrative Review on Beyond [18F]F-FDG Tracers - Part I. PSMA, Choline, and DOTA Radiotracers

Artificial intelligence (AI) has evolved significantly in the past few decades. This thriving trend has also been seen in medicine in recent years, particularly in the field of imaging. Machine learning (ML), deep learning (DL), and their methods (eg, SVM, CNN), as well as radiomics, are the terminologies that have been introduced to this field and, to some extent, become familiar to the expert clinicians. PET is one of the modalities that has been enhanced via these state-of-the-art algorithms. This robust imaging technique further merged with anatomical modalities, such as computed tomography (CT) and magnetic resonance imaging (MRI), to provide reliable hybrid modalities, PET/CT and PET/MRI. Applying AI-based algorithms on the different components (PET, CT, and MRI) has resulted in promising results, maximizing the value of PET imaging. However, [18F]F-FDG, the most commonly utilized tracer in molecular imaging, has been mainly in the spotlight. Thus, we aimed to look into the less discussed tracers in this review, moving beyond [18F]F-FDG. The novel non-[18F]F-FDG agents also showed to be valuable in various clinical tasks, including lesion detection and tumor characterization, accurate delineation, and prognostic impact. Regarding prostate patients, PSMA-based models were highly accurate in determining tumoral lesions' location and delineating them, particularly within the prostate gland. However, they also could assess whole-body images to detect extra-prostatic lesions in a patient automatically. Considering the prognostic value of prostate-specific membrane antigen (PSMA) PET using AI, it could predict response to treatment and patient survival, which are crucial in patient management. Choline imaging, another non-[18F]F-FDG tracer, similarly showed acceptable results that may be of benefit in the clinic, though the current evidence is significantly more limited than PSMA. Lastly, different subtypes of DOTA ligands were found to be valuable. They could diagnose tumoral lesions in challenging sites and even predict histopathology grade, being a highly advantageous noninvasive tool. In conclusion, the current limited investigations have shown promising results, leading us to a bright future for AI in molecular imaging beyond [18F]F-FDG.

Artificial Intelligence in Oncological Hybrid Imaging

BACKGROUND

 Artificial intelligence (AI) applications have become increasingly relevant across a broad spectrum of settings in medical imaging. Due to the large amount of imaging data that is generated in oncological hybrid imaging, AI applications are desirable for lesion detection and characterization in primary staging, therapy monitoring, and recurrence detection. Given the rapid developments in machine learning (ML) and deep learning (DL) methods, the role of AI will have significant impact on the imaging workflow and will eventually improve clinical decision making and outcomes.

METHODS AND RESULTS

 The first part of this narrative review discusses current research with an introduction to artificial intelligence in oncological hybrid imaging and key concepts in data science. The second part reviews relevant examples with a focus on applications in oncology as well as discussion of challenges and current limitations.

CONCLUSION

 AI applications have the potential to leverage the diagnostic data stream with high efficiency and depth to facilitate automated lesion detection, characterization, and therapy monitoring to ultimately improve quality and efficiency throughout the medical imaging workflow. The goal is to generate reproducible, structured, quantitative diagnostic data for evidence-based therapy guidance in oncology. However, significant challenges remain regarding application development, benchmarking, and clinical implementation.

KEY POINTS

  · Hybrid imaging generates a large amount of multimodality medical imaging data with high complexity and depth.. · Advanced tools are required to enable fast and cost-efficient processing along the whole radiology value chain.. · AI applications promise to facilitate the assessment of oncological disease in hybrid imaging with high quality and efficiency for lesion detection, characterization, and response assessment. The goal is to generate reproducible, structured, quantitative diagnostic data for evidence-based oncological therapy guidance.. · Selected applications in three oncological entities (lung, prostate, and neuroendocrine tumors) demonstrate how AI algorithms may impact imaging-based tasks in hybrid imaging and potentially guide clinical decision making..

AI-based quantification of whole-body tumour burden on somatostatin receptor PET/CT

BACKGROUND

Segmenting the whole-body somatostatin receptor-expressing tumour volume (SRETVwb) on positron emission tomography/computed tomography (PET/CT) images is highly time-consuming but has shown value as an independent prognostic factor for survival. An automatic method to measure SRETVwb could improve disease status assessment and provide a tool for prognostication. This study aimed to develop an artificial intelligence (AI)-based method to detect and quantify SRETVwb and total lesion somatostatin receptor expression (TLSREwb) from [68Ga]Ga-DOTA-TOC/TATE PET/CT images.

METHODS

A UNet3D convolutional neural network (CNN) was used to train an AI model with [68Ga]Ga-DOTA-TOC/TATE PET/CT images, where all tumours were manually segmented with a semi-automatic method. The training set consisted of 148 patients, of which 108 had PET-positive tumours. The test group consisted of 30 patients, of which 25 had PET-positive tumours. Two physicians segmented tumours in the test group for comparison with the AI model.

RESULTS

There were good correlations between the segmented SRETVwb and TLSREwb by the AI model and the physicians, with Spearman rank correlation coefficients of r = 0.78 and r = 0.73, respectively, for SRETVwb and r = 0.83 and r = 0.81, respectively, for TLSREwb. The sensitivity on a lesion detection level was 80% and 79%, and the positive predictive value was 83% and 84% when comparing the AI model with the two physicians.

CONCLUSION

It was possible to develop an AI model to segment SRETVwb and TLSREwb with high performance. A fully automated method makes quantification of tumour burden achievable and has the potential to be more widely used when assessing PET/CT images.

Applications of Artificial Intelligence and Radiomics in Molecular Hybrid Imaging and Theragnostics for Neuro-Endocrine Neoplasms (NENs)

Nuclear medicine has acquired a crucial role in the management of patients with neuroendocrine neoplasms (NENs) by improving the accuracy of diagnosis and staging as well as their risk stratification and personalized therapies, including radioligand therapies (RLT). Artificial intelligence (AI) and radiomics can enable physicians to further improve the overall efficiency and accuracy of the use of these tools in both diagnostic and therapeutic settings by improving the prediction of the tumor grade, differential diagnosis from other malignancies, assessment of tumor behavior and aggressiveness, and prediction of treatment response. This systematic review aims to describe the state-of-the-art AI and radiomics applications in the molecular imaging of NENs.

Clinical application of AI-based PET images in oncological patients

Based on the advantages of revealing the functional status and molecular expression of tumor cells, positron emission tomography (PET) imaging has been performed in numerous types of malignant diseases for diagnosis and monitoring. However, insufficient image quality, the lack of a convincing evaluation tool and intra- and interobserver variation in human work are well-known limitations of nuclear medicine imaging and restrict its clinical application. Artificial intelligence (AI) has gained increasing interest in the field of medical imaging due to its powerful information collection and interpretation ability. The combination of AI and PET imaging potentially provides great assistance to physicians managing patients. Radiomics, an important branch of AI applied in medical imaging, can extract hundreds of abstract mathematical features of images for further analysis. In this review, an overview of the applications of AI in PET imaging is provided, focusing on image enhancement, tumor detection, response and prognosis prediction and correlation analyses with pathology or specific gene mutations in several types of tumors. Our aim is to describe recent clinical applications of AI-based PET imaging in malignant diseases and to focus on the description of possible future developments.

Artificial Intelligence in Oncological Hybrid Imaging

BACKGROUND

Artificial intelligence (AI) applications have become increasingly relevant across a broad spectrum of settings in medical imaging. Due to the large amount of imaging data that is generated in oncological hybrid imaging, AI applications are desirable for lesion detection and characterization in primary staging, therapy monitoring, and recurrence detection. Given the rapid developments in machine learning (ML) and deep learning (DL) methods, the role of AI will have significant impact on the imaging workflow and will eventually improve clinical decision making and outcomes.

METHODS AND RESULTS

The first part of this narrative review discusses current research with an introduction to artificial intelligence in oncological hybrid imaging and key concepts in data science. The second part reviews relevant examples with a focus on applications in oncology as well as discussion of challenges and current limitations.

CONCLUSION

AI applications have the potential to leverage the diagnostic data stream with high efficiency and depth to facilitate automated lesion detection, characterization, and therapy monitoring to ultimately improve quality and efficiency throughout the medical imaging workflow. The goal is to generate reproducible, structured, quantitative diagnostic data for evidence-based therapy guidance in oncology. However, significant challenges remain regarding application development, benchmarking, and clinical implementation.

KEY POINTS

· Hybrid imaging generates a large amount of multimodality medical imaging data with high complexity and depth.. · Advanced tools are required to enable fast and cost-efficient processing along the whole radiology value chain.. · AI applications promise to facilitate the assessment of oncological disease in hybrid imaging with high quality and efficiency for lesion detection, characterization, and response assessment. The goal is to generate reproducible, structured, quantitative diagnostic data for evidence-based oncological therapy guidance.. · Selected applications in three oncological entities (lung, prostate, and neuroendocrine tumors) demonstrate how AI algorithms may impact imaging-based tasks in hybrid imaging and potentially guide clinical decision making..

CITATION FORMAT

· Feuerecker B, Heimer M, Geyer T et al. Artificial Intelligence in Oncological Hybrid Imaging. Fortschr Röntgenstr 2023; 195: 105 - 114.

Modeling contrast-to-noise ratio from list mode reconstructions of 68Ga DOTATATE PET/CT: predicting detectability of hepatic metastases in shorter acquisition PET reconstructions

BACKGROUND

Deep learning (DL) algorithms have shown promise in identifying and quantifying lesions in PET/CT. However, the accuracy and generalizability of these algorithms relies on large, diverse datasets which are time and labor intensive to curate. Modern PET/CT scanners may acquire data in list mode, allowing for multiple reconstructions of the same datasets with different parameters and imaging times. These reconstructions may provide a wide range of image characteristics to increase the size and diversity of datasets. Training algorithms with shorter imaging times and higher noise properties requires that lesions remain detectable. The purpose of this study is to model and predict the contrast-to-noise ratio (CNR) for shorter imaging times based on CNR from longer duration, lower noise images for ⁶⁸Ga DOTATATE PET hepatic lesions and identify a threshold above which lesions remain detectable.

METHODS

⁶⁸Ga DOTATATE subjects (n=20) with hepatic lesions were divided into two subgroups. The "Model" group (n=4 subjects; n=9 lesions; n=36 datapoints) was used to identify the relationship between CNR and imaging time. The "Test" group (n=16 subjects; n=44 lesions; n=176 datapoints) was used to evaluate the prediction provided by the model.

RESULTS

CNR plotted as a function of imaging time for a subset of identified subjects was very well fit with a quadratic model. For the remaining subjects, the measured CNR showed a very high linear correlation with the predicted CNR for these lesions (R² > 0.97) for all imaging durations. From the model, a threshold CNR=6.9 at 5-minutes predicted CNR > 5 at 2-minutes. Visual inspection of lesions in 2-minute images with CNR above the threshold in 5-minute images were assessed and rated as a 4 or 5 (probably positive or definitely positive) confirming 100% lesion detectability on the shorter 2-minute PET images.

CONCLUSIONS

CNR for shorter DOTATATE PET imaging times may be accurately predicted using list mode reconstructions of longer acquisitions. A threshold CNR may be applied to longer duration images to ensure lesion detectability of shorter duration reconstructions. This method can aid in the selection of lesions to include in novel data augmentation techniques for deep learning.

Differential Detection of Hepatic Metastases on 68 Ga-DOTATATE PET/CT and 177 Lu-DOTATATE SPECT/CT

ABSTRACT

68 Ga-DOTATATE PET/CT is indicated for selecting patients for peptide receptor radionuclide therapy (PRRT). Although highly sensitive, the detectability of smaller lesions, particularly in the liver, is lower. We present the case of a 58-year-old man with metastatic well-differentiated pancreatic neuroendocrine tumor whose MRI revealed progression of hepatic metastases. 68 Ga-DOTATATE PET/CT performed to determine eligibility for PRRT did not demonstrate DOTATATE-avid disease within the liver. 18 F-FDG PET/CT was also negative at the liver and the patient proceeded to 177 Lu-DOTATATE PRRT, where multi-time point posttherapy planar imaging and SPECT/CT showed intense uptake in the known liver metastases.

Current status and future of targeted peptide receptor radionuclide positron emission tomography imaging and therapy of gastroenteropancreatic-neuroendocrine tumors

Theranostics is the highly targeted molecular imaging and therapy of tumors. Targeted peptide receptor radionuclide therapy has taken the lead in demonstrating the safety and effectiveness of this molecular approach to treating cancers. Metastatic, well-differentiated gastroenteropancreatic neuroendocrine tumors may be most effectively imaged and treated with DOTATATE ligands. We review the current practice, safety, advantages, and limitations of DOTATATE based theranostics. Finally, we briefly describe the exciting new areas of development and future directions of gastroenteropancreatic neuroendocrine tumor theranostics.

Artificial Intelligence and Machine Learning in the Diagnosis and Management of Gastroenteropancreatic Neuroendocrine Neoplasms—A Scoping Review

Neuroendocrine neoplasms (NENs) and tumors (NETs) are rare neoplasms that may affect any part of the gastrointestinal system. In this scoping review, we attempt to map existing evidence on the role of artificial intelligence, machine learning and deep learning in the diagnosis and management of NENs of the gastrointestinal system. After implementation of inclusion and exclusion criteria, we retrieved 44 studies with 53 outcome analyses. We then classified the papers according to the type of studied NET (26 Pan-NETs, 59.1%; 3 metastatic liver NETs (6.8%), 2 small intestinal NETs, 4.5%; colorectal, rectal, non-specified gastroenteropancreatic and non-specified gastrointestinal NETs had from 1 study each, 2.3%). The most frequently used AI algorithms were Supporting Vector Classification/Machine (14 analyses, 29.8%), Convolutional Neural Network and Random Forest (10 analyses each, 21.3%), Random Forest (9 analyses, 19.1%), Logistic Regression (8 analyses, 17.0%), and Decision Tree (6 analyses, 12.8%). There was high heterogeneity on the description of the prediction model, structure of datasets, and performance metrics, whereas the majority of studies did not report any external validation set. Future studies should aim at incorporating a uniform structure in accordance with existing guidelines for purposes of reproducibility and research quality, which are prerequisites for integration into clinical practice.

An update on the diagnosis of gastroenteropancreatic neuroendocrine neoplasms

Gastroenteropancreatic neuroendocrine neoplasms (GEP-NENs) arise from neuroendocrine cells found throughout the gastrointestinal tract and islet cells of the pancreas. The incidence and prevalence of GEP-NENs have been increasing each year due to higher awareness, improved diagnostic modalities, and increased incidental detection on cross-sectional imaging and endoscopy for cancer screening and other conditions and symptoms. GEP-NENs are a heterogeneous group of tumors and have a wide range in clinical presentation, histopathologic features, and molecular biology. Clinical presentation most commonly depends on whether the GEP-NEN secretes an active hormone. The World Health Organization recently updated the classification of GEP-NENs to introduce a distinction between high-grade neuroendocrine tumors and neuroendocrine carcinomas, which can be identified using histology and molecular studies and are more aggressive with a worse prognosis compared to high-grade neuroendocrine tumors. As our understanding of the biology of GEP-NENs has grown, new and improved diagnostic modalities can be developed and optimized. Here, we discuss clinical features and updates in diagnosis, including histopathological analysis, biomarkers, molecular techniques, and radiology of GEP-NENs. We review established diagnostic tests and discuss promising novel diagnostic tests that are currently in development or require further investigation and validation prior to broad utilization in patient care.