Repeatability of two semi-automatic artificial intelligence approaches for tumor segmentation in PET

Challenges and limitations in applying radiomics to PET imaging: Possible opportunities and avenues for research

Radiomics, the high-throughput extraction of quantitative imaging features from medical images, holds immense potential for advancing precision medicine in oncology and beyond. While radiomics applied to positron emission tomography (PET) imaging offers unique insights into tumor biology and treatment response, it is imperative to elucidate the challenges and constraints inherent in this domain to facilitate their translation into clinical practice. This review examines the challenges and limitations of applying radiomics to PET imaging, synthesizing findings from the last five years (2019-2023) and highlights the significance of addressing these challenges to realize the full clinical potential of radiomics in oncology and molecular imaging. A comprehensive search was conducted across multiple electronic databases, including PubMed, Scopus, and Web of Science, using keywords relevant to radiomics issues in PET imaging. Only studies published in peer-reviewed journals were eligible for inclusion in this review. Although many studies have highlighted the potential of radiomics in predicting treatment response, assessing tumor heterogeneity, enabling risk stratification, and personalized therapy selection, various challenges regarding the practical implementation of the proposed models still need to be addressed. This review illustrates the challenges and limitations of radiomics in PET imaging across various cancer types, encompassing both phantom and clinical investigations. The analyzed studies highlight the importance of reproducible segmentation methods, standardized pre-processing and post-processing methodologies, and the need to create large multicenter studies registered in a centralized database to promote the continuous validation and clinical integration of radiomics into PET imaging.

Topological state-space estimation of functional human brain networks

We introduce an innovative, data-driven topological data analysis (TDA) technique for estimating the state spaces of dynamically changing functional human brain networks at rest. Our method utilizes the Wasserstein distance to measure topological differences, enabling the clustering of brain networks into distinct topological states. This technique outperforms the commonly used k-means clustering in identifying brain network state spaces by effectively incorporating the temporal dynamics of the data without the need for explicit model specification. We further investigate the genetic underpinnings of these topological features using a twin study design, examining the heritability of such state changes. Our findings suggest that the topology of brain networks, particularly in their dynamic state changes, may hold significant hidden genetic information.

Automated segmentation of lesions and organs at risk on [68Ga]Ga-PSMA-11 PET/CT images using self-supervised learning with Swin UNETR

BACKGROUND

Prostate-specific membrane antigen (PSMA) PET/CT imaging is widely used for quantitative image analysis, especially in radioligand therapy (RLT) for metastatic castration-resistant prostate cancer (mCRPC). Unknown features influencing PSMA biodistribution can be explored by analyzing segmented organs at risk (OAR) and lesions. Manual segmentation is time-consuming and labor-intensive, so automated segmentation methods are desirable. Training deep-learning segmentation models is challenging due to the scarcity of high-quality annotated images. Addressing this, we developed shifted windows UNEt TRansformers (Swin UNETR) for fully automated segmentation. Within a self-supervised framework, the model's encoder was pre-trained on unlabeled data. The entire model was fine-tuned, including its decoder, using labeled data.

METHODS

In this work, 752 whole-body [68Ga]Ga-PSMA-11 PET/CT images were collected from two centers. For self-supervised model pre-training, 652 unlabeled images were employed. The remaining 100 images were manually labeled for supervised training. In the supervised training phase, 5-fold cross-validation was used with 64 images for model training and 16 for validation, from one center. For testing, 20 hold-out images, evenly distributed between two centers, were used. Image segmentation and quantification metrics were evaluated on the test set compared to the ground-truth segmentation conducted by a nuclear medicine physician.

RESULTS

The model generates high-quality OARs and lesion segmentation in lesion-positive cases, including mCRPC. The results show that self-supervised pre-training significantly improved the average dice similarity coefficient (DSC) for all classes by about 3%. Compared to nnU-Net, a well-established model in medical image segmentation, our approach outperformed with a 5% higher DSC. This improvement was attributed to our model's combined use of self-supervised pre-training and supervised fine-tuning, specifically when applied to PET/CT input. Our best model had the lowest DSC for lesions at 0.68 and the highest for liver at 0.95.

CONCLUSIONS

We developed a state-of-the-art neural network using self-supervised pre-training on whole-body [68Ga]Ga-PSMA-11 PET/CT images, followed by fine-tuning on a limited set of annotated images. The model generates high-quality OARs and lesion segmentation for PSMA image analysis. The generalizable model holds potential for various clinical applications, including enhanced RLT and patient-specific internal dosimetry.

Prognostic model using 18F-FDG PET radiomics predicts progression-free survival in relapsed/refractory Hodgkin lymphoma

Investigating prognostic factors in patients with relapsed or primary refractory classical Hodgkin lymphoma (R/R cHL) is essential to optimize risk-adapted treatment strategies. We built a prognostic model using baseline quantitative 18F-fluorodeoxyglucose positron emission tomography (PET) radiomics features and clinical characteristics to predict the progression-free survival (PFS) among patients with R/R cHL treated with salvage chemotherapy followed by autologous stem cell transplantation. Metabolic tumor volume and several novel radiomics dissemination features, representing interlesional differences in distance, volume, and standard uptake value, were extracted from the baseline PET. Machine learning using backward selection and logistic regression were applied to develop and train the model on a total of 113 patients from 2 clinical trials. The model was validated on an independent external cohort of 69 patients. In addition, we validated 4 different PET segmentation methods to calculate radiomics features. We identified a subset of patients at high risk for progression with significant inferior 3-year PFS outcomes of 38.1% vs 88.4% for patients in the low-risk group in the training cohort (P < .001) and 38.5% vs 75.0% in the validation cohort (P = .015), respectively. The overall survival was also significantly better in the low-risk group (P = .022 and P < .001). We provide a formula to calculate a risk score for individual patients based on the model. In conclusion, we developed a prognostic model for PFS combining radiomics and clinical features in a large cohort of patients with R/R cHL. This model calculates a PET-based risk profile and can be applied to develop risk-stratified treatment strategies for patients with R/R cHL. These trials were registered at www.clinicaltrials.gov as #NCT02280993, #NCT00255723, and #NCT01508312.

A robust semi-automatic delineation workflow using denoised diffusion weighted magnetic resonance imaging for response assessment of patients with esophageal cancer treated with neoadjuvant chemoradiotherapy

Background and Purpose

Diffusion weighted magnetic resonance imaging (DW-MRI) can be prognostic for response to neoadjuvant chemotherapy (nCRT) in patients with esophageal cancer. However, manual tumor delineation is labor intensive and subjective. Furthermore, noise in DW-MRI images will propagate into the corresponding apparent diffusion coefficient (ADC) signal. In this study a workflow is investigated that combines a denoising algorithm with semi-automatic segmentation for quantifying ADC changes.

Materials and Methods

Twenty patients with esophageal cancer who underwent nCRT before esophagectomy were included. One baseline and five weekly DW-MRI scans were acquired for every patient during nCRT. A self-supervised learning denoising algorithm, Patch2Self, was used to denoise the DWI-MRI images. A semi-automatic delineation workflow (SADW) was next developed and compared with a manually adjusted workflow (MAW). The agreement between workflows was determined using the Dice coefficients and Brand Altman plots. The prognostic value of ADCmean increases (%/week) for pathologic complete response (pCR) was assessed using c-statistics.

Results

The median Dice coefficient between the SADW and MAW was 0.64 (interquartile range 0.20). For the MAW, the c-statistic for predicting pCR was 0.80 (95% confidence interval (CI):0.56-1.00). The SADW showed a c-statistic of 0.84 (95%CI:0.63-1.00) after denoising. No statistically significant differences in c-statistics were observed between the workflows or after applying denoising.

Conclusions

The SADW resulted in non-inferior prognostic value for pCR compared to the more laborious MAW, allowing broad scale applications. The effect of denoising on the prognostic value for pCR needs to be investigated in larger cohorts.

An Automatic Segmentation Method for Lung Tumor Based on Improved Region Growing Algorithm

In medical image processing, accurate segmentation of lung tumors is very important. Computer-aided accurate segmentation can effectively assist doctors in surgery planning and treatment decisions. Although the accurate segmentation results of lung tumors can provide a reliable basis for clinical treatment, the key to obtaining accurate segmentation results is how to improve the segmentation performance of the algorithm. We propose an automatic segmentation method for lung tumors based on an improved region growing algorithm, which uses the prior information on lung tumors to achieve an automatic selection of the initial seed point. The proposed method includes a seed point expansion mechanism and an automatic threshold update mechanism and takes the combination of multiple segmentation results as the final segmentation result. In the lung image database consortium (LIDC-IDRI) dataset, we designed 10 experiments to test the proposed method and compare it with 4 popular segmentation methods. The experimental results show that the average dice coefficient obtained by the proposed method is 0.936 ± 0.027, and the average Jaccard distance is 0.114 ± 0.049. The average dice coefficient obtained by the proposed method is 0.107, 0.053, 0.040, and 0.156, higher than that of the other four methods, respectively. This study proves that the proposed method can automatically segment lung tumors in CT slices and has suitable segmentation performance.

Convolutional neural networks for automatic image quality control and EARL compliance of PET images

Background

Machine learning studies require a large number of images often obtained on different PET scanners. When merging these images, the use of harmonized images following EARL-standards is essential. However, when including retrospective images, EARL accreditation might not have been in place. The aim of this study was to develop a convolutional neural network (CNN) that can identify retrospectively if an image is EARL compliant and if it is meeting older or newer EARL-standards.

Materials and methods

96 PET images acquired on three PET/CT systems were included in the study. All images were reconstructed with the locally clinically preferred, EARL1, and EARL2 compliant reconstruction protocols. After image pre-processing, one CNN was trained to separate clinical and EARL compliant reconstructions. A second CNN was optimized to identify EARL1 and EARL2 compliant images. The accuracy of both CNNs was assessed using fivefold cross-validation. The CNNs were validated on 24 images acquired on a PET scanner not included in the training data. To assess the impact of image noise on the CNN decision, the 24 images were reconstructed with different scan durations.

Results

In the cross-validation, the first CNN classified all images correctly. When identifying EARL1 and EARL2 compliant images, the second CNN identified 100% EARL1 compliant and 85% EARL2 compliant images correctly. The accuracy in the independent dataset was comparable to the cross-validation accuracy. The scan duration had almost no impact on the results.

Conclusion

The two CNNs trained in this study can be used to retrospectively include images in a multi-center setting by, e.g., adding additional smoothing. This method is especially important for machine learning studies where the harmonization of images from different PET systems is essential.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40658-022-00468-w.

[18F]FDG-PET/CT radiomics and artificial intelligence in lung cancer: Technical aspects and potential clinical applications

Lung cancer is the second most common cancer and the leading cause of cancer-related death worldwide. Molecular imaging using [¹⁸F]fluorodeoxyglucose Positron Emission Tomography and/or Computed Tomography ([¹⁸F]FDG-PET/CT) plays an essential role in the diagnosis, evaluation of response to treatment, and prediction of outcomes. The images are evaluated using qualitative and conventional quantitative indices. However, there is far more information embedded in the images, which can be extracted by sophisticated algorithms. Recently, the concept of uncovering and analyzing the invisible data extracted from medical images, called radiomics, is gaining more attention. Currently, [¹⁸F]FDG-PET/CT radiomics is growingly evaluated in lung cancer to discover if it enhances the diagnostic performance or implication of [¹⁸F]FDG-PET/CT in the management of lung cancer. In this review, we provide a short overview of the technical aspects, as they are discussed in different articles of this special issue. We mainly focus on the diagnostic performance of the [¹⁸F]FDG-PET/CT-based radiomics and the role of artificial intelligence in non-small cell lung cancer, impacting the early detection, staging, prediction of tumor subtypes, biomarkers, and patient's outcomes.

Automatic classification of lymphoma lesions in FDG-PET–Differentiation between tumor and non-tumor uptake

Introduction

The automatic classification of lymphoma lesions in PET is a main topic of ongoing research. An automatic algorithm would enable the swift evaluation of PET parameters, like texture and heterogeneity markers, concerning their prognostic value for patients outcome in large datasets. Moreover, the determination of the metabolic tumor volume would be facilitated. The aim of our study was the development and evaluation of an automatic algorithm for segmentation and classification of lymphoma lesions in PET.

Methods

Pre-treatment PET scans from 60 Hodgkin lymphoma patients from the EuroNet-PHL-C1 trial were evaluated. A watershed algorithm was used for segmentation. For standardization of the scan length, an automatic cropping algorithm was developed. All segmented volumes were manually classified into one of 14 categories. The random forest method and a nested cross-validation was used for automatic classification and evaluation.

Results

Overall, 853 volumes were segmented and classified. 203/246 tumor lesions and 554/607 non-tumor volumes were classified correctly by the automatic algorithm, corresponding to a sensitivity, a specificity, a positive and a negative predictive value of 83%, 91%, 79% and 93%. In 44/60 (73%) patients, all tumor lesions were correctly classified. In ten out of the 16 patients with misclassified tumor lesions, only one false-negative tumor lesion occurred. The automatic classification of focal gastrointestinal uptake, brown fat tissue and composed volumes consisting of more than one tissue was challenging.

Conclusion

Our algorithm, trained on a small number of patients and on PET information only, showed a good performance and is suitable for automatic lymphoma classification.

An update on the diagnosis of gastroenteropancreatic neuroendocrine neoplasms

Gastroenteropancreatic neuroendocrine neoplasms (GEP-NENs) arise from neuroendocrine cells found throughout the gastrointestinal tract and islet cells of the pancreas. The incidence and prevalence of GEP-NENs have been increasing each year due to higher awareness, improved diagnostic modalities, and increased incidental detection on cross-sectional imaging and endoscopy for cancer screening and other conditions and symptoms. GEP-NENs are a heterogeneous group of tumors and have a wide range in clinical presentation, histopathologic features, and molecular biology. Clinical presentation most commonly depends on whether the GEP-NEN secretes an active hormone. The World Health Organization recently updated the classification of GEP-NENs to introduce a distinction between high-grade neuroendocrine tumors and neuroendocrine carcinomas, which can be identified using histology and molecular studies and are more aggressive with a worse prognosis compared to high-grade neuroendocrine tumors. As our understanding of the biology of GEP-NENs has grown, new and improved diagnostic modalities can be developed and optimized. Here, we discuss clinical features and updates in diagnosis, including histopathological analysis, biomarkers, molecular techniques, and radiology of GEP-NENs. We review established diagnostic tests and discuss promising novel diagnostic tests that are currently in development or require further investigation and validation prior to broad utilization in patient care.

Early molecular imaging response assessment based on determination of total viable tumor burden in [68Ga]Ga-PSMA-11 PET/CT independently predicts overall survival in [177Lu]Lu-PSMA-617 radioligand therapy

Purpose

In patients with metastatic castration-resistant prostate cancer (mCRPC) treated with prostate-specific membrane antigen-targeted radioligand therapy (PSMA-RLT), the predictive value of PSMA PET/CT-derived response is still under investigation. Early molecular imaging response based on total viable tumor burden and its association with overall survival (OS) was explored in this study.

Methods

Sixty-six mCRPC patients who received [¹⁷⁷Lu]Lu-PSMA-617 RLT within a prospective patient registry (REALITY Study, NCT04833517) were analyzed. Patients received a [⁶⁸Ga]Ga-PSMA-11 PET/CT scan before the first and after the second cycle of PSMA-RLT. Total lesion PSMA (TLP) was determined by semiautomatic whole-body tumor segmentation. Molecular imaging response was assessed by change in TLP and modified PERCIST criteria. Biochemical response was assessed using standard serum PSA and PCWG3 criteria. Both response assessment methods and additional baseline parameters were analyzed regarding their association with OS by univariate and multivariable analysis.

Results

By molecular imaging, 40/66 (60.6%) patients showed partial remission (PR), 19/66 (28.7%) stable disease (SD), and 7/66 (10.6%) progressive disease (PD). Biochemical response assessment revealed PR in 34/66 (51.5%) patients, SD in 20/66 (30.3%), and PD in 12/66 (18.2%). Response assessments were concordant in 49/66 (74.3%) cases. On univariate analysis, both molecular and biochemical response (p = 0.001 and 0.008, respectively) as well as two baseline characteristics (ALP and ECOG) were each significantly associated with OS. The median OS of patients showing molecular PR was 24.6 versus 10.7 months in the remaining patients (with SD or PD). On multivariable analysis molecular imaging response remained an independent predictor of OS (p = 0.002), eliminating biochemical response as insignificant (p = 0.515).

Conclusion

The new whole-body molecular imaging–derived biomarker, early change of total lesion PSMA (TLP), independently predicts overall survival in [¹⁷⁷Lu]Lu-PSMA-617 RLT in mCRPC, outperforming conventional PSA-based response assessment. TLP might be considered a more distinguished and advanced biomarker for monitoring PSMA-RLT over commonly used serum PSA.

Automated liver lesion detection in 68Ga DOTATATE PET/CT using a deep fully convolutional neural network

BACKGROUND

Gastroenteropancreatic neuroendocrine tumors most commonly metastasize to the liver; however, high normal background ⁶⁸Ga-DOTATATE activity and high image noise make metastatic lesions difficult to detect. The purpose of this study is to develop a rapid, automated and highly specific method to identify ⁶⁸Ga-DOTATATE PET/CT hepatic lesions using a 2D U-Net convolutional neural network.

METHODS

A retrospective study of ⁶⁸Ga-DOTATATE PET/CT patient studies (n = 125; 57 with ⁶⁸Ga-DOTATATE hepatic lesions and 68 without) was evaluated. The dataset was randomly divided into 75 studies for the training set (36 abnormal, 39 normal), 25 for the validation set (11 abnormal, 14 normal) and 25 for the testing set (11 abnormal, 14 normal). Hepatic lesions were physician annotated using a modified PERCIST threshold, and boundary definition by gradient edge detection. The 2D U-Net was trained independently five times for 100,000 iterations using a linear combination of binary cross-entropy and dice losses with a stochastic gradient descent algorithm. Performance metrics included: positive predictive value (PPV), sensitivity, F₁ score and area under the precision-recall curve (PR-AUC). Five different pixel area thresholds were used to filter noisy predictions.

RESULTS

A total of 233 lesions were annotated with each abnormal study containing a mean of 4 ± 2.75 lesions. A pixel filter of 20 produced the highest mean PPV 0.94 ± 0.01. A pixel filter of 5 produced the highest mean sensitivity 0.74 ± 0.02. The highest mean F₁ score 0.79 ± 0.01 was produced with a 20 pixel filter. The highest mean PR-AUC 0.73 ± 0.03 was produced with a 15 pixel filter.

CONCLUSION

Deep neural networks can automatically detect hepatic lesions in ⁶⁸Ga-DOTATATE PET. Ongoing improvements in data annotation methods, increasing sample sizes and training methods are anticipated to further improve detection performance.

Toward High-Throughput Artificial Intelligence-Based Segmentation in Oncological PET Imaging

Artificial intelligence (AI) techniques for image-based segmentation have garnered much attention in recent years. Convolutional neural networks have shown impressive results and potential toward fully automated segmentation in medical imaging, and particularly PET imaging. To cope with the limited access to annotated data needed in supervised AI methods, given tedious and prone-to-error manual delineations, semi-supervised and unsupervised AI techniques have also been explored for segmentation of tumors or normal organs in single- and bimodality scans. This work reviews existing AI techniques for segmentation tasks and the evaluation criteria for translational AI-based segmentation efforts toward routine adoption in clinical workflows.

Whole-body uptake classification and prostate cancer staging in 68Ga-PSMA-11 PET/CT using dual-tracer learning

Purpose

In PSMA-ligand PET/CT imaging, standardized evaluation frameworks and image-derived parameters are increasingly used to support prostate cancer staging. Clinical applicability remains challenging wherever manual measurements of numerous suspected lesions are required. Deep learning methods are promising for automated image analysis, typically requiring extensive expert-annotated image datasets to reach sufficient accuracy. We developed a deep learning method to support image-based staging, investigating the use of training information from two radiotracers.

Methods

In 173 subjects imaged with ⁶⁸Ga-PSMA-11 PET/CT, divided into development (121) and test (52) sets, we trained and evaluated a convolutional neural network to both classify sites of elevated tracer uptake as nonsuspicious or suspicious for cancer and assign them an anatomical location. We evaluated training strategies to leverage information from a larger dataset of ¹⁸F-FDG PET/CT images and expert annotations, including transfer learning and combined training encoding the tracer type as input to the network. We assessed the agreement between the N and M stage assigned based on the network annotations and expert annotations, according to the PROMISE miTNM framework.

Results

In the development set, including ¹⁸F-FDG training data improved classification performance in four-fold cross validation. In the test set, compared to expert assessment, training with ¹⁸F-FDG data and the development set yielded 80.4% average precision [confidence interval (CI): 71.1–87.8] for identification of suspicious uptake sites, 77% (CI: 70.0–83.4) accuracy for anatomical location classification of suspicious findings, 81% agreement for identification of regional lymph node involvement, and 77% agreement for identification of metastatic stage.

Conclusion

The evaluated algorithm showed good agreement with expert assessment for identification and anatomical location classification of suspicious uptake sites in whole-body ⁶⁸Ga-PSMA-11 PET/CT. With restricted PSMA-ligand data available, the use of training examples from a different radiotracer improved performance. The investigated methods are promising for enabling efficient assessment of cancer stage and tumor burden.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00259-021-05473-2.