Standardised lesion segmentation for imaging biomarker quantitation: a consensus recommendation from ESR and EORTC

Radiomics for clinical decision support in radiation oncology

Radiomics is a promising tool for the development of quantitative biomarkers to support clinical decision-making. It has been shown to improve the prediction of response to treatment and outcome in different settings, particularly in the field of radiation oncology by optimising the dose delivery solutions and reducing the rate of radiation-induced side effects, leading to a fully personalised approach. Despite the promising results offered by radiomics at each of these stages, standardised methodologies, reproducibility and interpretability of results are still lacking, limiting the potential clinical impact of these tools. In this review, we briefly describe the principles of radiomics and the most relevant applications of radiomics at each stage of cancer management in the framework of radiation oncology. Furthermore, the integration of radiomics into clinical decision support systems is analysed, defining the challenges and offering possible solutions for translating radiomics into a clinically applicable tool.

Imaging biomarkers and radiomics in pediatric oncology: a view from the PRIMAGE (PRedictive In silico Multiscale Analytics to support cancer personalized diaGnosis and prognosis, Empowered by imaging biomarkers) project

This review paper presents the practical development of imaging biomarkers in the scope of the PRIMAGE (PRedictive In silico Multiscale Analytics to support cancer personalized diaGnosis and prognosis, Empowered by imaging biomarkers) project, as a noninvasive and reliable way to improve the diagnosis and prognosis in pediatric oncology. The PRIMAGE project is a European multi-center research initiative that focuses on developing medical imaging-derived artificial intelligence (AI) solutions designed to enhance overall management and decision-making for two types of pediatric cancer: neuroblastoma and diffuse intrinsic pontine glioma. To allow this, the PRIMAGE project has created an open-cloud platform that combines imaging, clinical, and molecular data together with AI models developed from this data, creating a comprehensive decision support environment for clinicians managing patients with these two cancers. In order to achieve this, a standardized data processing and analysis workflow was implemented to generate robust and reliable predictions for different clinical endpoints. Magnetic resonance (MR) image harmonization and registration was performed as part of the workflow. Subsequently, an automated tool for the detection and segmentation of tumors was trained and internally validated. The Dice similarity coefficient obtained for the independent validation dataset was 0.997, indicating compatibility with the manual segmentation variability. Following this, radiomics and deep features were extracted and correlated with clinical endpoints. Finally, reproducible and relevant imaging quantitative features were integrated with clinical and molecular data to enrich both the predictive models and a set of visual analytics tools, making the PRIMAGE platform a complete clinical decision aid system. In order to ensure the advancement of research in this field and to foster engagement with the wider research community, the PRIMAGE data repository and platform are currently being integrated into the European Federation for Cancer Images (EUCAIM), which is the largest European cancer imaging research infrastructure created to date.

Identification of impactful imaging biomarker: Clinical applications for breast and prostate carcinoma

BACKGROUND

 Imaging biomarkers are quantitative parameters from imaging modalities, which are collected noninvasively, allow conclusions about physiological and pathophysiological processes, and may consist of single (monoparametric) or multiple parameters (bi- or multiparametric).

METHOD

 This review aims to present the state of the art for the quantification of multimodal and multiparametric imaging biomarkers. Here, the use of biomarkers using artificial intelligence will be addressed and the clinical application of imaging biomarkers in breast and prostate cancers will be explained. For the preparation of the review article, an extensive literature search was performed based on Pubmed, Web of Science and Google Scholar. The results were evaluated and discussed for consistency and generality.

RESULTS AND CONCLUSION

 Different imaging biomarkers (multiparametric) are quantified based on the use of complementary imaging modalities (multimodal) from radiology, nuclear medicine, or hybrid imaging. From these techniques, parameters are determined at the morphological (e. g., size), functional (e. g., vascularization or diffusion), metabolic (e. g., glucose metabolism), or molecular (e. g., expression of prostate specific membrane antigen, PSMA) level. The integration and weighting of imaging biomarkers are increasingly being performed with artificial intelligence, using machine learning algorithms. In this way, the clinical application of imaging biomarkers is increasing, as illustrated by the diagnosis of breast and prostate cancers.

KEY POINTS

· Imaging biomarkers are quantitative parameters to detect physiological and pathophysiological processes.. · Imaging biomarkers from multimodality and multiparametric imaging are integrated using artificial intelligence algorithms.. · Quantitative imaging parameters are a fundamental component of diagnostics for all tumor entities, such as for mammary and prostate carcinomas..

CITATION FORMAT

· Bäuerle T, Dietzel M, Pinker K et al. Identification of impactful imaging biomarker: Clinical applications for breast and prostate carcinoma. Fortschr Röntgenstr 2024; 196: 354 - 362.

Reference standard for the evaluation of automatic segmentation algorithms: Quantification of inter observer variability of manual delineation of prostate contour on MRI

PURPOSE

The purpose of this study was to investigate the relationship between inter-reader variability in manual prostate contour segmentation on magnetic resonance imaging (MRI) examinations and determine the optimal number of readers required to establish a reliable reference standard.

MATERIALS AND METHODS

Seven radiologists with various experiences independently performed manual segmentation of the prostate contour (whole-gland [WG] and transition zone [TZ]) on 40 prostate MRI examinations obtained in 40 patients. Inter-reader variability in prostate contour delineations was estimated using standard metrics (Dice similarity coefficient [DSC], Hausdorff distance and volume-based metrics). The impact of the number of readers (from two to seven) on segmentation variability was assessed using pairwise metrics (consistency) and metrics with respect to a reference segmentation (conformity), obtained either with majority voting or simultaneous truth and performance level estimation (STAPLE) algorithm.

RESULTS

The average segmentation DSC for two readers in pairwise comparison was 0.919 for WG and 0.876 for TZ. Variability decreased with the number of readers: the interquartile ranges of the DSC were 0.076 (WG) / 0.021 (TZ) for configurations with two readers, 0.005 (WG) / 0.012 (TZ) for configurations with three readers, and 0.002 (WG) / 0.0037 (TZ) for configurations with six readers. The interquartile range decreased slightly faster between two and three readers than between three and six readers. When using consensus methods, variability often reached its minimum with three readers (with STAPLE, DSC = 0.96 [range: 0.945-0.971] for WG and DSC = 0.94 [range: 0.912-0.957] for TZ, and interquartile range was minimal for configurations with three readers.

CONCLUSION

The number of readers affects the inter-reader variability, in terms of inter-reader consistency and conformity to a reference. Variability is minimal for three readers, or three readers represent a tipping point in the variability evolution, with both pairwise-based metrics or metrics with respect to a reference. Accordingly, three readers may represent an optimal number to determine references for artificial intelligence applications.

METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII

PURPOSE

To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies.

METHODS

We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated.

RESULT

In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community.

CONCLUSION

In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers.

CRITICAL RELEVANCE STATEMENT

A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning.

KEY POINTS

• A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score ( https://metricsscore.github.io/metrics/METRICS.html ) and a repository created to collect feedback from the radiomics community ( https://github.com/metricsscore/metrics ).

Validity of a multiphase CT-based radiomics model in predicting the Leibovich risk groups for localized clear cell renal cell carcinoma: an exploratory study

OBJECTIVE

To develop and validate a multiphase CT-based radiomics model for preoperative risk stratification of patients with localized clear cell renal cell carcinoma (ccRCC).

METHODS

A total of 425 patients with localized ccRCC were enrolled and divided into training, validation, and external testing cohorts. Radiomics features were extracted from three-phase CT images (unenhanced, arterial, and venous), and radiomics signatures were constructed by the least absolute shrinkage and selection operator (LASSO) regression algorithm. The radiomics score (Rad-score) for each patient was calculated. The radiomics model was established and visualized as a nomogram by incorporating significant clinical factors and Rad-score. The predictive performance of the radiomics model was evaluated by the receiver operating characteristic curve, calibration curve, and decision curve analysis (DCA).

RESULTS

The AUC of the triphasic radiomics signature reached 0.862 (95% CI: 0.809-0.914), 0.853 (95% CI: 0.785-0.921), and 0.837 (95% CI: 0.714-0.959) in three cohorts, respectively, which were higher than arterial, venous, and unenhanced radiomics signatures. Multivariate logistic regression analysis showed that Rad-score (OR: 4.066, 95% CI: 3.495-8.790) and renal vein invasion (OR: 12.914, 95% CI: 1.118-149.112) were independent predictors and used to develop the radiomics model. The radiomics model showed good calibration and discrimination and yielded an AUC of 0.872 (95% CI: 0.821-0.923), 0.865 (95% CI: 0.800-0.930), and 0.848 (95% CI: 0.728-0.967) in three cohorts, respectively. DCA showed the clinical usefulness of the radiomics model in predicting the Leibovich risk groups.

CONCLUSIONS

The radiomics model can be used as a non-invasive and useful tool to predict the Leibovich risk groups for localized ccRCC patients.

CRITICAL RELEVANCE STATEMENT

The triphasic CT-based radiomics model achieved favorable performance in preoperatively predicting the Leibovich risk groups in patients with localized ccRCC. Therefore, it can be used as a non-invasive and effective tool for preoperative risk stratification of patients with localized ccRCC.

KEY POINTS

• The triphasic CT-based radiomics signature achieves better performance than the single-phase radiomics signature. • Radiomics holds prospects in preoperatively predicting the Leibovich risk groups for ccRCC. • This study provides a non-invasive method to stratify patients with localized ccRCC.

REliability of consensus-based segMentatIoN in raDiomic feature reproducibility (REMIND): A word of caution

OBJECTIVE

To evaluate the reliability of consensus-based segmentation in terms of reproducibility of radiomic features.

METHODS

In this retrospective study, three tumor data sets were investigated: breast cancer (n = 30), renal cell carcinoma (n = 30), and pituitary macroadenoma (n = 30). MRI was utilized for breast and pituitary data sets, while CT was used for renal data set. 12 readers participated in the segmentation process. Consensus segmentation was created by making corrections on a previous region or volume of interest. Four experiments were designed to evaluate the reproducibility of radiomic features. Reliability was assessed with intraclass correlation coefficient (ICC) with two cut-off values: 0.75 and 0.9.

RESULTS

Considering the lower bound of the 95% confidence interval and the ICC threshold of 0.90, at least 61% of the radiomic features were not reproducible in the inter-consensus analysis. In the susceptibility experiment, at least half (54%) became non-reproducible when the first reader is replaced with a different reader. In the intra-consensus analysis, at least about one-third (32%) were non-reproducible when the same second reader segmented the image over the same first reader two weeks later. Compared to inter-reader analysis based on independent single readers, the inter-consensus analysis did not statistically significantly improve the rates of reproducible features in all data sets and analyses.

CONCLUSIONS

Despite the positive connotation of the word "consensus", it is essential to REMIND that consensus-based segmentation has significant reproducibility issues. Therefore, the usage of consensus-based segmentation alone should be avoided unless a reliability analysis is performed, even if it is not practical in clinical settings.

Advanced MR Techniques for Preoperative Glioma Characterization: Part 1

Preoperative clinical magnetic resonance imaging (MRI) protocols for gliomas, brain tumors with dismal outcomes due to their infiltrative properties, still rely on conventional structural MRI, which does not deliver information on tumor genotype and is limited in the delineation of diffuse gliomas. The GliMR COST action wants to raise awareness about the state of the art of advanced MRI techniques in gliomas and their possible clinical translation or lack thereof. This review describes current methods, limits, and applications of advanced MRI for the preoperative assessment of glioma, summarizing the level of clinical validation of different techniques. In this first part, we discuss dynamic susceptibility contrast and dynamic contrast-enhanced MRI, arterial spin labeling, diffusion-weighted MRI, vessel imaging, and magnetic resonance fingerprinting. The second part of this review addresses magnetic resonance spectroscopy, chemical exchange saturation transfer, susceptibility-weighted imaging, MRI-PET, MR elastography, and MR-based radiomics applications. Evidence Level: 3 Technical Efficacy: Stage 2.

Independent Validation of a Deep Learning nnU-Net Tool for Neuroblastoma Detection and Segmentation in MR Images

OBJECTIVES

To externally validate and assess the accuracy of a previously trained fully automatic nnU-Net CNN algorithm to identify and segment primary neuroblastoma tumors in MR images in a large children cohort.

METHODS

An international multicenter, multivendor imaging repository of patients with neuroblastic tumors was used to validate the performance of a trained Machine Learning (ML) tool to identify and delineate primary neuroblastoma tumors. The dataset was heterogeneous and completely independent from the one used to train and tune the model, consisting of 300 children with neuroblastic tumors having 535 MR T2-weighted sequences (486 sequences at diagnosis and 49 after finalization of the first phase of chemotherapy). The automatic segmentation algorithm was based on a nnU-Net architecture developed within the PRIMAGE project. For comparison, the segmentation masks were manually edited by an expert radiologist, and the time for the manual editing was recorded. Different overlaps and spatial metrics were calculated to compare both masks.

RESULTS

The median Dice Similarity Coefficient (DSC) was high 0.997; 0.944-1.000 (median; Q1-Q3). In 18 MR sequences (6%), the net was not able neither to identify nor segment the tumor. No differences were found regarding the MR magnetic field, type of T2 sequence, or tumor location. No significant differences in the performance of the net were found in patients with an MR performed after chemotherapy. The time for visual inspection of the generated masks was 7.9 ± 7.5 (mean ± Standard Deviation (SD)) seconds. Those cases where manual editing was needed (136 masks) required 124 ± 120 s.

CONCLUSIONS

The automatic CNN was able to locate and segment the primary tumor on the T2-weighted images in 94% of cases. There was an extremely high agreement between the automatic tool and the manually edited masks. This is the first study to validate an automatic segmentation model for neuroblastic tumor identification and segmentation with body MR images. The semi-automatic approach with minor manual editing of the deep learning segmentation increases the radiologist's confidence in the solution with a minor workload for the radiologist.

Index lesion contouring on prostate MRI for targeted MRI/US fusion biopsy - Evaluation of mismatch between radiologists and urologists

PURPOSE

Mistargeting of focal lesions due to inaccurate segmentations can lead to false-negative findings on MRI-guided targeted biopsies. The purpose of this retrospective study was to examine inter-reader agreement of prostate index lesion segmentations from actual biopsy data between urologists and radiologists.

METHOD

Consecutive patients undergoing transperineal MRI-targeted prostate biopsy for PI-RADS 3-5 lesions between January 2020 and December 2021 were included. Agreement between segmentations on T2w-images between urologists and radiologists was assessed with Dice similarity coefficient (DSC) and 95 % Hausdorff distance (95 % HD). Differences in similarity scores were compared using Wilcoxon test. Differences depending on lesion features (size, zonal location, PI-RADS scores, lesion distinctness) were tested with Mann-Whitney U test. Correlation with prostate signal-intensity homogeneity score (PSHS) and lesion size was tested with Spearman's rank correlation.

RESULTS

Ninety-three patients (mean age 64.9 ± 7.1y, median serum PSA 6.5 [4.33-10.00]) were included. Mean similarity scores were statistically significantly lower between urologists and radiologists compared to radiologists only (DSC 0.41 ± 0.24 vs. 0.59 ± 0.23, p < 0.01; 95 %HD 6.38 ± 5.45 mm vs. 4.47 ± 4.12 mm, p < 0.01). There was a moderate and strong positive correlation between DSC scores and lesion size for segmentations from urologists and radiologists (ρ = 0.331, p = 0.002) and radiologists only (ρ = 0.501, p < 0.001). Similarity scores were worse in lesions ≤ 10 mm while other lesion features did not significantly influence similarity scores.

CONCLUSION

There is significant mismatch of prostate index lesion segmentations between urologists and radiologists. Segmentation agreement positively correlates with lesion size. PI-RADS scores, zonal location, lesion distinctness, and PSHS show no significant impact on segmentation agreement. These findings could underpin benefits of perilesional biopsies.