Longitudinal acquisition repeatability of MRI radiomics features: An ACR MRI phantom study on two MRI scanners using a 3D T1W TSE sequence

Quantifying the reproducibility and longitudinal repeatability of radiomics features in magnetic resonance Image-Guide accelerator Imaging: A phantom study

OBJECTIVE

This study aimed to quantitatively evaluate the inter-platform reproducibility and longitudinal acquisition repeatability of MRI radiomics features in Fluid-Attenuated Inversion Recovery (FLAIR), T2-weighted (T2W), and T1-weighted (T1W) sequences on MR-Linac systems using an American College of Radiology (ACR) phantom.

MATERIALS AND METHODS

This study used two MR-Linac systems (A and B) in different cancer centers. The ACR phantom was scanned on system A daily for 30 consecutive days to evaluate longitudinal repeatability. Additionally, retest data were collected after repositioning the phantom. Inter-platform reproducibility was assessed by conducting scans under identical conditions using system B. Regions of interest were delineated on the T1W sequence from system A and mapped to other sequences via rigid registration. Intra-observer and inter-observer comparisons were conducted. Repeatability and reproducibility were assessed using the intraclass correlation coefficient (ICC) and coefficient of variation (CV). Robust radiomics features were identified based on ICC>0.9 and CV<10 %.

RESULTS

Analysis showed that a higher proportion of radiomics features derived from longitudinal FLAIR sequence (51.65 %) met robustness criteria compared to T2W (48.35 %) and T1W (43.96 %). Additionally, more inter-platform features from the FLAIR sequence (62.64 %) were robust compared to T2W (42.86 %) and T1W (39.56 %). Test-retest and intra-observer repeatability were excellent across all sequences, with a median ICC of 0.99 and CV<5%. However, inter-observer reproducibility was inferior, especially for the T1W sequence.

CONCLUSIONS

Different sequences show variations in repeatability and reproducibility. The FLAIR sequence demonstrated advantages in both longitudinal repeatability and inter-platform reproducibility. Caution is warranted when interpreting data, particularly in longitudinal or multiplatform radiomics studies.

Robustness of radiomics features on 0.35 T magnetic resonance imaging for magnetic resonance-guided radiotherapy

Background and purpose

MR-guided radiotherapy adds the precision of magnetic resonance imaging (MRI) to the therapeutic benefits of a linear accelerator. Prior to each therapeutic session, an MRI generates a significant volume of imaging data ripe for analysis. Radiomics stands at the forefront of medical imaging and oncology research, dedicated to mining quantitative imaging attributes to forge predictive models. However, the robustness of these models is often challenged.

Materials and methods

To assess the robustness of feature extraction, we conducted reproducibility studies using a 0.35 T MR-linac system, employing both a specialized phantom and patient-derived images, focusing on cases of pancreatic cancer. We extracted shape-based, first-order and textural features from patient-derived images and only first-order and textural features from phantom-derived images. The impact of the delay between simulation and first fraction images was also assessed with an equivalence test.

Results

From 107 features evaluated, 58 (54 %) were considered as non-reproducible: 18 were uniformly inconsistent across both phantom and patient images, 9 were specific to phantom-based analysis, and 31 to patient-derived data.

Conclusion

Our findings show that a significant proportion of radiomic features extracted from this dual dataset were unreliable. It is essential to discard these non-reproducible elements to refine and enhance radiomic model development, particularly for MR-guided radiotherapy in pancreatic cancer.

The impact of the combat method on radiomics feature compensation and analysis of scanners from different manufacturers

BACKGROUND

This study investigated whether the Combat compensation method can remove the variability of radiomic features extracted from different scanners, while also examining its impact on the subsequent predictive performance of machine learning models.

MATERIALS AND METHODS

135 CT images of Credence Cartridge Radiomic phantoms were collected and screened from three scanners manufactured by Siemens, Philips, and GE. 100 radiomic features were extracted and 20 radiomic features were screened according to the Lasso regression method. The radiomic features extracted from the rubber and resin-filled regions in the cartridges were labeled into different categories for evaluating the performance of the machine learning model. Radiomics features were divided into three groups based on the different scanner manufacturers. The radiomic features were randomly divided into training and test sets with a ratio of 8:2. Five machine learning models (lasso, logistic regression, random forest, support vector machine, neural network) were employed to evaluate the impact of Combat on radiomic features. The variability among radiomic features were assessed using analysis of variance (ANOVA) and principal component analysis (PCA). Accuracy, precision, recall, and area under the receiver curve (AUC) were used as evaluation metrics for model classification.

RESULTS

The principal component and ANOVA analysis results show that the variability of different scanner manufacturers in radiomic features was removed (P˃0.05). After harmonization with the Combat algorithm, the distributions of radiomic features were aligned in terms of location and scale. The performance of machine learning models for classification improved, with the Random Forest model showing the most significant enhancement. The AUC value increased from 0.88 to 0.92.

CONCLUSIONS

The Combat algorithm has reduced variability in radiomic features from different scanners. In the phantom CT dataset, it appears that the machine learning model's classification performance may have improved after Combat harmonization. However, further investigation and validation are required to fully comprehend Combat's impact on radiomic features in medical imaging.

Phantom study investigating the repeatability of radiomic features with alteration of image acquisition parameters in magnetic resonance imaging

BACKGROUND

Magnetic resonance imaging (MRI) has many different alterable parameters that affect how an image appears. This is relevant in radiomics which produces quantitative features through analysis of medical images. One significant acknowledged limitation of radiomics is repeatability. This phantom study aims to further investigate the repeatability of radiomic features (RaF), within MRI, across a range of different echo (TE) and repetition times (TR).

METHODS

A phantom was scanned 10 times under identical conditions on a 3T scanner using head coil over 4 months. The TE ranged from 80 to 110 ms while the TR from 3000 to 5000 ms. Radiomics analysis was performed on the same segmented section of the phantom across all TE and TR combinations. Intraclass Correlation Coefficient (ICC) was calculated across the different TE and TR ranges to investigate the repeatability of RaF.

RESULTS

Of 1596 features calculated, 187 features had ICC >0.9 across the range of TE, while 82 features had an ICC >0.9 across a range of TR. 664 had ICC >0.75 across the range of TEs, with 541 across the range of TR values. There was an overlap of 51 features with ICC >0.9.

CONCLUSION

Repeatability of RaF in MRI is dependent on imaging parameters and careful consideration of these, in combination with variable selection, is required when applying radiomics to MRI.

The Impact of Exercise Training and Supplemental Oxygen on Peripheral Muscles in Chronic Obstructive Pulmonary Disease: A Randomized Controlled Trial

OBJECTIVE

Exercise training is a cornerstone of the treatment of chronic obstructive pulmonary disease, whereas the related interindividual heterogeneity in skeletal muscle dysfunction and adaptations are not yet fully understood. We set out to investigate the effects of exercise training and supplemental oxygen on functional and structural peripheral muscle adaptation.

METHODS

In this prospective, randomized, controlled, double-blind study, 28 patients with nonhypoxemic chronic obstructive pulmonary disease (forced expiratory volume in 1 second, 45.92% ± 9.06%) performed 6 wk of combined endurance and strength training, three times a week while breathing either supplemental oxygen or medical air. The impact on exercise capacity, muscle strength, and quadriceps femoris muscle cross-sectional area (CSA) was assessed by maximal cardiopulmonary exercise testing, 10-repetition maximum strength test of knee extension, and magnetic resonance imaging, respectively.

RESULTS

After exercise training, patients demonstrated a significant increase in functional capacity, aerobic capacity, exercise tolerance, quadriceps muscle strength, and bilateral CSA. Supplemental oxygen affected significantly the training impact on peak work rate when compared with medical air (+0.20 ± 0.03 vs +0.12 ± 0.03 W·kg -1 , P = 0.047); a significant increase in CSA (+3.9 ± 1.3 cm 2 , P = 0.013) was only observed in the training group using oxygen. Supplemental oxygen and exercise-induced peripheral desaturation were identified as significant opposing determinants of muscle gain during this exercise training intervention, which led to different adaptations of CSA between the respective subgroups.

CONCLUSIONS

The heterogenous functional and structural muscle adaptations seem determined by supplemental oxygen and exercise-induced hypoxia. Indeed, supplemental oxygen may facilitate muscular training adaptations, particularly in limb muscle dysfunction, thereby contributing to the enhanced training responses on maximal aerobic and functional capacity.

Determining the reliable feature change in longitudinal radiomics studies: A methodological approach using the reliable change index

PURPOSE

Determination of reliable change of radiomics feature over time is essential and vital in delta-radiomics, but has not yet been rigorously examined. This study attempts to propose a methodological approach using reliable change index (RCI), a statistical metric to determine the reliability of quantitative biomarker changes by accounting for the baseline measurement standard error, in delta-radiomics. The use of RCI was demonstrated with the MRI data acquired from a group of prostate cancer (PCa) patients treated by 1.5 T MRI-guided radiotherapy (MRgRT).

METHODS

Fifty consecutive PCa patients who underwent five-fractionated MRgRT were retrospectively included, and 1023 radiomics features were extracted from the clinical target volume (CTV) and planning target volume (PTV). The two MRI datasets acquired at the first fraction (MRI11 and MRI21) were used to calculate the baseline feature reliability against image acquisition using intraclass correlation coefficient (ICC). The RCI was constructed based on the baseline feature measurement standard deviation, ICC, and feature value differences at two time points between the fifth (MRI51) and the first fraction MRI (MRI11). The reliable change of features was determined in each patient only if the calculated RCI was over 1.96 or smaller than -1.96. The feature changes between MRI51 and MRI11 were correlated to two patient-reported quality-of-life clinical endpoints of urinary domain summary score (UDSS) and bowel domain summary score (BDSS) in 35 patients using the Spearman correlation test. Only the significant correlations between a feature that was reliably changed in ≥7 patients (20%) by RCI and an endpoint were considered as true significant correlations.

RESULTS

The 352 (34.4%) and 386 (37.7%) features among all 1023 features were determined by RCI to be reliably changed in more than five (10%) patients in the CTV and PTV, respectively. Nineteen features were found reliably changed in the CTV and 31 features in the PTV, respectively, in 10 (20%) or more patients. These features were not necessarily associated with significantly different longitudinal feature values (group p-value < 0.05). Most reliably changed features in more than 10 patients had excellent or good baseline test-retest reliability ICC, while none showed poor reliability. The RCI method ruled out the features to be reliably changed when substantial feature measurement bias was presented. After applying the RCI criterion, only four and five true significant correlations were confirmed with UDSS and BDSS in the CTV, respectively, with low true significance correlation rates of 10.8% (4/37) and 17.9% (5/28). No true significant correlations were found in the PTV.

CONCLUSIONS

The RCI method was proposed for delta-radiomics and demonstrated using PCa MRgRT data. The RCI has advantages over some other statistical metrics commonly used in the previous delta-radiomics studies, and is useful to reliably identify the longitudinal radiomics feature change on an individual basis. This proposed RCI method should be helpful for the development of essential feature selection methodology in delta-radiomics.

Phantom Study on the Robustness of MR Radiomics Features: Comparing the Applicability of 3D Printed and Biological Phantoms

The objectives of our study were to (a) evaluate the feasibility of using 3D printed phantoms in magnetic resonance imaging (MR) in assessing the robustness and repeatability of radiomic parameters and (b) to compare the results obtained from the 3D printed phantoms to metrics obtained in biological phantoms. To this end, three different 3D phantoms were printed: a Hilbert cube (5 × 5 × 5 cm³) and two cubic quick response (QR) code phantoms (a large phantom (large QR) (5 × 5 × 4 cm³) and a small phantom (small QR) (4 × 4 × 3 cm³)). All 3D printed and biological phantoms (kiwis, tomatoes, and onions) were scanned thrice on clinical 1.5 T and 3 T MR with 1 mm and 2 mm isotropic resolution. Subsequent analyses included analyses of several radiomics indices (RI), their repeatability and reliability were calculated using the coefficient of variation (CV), the relative percentage difference (RPD), and the interclass coefficient (ICC) parameters. Additionally, the readability of QR codes obtained from the MR images was examined with several mobile phones and algorithms. The best repeatability (CV ≤ 10%) is reported for the acquisition protocols with the highest spatial resolution. In general, the repeatability and reliability of RI were better in data obtained at 1.5 T (CV = 1.9) than at 3 T (CV = 2.11). Furthermore, we report good agreements between results obtained for the 3D phantoms and biological phantoms. Finally, analyses of the read-out rate of the QR code revealed better texture analyses for images with a spatial resolution of 1 mm than 2 mm. In conclusion, 3D printing techniques offer a unique solution to create textures for analyzing the reliability of radiomic data from MR scans.

Acquisition repeatability of MRI radiomics features in the head and neck: a dual-3D-sequence multi-scan study

Radiomics has increasingly been investigated as a potential biomarker in quantitative imaging to facilitate personalized diagnosis and treatment of head and neck cancer (HNC), a group of malignancies associated with high heterogeneity. However, the feature reliability of radiomics is a major obstacle to its broad validity and generality in application to the highly heterogeneous head and neck (HN) tissues. In particular, feature repeatability of radiomics in magnetic resonance imaging (MRI) acquisition, which is considered a crucial confounding factor of radiomics feature reliability, is still sparsely investigated. This study prospectively investigated the acquisition repeatability of 93 MRI radiomics features in ten HN tissues of 15 healthy volunteers, aiming for potential magnetic resonance-guided radiotherapy (MRgRT) treatment of HNC. Each subject underwent four MRI acquisitions with MRgRT treatment position and immobilization using two pulse sequences of 3D T1-weighed turbo spin-echo and 3D T2-weighed turbo spin-echo on a 1.5 T MRI simulator. The repeatability of radiomics feature acquisition was evaluated in terms of the intraclass correlation coefficient (ICC), whereas within-subject acquisition variability was evaluated in terms of the coefficient of variation (CV). The results showed that MRI radiomics features exhibited heterogeneous acquisition variability and uncertainty dependent on feature types, tissues, and pulse sequences. Only a small fraction of features showed excellent acquisition repeatability (ICC > 0.9) and low within-subject variability. Multiple MRI scans improved the accuracy and confidence of the identification of reliable features concerning MRI acquisition compared to simple test-retest repeated scans. This study contributes to the literature on the reliability of radiomics features with respect to MRI acquisition and the selection of reliable radiomics features for use in modeling in future HNC MRgRT applications.

Robustness Assessment of Images From a 0.35T Scanner of an Integrated MRI-Linac: Characterization of Radiomics Features in Phantom and Patient Data

Purpose: Radiomics entails the extraction of quantitative imaging biomarkers (or radiomics features) hypothesized to provide additional pathophysiological and/or clinical information compared to qualitative visual observation and interpretation. This retrospective study explores the variability of radiomics features extracted from images acquired with the 0.35 T scanner of an integrated MRI-Linac. We hypothesized we would be able to identify features with high repeatability and reproducibility over various imaging conditions using phantom and patient imaging studies. We also compared findings from the literature relevant to our results. Methods: Eleven scans of a Magphan^® RT phantom over 13 months and 11 scans of a ViewRay Daily QA phantom over 11 days constituted the phantom data. Patient datasets included 50 images from ten anonymized stereotactic body radiation therapy (SBRT) pancreatic cancer patients (50 Gy in 5 fractions). A True Fast Imaging with Steady-State Free Precession (TRUFI) pulse sequence was selected, using a voxel resolution of 1.5 mm × 1.5 mm × 1.5 mm and 1.5 mm × 1.5 mm × 3.0 mm for phantom and patient data, respectively. A total of 1087 shape-based, first, second, and higher order features were extracted followed by robustness analysis. Robustness was assessed with the Coefficient of Variation (CoV < 5%). Results: We identified 130 robust features across the datasets. Robust features were found within each category, except for 2 second-order sub-groups, namely, Gray Level Size Zone Matrix (GLSZM) and Neighborhood Gray Tone Difference Matrix (NGTDM). Additionally, several robust features agreed with findings from other stability assessments or predictive performance studies in the literature. Conclusion: We verified the stability of the 0.35 T scanner of an integrated MRI-Linac for longitudinal radiomics phantom studies and identified robust features over various imaging conditions. We conclude that phantom measurements can be used to identify robust radiomics features. More stability assessment research is warranted.

Delta radiomics: a systematic review

BACKGROUND

Radiomics can provide quantitative features from medical imaging that can be correlated with various biological features and clinical endpoints. Delta radiomics, on the other hand, consists in the analysis of feature variation at different acquisition time points, usually before and after therapy. The aim of this study was to provide a systematic review of the different delta radiomics approaches.

METHODS

Eligible articles were searched in Embase, PubMed, and ScienceDirect using a search string that included free text and/or Medical Subject Headings (MeSH) with three key search terms: "radiomics", "texture", and "delta". Studies were analysed using QUADAS-2 and the RQS tool.

RESULTS

Forty-eight studies were finally included. The studies were divided into preclinical/methodological (five studies, 10.4%); rectal cancer (six studies, 12.5%); lung cancer (twelve studies, 25%); sarcoma (five studies, 10.4%); prostate cancer (three studies, 6.3%), head and neck cancer (six studies, 12.5%); gastrointestinal malignancies excluding rectum (seven studies, 14.6%), and other disease sites (four studies, 8.3%). The median RQS of all studies was 25% (mean 21% ± 12%), with 13 studies (30.2%) achieving a quality score < 10% and 22 studies (51.2%) < 25%.

CONCLUSIONS

Delta radiomics shows potential benefit for several clinical endpoints in oncology (differential diagnosis, prognosis and prediction of treatment response, and evaluation of side effects). Nevertheless, the studies included in this systematic review suffer from the bias of overall low quality, so that the conclusions are currently heterogeneous, not robust, and not replicable. Further research with prospective and multicentre studies is needed for the clinical validation of delta radiomics approaches.

Robustness of radiomic features of benign breast lesions and hormone receptor positive/HER2-negative cancers across DCE-MR magnet strengths

Radiomic features extracted from breast lesion images have shown potential in diagnosis and prognosis of breast cancer. As medical centers transition from 1.5 T to 3.0 T magnetic resonance (MR) imaging, it is beneficial to identify potentially robust radiomic features across field strengths because images acquired at different field strengths could be used in machine learning models. Dynamic contrast-enhanced MR images of benign breast lesions and hormone receptor positive/HER2-negative (HR+/HER2-) breast cancers were acquired retrospectively, yielding 612 unique cases: 150 and 99 benign lesions imaged at 1.5 T and 3.0 T, and 223 and 140 HR+/HER2- cancerous lesions imaged at 1.5 T and 3.0 T, respectively. In addition, an independent set of seven lesions imaged at both field strengths, three benign lesions and four HR+/HER2- cancers, was analyzed separately. Lesions were automatically segmented using a 4D fuzzy c-means method; thirty-eight radiomic features were extracted. Feature value distributions were compared by cancer status and imaging field strength using the Kolmogorov-Smirnov test. Features that did not demonstrate a statistically significant difference were considered to be potentially robust. The area under the receiver operating characteristic curve (AUC), for the task of classifying lesions as benign or HR+/HER2- cancer, was determined for each feature at each field strength. Three features were found to be both potentially robust across field strength and of high classification performance, i.e., AUCs statistically greater than 0.5 in the classification task: one shape feature (irregularity), one texture feature (sum average) and one enhancement variance kinetics features (enhancement variance increasing rate). In the demonstration set of lesions imaged at both field strengths, two of the three potentially robust features showed qualitative agreement across field strength. These findings may contribute to the development of computer-aided diagnosis models that are robust across field strength for this classification task.

Reliability of MRI radiomics features in MR-guided radiotherapy for prostate cancer: Repeatability, reproducibility, and within-subject agreement

PURPOSE

The MR-guided radiotherapy (MRgRT) images on the integrated MRI and linear accelerator (MR-LINAC) might facilitate radiomics analysis for longitudinal treatment response assessment. However, the reliability of MRgRT radiomics features is largely unknown. This study aims to investigate MRgRT radiomics feature reliability acquired using a standardized 3D-T2W-TSE sequence in terms of repeatability, reproducibility, and within-subject feature agreement on a 1.5T MR-simulator and a 1.5T MR-LINAC for prostate cancer (PC).

METHODS

Twenty-six consecutive PC patients who underwent one MRI-simulator scan and two MR-LINAC scans before dose delivery were retrospectively included. The three MRI datasets were rigidly co-registered. 1023 first-order and texture radiomics features were extracted with different intensity bin widths for each scan in the manually segmented clinical target volume (CTV) and planning target volume (PTV) by an experienced radiation oncologist. Intraclass correlation coefficient (ICC) was used to evaluate feature repeatability between MR-LINAC scans and reproducibility between MRI-simulator and MR-LINAC scans. The within-subject feature value agreements were evaluated using Bland-Altman analysis. The impact of inter-observer segmentation on the radiomics feature reliability was also examined based on the second manual segmentation of CTV and PTV by an MRI researcher.

RESULTS

Based on the segmentation by the radiation oncologist and the default bin width of 25, 9.6%, 24.1%, 49.6%, and 16.8% of the total 1023 features exhibited excellent (ICC > 0.9), good (0.9 > ICC > 0.75), moderate (0.75 > ICC > 0.5), and poor (ICC < 0.5) repeatability in the CTV, and 9.2%, 26.8%, 50.5%, and 13.5% in the PTV, respectively. For reproducibility, the corresponding feature percentages were 8.9%, 19.7%, 41.9%, and 29.6% in the CTV, and 8.4%, 17.8%, 47.9%, and 26% in the PTV. Feature reliability was not notably influenced by intensity bin width for discretization. BA analysis revealed wide 95% limit-of-agreements and substantial biases of feature values between CTV and PTV and between any two MRI scans. The features even with excellent ICC were still subjected to considerable inter-scan feature variations in each individual subject. The analysis on the second segmentation by the MRI researcher showed insignificantly different feature repeatability and reproducibility in terms of ICC values.

CONCLUSIONS

Only a small proportion of features exhibited excellent/good repeatability and reproducibility, highlighting the importance of reliable MRgRT feature selection. The within-subject feature values were subjected to considerable inter-scan variations, imposing a challenge on the determination of the smallest detectable change in future MRgRT delta-radiomics studies.

Contrast-Enhanced CT-Based Radiomics Analysis in Predicting Lymphovascular Invasion in Esophageal Squamous Cell Carcinoma

Objectives

To develop a radiomics model based on contrast-enhanced CT (CECT) to predict the lymphovascular invasion (LVI) in esophageal squamous cell carcinoma (ESCC) and provide decision-making support for clinicians.

Patients and Methods

This retrospective study enrolled 334 patients with surgically resected and pathologically confirmed ESCC, including 96 patients with LVI and 238 patients without LVI. All enrolled patients were randomly divided into a training cohort and a testing cohort at a ratio of 7:3, with the training cohort containing 234 patients (68 patients with LVI and 166 without LVI) and the testing cohort containing 100 patients (28 patients with LVI and 72 without LVI). All patients underwent preoperative CECT scans within 2 weeks before operation. Quantitative radiomics features were extracted from CECT images, and the least absolute shrinkage and selection operator (LASSO) method was applied to select radiomics features. Logistic regression (Logistic), support vector machine (SVM), and decision tree (Tree) methods were separately used to establish radiomics models to predict the LVI status in ESCC, and the best model was selected to calculate Radscore, which combined with two clinical CT predictors to build a combined model. The clinical model was also developed by using logistic regression. The receiver characteristic curve (ROC) and decision curve (DCA) analysis were used to evaluate the model performance in predicting the LVI status in ESCC.

Results

In the radiomics model, Sphericity and gray-level non-uniformity (GLNU) were the most significant radiomics features for predicting LVI. In the clinical model, the maximum tumor thickness based on CECT (cThick) in patients with LVI was significantly greater than that in patients without LVI (P<0.001). Patients with LVI had higher clinical N stage based on CECT (cN stage) than patients without LVI (P<0.001). The ROC analysis showed that both the radiomics model (AUC values were 0.847 and 0.826 in the training and testing cohort, respectively) and the combined model (0.876 and 0.867, respectively) performed better than the clinical model (0.775 and 0.798, respectively), with the combined model exhibiting the best performance.

Conclusions

The combined model incorporating radiomics features and clinical CT predictors may potentially predict the LVI status in ESCC and provide support for clinical treatment decisions.