Test-Retest Data for the Assessment of Breast MRI Radiomic Feature Repeatability

Impact of Preprocessing Parameters in Medical Imaging-Based Radiomic Studies: A Systematic Review

BACKGROUND

Lately, radiomic studies featuring the development of a signature to use in prediction models in diagnosis or prognosis outcomes have been increasingly published. While the results are shown to be promising, these studies still have many pitfalls and limitations. One of the main issues of these studies is that radiomic features depend on how the images are preprocessed before their computation. Since, in widely known and used software for radiomic features calculation, it is possible to set these preprocessing parameters before the calculation of the radiomic feature, there are ongoing studies assessing the stability and repeatability of radiomic features to find the most suitable preprocessing parameters for every used imaging modality.

MATERIALS AND METHODS

We performed a comprehensive literature search using four electronic databases: PubMed, Cochrane Library, Embase, and Scopus. Mesh terms and free text were modeled in search strategies for databases. The inclusion criteria were studies where preprocessing parameters' influence on feature values and model predictions was addressed. Records lacking information on image acquisition parameters were excluded, and any eligible studies with full-text versions were included in the review process, while conference proceedings and monographs were disregarded. We used the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2) tool to investigate the risk of bias. We synthesized our data in a table divided by the imaging modalities subgroups.

RESULTS

After applying the inclusion and exclusion criteria, we selected 43 works. This review examines the impact of preprocessing parameters on the reproducibility and reliability of radiomic features extracted from multimodality imaging (CT, MRI, CBCT, and PET/CT). Standardized preprocessing is crucial for consistent radiomic feature extraction. Key preprocessing steps include voxel resampling, normalization, and discretization, which influence feature robustness and reproducibility. In total, 44% of the included works studied the effects of an isotropic voxel resampling, and most studies opted to employ a discretization strategy. From 2021, several studies started selecting the best set of preprocessing parameters based on models' best performance. As for comparison metrics, ICC was the most used in MRI studies in 58% of the screened works.

CONCLUSIONS

From our work, we highlighted the need to harmonize the use of preprocessing parameters and their values, especially in light of future studies of prospective studies, which are still lacking in the current literature.

Machine learning and radiomics analysis by computed tomography in colorectal liver metastases patients for RAS mutational status prediction

PURPOSE

To assess the efficacy of machine learning and radiomics analysis by computed tomography (CT) in presurgical setting, to predict RAS mutational status in colorectal liver metastases.

METHODS

Patient selection in a retrospective study was carried out from January 2018 to May 2021 considering the following inclusion criteria: patients subjected to surgical resection for liver metastases; proven pathological liver metastases; patients subjected to enhanced CT examination in the presurgical setting with a good quality of images; and RAS assessment as standard reference. A total of 851 radiomics features were extracted using the PyRadiomics Python package from the Slicer 3D image computing platform after slice-by-slice segmentation on CT portal phase by two expert radiologists of each individual liver metastasis performed first independently by the individual reader and then in consensus. Balancing technique was performed, and inter- and intraclass correlation coefficients were calculated to assess the between-observer and within-observer reproducibility of features. Receiver operating characteristics (ROC) analysis with the calculation of area under the ROC curve (AUC), sensitivity (SENS), specificity (SPEC), positive predictive value (PPV), negative predictive value (NPV) and accuracy (ACC) were assessed for each parameter. Linear and non-logistic regression model (LRM and NLRM) and different machine learning-based classifiers were considered. Moreover, features selection was performed before and after a normalized procedure using two different methods (3-sigma and z-score).

RESULTS

Seventy-seven liver metastases in 28 patients with a mean age of 60 years (range 40-80 years) were analyzed. The best predictors, at univariate analysis for both normalized procedures, were original_shape_Maximum2DDiameter and wavelet_HLL_glcm_InverseVariance that reached an accuracy of 80%, an AUC ≥ 0.75, a sensitivity ≥ 80% and a specificity ≥ 70% (p value <  < 0.01). However, a multivariate analysis significantly increased the accuracy in RAS prediction when a linear regression model (LRM) was used. The best performance was obtained using a LRM combining linearly 12 robust features after a z-score normalization procedure: AUC of 0.953, accuracy 98%, sensitivity 96%, specificity of 100%, PPV 100% and NPV 96% (p value <  < 0.01). No statistically significant increase was obtained considering the tested machine learning both without normalization and with normalization methods.

CONCLUSIONS

Normalized approach in CT radiomics analysis allows to predict RAS mutational status in colorectal liver metastases patients.

Does consensus contour improve robustness and accuracy in 18F-FDG PET radiomic features?

PURPOSE

The purpose of our study is to validate the robustness and accuracy of consensus contour in 2-deoxy-2-[18 F]fluoro-D-glucose (18 F-FDG) PET radiomic features.

METHODS

225 nasopharyngeal carcinoma (NPC) and 13 extended cardio-torso (XCAT) simulated data were enrolled. All segmentation were performed with four segmentation methods under two different initial masks, respectively. Consensus contour (ConSeg) was then developed using the majority vote rule. 107 radiomic features were extracted by Pyradiomics based on segmentation and the intraclass correlation coefficient (ICC) was calculated for each feature between masks or among segmentation, respectively. In XCAT ICC between segmentation and simulated ground truth were also calculated to access the accuracy.

RESULTS

ICC varied with the dataset, segmentation method, initial mask and feature type. ConSeg presented higher ICC for radiomic features in robustness tests and similar ICC in accuracy tests, compared with the average of four segmentation results. Higher ICC were also generally observed in irregular initial masks compared with rectangular masks in both robustness and accuracy tests. Furthermore, 19 features (17.76%) had ICC ≥ 0.75 in both robustness and accuracy tests for any of the segmentation methods or initial masks. The dataset was observed to have a large impact on the correlation relationships between radiomic features, but not the segmentation method or initial mask.

CONCLUSIONS

The consensus contour combined with irregular initial mask could improve the robustness and accuracy in radiomic analysis to some extent. The correlation relationships between radiomic features and feature clusters largely depended on the dataset, but not segmentation method or initial mask.

Machine learning-based radiomics analysis in predicting RAS mutational status using magnetic resonance imaging

PURPOSE

To assess the efficacy of radiomics features, obtained by magnetic resonance imaging (MRI) with hepatospecific contrast agent, in pre-surgical setting, to predict RAS mutational status in liver metastases.

METHODS

Patients with MRI in pre-surgical setting were enrolled in a retrospective study. Manual segmentation was made by means 3D Slicer image computing, and 851 radiomics features were extracted as median values using the PyRadiomics Python package. The features were extracted considering the agreement with the Imaging Biomarker Standardization Initiative (IBSI). Balancing was performed through synthesis of samples for the underrepresented classes using the self-adaptive synthetic oversampling (SASYNO) approach. Inter- and intraclass correlation coefficients (ICC) were calculated to assess the between-observer and within-observer reproducibility of all radiomics characteristics. For continuous variables, nonparametric Wilcoxon-Mann-Whitney test was utilized. Benjamini and Hochberg's false discovery rate (FDR) adjustment for multiple testing was used. Receiver operating characteristics (ROC) analysis with the calculation of area under the ROC curve (AUC), sensitivity (SENS), specificity (SPEC), positive predictive value (PPV), negative predictive value (NPV) and accuracy (ACC) were assessed for each parameter. Linear and non-logistic regression model (LRM and NLRM) and different machine learning-based classifiers including decision tree (DT), k-nearest neighbor (KNN) and support vector machine (SVM) were considered. Moreover, features selection were performed before and after a normalized procedure using two different methods (3-sigma and z-score). McNemar test was used to assess differences statistically significant between dichotomic tables. All statistical procedures were done using MATLAB R2021b Statistics and Machine Toolbox (MathWorks, Natick, MA, USA).

RESULTS

Seven normalized radiomics features, extracted from arterial phase, 11 normalized radiomics features, from portal phase, 12 normalized radiomics features from hepatobiliary phase and 12 normalized features from T2-W SPACE sequence were robust predictors of RAS mutational status. The multivariate analysis increased significantly the accuracy in RAS prediction when a LRM was used, combining 12 robust normalized features extracted by VIBE hepatobiliary phase reaching an accuracy of 99%, a sensitivity 97%, a specificity of 100%, a PPV of 100% and a NPV of 98%. No statistically significant increase was obtained, considering the tested classifiers DT, KNN and SVM, both without normalization and with normalization methods.

CONCLUSIONS

Normalized approach in MRI radiomics analysis allows to predict RAS mutational status.

Prior Clinico-Radiological Features Informed Multi-Modal MR Images Convolution Neural Network: A novel deep learning framework for prediction of lymphovascular invasion in breast cancer

BACKGROUND

Current methods utilizing preoperative magnetic resonance imaging (MRI)-based radiomics for assessing lymphovascular invasion (LVI) in patients with early-stage breast cancer lack precision, limiting the options for surgical planning.

PURPOSE

This study aimed to develop a sophisticated deep learning framework called "Prior Clinico-Radiological Features Informed Multi-Modal MR Images Convolutional Neural Network (PCMM-Net)" to improve the accuracy of LVI prediction in breast cancer. By incorporating multiparameter MRI and prior clinical knowledge, PCMM-Net should enhance the precision of LVI assessment.

METHODS

A total of 341 patients with breast cancer were randomly divided into training and validation groups at a ratio of 7:3. Imaging features were extracted from T1-weighted, T2-weighted, and contrast-enhanced T1-weighted MRI sequences. Stepwise univariate and multivariate logistic regression were employed to establish a clinico-radiological model for LVI prediction. The radiomics model was built using redundancy and the least absolute shrinkage and selection operator. Then, two deep learning frameworks were developed: the Multi-Modal MR Images Convolutional Neural Network (MM-Net), which does not consider prior radiological features, and PCMM-Net, which incorporates multiparameter MRI and prior clinical knowledge. Receiver operating characteristic curves were used, and the corresponding areas under the curves (AUCs) were calculated for evaluation.

RESULTS

PCMM-Net achieved the highest AUC of 0.843. The clinico-radiological features displayed the lowest AUC value of 0.743, followed by MM-Net with an AUC of 0.774, and radiomics with an AUC of 0.795.

CONCLUSIONS

This study introduces PCMM-Net, an innovative deep learning framework that integrates prior clinico-radiological features for accurate LVI prediction in breast cancer. PCMM-Net demonstrates excellent diagnostic performance and facilitates the application of precision medicine.

Comparing effectiveness of image perturbation and test retest imaging in improving radiomic model reliability

Image perturbation is a promising technique to assess radiomic feature repeatability, but whether it can achieve the same effect as test-retest imaging on model reliability is unknown. This study aimed to compare radiomic model reliability based on repeatable features determined by the two methods using four different classifiers. A 191-patient public breast cancer dataset with 71 test-retest scans was used with pre-determined 117 training and 74 testing samples. We collected apparent diffusion coefficient images and manual tumor segmentations for radiomic feature extraction. Random translations, rotations, and contour randomizations were performed on the training images, and intra-class correlation coefficient (ICC) was used to filter high repeatable features. We evaluated model reliability in both internal generalizability and robustness, which were quantified by training and testing AUC and prediction ICC. Higher testing performance was found at higher feature ICC thresholds, but it dropped significantly at ICC = 0.95 for the test-retest model. Similar optimal reliability can be achieved with testing AUC = 0.7-0.8 and prediction ICC > 0.9 at the ICC threshold of 0.9. It is recommended to include feature repeatability analysis using image perturbation in any radiomic study when test-retest is not feasible, but care should be taken when deciding the optimal feature repeatability criteria.

MSCDA: Multi-level semantic-guided contrast improves unsupervised domain adaptation for breast MRI segmentation in small datasets

Deep learning (DL) applied to breast tissue segmentation in magnetic resonance imaging (MRI) has received increased attention in the last decade, however, the domain shift which arises from different vendors, acquisition protocols, and biological heterogeneity, remains an important but challenging obstacle on the path towards clinical implementation. In this paper, we propose a novel Multi-level Semantic-guided Contrastive Domain Adaptation (MSCDA) framework to address this issue in an unsupervised manner. Our approach incorporates self-training with contrastive learning to align feature representations between domains. In particular, we extend the contrastive loss by incorporating pixel-to-pixel, pixel-to-centroid, and centroid-to-centroid contrasts to better exploit the underlying semantic information of the image at different levels. To resolve the data imbalance problem, we utilize a category-wise cross-domain sampling strategy to sample anchors from target images and build a hybrid memory bank to store samples from source images. We have validated MSCDA with a challenging task of cross-domain breast MRI segmentation between datasets of healthy volunteers and invasive breast cancer patients. Extensive experiments show that MSCDA effectively improves the model's feature alignment capabilities between domains, outperforming state-of-the-art methods. Furthermore, the framework is shown to be label-efficient, achieving good performance with a smaller source dataset. The code is publicly available at https://github.com/ShengKuangCN/MSCDA.

In Vivo Repeatability and Multiscanner Reproducibility of MRI Radiomics Features in Patients With Monoclonal Plasma Cell Disorders: A Prospective Bi-institutional Study

OBJECTIVES

Despite the extensive number of publications in the field of radiomics, radiomics algorithms barely enter large-scale clinical application. Supposedly, the low external generalizability of radiomics models is one of the main reasons, which hinders the translation from research to clinical application. The objectives of this study were to investigate reproducibility of radiomics features (RFs) in vivo under variation of patient positioning, magnetic resonance imaging (MRI) sequence, and MRI scanners, and to identify a subgroup of RFs that shows acceptable reproducibility across all different acquisition scenarios.

MATERIALS AND METHODS

Between November 30, 2020 and February 16, 2021, 55 patients with monoclonal plasma cell disorders were included in this prospective, bi-institutional, single-vendor study. Participants underwent one reference scan at a 1.5 T MRI scanner and several retest scans: once after simple repositioning, once with a second MRI protocol, once at another 1.5 T scanner, and once at a 3 T scanner. Radiomics feature from the bone marrow of the left hip bone were extracted, both from original scans and after different image normalizations. Intraclass correlation coefficient (ICC) was used to assess RF repeatability and reproducibility.

RESULTS

Fifty-five participants (mean age, 59 ± 7 years; 36 men) were enrolled. For T1-weighted images after muscle normalization, in the simple test-retest experiment, 110 (37%) of 295 RFs showed an ICC ≥0.8: 54 (61%) of 89 first-order features (FOFs), 35 (95%) of 37 volume and shape features, and 21 (12%) of 169 texture features (TFs). When the retest was performed with different technical settings, even after muscle normalization, the number of FOF/TF with an ICC ≥0.8 declined to 58/13 for the second protocol, 29/7 for the second 1.5 T scanner, and 49/7 for the 3 T scanner, respectively. Twenty-five (28%) of the 89 FOFs and 6 (4%) of the 169 TFs from muscle-normalized T1-weighted images showed an ICC ≥0.8 throughout all repeatability and reproducibility experiments.

CONCLUSIONS

In vivo, only few RFs are reproducible with different MRI sequences or different MRI scanners, even after application of a simple image normalization. Radiomics features selected by a repeatability experiment only are not necessarily suited to build radiomics models for multicenter clinical application. This study isolated a subset of RFs, which are robust to variations in MRI acquisition observed in scanners from 1 vendor, and therefore are candidates to build reproducible radiomics models for monoclonal plasma cell disorders for multicentric applications, at least when centers are equipped with scanners from this vendor.

A Two-Step Feature Selection Radiomic Approach to Predict Molecular Outcomes in Breast Cancer

Breast Cancer (BC) is the most common cancer among women worldwide and is characterized by intra- and inter-tumor heterogeneity that strongly contributes towards its poor prognosis. The Estrogen Receptor (ER), Progesterone Receptor (PR), Human Epidermal Growth Factor Receptor 2 (HER2), and Ki67 antigen are the most examined markers depicting BC heterogeneity and have been shown to have a strong impact on BC prognosis. Radiomics can noninvasively predict BC heterogeneity through the quantitative evaluation of medical images, such as Magnetic Resonance Imaging (MRI), which has become increasingly important in the detection and characterization of BC. However, the lack of comprehensive BC datasets in terms of molecular outcomes and MRI modalities, and the absence of a general methodology to build and compare feature selection approaches and predictive models, limit the routine use of radiomics in the BC clinical practice. In this work, a new radiomic approach based on a two-step feature selection process was proposed to build predictors for ER, PR, HER2, and Ki67 markers. An in-house dataset was used, containing 92 multiparametric MRIs of patients with histologically proven BC and all four relevant biomarkers available. Thousands of radiomic features were extracted from post-contrast and subtracted Dynamic Contrast-Enanched (DCE) MRI images, Apparent Diffusion Coefficient (ADC) maps, and T2-weighted (T2) images. The two-step feature selection approach was used to identify significant radiomic features properly and then to build the final prediction models. They showed remarkable results in terms of F1-score for all the biomarkers: 84%, 63%, 90%, and 72% for ER, HER2, Ki67, and PR, respectively. When possible, the models were validated on the TCGA/TCIA Breast Cancer dataset, returning promising results (F1-score = 88% for the ER+/ER- classification task). The developed approach efficiently characterized BC heterogeneity according to the examined molecular biomarkers.

Fetal MRI radiomics: non-invasive and reproducible quantification of human lung maturity

OBJECTIVES

To assess the reproducibility of radiomics features extracted from the developing lung in repeated in-vivo fetal MRI acquisitions.

METHODS

In-vivo MRI (1.5 Tesla) scans of 30 fetuses, each including two axial and one coronal T2-weighted sequences of the whole lung with all other acquisition parameters kept constant, were retrospectively identified. Manual segmentation of the lungs was performed using ITK-Snap. One hundred radiomics features were extracted from fetal lung MRI data using Pyradiomics, resulting in 90 datasets. Intra-class correlation coefficients (ICC) of radiomics features were calculated between baseline and repeat axial acquisitions and between baseline axial and coronal acquisitions.

RESULTS

MRI data of 30 fetuses (12 [40%] females, 18 [60%] males) at a median gestational age of 24 + 5 gestational weeks plus days (GW) (interquartile range [IQR] 3 + 3 GW, range 21 + 1 to 32 + 6 GW) were included. Median ICC of radiomics features between baseline and repeat axial MR acquisitions was 0.92 (IQR 0.13, range 0.33 to 1), with 60 features exhibiting excellent (ICC > 0.9), 27 good (> 0.75-0.9), twelve moderate (0.5-0.75), and one poor (ICC < 0.5) reproducibility. Median ICC of radiomics features between baseline axial and coronal MR acquisitions was 0.79 (IQR 0.15, range 0.2 to 1), with 20 features exhibiting excellent, 47 good, 29 moderate, and four poor reproducibility.

CONCLUSION

Standardized in-vivo fetal MRI allows reproducible extraction of lung radiomics features. In the future, radiomics analysis may improve diagnostic and prognostic yield of fetal MRI in normal and pathologic lung development.

KEY POINTS

• Non-invasive fetal MRI acquired using a standardized protocol allows reproducible extraction of radiomics features from the developing lung for objective tissue characterization. • Alteration of imaging plane between fetal MRI acquisitions has a negative impact on lung radiomics feature reproducibility. • Fetal MRI radiomics features reflecting the microstructure and shape of the fetal lung could complement observed-to-expected lung volume in the prediction of postnatal outcome and optimal treatment of fetuses with abnormal lung development in the future.

CT Reconstruction Kernels and the Effect of Pre- and Post-Processing on the Reproducibility of Handcrafted Radiomic Features

Handcrafted radiomics features (HRFs) are quantitative features extracted from medical images to decode biological information to improve clinical decision making. Despite the potential of the field, limitations have been identified. The most important identified limitation, currently, is the sensitivity of HRF to variations in image acquisition and reconstruction parameters. In this study, we investigated the use of Reconstruction Kernel Normalization (RKN) and ComBat harmonization to improve the reproducibility of HRFs across scans acquired with different reconstruction kernels. A set of phantom scans (n = 28) acquired on five different scanner models was analyzed. HRFs were extracted from the original scans, and scans were harmonized using the RKN method. ComBat harmonization was applied on both sets of HRFs. The reproducibility of HRFs was assessed using the concordance correlation coefficient. The difference in the number of reproducible HRFs in each scenario was assessed using McNemar’s test. The majority of HRFs were found to be sensitive to variations in the reconstruction kernels, and only six HRFs were found to be robust with respect to variations in reconstruction kernels. The use of RKN resulted in a significant increment in the number of reproducible HRFs in 19 out of the 67 investigated scenarios (28.4%), while the ComBat technique resulted in a significant increment in 36 (53.7%) scenarios. The combination of methods resulted in a significant increment in 53 (79.1%) scenarios compared to the HRFs extracted from original images. Since the benefit of applying the harmonization methods depended on the data being harmonized, reproducibility analysis is recommended before performing radiomics analysis. For future radiomics studies incorporating images acquired with similar image acquisition and reconstruction parameters, except for the reconstruction kernels, we recommend the systematic use of the pre- and post-processing approaches (respectively, RKN and ComBat).