1
|
Hatamikia S, George G, Schwarzhans F, Mahbod A, Woitek R. Breast MRI radiomics and machine learning-based predictions of response to neoadjuvant chemotherapy - How are they affected by variations in tumor delineation? Comput Struct Biotechnol J 2024; 23:52-63. [PMID: 38125296 PMCID: PMC10730996 DOI: 10.1016/j.csbj.2023.11.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2023] [Revised: 11/08/2023] [Accepted: 11/08/2023] [Indexed: 12/23/2023] Open
Abstract
Manual delineation of volumes of interest (VOIs) by experts is considered the gold-standard method in radiomics analysis. However, it suffers from inter- and intra-operator variability. A quantitative assessment of the impact of variations in these delineations on the performance of the radiomics predictors is required to develop robust radiomics based prediction models. In this study, we developed radiomics models for the prediction of pathological complete response to neoadjuvant chemotherapy in patients with two different breast cancer subtypes based on contrast-enhanced magnetic resonance imaging acquired prior to treatment (baseline MRI scans). Different mathematical operations such as erosion, smoothing, dilation, randomization, and ellipse fitting were applied to the original VOIs delineated by experts to simulate variations of segmentation masks. The effects of such VOI modifications on various steps of the radiomics workflow, including feature extraction, feature selection, and prediction performance, were evaluated. Using manual tumor VOIs and radiomics features extracted from baseline MRI scans, an AUC of up to 0.96 and 0.89 was achieved for human epidermal growth receptor 2 positive and triple-negative breast cancer, respectively. For smoothing and erosion, VOIs yielded the highest number of robust features and the best prediction performance, while ellipse fitting and dilation lead to the lowest robustness and prediction performance for both breast cancer subtypes. At most 28% of the selected features were similar to manual VOIs when different VOI delineation data were used. Differences in VOI delineation affect different steps of radiomics analysis, and their quantification is therefore important for development of standardized radiomics research.
Collapse
Affiliation(s)
- Sepideh Hatamikia
- Danube Private University, Krems, Rathausplatz 1, Krems-Stein, AT-3500, Austria
- Austrian Center for Medical Innovation and Technology (ACMIT), Viktor Kaplan-Straße 2/1, Wiener Neustadt 2700, Austria
| | - Geevarghese George
- Danube Private University, Krems, Rathausplatz 1, Krems-Stein, AT-3500, Austria
| | - Florian Schwarzhans
- Danube Private University, Krems, Rathausplatz 1, Krems-Stein, AT-3500, Austria
| | - Amirreza Mahbod
- Danube Private University, Krems, Rathausplatz 1, Krems-Stein, AT-3500, Austria
| | - Ramona Woitek
- Danube Private University, Krems, Rathausplatz 1, Krems-Stein, AT-3500, Austria
| |
Collapse
|
2
|
Demircioğlu A. radMLBench: A dataset collection for benchmarking in radiomics. Comput Biol Med 2024; 182:109140. [PMID: 39270457 DOI: 10.1016/j.compbiomed.2024.109140] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2024] [Revised: 08/20/2024] [Accepted: 09/08/2024] [Indexed: 09/15/2024]
Abstract
BACKGROUND New machine learning methods and techniques are frequently introduced in radiomics, but they are often tested on a single dataset, which makes it challenging to assess their true benefit. Currently, there is a lack of a larger, publicly accessible dataset collection on which such assessments could be performed. In this study, a collection of radiomics datasets with binary outcomes in tabular form was curated to allow benchmarking of machine learning methods and techniques. METHODS A variety of journals and online sources were searched to identify tabular radiomics data with binary outcomes, which were then compiled into a homogeneous data collection that is easily accessible via Python. To illustrate the utility of the dataset collection, it was applied to investigate whether feature decorrelation prior to feature selection could improve predictive performance in a radiomics pipeline. RESULTS A total of 50 radiomic datasets were collected, with sample sizes ranging from 51 to 969 and 101 to 11165 features. Using this data, it was observed that decorrelating features did not yield any significant improvement on average. CONCLUSIONS A large collection of datasets, easily accessible via Python, suitable for benchmarking and evaluating new machine learning techniques and methods was curated. Its utility was exemplified by demonstrating that feature decorrelation prior to feature selection does not, on average, lead to significant performance gains and could be omitted, thereby increasing the robustness and reliability of the radiomics pipeline.
Collapse
Affiliation(s)
- Aydin Demircioğlu
- Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Hufelandstraße 55, D-45147, Essen, Germany.
| |
Collapse
|
3
|
Nowakowska S, Borkowski K, Ruppert C, Hejduk P, Ciritsis A, Landsmann A, Marcon M, Berger N, Boss A, Rossi C. Explainable Precision Medicine in Breast MRI: A Combined Radiomics and Deep Learning Approach for the Classification of Contrast Agent Uptake. Bioengineering (Basel) 2024; 11:556. [PMID: 38927793 PMCID: PMC11200390 DOI: 10.3390/bioengineering11060556] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2024] [Revised: 05/24/2024] [Accepted: 05/29/2024] [Indexed: 06/28/2024] Open
Abstract
In DCE-MRI, the degree of contrast uptake in normal fibroglandular tissue, i.e., background parenchymal enhancement (BPE), is a crucial biomarker linked to breast cancer risk and treatment outcome. In accordance with the Breast Imaging Reporting & Data System (BI-RADS), it should be visually classified into four classes. The susceptibility of such an assessment to inter-reader variability highlights the urgent need for a standardized classification algorithm. In this retrospective study, the first post-contrast subtraction images for 27 healthy female subjects were included. The BPE was classified slice-wise by two expert radiologists. The extraction of radiomic features from segmented BPE was followed by dataset splitting and dimensionality reduction. The latent representations were then utilized as inputs to a deep neural network classifying BPE into BI-RADS classes. The network's predictions were elucidated at the radiomic feature level with Shapley values. The deep neural network achieved a BPE classification accuracy of 84 ± 2% (p-value < 0.00001). Most of the misclassifications involved adjacent classes. Different radiomic features were decisive for the prediction of each BPE class underlying the complexity of the decision boundaries. A highly precise and explainable pipeline for BPE classification was achieved without user- or algorithm-dependent radiomic feature selection.
Collapse
Affiliation(s)
- Sylwia Nowakowska
- Diagnostic and Interventional Radiology, University Hospital Zürich, University Zürich, Rämistrasse 100, 8091 Zürich, Switzerland (C.R.)
| | | | - Carlotta Ruppert
- Diagnostic and Interventional Radiology, University Hospital Zürich, University Zürich, Rämistrasse 100, 8091 Zürich, Switzerland (C.R.)
- b-rayZ AG, Wagistrasse 21, 8952 Schlieren, Switzerland
| | - Patryk Hejduk
- Diagnostic and Interventional Radiology, University Hospital Zürich, University Zürich, Rämistrasse 100, 8091 Zürich, Switzerland (C.R.)
| | - Alexander Ciritsis
- Diagnostic and Interventional Radiology, University Hospital Zürich, University Zürich, Rämistrasse 100, 8091 Zürich, Switzerland (C.R.)
- b-rayZ AG, Wagistrasse 21, 8952 Schlieren, Switzerland
| | - Anna Landsmann
- Diagnostic and Interventional Radiology, University Hospital Zürich, University Zürich, Rämistrasse 100, 8091 Zürich, Switzerland (C.R.)
| | - Magda Marcon
- Diagnostic and Interventional Radiology, University Hospital Zürich, University Zürich, Rämistrasse 100, 8091 Zürich, Switzerland (C.R.)
| | - Nicole Berger
- Diagnostic and Interventional Radiology, University Hospital Zürich, University Zürich, Rämistrasse 100, 8091 Zürich, Switzerland (C.R.)
| | - Andreas Boss
- Diagnostic and Interventional Radiology, University Hospital Zürich, University Zürich, Rämistrasse 100, 8091 Zürich, Switzerland (C.R.)
| | - Cristina Rossi
- Diagnostic and Interventional Radiology, University Hospital Zürich, University Zürich, Rämistrasse 100, 8091 Zürich, Switzerland (C.R.)
- b-rayZ AG, Wagistrasse 21, 8952 Schlieren, Switzerland
| |
Collapse
|
4
|
Ansari G, Mirza-Aghazadeh-Attari M, Afyouni S, Mohseni A, Shahbazian H, Kamel IR. Utilization of texture features of volumetric ADC maps in differentiating between serous cystadenoma and intraductal papillary neoplasms. Abdom Radiol (NY) 2024; 49:1175-1184. [PMID: 38378839 DOI: 10.1007/s00261-024-04187-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Revised: 01/04/2024] [Accepted: 01/07/2024] [Indexed: 02/22/2024]
Abstract
INTRODUCTION The rising incidence of incidental detection of pancreatic cystic neoplasms has compelled radiologists to determine new diagnostic methods for the differentiation of various kinds of lesions. We aim to demonstrate the utility of texture features extracted from ADC maps in differentiating intraductal papillary mucinous neoplasms (IPMN) from serous cystadenomas (SCA). METHODS This retrospective study was performed on 136 patients (IPMN = 87, SCA = 49) split into testing and training datasets. A total of 851 radiomics features were extracted from volumetric contours drawn by an expert radiologist on ADC maps of the lesions. LASSO regression analysis was used to determine the most predictive set of features and a radiomics score was developed based on their respective coefficients. A hyper-optimized support vector machine was then utilized to classify the lesions based on their radiomics score. RESULTS A total of four Wavelet features (LHL/GLCM/LCM2, HLL/GLCM/LCM2, /LLL/First Order/90percent, /LLL/GLCM/MCC) were selected from all of the features to be included in our classifier. The classifier was optimized by altering hyperparameters and the trained model was applied to the validation dataset. The model achieved a sensitivity of 92.8, specificity of 90%, and an AUC of 0.97 in the training data set, and a sensitivity of 83.3%, specificity of 66.7%, and AUC of 0.90 in the testing dataset. CONCLUSION A support vector machine model trained and validated on volumetric texture features extracted from ADC maps showed the possible beneficence of these features in differentiating IPMNs from SCAs. These results are in line with previous regarding the role of ADC maps in classifying cystic lesions and offers new evidence regarding the role of texture features in differentiation of potentially neoplastic and benign lesions.
Collapse
Affiliation(s)
- Golnoosh Ansari
- Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins University, 600 North Wolfe Street, MRI 143, Baltimore, MD, 21287, USA
| | - Mohammad Mirza-Aghazadeh-Attari
- Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins University, 600 North Wolfe Street, MRI 143, Baltimore, MD, 21287, USA
| | - Shadi Afyouni
- Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins University, 600 North Wolfe Street, MRI 143, Baltimore, MD, 21287, USA
| | - Alireza Mohseni
- Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins University, 600 North Wolfe Street, MRI 143, Baltimore, MD, 21287, USA
| | - Haneyeh Shahbazian
- Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins University, 600 North Wolfe Street, MRI 143, Baltimore, MD, 21287, USA
| | - Ihab R Kamel
- Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins University, 600 North Wolfe Street, MRI 143, Baltimore, MD, 21287, USA.
| |
Collapse
|
5
|
Demircioğlu A. The effect of data resampling methods in radiomics. Sci Rep 2024; 14:2858. [PMID: 38310165 PMCID: PMC10838284 DOI: 10.1038/s41598-024-53491-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2023] [Accepted: 02/01/2024] [Indexed: 02/05/2024] Open
Abstract
Radiomic datasets can be class-imbalanced, for instance, when the prevalence of diseases varies notably, meaning that the number of positive samples is much smaller than that of negative samples. In these cases, the majority class may dominate the model's training and thus negatively affect the model's predictive performance, leading to bias. Therefore, resampling methods are often utilized to class-balance the data. However, several resampling methods exist, and neither their relative predictive performance nor their impact on feature selection has been systematically analyzed. In this study, we aimed to measure the impact of nine resampling methods on radiomic models utilizing a set of fifteen publicly available datasets regarding their predictive performance. Furthermore, we evaluated the agreement and similarity of the set of selected features. Our results show that applying resampling methods did not improve the predictive performance on average. On specific datasets, slight improvements in predictive performance (+ 0.015 in AUC) could be seen. A considerable disagreement on the set of selected features was seen (only 28.7% of features agreed), which strongly impedes feature interpretability. However, selected features are similar when considering their correlation (82.9% of features correlated on average).
Collapse
Affiliation(s)
- Aydin Demircioğlu
- Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Hufelandstraße 55, 45147, Essen, Germany.
| |
Collapse
|
6
|
Demircioğlu A. The effect of feature normalization methods in radiomics. Insights Imaging 2024; 15:2. [PMID: 38185786 PMCID: PMC10772134 DOI: 10.1186/s13244-023-01575-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Accepted: 11/25/2023] [Indexed: 01/09/2024] Open
Abstract
OBJECTIVES In radiomics, different feature normalization methods, such as z-Score or Min-Max, are currently utilized, but their specific impact on the model is unclear. We aimed to measure their effect on the predictive performance and the feature selection. METHODS We employed fifteen publicly available radiomics datasets to compare seven normalization methods. Using four feature selection and classifier methods, we used cross-validation to measure the area under the curve (AUC) of the resulting models, the agreement of selected features, and the model calibration. In addition, we assessed whether normalization before cross-validation introduces bias. RESULTS On average, the difference between the normalization methods was relatively small, with a gain of at most + 0.012 in AUC when comparing the z-Score (mean AUC: 0.707 ± 0.102) to no normalization (mean AUC: 0.719 ± 0.107). However, on some datasets, the difference reached + 0.051. The z-Score performed best, while the tanh transformation showed the worst performance and even decreased the overall predictive performance. While quantile transformation performed, on average, slightly worse than the z-Score, it outperformed all other methods on one out of three datasets. The agreement between the features selected by different normalization methods was only mild, reaching at most 62%. Applying the normalization before cross-validation did not introduce significant bias. CONCLUSION The choice of the feature normalization method influenced the predictive performance but depended strongly on the dataset. It strongly impacted the set of selected features. CRITICAL RELEVANCE STATEMENT Feature normalization plays a crucial role in the preprocessing and influences the predictive performance and the selected features, complicating feature interpretation. KEY POINTS • The impact of feature normalization methods on radiomic models was measured. • Normalization methods performed similarly on average, but differed more strongly on some datasets. • Different methods led to different sets of selected features, impeding feature interpretation. • Model calibration was not largely affected by the normalization method.
Collapse
Affiliation(s)
- Aydin Demircioğlu
- Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Hufelandstrasse 55, 45147, Essen, Germany.
| |
Collapse
|
7
|
Bai J, He M, Gao E, Yang G, Yang H, Dong J, Ma X, Gao Y, Zhang H, Yan X, Zhang Y, Cheng J, Zhao G. Radiomic texture analysis based on neurite orientation dispersion and density imaging to differentiate glioblastoma from solitary brain metastasis. BMC Cancer 2023; 23:1231. [PMID: 38098041 PMCID: PMC10722697 DOI: 10.1186/s12885-023-11718-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Accepted: 12/05/2023] [Indexed: 12/17/2023] Open
Abstract
BACKGROUND We created discriminative models of different regions of interest (ROIs) using radiomic texture features of neurite orientation dispersion and density imaging (NODDI) and evaluated the feasibility of each model in differentiating glioblastoma multiforme (GBM) from solitary brain metastasis (SBM). METHODS We conducted a retrospective study of 204 patients with GBM (n = 146) or SBM (n = 58). Radiomic texture features were extracted from five ROIs based on three metric maps (intracellular volume fraction, orientation dispersion index, and isotropic volume fraction of NODDI), including necrosis, solid tumors, peritumoral edema, tumor bulk volume (TBV), and abnormal bulk volume. Four feature selection methods and eight classifiers were used for the radiomic texture feature selection and model construction. Receiver operating characteristic (ROC) curve analysis was used to evaluate the diagnostic performance of the models. Routine magnetic resonance imaging (MRI) radiomic texture feature models generated in the same manner were used for the horizontal comparison. RESULTS NODDI-radiomic texture analysis based on TBV subregions exhibited the highest accuracy (although nonsignificant) in differentiating GBM from SBM, with area under the ROC curve (AUC) values of 0.918 and 0.882 in the training and test datasets, respectively, compared to necrosis (AUCtraining:0.845, AUCtest:0.714), solid tumor (AUCtraining:0.852, AUCtest:0.821), peritumoral edema (AUCtraining:0.817, AUCtest:0.762), and ABV (AUCtraining:0.834, AUCtest:0.779). The performance of the five ROI radiomic texture models in routine MRI was inferior to that of the NODDI-radiomic texture model. CONCLUSION Preoperative NODDI-radiomic texture analysis based on TBV subregions shows great potential for distinguishing GBM from SBM.
Collapse
Affiliation(s)
- Jie Bai
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, No. Jianshe Dong Road, Zhengzhou, 450052, China
| | - Mengyang He
- School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou, 450001, China
| | - Eryuan Gao
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, No. Jianshe Dong Road, Zhengzhou, 450052, China
| | - Guang Yang
- Shanghai Key Laboratory of Magnetic Resonance, East China Normal University, Shanghai, 200062, China
| | - Hongxi Yang
- Shanghai Key Laboratory of Magnetic Resonance, East China Normal University, Shanghai, 200062, China
| | - Jie Dong
- School of Information Engineering, North China University of Water Resources and Electric Power, Zhengzhou, 450046, China
| | - Xiaoyue Ma
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, No. Jianshe Dong Road, Zhengzhou, 450052, China
| | - Yufei Gao
- School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou, 450001, China
| | - Huiting Zhang
- MR Research Collaboration, Siemens Healthineers, Wuhan, 201318, China
| | - Xu Yan
- MR Research Collaboration, Siemens Healthineers, Wuhan, 201318, China
| | - Yong Zhang
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, No. Jianshe Dong Road, Zhengzhou, 450052, China
| | - Jingliang Cheng
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, No. Jianshe Dong Road, Zhengzhou, 450052, China
| | - Guohua Zhao
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, No. Jianshe Dong Road, Zhengzhou, 450052, China.
| |
Collapse
|
8
|
Li L, Wang M, Jiang X, Lin Y. Universal multi-factor feature selection method for radiomics-based brain tumor classification. Comput Biol Med 2023; 164:107122. [PMID: 37517322 DOI: 10.1016/j.compbiomed.2023.107122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 05/19/2023] [Accepted: 05/30/2023] [Indexed: 08/01/2023]
Abstract
Brain tumor mortality is high, and accurate classification before treatment can improve patient prognosis. Radiomics, which extracts numerous features from medical images, has been widely applied in brain tumor classification studies. Feature selection (FS) is a critical step in radiomics because it reduces redundant information and enhances classification performance. However, the lack of universal FS methods limits the development of radiomics-based brain tumor classification studies. To address this issue, we summarize the characteristics of the FS methods used in related studies and propose a universal method based on three selection factors called triple-factor cascaded selection (TFCS). Particularly, these factors correspond to the correlation between features and task labels, interdependence among features, and role of features in the model. The TFCS method divides FS into two steps. First, it utilizes mutual information to select features that are strongly correlated with the task and contain less redundant information. Recursive feature elimination is then employed to obtain the subset with the best classification performance. To validate the universality of the TFCS, we conducted experiments on seven datasets containing 13 brain tumor classification tasks and evaluated the overall performance using five types of indicators. Results: TFCS exhibited excellent overall performance for all tasks. Compared to the 13 related methods, it takes less time, has moderate parsimony, the best classification performance, adaptability, and stability, and shows better universality. Our study demonstrates that the reasonable utilization of multiple factors can enhance FS performance and provide new insights for future method design.
Collapse
Affiliation(s)
- Longfei Li
- School of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou, China; Collaborative Innovation Center for Internet Healthcare, Zhengzhou University, Zhengzhou, China
| | - Meiyun Wang
- Collaborative Innovation Center for Internet Healthcare, Zhengzhou University, Zhengzhou, China; Department of Radiology, People's Hospital of Zhengzhou University, Zhengzhou, China
| | - Xiaoming Jiang
- Chongqing Engineering Research Center of Medical Electronics and Information Technology, Chongqing University of Post and Telecommunications, Chongqing, China.
| | - Yusong Lin
- Collaborative Innovation Center for Internet Healthcare, Zhengzhou University, Zhengzhou, China; School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou, China; Hanwei IoT Institute, Zhengzhou University, Zhengzhou, China.
| |
Collapse
|
9
|
Kunkyab T, Mou B, Jirasek A, Haston C, Andrews J, Thomas S, Hyde D. Radiomic analysis for early differentiation of lung cancer recurrence from fibrosis in patients treated with lung stereotactic ablative radiotherapy. Phys Med Biol 2023; 68:165015. [PMID: 37164024 DOI: 10.1088/1361-6560/acd431] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2022] [Accepted: 05/10/2023] [Indexed: 05/12/2023]
Abstract
Objective. The development of radiation-induced fibrosis after stereotactic ablative radiotherapy (SABR) can obscure follow-up images and delay detection of a local recurrence in early-stage lung cancer patients. The objective of this study was to develop a radiomics model for computer-assisted detection of local recurrence and fibrosis for an earlier timepoint (<1 year) after the SABR treatment.Approach. This retrospective clinical study included CT images (n= 107) of 66 patients treated with SABR. A z-score normalization technique was used for radiomic feature standardization across scanner protocols. The training set for the radiomics model consisted of CT images (66 patients; 22 recurrences and 44 fibrosis) obtained at 24 months (median) follow-up. The test set included CT-images of 41 patients acquired at 5-12 months follow-up. Combinations of four widely used machine learning techniques (support vector machines, gradient boosting, random forests (RF), and logistic regression) and feature selection methods (Relief feature scoring, maximum relevance minimum redundancy, mutual information maximization, forward feature selection, and LASSO) were investigated. Pyradiomics was used to extract 106 radiomic features from the CT-images for feature selection and classification.Main results. An RF + LASSO model scored the highest in terms of AUC (0.87) and obtained a sensitivity of 75% and a specificity of 88% in identifying a local recurrence in the test set. In the training set, 86% accuracy was achieved using five-fold cross-validation. Delong's test indicated that AUC achieved by the RF+LASSO is significantly better than 11 other machine learning models presented here. The top three radiomic features: interquartile range (first order), Cluster Prominence (GLCM), and Autocorrelation (GLCM), were revealed as differentiating a recurrence from fibrosis with this model.Significance. The radiomics model selected, out of multiple machine learning and feature selection algorithms, was able to differentiate a recurrence from fibrosis in earlier follow-up CT-images with a high specificity rate and satisfactory sensitivity performance.
Collapse
Affiliation(s)
- Tenzin Kunkyab
- Department of Physics, University of British Columbia Okanagan, Kelowna, British Columbia, Canada
| | | | - Andrew Jirasek
- Department of Physics, University of British Columbia Okanagan, Kelowna, British Columbia, Canada
| | - Christina Haston
- Department of Physics, University of British Columbia Okanagan, Kelowna, British Columbia, Canada
| | - Jeff Andrews
- Department of Statistics, University of British Columbia Okanagan, Kelowna, British Columbia, Canada
| | | | - Derek Hyde
- Department of Physics, University of British Columbia Okanagan, Kelowna, British Columbia, Canada
- BC Cancer-Kelowna, Canada
| |
Collapse
|
10
|
Zhong J, Xing Y, Zhang G, Hu Y, Ding D, Ge X, Pan Z, Yin Q, Zhang H, Yang Q, Zhang H, Yao W. A systematic review of radiomics in giant cell tumor of bone (GCTB): the potential of analysis on individual radiomics feature for identifying genuine promising imaging biomarkers. J Orthop Surg Res 2023; 18:414. [PMID: 37287036 DOI: 10.1186/s13018-023-03863-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/22/2023] [Accepted: 05/16/2023] [Indexed: 06/09/2023] Open
Abstract
PURPOSE To systematically assess the quality of radiomics research in giant cell tumor of bone (GCTB) and to test the feasibility of analysis at the level of radiomics feature. METHODS We searched PubMed, Embase, Web of Science, China National Knowledge Infrastructure, and Wanfang Data to identify articles of GCTB radiomics until 31 July 2022. The studies were assessed by radiomics quality score (RQS), transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement, checklist for artificial intelligence in medical imaging (CLAIM), and modified quality assessment of diagnostic accuracy studies (QUADAS-2) tool. The radiomic features selected for model development were documented. RESULTS Nine articles were included. The average of the ideal percentage of RQS, the TRIPOD adherence rate and the CLAIM adherence rate were 26%, 56%, and 57%, respectively. The risk of bias and applicability concerns were mainly related to the index test. The shortness in external validation and open science were repeatedly emphasized. In GCTB radiomics models, the gray level co-occurrence matrix features (40%), first order features (28%), and gray-level run-length matrix features (18%) were most selected features out of all reported features. However, none of the individual feature has appeared repeatably in multiple studies. It is not possible to meta-analyze radiomics features at present. CONCLUSION The quality of GCTB radiomics studies is suboptimal. The reporting of individual radiomics feature data is encouraged. The analysis at the level of radiomics feature has potential to generate more practicable evidence for translating radiomics into clinical application.
Collapse
Affiliation(s)
- Jingyu Zhong
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Yue Xing
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Guangcheng Zhang
- Department of Sports Medicine, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200233, China
| | - Yangfan Hu
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Defang Ding
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Xiang Ge
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Zhen Pan
- Department of Orthopedics, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Qian Yin
- Department of Pathology, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200233, China
| | - Huizhen Zhang
- Department of Pathology, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200233, China
| | - Qingcheng Yang
- Department of Orthopedics, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200233, China
| | - Huan Zhang
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University of Medicine, Shanghai, 200025, China.
| | - Weiwu Yao
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China.
| |
Collapse
|
11
|
Varghese AJ, Gouthamchand V, Sasidharan BK, Wee L, Sidhique SK, Rao JP, Dekker A, Hoebers F, Devakumar D, Irodi A, Balasingh TP, Godson HF, Joel T, Mathew M, Gunasingam Isiah R, Pavamani SP, Thomas HMT. Multi-centre radiomics for prediction of recurrence following radical radiotherapy for head and neck cancers: Consequences of feature selection, machine learning classifiers and batch-effect harmonization. Phys Imaging Radiat Oncol 2023; 26:100450. [PMID: 37260438 PMCID: PMC10227455 DOI: 10.1016/j.phro.2023.100450] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 05/01/2023] [Accepted: 05/02/2023] [Indexed: 06/02/2023] Open
Abstract
Background and purpose Radiomics models trained with limited single institution data are often not reproducible and generalisable. We developed radiomics models that predict loco-regional recurrence within two years of radiotherapy with private and public datasets and their combinations, to simulate small and multi-institutional studies and study the responsiveness of the models to feature selection, machine learning algorithms, centre-effect harmonization and increased dataset sizes. Materials and methods 562 patients histologically confirmed and treated for locally advanced head-and-neck cancer (LA-HNC) from two public and two private datasets; one private dataset exclusively reserved for validation. Clinical contours of primary tumours were not recontoured and were used for Pyradiomics based feature extraction. ComBat harmonization was applied, and LASSO-Logistic Regression (LR) and Support Vector Machine (SVM) models were built. 95% confidence interval (CI) of 1000 bootstrapped area-under-the-Receiver-operating-curves (AUC) provided predictive performance. Responsiveness of the models' performance to the choice of feature selection methods, ComBat harmonization, machine learning classifier, single and pooled data was evaluated. Results LASSO and SelectKBest selected 14 and 16 features, respectively; three were overlapping. Without ComBat, the LR and SVM models for three institutional data showed AUCs (CI) of 0.513 (0.481-0.559) and 0.632 (0.586-0.665), respectively. Performances following ComBat revealed AUCs of 0.559 (0.536-0.590) and 0.662 (0.606-0.690), respectively. Compared to single cohort AUCs (0.562-0.629), SVM models from pooled data performed significantly better at AUC = 0.680. Conclusions Multi-institutional retrospective data accentuates the existing variabilities that affect radiomics. Carefully designed prospective, multi-institutional studies and data sharing are necessary for clinically relevant head-and-neck cancer prognostication models.
Collapse
Affiliation(s)
- Amal Joseph Varghese
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| | - Varsha Gouthamchand
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | | | - Leonard Wee
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Sharief K Sidhique
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| | | | - Andre Dekker
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Frank Hoebers
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Devadhas Devakumar
- Department of Nuclear Medicine, Christian Medical College, Vellore, Tamil Nadu, India
| | - Aparna Irodi
- Department of Radiology, Christian Medical College, Vellore, Tamil Nadu, India
| | | | - Henry Finlay Godson
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| | - T Joel
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| | - Manu Mathew
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| | | | | | - Hannah Mary T Thomas
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| |
Collapse
|
12
|
Demircioğlu A. Are deep models in radiomics performing better than generic models? A systematic review. Eur Radiol Exp 2023; 7:11. [PMID: 36918479 PMCID: PMC10014394 DOI: 10.1186/s41747-023-00325-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Accepted: 01/13/2023] [Indexed: 03/16/2023] Open
Abstract
BACKGROUND Application of radiomics proceeds by extracting and analysing imaging features based on generic morphological, textural, and statistical features defined by formulas. Recently, deep learning methods were applied. It is unclear whether deep models (DMs) can outperform generic models (GMs). METHODS We identified publications on PubMed and Embase to determine differences between DMs and GMs in terms of receiver operating area under the curve (AUC). RESULTS Of 1,229 records (between 2017 and 2021), 69 studies were included, 61 (88%) on tumours, 68 (99%) retrospective, and 39 (56%) single centre; 30 (43%) used an internal validation cohort; and 18 (26%) applied cross-validation. Studies with independent internal cohort had a median training sample of 196 (range 41-1,455); those with cross-validation had only 133 (43-1,426). Median size of validation cohorts was 73 (18-535) for internal and 94 (18-388) for external. Considering the internal validation, in 74% (49/66), the DMs performed better than the GMs, vice versa in 20% (13/66); no difference in 6% (4/66); and median difference in AUC 0.045. On the external validation, DMs were better in 65% (13/20), GMs in 20% (4/20) cases; no difference in 3 (15%); and median difference in AUC 0.025. On internal validation, fused models outperformed GMs and DMs in 72% (20/28), while they were worse in 14% (4/28) and equal in 14% (4/28); median gain in AUC was + 0.02. On external validation, fused model performed better in 63% (5/8), worse in 25% (2/8), and equal in 13% (1/8); median gain in AUC was + 0.025. CONCLUSIONS Overall, DMs outperformed GMs but in 26% of the studies, DMs did not outperform GMs.
Collapse
Affiliation(s)
- Aydin Demircioğlu
- Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Hufelandstraße 55, 45147, Essen, Germany.
| |
Collapse
|