1
|
Hosseini SA, Shiri I, Ghaffarian P, Hajianfar G, Avval AH, Seyfi M, Servaes S, Rosa-Neto P, Zaidi H, Ay MR. The effect of harmonization on the variability of PET radiomic features extracted using various segmentation methods. Ann Nucl Med 2024; 38:493-507. [PMID: 38575814 PMCID: PMC11217131 DOI: 10.1007/s12149-024-01923-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2023] [Accepted: 03/07/2024] [Indexed: 04/06/2024]
Abstract
PURPOSE This study aimed to examine the robustness of positron emission tomography (PET) radiomic features extracted via different segmentation methods before and after ComBat harmonization in patients with non-small cell lung cancer (NSCLC). METHODS We included 120 patients (positive recurrence = 46 and negative recurrence = 74) referred for PET scanning as a routine part of their care. All patients had a biopsy-proven NSCLC. Nine segmentation methods were applied to each image, including manual delineation, K-means (KM), watershed, fuzzy-C-mean, region-growing, local active contour (LAC), and iterative thresholding (IT) with 40, 45, and 50% thresholds. Diverse image discretizations, both without a filter and with different wavelet decompositions, were applied to PET images. Overall, 6741 radiomic features were extracted from each image (749 radiomic features from each segmented area). Non-parametric empirical Bayes (NPEB) ComBat harmonization was used to harmonize the features. Linear Support Vector Classifier (LinearSVC) with L1 regularization For feature selection and Support Vector Machine classifier (SVM) with fivefold nested cross-validation was performed using StratifiedKFold with 'n_splits' set to 5 to predict recurrence in NSCLC patients and assess the impact of ComBat harmonization on the outcome. RESULTS From 749 extracted radiomic features, 206 (27%) and 389 (51%) features showed excellent reliability (ICC ≥ 0.90) against segmentation method variation before and after NPEB ComBat harmonization, respectively. Among all, 39 features demonstrated poor reliability, which declined to 10 after ComBat harmonization. The 64 fixed bin widths (without any filter) and wavelets (LLL)-based radiomic features set achieved the best performance in terms of robustness against diverse segmentation techniques before and after ComBat harmonization. The first-order and GLRLM and also first-order and NGTDM feature families showed the largest number of robust features before and after ComBat harmonization, respectively. In terms of predicting recurrence in NSCLC, our findings indicate that using ComBat harmonization can significantly enhance machine learning outcomes, particularly improving the accuracy of watershed segmentation, which initially had fewer reliable features than manual contouring. Following the application of ComBat harmonization, the majority of cases saw substantial increase in sensitivity and specificity. CONCLUSION Radiomic features are vulnerable to different segmentation methods. ComBat harmonization might be considered a solution to overcome the poor reliability of radiomic features.
Collapse
Affiliation(s)
- Seyyed Ali Hosseini
- Translational Neuroimaging Laboratory, The McGill University Research Centre for Studies in Aging, Douglas Hospital, McGill University, Montréal, QC, Canada
- Department of Neurology and Neurosurgery, Faculty of Medicine, McGill University, Montréal, QC, Canada
| | - Isaac Shiri
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, 1211, Geneva 4, Switzerland
| | - Pardis Ghaffarian
- Chronic Respiratory Diseases Research Center, National Research Institute of Tuberculosis and Lung Diseases (NRITLD), Shahid Beheshti University of Medical Sciences, Tehran, Iran
- PET/CT and Cyclotron Center, Masih Daneshvari Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Ghasem Hajianfar
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, 1211, Geneva 4, Switzerland
| | | | - Milad Seyfi
- Department of Medical Physics and Biomedical Engineering, Tehran University of Medical Sciences, Tehran, Iran
- Research Center for Molecular and Cellular Imaging, Tehran University of Medical Sciences, Tehran, Iran
| | - Stijn Servaes
- Translational Neuroimaging Laboratory, The McGill University Research Centre for Studies in Aging, Douglas Hospital, McGill University, Montréal, QC, Canada
- Department of Neurology and Neurosurgery, Faculty of Medicine, McGill University, Montréal, QC, Canada
| | - Pedro Rosa-Neto
- Translational Neuroimaging Laboratory, The McGill University Research Centre for Studies in Aging, Douglas Hospital, McGill University, Montréal, QC, Canada
- Department of Neurology and Neurosurgery, Faculty of Medicine, McGill University, Montréal, QC, Canada
| | - Habib Zaidi
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, 1211, Geneva 4, Switzerland.
- Department of Nuclear Medicine and Molecular Imaging, University of Groningen, University Medical Center Groningen, 9700 RB, Groningen, Netherlands.
- Department of Nuclear Medicine, University of Southern Denmark, 500, Odense, Denmark.
- University Research and Innovation Center, Óbudabuda University, Budapest, Hungary.
| | - Mohammad Reza Ay
- Department of Medical Physics and Biomedical Engineering, Tehran University of Medical Sciences, Tehran, Iran
- Research Center for Molecular and Cellular Imaging, Tehran University of Medical Sciences, Tehran, Iran
| |
Collapse
|
2
|
Zhang X, Iqbal Bin Saripan M, Wu Y, Wang Z, Wen D, Cao Z, Wang B, Xu S, Liu Y, Marhaban MH, Dong X. The impact of the combat method on radiomics feature compensation and analysis of scanners from different manufacturers. BMC Med Imaging 2024; 24:137. [PMID: 38844854 PMCID: PMC11157873 DOI: 10.1186/s12880-024-01306-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2023] [Accepted: 05/23/2024] [Indexed: 06/09/2024] Open
Abstract
BACKGROUND This study investigated whether the Combat compensation method can remove the variability of radiomic features extracted from different scanners, while also examining its impact on the subsequent predictive performance of machine learning models. MATERIALS AND METHODS 135 CT images of Credence Cartridge Radiomic phantoms were collected and screened from three scanners manufactured by Siemens, Philips, and GE. 100 radiomic features were extracted and 20 radiomic features were screened according to the Lasso regression method. The radiomic features extracted from the rubber and resin-filled regions in the cartridges were labeled into different categories for evaluating the performance of the machine learning model. Radiomics features were divided into three groups based on the different scanner manufacturers. The radiomic features were randomly divided into training and test sets with a ratio of 8:2. Five machine learning models (lasso, logistic regression, random forest, support vector machine, neural network) were employed to evaluate the impact of Combat on radiomic features. The variability among radiomic features were assessed using analysis of variance (ANOVA) and principal component analysis (PCA). Accuracy, precision, recall, and area under the receiver curve (AUC) were used as evaluation metrics for model classification. RESULTS The principal component and ANOVA analysis results show that the variability of different scanner manufacturers in radiomic features was removed (P˃0.05). After harmonization with the Combat algorithm, the distributions of radiomic features were aligned in terms of location and scale. The performance of machine learning models for classification improved, with the Random Forest model showing the most significant enhancement. The AUC value increased from 0.88 to 0.92. CONCLUSIONS The Combat algorithm has reduced variability in radiomic features from different scanners. In the phantom CT dataset, it appears that the machine learning model's classification performance may have improved after Combat harmonization. However, further investigation and validation are required to fully comprehend Combat's impact on radiomic features in medical imaging.
Collapse
Affiliation(s)
- Xiaolei Zhang
- Faculty of Engineering, Universiti Putra Malaysia, Serdang, Malaysia.
- Hebei International Research Center of Medical Engineering, Chengde Medical University, Chengde City, Hebei Province, China.
- Department of Biomedical Engineering, Chengde Medical University, Chengde City, Hebei Province, China.
| | | | - Yanjun Wu
- Hebei International Research Center of Medical Engineering, Chengde Medical University, Chengde City, Hebei Province, China
| | - Zhongxiao Wang
- Hebei International Research Center of Medical Engineering, Chengde Medical University, Chengde City, Hebei Province, China
| | - Dong Wen
- Institute of Artificial Intelligence, University of Science and Technology Beijing, Beijing, China
| | - Zhendong Cao
- Department of Radiology, the Affiliated Hospital of Chengde Medical University, Chengde, Hebei, China
| | - Bingzhen Wang
- Faculty of Engineering, Universiti Putra Malaysia, Serdang, Malaysia
- Hebei International Research Center of Medical Engineering, Chengde Medical University, Chengde City, Hebei Province, China
| | - Shiqi Xu
- Hebei International Research Center of Medical Engineering, Chengde Medical University, Chengde City, Hebei Province, China
| | - Yanli Liu
- Hebei International Research Center of Medical Engineering, Chengde Medical University, Chengde City, Hebei Province, China
| | | | - Xianling Dong
- Hebei International Research Center of Medical Engineering, Chengde Medical University, Chengde City, Hebei Province, China.
- Hebei Provincial Key Laboratory of Nerve Injury and Repair, Chengde Medical University, Chengde City, Hebei Province, China.
| |
Collapse
|
3
|
Scapicchio C, Imbriani M, Lizzi F, Quattrocchi M, Retico A, Saponaro S, Tenerani MI, Tofani A, Zafaranchi A, Fantacci ME. Investigation of a potential upstream harmonization based on image appearance matching to improve radiomics features robustness: a phantom study. Biomed Phys Eng Express 2024; 10:045006. [PMID: 38653209 DOI: 10.1088/2057-1976/ad41e7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 04/23/2024] [Indexed: 04/25/2024]
Abstract
Objective. Radiomics is a promising valuable analysis tool consisting in extracting quantitative information from medical images. However, the extracted radiomics features are too sensitive to variations in used image acquisition and reconstruction parameters. This limited robustness hinders the generalizable validity of radiomics-assisted models. Our aim is to investigate a possible harmonization strategy based on matching image quality to improve feature robustness.Approach.We acquired CT scans of a phantom with two scanners across different dose levels and percentages of Iterative Reconstruction algorithms. The detectability index was used as a comprehensive task-based image quality metric. A statistical analysis based on the Intraclass Correlation Coefficient was performed to determine if matching image quality/appearance could enhance the robustness of radiomics features extracted from the phantom images. Additionally, an Artificial Neural Network was trained on these features to automatically classify the scanner used for image acquisition.Main results.We found that the ICC of the features across protocols providing a similar detectability index improves with respect to the ICC of the features across protocols providing a different detectability index. This improvement was particularly noticeable in features relevant for distinguishing between scanners.Significance.This preliminary study demonstrates that a harmonization based on image quality/appearance matching could improve radiomics features robustness and heterogeneous protocols can be used to obtain a similar image appearance in terms of the detectability index. Thus protocols with a lower dose level could be selected to reduce the amount of radiation dose delivered to the patient and simultaneously obtain a more robust quantitative analysis.
Collapse
Affiliation(s)
- Camilla Scapicchio
- Department of Physics, University of Pisa, Pisa, Italy
- National Institute for Nuclear Physics, Pisa Division, Italy
| | | | - Francesca Lizzi
- National Institute for Nuclear Physics, Pisa Division, Italy
| | | | | | - Sara Saponaro
- National Institute for Nuclear Physics, Pisa Division, Italy
| | - Maria Irene Tenerani
- Department of Physics, University of Pisa, Pisa, Italy
- National Institute for Nuclear Physics, Pisa Division, Italy
| | - Alessandro Tofani
- Medical Physics Department, Azienda Toscana Nord Ovest Area Nord, Lucca, Italy
| | - Arman Zafaranchi
- Department of Physics, University of Pisa, Pisa, Italy
- National Institute for Nuclear Physics, Pisa Division, Italy
- Department of Computer Science, University of Pisa, Pisa, Italy
| | - Maria Evelina Fantacci
- Department of Physics, University of Pisa, Pisa, Italy
- National Institute for Nuclear Physics, Pisa Division, Italy
| |
Collapse
|
4
|
Thulasi Seetha S, Garanzini E, Tenconi C, Marenghi C, Avuzzi B, Catanzaro M, Stagni S, Villa S, Chiorda BN, Badenchini F, Bertocchi E, Sanduleanu S, Pignoli E, Procopio G, Valdagni R, Rancati T, Nicolai N, Messina A. Stability of Multi-Parametric Prostate MRI Radiomic Features to Variations in Segmentation. J Pers Med 2023; 13:1172. [PMID: 37511785 PMCID: PMC10381192 DOI: 10.3390/jpm13071172] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Revised: 07/13/2023] [Accepted: 07/18/2023] [Indexed: 07/30/2023] Open
Abstract
Stability analysis remains a fundamental step in developing a successful imaging biomarker to personalize oncological strategies. This study proposes an in silico contour generation method for simulating segmentation variations to identify stable radiomic features. Ground-truth annotation provided for the whole prostate gland on the multi-parametric MRI sequences (T2w, ADC, and SUB-DCE) were perturbed to mimic segmentation differences observed among human annotators. In total, we generated 15 synthetic contours for a given image-segmentation pair. One thousand two hundred twenty-four unfiltered/filtered radiomic features were extracted applying Pyradiomics, followed by stability assessment using ICC(1,1). Stable features identified in the internal population were then compared with an external population to discover and report robust features. Finally, we also investigated the impact of a wide range of filtering strategies on the stability of features. The percentage of unfiltered (filtered) features that remained robust subjected to segmentation variations were T2w-36% (81%), ADC-36% (94%), and SUB-43% (93%). Our findings suggest that segmentation variations can significantly impact radiomic feature stability but can be mitigated by including pre-filtering strategies as part of the feature extraction pipeline.
Collapse
Affiliation(s)
- Sithin Thulasi Seetha
- Prostate Cancer Program, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (S.T.S.); (R.V.)
- Department of Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University, 6211 LK Maastricht, The Netherlands
| | - Enrico Garanzini
- Department of Radiology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (E.G.); (A.M.)
| | - Chiara Tenconi
- Department of Medical Physics, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy;
- Department of Oncology and Hematooncology, Università degli Studi di Milano, 20133 Milan, Italy
| | - Cristina Marenghi
- Unit of Genito-Urinary Medical Oncology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (C.M.); (F.B.); (E.B.); (G.P.)
| | - Barbara Avuzzi
- Department of Radiation Oncology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (B.A.); (S.V.); (B.N.C.)
| | - Mario Catanzaro
- Department of Urology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (M.C.); (S.S.); (N.N.)
| | - Silvia Stagni
- Department of Urology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (M.C.); (S.S.); (N.N.)
| | - Sergio Villa
- Department of Radiation Oncology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (B.A.); (S.V.); (B.N.C.)
| | - Barbara Noris Chiorda
- Department of Radiation Oncology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (B.A.); (S.V.); (B.N.C.)
| | - Fabio Badenchini
- Unit of Genito-Urinary Medical Oncology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (C.M.); (F.B.); (E.B.); (G.P.)
| | - Elena Bertocchi
- Unit of Genito-Urinary Medical Oncology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (C.M.); (F.B.); (E.B.); (G.P.)
| | - Sebastian Sanduleanu
- Department of Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University, 6211 LK Maastricht, The Netherlands
| | - Emanuele Pignoli
- Department of Medical Physics, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy;
| | - Giuseppe Procopio
- Unit of Genito-Urinary Medical Oncology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (C.M.); (F.B.); (E.B.); (G.P.)
| | - Riccardo Valdagni
- Prostate Cancer Program, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (S.T.S.); (R.V.)
- Department of Oncology and Hematooncology, Università degli Studi di Milano, 20133 Milan, Italy
| | - Tiziana Rancati
- Data Science Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy
| | - Nicola Nicolai
- Department of Urology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (M.C.); (S.S.); (N.N.)
| | - Antonella Messina
- Department of Radiology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (E.G.); (A.M.)
| |
Collapse
|
5
|
Zhang YP, Zhang XY, Cheng YT, Li B, Teng XZ, Zhang J, Lam S, Zhou T, Ma ZR, Sheng JB, Tam VCW, Lee SWY, Ge H, Cai J. Artificial intelligence-driven radiomics study in cancer: the role of feature engineering and modeling. Mil Med Res 2023; 10:22. [PMID: 37189155 DOI: 10.1186/s40779-023-00458-8] [Citation(s) in RCA: 17] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Accepted: 05/04/2023] [Indexed: 05/17/2023] Open
Abstract
Modern medicine is reliant on various medical imaging technologies for non-invasively observing patients' anatomy. However, the interpretation of medical images can be highly subjective and dependent on the expertise of clinicians. Moreover, some potentially useful quantitative information in medical images, especially that which is not visible to the naked eye, is often ignored during clinical practice. In contrast, radiomics performs high-throughput feature extraction from medical images, which enables quantitative analysis of medical images and prediction of various clinical endpoints. Studies have reported that radiomics exhibits promising performance in diagnosis and predicting treatment responses and prognosis, demonstrating its potential to be a non-invasive auxiliary tool for personalized medicine. However, radiomics remains in a developmental phase as numerous technical challenges have yet to be solved, especially in feature engineering and statistical modeling. In this review, we introduce the current utility of radiomics by summarizing research on its application in the diagnosis, prognosis, and prediction of treatment responses in patients with cancer. We focus on machine learning approaches, for feature extraction and selection during feature engineering and for imbalanced datasets and multi-modality fusion during statistical modeling. Furthermore, we introduce the stability, reproducibility, and interpretability of features, and the generalizability and interpretability of models. Finally, we offer possible solutions to current challenges in radiomics research.
Collapse
Affiliation(s)
- Yuan-Peng Zhang
- Department of Medical Informatics, Nantong University, Nantong, 226001, Jiangsu, China
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
- The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen, 518000, Guangdong, China
| | - Xin-Yun Zhang
- Department of Medical Informatics, Nantong University, Nantong, 226001, Jiangsu, China
| | - Yu-Ting Cheng
- Department of Medical Informatics, Nantong University, Nantong, 226001, Jiangsu, China
| | - Bing Li
- Department of Radiation Oncology, the Affiliated Cancer Hospital of Zhengzhou University and Henan Cancer Hospital, Zhengzhou, 450008, Henan, China
| | - Xin-Zhi Teng
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Jiang Zhang
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Saikit Lam
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Ta Zhou
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Zong-Rui Ma
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Jia-Bao Sheng
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Victor C W Tam
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Shara W Y Lee
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Hong Ge
- Department of Radiation Oncology, the Affiliated Cancer Hospital of Zhengzhou University and Henan Cancer Hospital, Zhengzhou, 450008, Henan, China
| | - Jing Cai
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China.
- The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen, 518000, Guangdong, China.
| |
Collapse
|
6
|
External validation of an MR-based radiomic model predictive of locoregional control in oropharyngeal cancer. Eur Radiol 2023; 33:2850-2860. [PMID: 36460924 DOI: 10.1007/s00330-022-09255-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2021] [Revised: 09/27/2022] [Accepted: 10/02/2022] [Indexed: 12/05/2022]
Abstract
OBJECTIVES To externally validate a pre-treatment MR-based radiomics model predictive of locoregional control in oropharyngeal squamous cell carcinoma (OPSCC) and to assess the impact of differences between datasets on the predictive performance. METHODS Radiomic features, as defined in our previously published radiomics model, were extracted from the primary tumor volumes of 157 OPSCC patients in a different institute. The developed radiomics model was validated using this cohort. Additionally, parameters influencing performance, such as patient subgroups, MRI acquisition, and post-processing steps on prediction performance will be investigated. For this analysis, matched subgroups (based on human papillomavirus (HPV) status of the tumor, T-stage, and tumor subsite) and a subgroup with only patients with 4-mm slice thickness were studied. Also the influence of harmonization techniques (ComBat harmonization, quantile normalization) and the impact of feature stability across observers and centers were studied. Model performances were assessed by area under the curve (AUC), sensitivity, and specificity. RESULTS Performance of the published model (AUC/sensitivity/specificity: 0.74/0.75/0.60) drops when applied on the validation cohort (AUC/sensitivity/specificity: 0.64/0.68/0.60). The performance of the full validation cohort improves slightly when the model is validated using a patient group with comparable HPV status of the tumor (AUC/sensitivity/specificity: 0.68/0.74/0.60), using patients acquired with a slice thickness of 4 mm (AUC/sensitivity/specificity: 0.67/0.73/0.57), or when quantile harmonization was performed (AUC/sensitivity/specificity: 0.66/0.69/0.60). CONCLUSION The previously published model shows its generalizability and can be applied on data acquired from different vendors and protocols. Harmonization techniques and subgroup definition influence performance of predictive radiomics models. KEY POINTS • Radiomics, a noninvasive quantitative image analysis technique, can support the radiologist by enhancing diagnostic accuracy and/or treatment decision-making. • A previously published model shows its generalizability and could be applied on data acquired from different vendors and protocols.
Collapse
|
7
|
Deep learning-based harmonization of CT reconstruction kernels towards improved clinical task performance. Eur Radiol 2023; 33:2426-2438. [PMID: 36355196 DOI: 10.1007/s00330-022-09229-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2022] [Revised: 08/11/2022] [Accepted: 10/09/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVES To develop a deep learning-based harmonization framework, assessing whether it can improve performance of radiomics models given different kernels in different clinical tasks and additionally generalize to mitigate the effects of new/unobserved kernels on radiomics features. METHODS Patient data with 2 reconstruction kernels and phantom data with 22 reconstruction kernels were included. Eighty-five patients were studied for lymph node metastasis (LNM) prediction, and 164 patients for differential diagnosis between lung cancer (LC) and pulmonary tuberculosis (TB). Two convolutional neural network (CNN) models were developed to convert images (i) from B70f to B30f (CNNa) and (ii) from B30f to B70f (CNNb). Model performance between the two kernels was evaluated using AUC and compared with other well-known harmonization methods. Patient-normalized feature difference (PNFD) was used to identify the incompatible kernels (i.e., kernel with median PNFD > 1) with baseline (B30f/B70f), and measure the ability of the CNN models to convert the non-comparable kernels. RESULTS For LC versus pulmonary TB diagnosis, AUCs of CNNa vs. others were 0.85 vs. 0.54-0.74 (p = 0.0001-0.0003), and for CNNb vs. others: 0.87 vs. 0.54-0.86 (p = 0.0001-0.55). For LNM prediction, AUCs of CNNa vs. others were 0.68 vs. 0.56-0.61 (p = 0.10-0.39), and for CNNb vs. others: 0.78 vs. 0.70-0.73 (p = 0.07-0.40). After CNN harmonization, 17 of 20 (85%) of investigated unknown kernels produced comparable radiomics feature values relative to baseline (median PNFD from 1.10-2.31 to 0.23-1.13). CONCLUSION The CNN harmonization effectively improved performance of radiomics models between reconstruction kernels in different clinical tasks, and reduced feature differences between unknown kernels vs. baseline. KEY POINTS • The soft (B30f) and sharp (B70f) kernels strongly affect radiomics reproducibility and generalizability. • The convolutional neural network (CNN) harmonization methods performed better than location-scale (ComBat and centering-scaling) and matrix factorization harmonization methods (based on singular value decomposition (SVD) and independent component analysis (ICA)) in both clinical tasks. • The CNN harmonization methods improve feature reproducibility not only between specific kernels (B30f and B70f) from the same scanner, but also between unobserved kernels from different scanners of different vendors.
Collapse
|
8
|
A Framework of Analysis to Facilitate the Harmonization of Multicenter Radiomic Features in Prostate Cancer. J Clin Med 2022; 12:jcm12010140. [PMID: 36614941 PMCID: PMC9821561 DOI: 10.3390/jcm12010140] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2022] [Revised: 12/16/2022] [Accepted: 12/21/2022] [Indexed: 12/28/2022] Open
Abstract
Pooling radiomic features coming from different centers in a statistical framework is challenging due to the variability in scanner models, acquisition protocols, and reconstruction settings. To remove technical variability, commonly called batch effects, different statistical harmonization strategies have been widely used in genomics but less considered in radiomics. The aim of this work was to develop a framework of analysis to facilitate the harmonization of multicenter radiomic features extracted from prostate T2-weighted magnetic resonance imaging (MRI) and to improve the power of radiomics for prostate cancer (PCa) management in order to develop robust non-invasive biomarkers translating into clinical practice. To remove technical variability and correct for batch effects, we investigated four different statistical methods (ComBat, SVA, Arsynseq, and mixed effect). The proposed approaches were evaluated using a dataset of 210 prostate cancer (PCa) patients from two centers. The impacts of the different statistical approaches were evaluated by principal component analysis and classification methods (LogitBoost, random forest, K-nearest neighbors, and decision tree). The ComBat method outperformed all other methods by achieving 70% accuracy and 78% AUC with the random forest method to automatically classify patients affected by PCa. The proposed statistical framework enabled us to define and develop a standardized pipeline of analysis to harmonize multicenter T2W radiomic features, yielding great promise to support PCa clinical practice.
Collapse
|
9
|
Singh A, Horng H, Chitalia R, Roshkovan L, Katz SI, Noël P, Shinohara RT, Kontos D. Resampling and harmonization for mitigation of heterogeneity in image parameters of baseline scans. Sci Rep 2022; 12:21505. [PMID: 36513760 PMCID: PMC9747915 DOI: 10.1038/s41598-022-26083-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Accepted: 12/09/2022] [Indexed: 12/15/2022] Open
Abstract
Our study investigates the effects of heterogeneity in image parameters on the reproducibility of prognostic performance of models built using radiomic biomarkers. We compare the prognostic performance of models derived from the heterogeneity-mitigated features with that of models obtained from raw features, to assess whether reproducibility of prognostic scores improves upon application of our methods. We used two datasets: The Breast I-SPY1 dataset-Baseline DCE-MRI scans of 156 women with locally advanced breast cancer, treated with neoadjuvant chemotherapy, publicly available via The Cancer Imaging Archive (TCIA); The NSCLC IO dataset-Baseline CT scans of 107 patients with stage 4 non-small cell lung cancer (NSCLC), treated with pembrolizumab immunotherapy at our institution. Radiomic features (n = 102) are extracted from the tumor ROIs. We use a variety of resampling and harmonization scenarios to mitigate the heterogeneity in image parameters. The patients were divided into groups based on batch variables. For each group, the radiomic phenotypes are combined with the clinical covariates into a prognostic model. The performance of the groups is assessed using the c-statistic, derived from a Cox proportional hazards model fitted on all patients within a group. The heterogeneity-mitigation scenario (radiomic features, derived from images that have been resampled to minimum voxel spacing, are harmonized using the image acquisition parameters as batch variables) gave models with highest prognostic scores (for e.g., IO dataset; batch variable: high kernel resolution-c-score: 0.66). The prognostic performance of patient groups is not comparable in case of models built using non-heterogeneity mitigated features (for e.g., I-SPY1 dataset; batch variable: small pixel spacing-c-score: 0.54, large pixel spacing-c-score: 0.65). The prognostic performance of patient groups is closer in case of heterogeneity-mitigated scenarios (for e.g., scenario-harmonize by voxel spacing parameters: IO dataset; thin slice-c-score: 0.62, thick slice-c-score: 0.60). Our results indicate that accounting for heterogeneity in image parameters is important to obtain more reproducible prognostic scores, irrespective of image site or modality. For non-heterogeneity mitigated models, the prognostic scores are not comparable across patient groups divided based on batch variables. This study can be a step in the direction of constructing reproducible radiomic biomarkers, thus increasing their application in clinical decision making.
Collapse
Affiliation(s)
- Apurva Singh
- grid.25879.310000 0004 1936 8972Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104 USA ,grid.25879.310000 0004 1936 8972Department of Bioengineering, University of Pennsylvania, Philadelphia, PA 19104 USA
| | - Hannah Horng
- grid.25879.310000 0004 1936 8972Department of Bioengineering, University of Pennsylvania, Philadelphia, PA 19104 USA
| | - Rhea Chitalia
- grid.25879.310000 0004 1936 8972Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104 USA ,grid.25879.310000 0004 1936 8972Department of Bioengineering, University of Pennsylvania, Philadelphia, PA 19104 USA
| | - Leonid Roshkovan
- grid.25879.310000 0004 1936 8972Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104 USA
| | - Sharyn I. Katz
- grid.25879.310000 0004 1936 8972Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104 USA
| | - Peter Noël
- grid.25879.310000 0004 1936 8972Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104 USA
| | - Russell T. Shinohara
- grid.25879.310000 0004 1936 8972Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania, Philadelphia, PA 19104 USA
| | - Despina Kontos
- grid.25879.310000 0004 1936 8972Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104 USA ,grid.25879.310000 0004 1936 8972Department of Radiology, Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Rm D702 Richards Bldg., 3700 Hamilton Walk, Philadelphia, PA 19104 USA
| |
Collapse
|
10
|
Stamoulou E, Spanakis C, Manikis GC, Karanasiou G, Grigoriadis G, Foukakis T, Tsiknakis M, Fotiadis DI, Marias K. Harmonization Strategies in Multicenter MRI-Based Radiomics. J Imaging 2022; 8:303. [PMID: 36354876 PMCID: PMC9695920 DOI: 10.3390/jimaging8110303] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Revised: 10/28/2022] [Accepted: 10/31/2022] [Indexed: 08/13/2023] Open
Abstract
Radiomics analysis is a powerful tool aiming to provide diagnostic and prognostic patient information directly from images that are decoded into handcrafted features, comprising descriptors of shape, size and textural patterns. Although radiomics is gaining momentum since it holds great promise for accelerating digital diagnostics, it is susceptible to bias and variation due to numerous inter-patient factors (e.g., patient age and gender) as well as inter-scanner ones (different protocol acquisition depending on the scanner center). A variety of image and feature based harmonization methods has been developed to compensate for these effects; however, to the best of our knowledge, none of these techniques has been established as the most effective in the analysis pipeline so far. To this end, this review provides an overview of the challenges in optimizing radiomics analysis, and a concise summary of the most relevant harmonization techniques, aiming to provide a thorough guide to the radiomics harmonization process.
Collapse
Affiliation(s)
- Elisavet Stamoulou
- Computational BioMedicine Laboratory (CBML), Foundation for Research and Technology—Hellas (FORTH), 700 13 Heraklion, Greece
| | - Constantinos Spanakis
- Computational BioMedicine Laboratory (CBML), Foundation for Research and Technology—Hellas (FORTH), 700 13 Heraklion, Greece
| | - Georgios C. Manikis
- Computational BioMedicine Laboratory (CBML), Foundation for Research and Technology—Hellas (FORTH), 700 13 Heraklion, Greece
- Department of Oncology-Pathology, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Georgia Karanasiou
- Unit of Medical Technology and Intelligent Information Systems, Department of Materials Science and Engineering, University of Ioannina, 451 10 Ioannina, Greece
| | - Grigoris Grigoriadis
- Unit of Medical Technology and Intelligent Information Systems, Department of Materials Science and Engineering, University of Ioannina, 451 10 Ioannina, Greece
| | - Theodoros Foukakis
- Department of Oncology-Pathology, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Manolis Tsiknakis
- Computational BioMedicine Laboratory (CBML), Foundation for Research and Technology—Hellas (FORTH), 700 13 Heraklion, Greece
- Department of Electrical & Computer Engineering, Hellenic Mediterranean University, 714 10 Heraklion, Greece
| | - Dimitrios I. Fotiadis
- Unit of Medical Technology and Intelligent Information Systems, Department of Materials Science and Engineering, University of Ioannina, 451 10 Ioannina, Greece
- Department of Biomedical Research, Institute of Molecular Biology and Biotechnology—FORTH, University Campus of Ioannina, 451 15 Ioannina, Greece
| | - Kostas Marias
- Computational BioMedicine Laboratory (CBML), Foundation for Research and Technology—Hellas (FORTH), 700 13 Heraklion, Greece
- Department of Electrical & Computer Engineering, Hellenic Mediterranean University, 714 10 Heraklion, Greece
| |
Collapse
|
11
|
Paquier Z, Chao SL, Bregni G, Sanchez AV, Guiot T, Dhont J, Gulyban A, Levillain H, Sclafani F, Reynaert N, Bali MA. Pre-trial quality assurance of diffusion-weighted MRI for radiomic analysis and the role of harmonisation. Phys Med 2022; 103:138-146. [DOI: 10.1016/j.ejmp.2022.10.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/16/2022] [Revised: 09/30/2022] [Accepted: 10/08/2022] [Indexed: 11/17/2022] Open
|
12
|
Ibrahim A, Lu L, Yang H, Akin O, Schwartz LH, Zhao B. The Impact of Image Acquisition Parameters and ComBat Harmonization on the Predictive Performance of Radiomics: A Renal Cell Carcinoma Model. APPLIED SCIENCES (BASEL, SWITZERLAND) 2022; 12:9824. [PMID: 37091743 PMCID: PMC10121203 DOI: 10.3390/app12199824] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 04/25/2023]
Abstract
Radiomics, one of the potential methods for developing clinical biomarker, is one of the exponentially growing research fields. In addition to its potential, several limitations have been identified in this field, and most importantly the effects of variations in imaging parameters on radiomic features (RFs). In this study, we investigate the potential of RFs to predict overall survival in patients with clear cell renal cell carcinoma, as well as the impact of ComBat harmonization on the performance of RF models. We assessed the robustness of the results by performing the analyses a thousand times. Publicly available CT scans of 179 patients were retrospectively collected and analyzed. The scans were acquired using different imaging vendors and parameters in different medical centers. The performance was calculated by averaging the metrics over all runs. On average, the clinical model significantly outperformed the radiomic models. The use of ComBat harmonization, on average, did not significantly improve the performance of radiomic models. Hence, the variability in image acquisition and reconstruction parameters significantly affect the performance of radiomic models. The development of radiomic specific harmonization techniques remain a necessity for the advancement of the field.
Collapse
Affiliation(s)
- Abdalla Ibrahim
- Department of Radiology, Columbia University Irving Medical Center, New York, NY 10032, USA
- Correspondence:
| | - Lin Lu
- Department of Radiology, Columbia University Irving Medical Center, New York, NY 10032, USA
| | - Hao Yang
- Department of Radiology, Columbia University Irving Medical Center, New York, NY 10032, USA
| | - Oguz Akin
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY 10065, USA
| | - Lawrence H. Schwartz
- Department of Radiology, Columbia University Irving Medical Center, New York, NY 10032, USA
| | - Binsheng Zhao
- Department of Radiology, Columbia University Irving Medical Center, New York, NY 10032, USA
| |
Collapse
|
13
|
Keek SA, Kayan E, Chatterjee A, Belderbos JSA, Bootsma G, van den Borne B, Dingemans AMC, Gietema HA, Groen HJM, Herder J, Pitz C, Praag J, De Ruysscher D, Schoenmaekers J, Smit HJM, Stigt J, Westenend M, Zeng H, Woodruff HC, Lambin P, Hendriks L. Investigation of the added value of CT-based radiomics in predicting the development of brain metastases in patients with radically treated stage III NSCLC. Ther Adv Med Oncol 2022; 14:17588359221116605. [PMID: 36032350 PMCID: PMC9403451 DOI: 10.1177/17588359221116605] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2022] [Accepted: 07/12/2022] [Indexed: 12/04/2022] Open
Abstract
Introduction: Despite radical intent therapy for patients with stage III non-small-cell
lung cancer (NSCLC), cumulative incidence of brain metastases (BM) reaches
30%. Current risk stratification methods fail to accurately identify these
patients. As radiomics features have been shown to have predictive value,
this study aims to develop a model combining clinical risk factors with
radiomics features for BM development in patients with radically treated
stage III NSCLC. Methods: Retrospective analysis of two prospective multicentre studies. Inclusion
criteria: adequately staged [18F-fluorodeoxyglucose positron
emission tomography-computed tomography (18-FDG-PET-CT), contrast-enhanced
chest CT, contrast-enhanced brain magnetic resonance imaging/CT] and
radically treated stage III NSCLC, exclusion criteria: second primary within
2 years of NSCLC diagnosis and prior prophylactic cranial irradiation.
Primary endpoint was BM development any time during follow-up (FU). CT-based
radiomics features (N = 530) were extracted from the
primary lung tumour on 18-FDG-PET-CT images, and a list of clinical features
(N = 8) was collected. Univariate feature selection
based on the area under the curve (AUC) of the receiver operating
characteristic was performed to identify relevant features. Generalized
linear models were trained using the selected features, and multivariate
predictive performance was assessed through the AUC. Results: In total, 219 patients were eligible for analysis. Median FU was 59.4 months
for the training cohort and 67.3 months for the validation cohort; 21 (15%)
and 17 (22%) patients developed BM in the training and validation cohort,
respectively. Two relevant clinical features (age and adenocarcinoma
histology) and four relevant radiomics features were identified as
predictive. The clinical model yielded the highest AUC value of 0.71 (95%
CI: 0.58–0.84), better than radiomics or a combination of clinical
parameters and radiomics (both an AUC of 0.62, 95% CIs of 0.47–076 and
0.48–0.76, respectively). Conclusion: CT-based radiomics features of primary NSCLC in the current setup could not
improve on a model based on clinical predictors (age and adenocarcinoma
histology) of BM development in radically treated stage III NSCLC
patients.
Collapse
Affiliation(s)
- Simon A Keek
- The D-Lab, Department of Precision Medicine, GROW - School for Oncology and Reproduction, Maastricht University, Maastricht, The Netherlands
| | - Esma Kayan
- The D-Lab, Department of Precision Medicine, GROW - School for Oncology and Reproduction, Maastricht University, Maastricht, The Netherlands
| | - Avishek Chatterjee
- The D-Lab, Department of Precision Medicine, GROW - School for Oncology and Reproduction, Maastricht University, Maastricht, The Netherlands
| | - José S A Belderbos
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Gerben Bootsma
- Department of Pulmonary Diseases, Zuyderland Hospital, Heerlen, The Netherlands
| | - Ben van den Borne
- Department of Pulmonary Diseases, Catharina Hospital, Eindhoven, The Netherlands
| | | | - Hester A Gietema
- Department of Radiology and Nuclear Medicine, GROW - School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Harry J M Groen
- Department of Pulmonary Diseases, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
| | - Judith Herder
- Department of Pulmonary Diseases, Meander Medical Center, Amersfoort, The Netherlands
| | - Cordula Pitz
- Department of Pulmonary Diseases, Laurentius Hospital, Roermond, The Netherlands
| | - John Praag
- Department of Radiotherapy, Erasmus MC, Rotterdam, The Netherlands
| | - Dirk De Ruysscher
- Department of Radiation Oncology (Maastro), GROW - School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Janna Schoenmaekers
- Department of Pulmonary Diseases, GROW - School for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
| | - Hans J M Smit
- Department of Pulmonary Diseases, Rijnstate, Arnhem, The Netherlands
| | - Jos Stigt
- Department of Pulmonary Diseases, Isala Hospital, Zwolle, The Netherlands
| | - Marcel Westenend
- Department of Pulmonary Diseases, VieCuri, Venlo, The Netherlands
| | - Haiyan Zeng
- Department of Radiation Oncology (Maastro), GROW - School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Henry C Woodruff
- The D-Lab, Department of Precision Medicine, GROW - School for Oncology and Reproduction, Maastricht University, Maastricht, The Netherlands
| | - Philippe Lambin
- The D-Lab, Department of Precision Medicine, GROW - School for Oncology and Reproduction, Maastricht University, Maastricht, The Netherlands
| | - Lizza Hendriks
- Department of Pulmonary Diseases, GROW - School for Oncology and Reproduction, Maastricht University Medical Centre+, P.O. Box 5800, 6202 AZ, Maastricht, The Netherlands
| |
Collapse
|
14
|
Cui Y, Yin FF. Impact of image quality on radiomics applications. Phys Med Biol 2022; 67. [DOI: 10.1088/1361-6560/ac7fd7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2022] [Accepted: 07/08/2022] [Indexed: 11/12/2022]
Abstract
Abstract
Radiomics features extracted from medical images have been widely reported to be useful in the patient specific outcome modeling for variety of assessment and prediction purposes. Successful application of radiomics features as imaging biomarkers, however, is dependent on the robustness of the approach to the variation in each step of the modeling workflow. Variation in the input image quality is one of the main sources that impacts the reproducibility of radiomics analysis when a model is applied to broader range of medical imaging data. The quality of medical image is generally affected by both the scanner related factors such as image acquisition/reconstruction settings and the patient related factors such as patient motion. This article aimed to review the published literatures in this field that reported the impact of various imaging factors on the radiomics features through the change in image quality. The literatures were categorized by different imaging modalities and also tabulated based on the imaging parameters and the class of radiomics features included in the study. Strategies for image quality standardization were discussed based on the relevant literatures and recommendations for reducing the impact of image quality variation on the radiomics in multi-institutional clinical trial were summarized at the end of this article.
Collapse
|
15
|
Nan Y, Ser JD, Walsh S, Schönlieb C, Roberts M, Selby I, Howard K, Owen J, Neville J, Guiot J, Ernst B, Pastor A, Alberich-Bayarri A, Menzel MI, Walsh S, Vos W, Flerin N, Charbonnier JP, van Rikxoort E, Chatterjee A, Woodruff H, Lambin P, Cerdá-Alberich L, Martí-Bonmatí L, Herrera F, Yang G. Data harmonisation for information fusion in digital healthcare: A state-of-the-art systematic review, meta-analysis and future research directions. AN INTERNATIONAL JOURNAL ON INFORMATION FUSION 2022; 82:99-122. [PMID: 35664012 PMCID: PMC8878813 DOI: 10.1016/j.inffus.2022.01.001] [Citation(s) in RCA: 38] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/24/2021] [Revised: 12/22/2021] [Accepted: 01/07/2022] [Indexed: 05/13/2023]
Abstract
Removing the bias and variance of multicentre data has always been a challenge in large scale digital healthcare studies, which requires the ability to integrate clinical features extracted from data acquired by different scanners and protocols to improve stability and robustness. Previous studies have described various computational approaches to fuse single modality multicentre datasets. However, these surveys rarely focused on evaluation metrics and lacked a checklist for computational data harmonisation studies. In this systematic review, we summarise the computational data harmonisation approaches for multi-modality data in the digital healthcare field, including harmonisation strategies and evaluation metrics based on different theories. In addition, a comprehensive checklist that summarises common practices for data harmonisation studies is proposed to guide researchers to report their research findings more effectively. Last but not least, flowcharts presenting possible ways for methodology and metric selection are proposed and the limitations of different methods have been surveyed for future research.
Collapse
Affiliation(s)
- Yang Nan
- National Heart and Lung Institute, Imperial College London, London, Northern Ireland UK
| | - Javier Del Ser
- Department of Communications Engineering, University of the Basque Country UPV/EHU, Bilbao 48013, Spain
- TECNALIA, Basque Research and Technology Alliance (BRTA), Derio 48160, Spain
| | - Simon Walsh
- National Heart and Lung Institute, Imperial College London, London, Northern Ireland UK
| | - Carola Schönlieb
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, Northern Ireland UK
| | - Michael Roberts
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, Northern Ireland UK
- Oncology R&D, AstraZeneca, Cambridge, Northern Ireland UK
| | - Ian Selby
- Department of Radiology, University of Cambridge, Cambridge, Northern Ireland UK
| | - Kit Howard
- Clinical Data Interchange Standards Consortium, Austin, TX, United States of America
| | - John Owen
- Clinical Data Interchange Standards Consortium, Austin, TX, United States of America
| | - Jon Neville
- Clinical Data Interchange Standards Consortium, Austin, TX, United States of America
| | - Julien Guiot
- University Hospital of Liège (CHU Liège), Respiratory medicine department, Liège, Belgium
- University of Liege, Department of clinical sciences, Pneumology-Allergology, Liège, Belgium
| | - Benoit Ernst
- University Hospital of Liège (CHU Liège), Respiratory medicine department, Liège, Belgium
- University of Liege, Department of clinical sciences, Pneumology-Allergology, Liège, Belgium
| | | | | | - Marion I. Menzel
- Technische Hochschule Ingolstadt, Ingolstadt, Germany
- GE Healthcare GmbH, Munich, Germany
| | - Sean Walsh
- Radiomics (Oncoradiomics SA), Liège, Belgium
| | - Wim Vos
- Radiomics (Oncoradiomics SA), Liège, Belgium
| | - Nina Flerin
- Radiomics (Oncoradiomics SA), Liège, Belgium
| | | | | | - Avishek Chatterjee
- Department of Precision Medicine, Maastricht University, Maastricht, The Netherlands
| | - Henry Woodruff
- Department of Precision Medicine, Maastricht University, Maastricht, The Netherlands
| | - Philippe Lambin
- Department of Precision Medicine, Maastricht University, Maastricht, The Netherlands
| | - Leonor Cerdá-Alberich
- Medical Imaging Department, Hospital Universitari i Politècnic La Fe, Valencia, Spain
| | - Luis Martí-Bonmatí
- Medical Imaging Department, Hospital Universitari i Politècnic La Fe, Valencia, Spain
| | - Francisco Herrera
- Department of Computer Sciences and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI) University of Granada, Granada, Spain
- Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| | - Guang Yang
- National Heart and Lung Institute, Imperial College London, London, Northern Ireland UK
- Cardiovascular Research Centre, Royal Brompton Hospital, London, Northern Ireland UK
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, Northern Ireland UK
| |
Collapse
|
16
|
CT Reconstruction Kernels and the Effect of Pre- and Post-Processing on the Reproducibility of Handcrafted Radiomic Features. J Pers Med 2022; 12:jpm12040553. [PMID: 35455668 PMCID: PMC9030848 DOI: 10.3390/jpm12040553] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2022] [Revised: 03/27/2022] [Accepted: 03/28/2022] [Indexed: 02/06/2023] Open
Abstract
Handcrafted radiomics features (HRFs) are quantitative features extracted from medical images to decode biological information to improve clinical decision making. Despite the potential of the field, limitations have been identified. The most important identified limitation, currently, is the sensitivity of HRF to variations in image acquisition and reconstruction parameters. In this study, we investigated the use of Reconstruction Kernel Normalization (RKN) and ComBat harmonization to improve the reproducibility of HRFs across scans acquired with different reconstruction kernels. A set of phantom scans (n = 28) acquired on five different scanner models was analyzed. HRFs were extracted from the original scans, and scans were harmonized using the RKN method. ComBat harmonization was applied on both sets of HRFs. The reproducibility of HRFs was assessed using the concordance correlation coefficient. The difference in the number of reproducible HRFs in each scenario was assessed using McNemar’s test. The majority of HRFs were found to be sensitive to variations in the reconstruction kernels, and only six HRFs were found to be robust with respect to variations in reconstruction kernels. The use of RKN resulted in a significant increment in the number of reproducible HRFs in 19 out of the 67 investigated scenarios (28.4%), while the ComBat technique resulted in a significant increment in 36 (53.7%) scenarios. The combination of methods resulted in a significant increment in 53 (79.1%) scenarios compared to the HRFs extracted from original images. Since the benefit of applying the harmonization methods depended on the data being harmonized, reproducibility analysis is recommended before performing radiomics analysis. For future radiomics studies incorporating images acquired with similar image acquisition and reconstruction parameters, except for the reconstruction kernels, we recommend the systematic use of the pre- and post-processing approaches (respectively, RKN and ComBat).
Collapse
|
17
|
Ibrahim A, Barufaldi B, Refaee T, Silva Filho TM, Acciavatti RJ, Salahuddin Z, Hustinx R, Mottaghy FM, Maidment ADA, Lambin P. MaasPenn Radiomics Reproducibility Score: A Novel Quantitative Measure for Evaluating the Reproducibility of CT-Based Handcrafted Radiomic Features. Cancers (Basel) 2022; 14:cancers14071599. [PMID: 35406372 PMCID: PMC8997100 DOI: 10.3390/cancers14071599] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Revised: 03/16/2022] [Accepted: 03/18/2022] [Indexed: 02/04/2023] Open
Abstract
Simple Summary The reproducibility of handcrafted radiomic features (HRFs) has been reported to be affected by variations in imaging acquisition and reconstruction parameters. However, to date, these effects have not been understood or quantified. In this study, we analyzed a significantly large number of scenarios in an effort to quantify the effects of variations on the reproducibility of HRFs. In addition, we assessed the performance of ComBat harmonization in each of the 31,375 investigated scenarios. We developed a novel score that can be considered the first attempt to objectively assess the number of reproducible HRFs in different scenario. Following further validation, the score could be used to decide on the inclusion of data acquired differently, as well as the assessment of the generalizability of developed radiomic signatures. Abstract The reproducibility of handcrafted radiomic features (HRFs) has been reported to be affected by variations in imaging parameters, which significantly affect the generalizability of developed signatures and translation to clinical practice. However, the collective effect of the variations in imaging parameters on the reproducibility of HRFs remains unclear, with no objective measure to assess it in the absence of reproducibility analysis. We assessed these effects of variations in a large number of scenarios and developed the first quantitative score to assess the reproducibility of CT-based HRFs without the need for phantom or reproducibility studies. We further assessed the potential of image resampling and ComBat harmonization for removing these effects. Our findings suggest a need for radiomics-specific harmonization methods. Our developed score should be considered as a first attempt to introduce comprehensive metrics to quantify the reproducibility of CT-based handcrafted radiomic features. More research is warranted to demonstrate its validity in clinical contexts and to further improve it, possibly by the incorporation of more realistic situations, which better reflect real patients’ situations.
Collapse
Affiliation(s)
- Abdalla Ibrahim
- The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, 6229 ER Maastricht, The Netherlands; (T.R.); (Z.S.); (P.L.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center+, 6229 HX Maastricht, The Netherlands;
- Division of Nuclear Medicine and Oncological Imaging, Department of Medical Physics, CHU de Liege, CRC In Vivo Imaging, University of Liège, 4000 Liege, Belgium;
- Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
- Correspondence:
| | - Bruno Barufaldi
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; (B.B.); (R.J.A.); (A.D.A.M.)
| | - Turkey Refaee
- The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, 6229 ER Maastricht, The Netherlands; (T.R.); (Z.S.); (P.L.)
- Department of Diagnostic Radiology, Faculty of Applied Medical Sciences, Jazan University, Jazan 45142, Saudi Arabia
| | - Telmo M. Silva Filho
- Department of Statistics, Federal University of Paraíba, João Pessoa 58051-900, Brazil;
| | - Raymond J. Acciavatti
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; (B.B.); (R.J.A.); (A.D.A.M.)
| | - Zohaib Salahuddin
- The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, 6229 ER Maastricht, The Netherlands; (T.R.); (Z.S.); (P.L.)
| | - Roland Hustinx
- Division of Nuclear Medicine and Oncological Imaging, Department of Medical Physics, CHU de Liege, CRC In Vivo Imaging, University of Liège, 4000 Liege, Belgium;
| | - Felix M. Mottaghy
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center+, 6229 HX Maastricht, The Netherlands;
- Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
| | - Andrew D. A. Maidment
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; (B.B.); (R.J.A.); (A.D.A.M.)
| | - Philippe Lambin
- The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, 6229 ER Maastricht, The Netherlands; (T.R.); (Z.S.); (P.L.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center+, 6229 HX Maastricht, The Netherlands;
| |
Collapse
|
18
|
Granzier RWY, Ibrahim A, Primakov S, Keek SA, Halilaj I, Zwanenburg A, Engelen SME, Lobbes MBI, Lambin P, Woodruff HC, Smidt ML. Test-Retest Data for the Assessment of Breast MRI Radiomic Feature Repeatability. J Magn Reson Imaging 2021; 56:592-604. [PMID: 34936160 PMCID: PMC9544420 DOI: 10.1002/jmri.28027] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2021] [Revised: 12/03/2021] [Accepted: 12/03/2021] [Indexed: 12/14/2022] Open
Abstract
Background Radiomic features extracted from breast MRI have potential for diagnostic, prognostic, and predictive purposes. However, before they can be used as biomarkers in clinical decision support systems, features need to be repeatable and reproducible. Objective Identify repeatable radiomics features within breast tissue on prospectively collected MRI exams through multiple test–retest measurements. Study Type Prospective. Population 11 healthy female volunteers. Field Strength/Sequence 1.5 T; MRI exams, comprising T2‐weighted turbo spin‐echo (T2W) sequence, native T1‐weighted turbo gradient‐echo (T1W) sequence, diffusion‐weighted imaging (DWI) sequence using b‐values 0/150/800, and corresponding derived ADC maps. Assessment 18 MRI exams (three test–retest settings, repeated on 2 days) per healthy volunteer were examined on an identical scanner using a fixed clinical breast protocol. For each scan, 91 features were extracted from the 3D manually segmented right breast using Pyradiomics, before and after image preprocessing. Image preprocessing consisted of 1) bias field correction (BFC); 2) z‐score normalization with and without BFC; 3) grayscale discretization using 32 and 64 bins with and without BFC; and 4) z‐score normalization + grayscale discretization using 32 and 64 bins with and without BFC. Statistical Tests Features' repeatability was assessed using concordance correlation coefficient(CCC) for each pair, i.e. each MRI was compared to each of the remaining 17 MRI with a cut‐off value of CCC > 0.90. Results Images without preprocessing produced the highest number of repeatable features for both T1W sequence and ADC maps with 15 of 91 (16.5%) and 8 of 91 (8.8%) repeatable features, respectively. Preprocessed images produced between 4 of 91 (4.4%) and 14 of 91 (15.4%), and 6 of 91 (6.6%) and 7 of 91 (7.7%) repeatable features, respectively for T1W and ADC maps. Z‐score normalization produced highest number of repeatable features, 26 of 91 (28.6%) in T2W sequences, in these images, no preprocessing produced 11 of 91 (12.1%) repeatable features. Data Conclusion Radiomic features extracted from T1W, T2W sequences and ADC maps from breast MRI exams showed a varying number of repeatable features, depending on the sequence. Effects of different preprocessing procedures on repeatability of features were different for each sequence. Level of Evidence 2 Technical Efficacy Stage 1
Collapse
Affiliation(s)
- R W Y Granzier
- Department of Surgery, Maastricht University Medical Centre+, Maastricht, The Netherlands.,GROW - School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands
| | - A Ibrahim
- GROW - School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands.,Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands.,The D-Lab, Department of Precision Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands.,Division of Nuclear Medicine and Oncological Imaging, Department of Medical Physics, Hospital Center Universitaire De Liege, Liege, Belgium.,Department of Nuclear Medicine and Comprehensive diagnostic center Aachen (CDCA), University Hospital RWTH Aachen University, Aachen, Germany
| | - S Primakov
- GROW - School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands.,Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands.,The D-Lab, Department of Precision Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - S A Keek
- GROW - School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands.,The D-Lab, Department of Precision Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - I Halilaj
- GROW - School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands.,The D-Lab, Department of Precision Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands.,Health Innovation Ventures, Maastricht, The Netherlands
| | - A Zwanenburg
- OncoRay - National Center for Radiation Research in Oncology, Faculty of Medicine and University Hospital Carl Gustav Carus, Technische Universität Dresden, Helmholtz-Zentrum Dresden, Rossendorf, Dresden, Germany.,National Center for Tumor Diseases (NCT), Partner Site Dresden, Germany: German Cancer Research Center (DKFZ), Heidelberg, Germany.,Faculty of Medicine and University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany.,Helmholtz Association/Helmholtz-Zentrum Dresden - Rossendorf (HZDR), Dresden, Germany
| | - S M E Engelen
- Department of Surgery, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - M B I Lobbes
- GROW - School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands.,Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands.,Department of Medical Imaging, Zuyderland Medical Center, Sittard-Geleen, the Netherlands
| | - P Lambin
- GROW - School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands.,Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands.,The D-Lab, Department of Precision Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - H C Woodruff
- GROW - School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands.,Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands.,The D-Lab, Department of Precision Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - M L Smidt
- Department of Surgery, Maastricht University Medical Centre+, Maastricht, The Netherlands.,GROW - School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
19
|
Ibrahim A, Widaatalla Y, Refaee T, Primakov S, Miclea RL, Öcal O, Fabritius MP, Ingrisch M, Ricke J, Hustinx R, Mottaghy FM, Woodruff HC, Seidensticker M, Lambin P. Reproducibility of CT-Based Hepatocellular Carcinoma Radiomic Features across Different Contrast Imaging Phases: A Proof of Concept on SORAMIC Trial Data. Cancers (Basel) 2021; 13:cancers13184638. [PMID: 34572870 PMCID: PMC8468150 DOI: 10.3390/cancers13184638] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2021] [Revised: 09/11/2021] [Accepted: 09/13/2021] [Indexed: 12/17/2022] Open
Abstract
Simple Summary Radiomics has been reported to have potential for correlating with clinical outcomes. However, handcrafted radiomic features (HRFs)—the quantitative features extracted from medical images—are limited by their sensitivity to variations in scanning parameters. Furthermore, radiomics analyses require big data with good quality to achieve desirable performances. In this study, we investigated the reproducibility of HRFs between scans acquired with the same scanning parameters except for the imaging phase (arterial and portal venous phases) to assess the possibilities of merging scans from different phases or replacing missing scans from a phase with other phases to increase data entries. Additionally, we assessed the potential of ComBat harmonization to remove batch effects attributed to this variation. Our results show that the majority of HRFs were not reproducible between the arterial and portal venous phases before or after ComBat harmonization. We provide a guide for analyzing scans of different imaging phases. Abstract Handcrafted radiomic features (HRFs) are quantitative imaging features extracted from regions of interest on medical images which can be correlated with clinical outcomes and biologic characteristics. While HRFs have been used to train predictive and prognostic models, their reproducibility has been reported to be affected by variations in scan acquisition and reconstruction parameters, even within the same imaging vendor. In this work, we evaluated the reproducibility of HRFs across the arterial and portal venous phases of contrast-enhanced computed tomography images depicting hepatocellular carcinomas, as well as the potential of ComBat harmonization to correct for this difference. ComBat harmonization is a method based on Bayesian estimates that was developed for gene expression arrays, and has been investigated as a potential method for harmonizing HRFs. Our results show that the majority of HRFs are not reproducible between the arterial and portal venous imaging phases, yet a number of HRFs could be used interchangeably between those phases. Furthermore, ComBat harmonization increased the number of reproducible HRFs across both phases by 1%. Our results guide the pooling of arterial and venous phases from different patients in an effort to increase cohort size, as well as joint analysis of the phases.
Collapse
Affiliation(s)
- Abdalla Ibrahim
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, 6200 MD Maastricht, The Netherlands; (Y.W.); (S.P.); (H.C.W.); (P.L.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, 6200 MD Maastricht, The Netherlands; (R.L.M.); (F.M.M.)
- Division of Nuclear Medicine and Oncological Imaging, Department of Medical Physics, University Hospital of Liege and GIGA CRC-In Vivo Imaging, University of Liege, 4000 Liege, Belgium;
- Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
- Correspondence: (A.I.); (T.R.)
| | - Yousif Widaatalla
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, 6200 MD Maastricht, The Netherlands; (Y.W.); (S.P.); (H.C.W.); (P.L.)
| | - Turkey Refaee
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, 6200 MD Maastricht, The Netherlands; (Y.W.); (S.P.); (H.C.W.); (P.L.)
- Department of Diagnostic Radiology, Faculty of Applied Medical Sciences, Jazan University, Jazan 45142, Saudi Arabia
- Correspondence: (A.I.); (T.R.)
| | - Sergey Primakov
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, 6200 MD Maastricht, The Netherlands; (Y.W.); (S.P.); (H.C.W.); (P.L.)
- Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
| | - Razvan L. Miclea
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, 6200 MD Maastricht, The Netherlands; (R.L.M.); (F.M.M.)
| | - Osman Öcal
- Department of Radiology, University Hospital, LMU Munich, 80336 Munich, Germany; (O.Ö.); (M.P.F.); (M.I.); (J.R.); (M.S.)
| | - Matthias P. Fabritius
- Department of Radiology, University Hospital, LMU Munich, 80336 Munich, Germany; (O.Ö.); (M.P.F.); (M.I.); (J.R.); (M.S.)
| | - Michael Ingrisch
- Department of Radiology, University Hospital, LMU Munich, 80336 Munich, Germany; (O.Ö.); (M.P.F.); (M.I.); (J.R.); (M.S.)
| | - Jens Ricke
- Department of Radiology, University Hospital, LMU Munich, 80336 Munich, Germany; (O.Ö.); (M.P.F.); (M.I.); (J.R.); (M.S.)
| | - Roland Hustinx
- Division of Nuclear Medicine and Oncological Imaging, Department of Medical Physics, University Hospital of Liege and GIGA CRC-In Vivo Imaging, University of Liege, 4000 Liege, Belgium;
| | - Felix M. Mottaghy
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, 6200 MD Maastricht, The Netherlands; (R.L.M.); (F.M.M.)
- Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
| | - Henry C. Woodruff
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, 6200 MD Maastricht, The Netherlands; (Y.W.); (S.P.); (H.C.W.); (P.L.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, 6200 MD Maastricht, The Netherlands; (R.L.M.); (F.M.M.)
| | - Max Seidensticker
- Department of Radiology, University Hospital, LMU Munich, 80336 Munich, Germany; (O.Ö.); (M.P.F.); (M.I.); (J.R.); (M.S.)
| | - Philippe Lambin
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, 6200 MD Maastricht, The Netherlands; (Y.W.); (S.P.); (H.C.W.); (P.L.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, 6200 MD Maastricht, The Netherlands; (R.L.M.); (F.M.M.)
| |
Collapse
|
20
|
Mali SA, Ibrahim A, Woodruff HC, Andrearczyk V, Müller H, Primakov S, Salahuddin Z, Chatterjee A, Lambin P. Making Radiomics More Reproducible across Scanner and Imaging Protocol Variations: A Review of Harmonization Methods. J Pers Med 2021; 11:842. [PMID: 34575619 PMCID: PMC8472571 DOI: 10.3390/jpm11090842] [Citation(s) in RCA: 68] [Impact Index Per Article: 22.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2021] [Revised: 08/21/2021] [Accepted: 08/24/2021] [Indexed: 12/13/2022] Open
Abstract
Radiomics converts medical images into mineable data via a high-throughput extraction of quantitative features used for clinical decision support. However, these radiomic features are susceptible to variation across scanners, acquisition protocols, and reconstruction settings. Various investigations have assessed the reproducibility and validation of radiomic features across these discrepancies. In this narrative review, we combine systematic keyword searches with prior domain knowledge to discuss various harmonization solutions to make the radiomic features more reproducible across various scanners and protocol settings. Different harmonization solutions are discussed and divided into two main categories: image domain and feature domain. The image domain category comprises methods such as the standardization of image acquisition, post-processing of raw sensor-level image data, data augmentation techniques, and style transfer. The feature domain category consists of methods such as the identification of reproducible features and normalization techniques such as statistical normalization, intensity harmonization, ComBat and its derivatives, and normalization using deep learning. We also reflect upon the importance of deep learning solutions for addressing variability across multi-centric radiomic studies especially using generative adversarial networks (GANs), neural style transfer (NST) techniques, or a combination of both. We cover a broader range of methods especially GANs and NST methods in more detail than previous reviews.
Collapse
Affiliation(s)
- Shruti Atul Mali
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
| | - Abdalla Ibrahim
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
- Department of Radiology and Nuclear Medicine, GROW—School for Oncology, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands
- Department of Medical Physics, Division of Nuclear Medicine and Oncological Imaging, Hospital Center Universitaire de Liege, 4000 Liege, Belgium
- Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
| | - Henry C. Woodruff
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
- Department of Radiology and Nuclear Medicine, GROW—School for Oncology, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands
| | - Vincent Andrearczyk
- Institute of Information Systems, University of Applied Sciences and Arts Western Switzerland (HES-SO), rue du Technopole 3, 3960 Sierre, Switzerland; (V.A.); (H.M.)
| | - Henning Müller
- Institute of Information Systems, University of Applied Sciences and Arts Western Switzerland (HES-SO), rue du Technopole 3, 3960 Sierre, Switzerland; (V.A.); (H.M.)
| | - Sergey Primakov
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
| | - Zohaib Salahuddin
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
| | - Avishek Chatterjee
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
| | - Philippe Lambin
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
- Department of Radiology and Nuclear Medicine, GROW—School for Oncology, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands
| |
Collapse
|
21
|
Ibrahim A, Refaee T, Primakov S, Barufaldi B, Acciavatti RJ, Granzier RWY, Hustinx R, Mottaghy FM, Woodruff HC, Wildberger JE, Lambin P, Maidment ADA. Reply to Orlhac, F.; Buvat, I. Comment on "Ibrahim et al. The Effects of In-Plane Spatial Resolution on CT-Based Radiomic Features' Stability with and without ComBat Harmonization. Cancers 2021, 13, 1848". Cancers (Basel) 2021; 13:cancers13123080. [PMID: 34205490 PMCID: PMC8235557 DOI: 10.3390/cancers13123080] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Accepted: 06/16/2021] [Indexed: 11/25/2022] Open
Affiliation(s)
- Abdalla Ibrahim
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands; (T.R.); (S.P.); (H.C.W.); (P.L.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, 6200 Maastricht, The Netherlands; (F.M.M.); (J.E.W.)
- Division of Nuclear Medicine and Oncological Imaging, Department of Medical Physics, University Hospital of Liège and GIGA CRC-In Vivo Imaging, University of Liège, 4000 Liege, Belgium;
- Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
- Correspondence:
| | - Turkey Refaee
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands; (T.R.); (S.P.); (H.C.W.); (P.L.)
- Department of Diagnostic Radiology, Faculty of Applied Medical Sciences, Jazan University, Jazan 45142, Saudi Arabia
| | - Sergey Primakov
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands; (T.R.); (S.P.); (H.C.W.); (P.L.)
- Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
| | - Bruno Barufaldi
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; (B.B.); (R.J.A.); (A.D.A.M.)
| | - Raymond J. Acciavatti
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; (B.B.); (R.J.A.); (A.D.A.M.)
| | - Renée W. Y. Granzier
- Department of Surgery, GROW—School for Oncology, Maastricht University Medical Centre+, 6200 Maastricht, The Netherlands;
| | - Roland Hustinx
- Division of Nuclear Medicine and Oncological Imaging, Department of Medical Physics, University Hospital of Liège and GIGA CRC-In Vivo Imaging, University of Liège, 4000 Liege, Belgium;
| | - Felix M. Mottaghy
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, 6200 Maastricht, The Netherlands; (F.M.M.); (J.E.W.)
- Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
| | - Henry C. Woodruff
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands; (T.R.); (S.P.); (H.C.W.); (P.L.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, 6200 Maastricht, The Netherlands; (F.M.M.); (J.E.W.)
| | - Joachim E. Wildberger
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, 6200 Maastricht, The Netherlands; (F.M.M.); (J.E.W.)
| | - Philippe Lambin
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, 6200 Maastricht, The Netherlands; (T.R.); (S.P.); (H.C.W.); (P.L.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, 6200 Maastricht, The Netherlands; (F.M.M.); (J.E.W.)
| | - Andrew D. A. Maidment
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; (B.B.); (R.J.A.); (A.D.A.M.)
| |
Collapse
|
22
|
Granzier RWY, Ibrahim A, Primakov SP, Samiei S, van Nijnatten TJA, de Boer M, Heuts EM, Hulsmans FJ, Chatterjee A, Lambin P, Lobbes MBI, Woodruff HC, Smidt ML. MRI-Based Radiomics Analysis for the Pretreatment Prediction of Pathologic Complete Tumor Response to Neoadjuvant Systemic Therapy in Breast Cancer Patients: A Multicenter Study. Cancers (Basel) 2021; 13:cancers13102447. [PMID: 34070016 PMCID: PMC8157878 DOI: 10.3390/cancers13102447] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2021] [Revised: 05/11/2021] [Accepted: 05/13/2021] [Indexed: 12/23/2022] Open
Abstract
This retrospective study investigated the value of pretreatment contrast-enhanced Magnetic Resonance Imaging (MRI)-based radiomics for the prediction of pathologic complete tumor response to neoadjuvant systemic therapy in breast cancer patients. A total of 292 breast cancer patients, with 320 tumors, who were treated with neo-adjuvant systemic therapy and underwent a pretreatment MRI exam were enrolled. As the data were collected in two different hospitals with five different MRI scanners and varying acquisition protocols, three different strategies to split training and validation datasets were used. Radiomics, clinical, and combined models were developed using random forest classifiers in each strategy. The analysis of radiomics features had no added value in predicting pathologic complete tumor response to neoadjuvant systemic therapy in breast cancer patients compared with the clinical models, nor did the combined models perform significantly better than the clinical models. Further, the radiomics features selected for the models and their performance differed with and within the different strategies. Due to previous and current work, we tentatively attribute the lack of improvement in clinical models following the addition of radiomics to the effects of variations in acquisition and reconstruction parameters. The lack of reproducibility data (i.e., test-retest or similar) meant that this effect could not be analyzed. These results indicate the need for reproducibility studies to preselect reproducible features in order to properly assess the potential of radiomics.
Collapse
Affiliation(s)
- Renée W. Y. Granzier
- Department of Surgery, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands; (S.S.); (E.M.H.); (M.L.S.)
- GROW-School for Oncology and Developmental Biology, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands; (A.I.); (S.P.P.); (M.d.B.); (A.C.); (P.L.); (M.B.I.L.); (H.C.W.)
- Correspondence: ; Tel.: +31-43-388-1575
| | - Abdalla Ibrahim
- GROW-School for Oncology and Developmental Biology, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands; (A.I.); (S.P.P.); (M.d.B.); (A.C.); (P.L.); (M.B.I.L.); (H.C.W.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands;
- The D-Lab, Department of Precision Medicine, Maastricht University, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands
- Division of Nuclear Medicine and Oncological Imaging, Department of Medical Physics, University Hospital of Liège and GIGA CRC-In Vivo Imaging, University of Liège, 4000 Liege, Belgium
- Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
| | - Sergey P. Primakov
- GROW-School for Oncology and Developmental Biology, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands; (A.I.); (S.P.P.); (M.d.B.); (A.C.); (P.L.); (M.B.I.L.); (H.C.W.)
- The D-Lab, Department of Precision Medicine, Maastricht University, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands
| | - Sanaz Samiei
- Department of Surgery, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands; (S.S.); (E.M.H.); (M.L.S.)
- GROW-School for Oncology and Developmental Biology, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands; (A.I.); (S.P.P.); (M.d.B.); (A.C.); (P.L.); (M.B.I.L.); (H.C.W.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands;
| | - Thiemo J. A. van Nijnatten
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands;
| | - Maaike de Boer
- GROW-School for Oncology and Developmental Biology, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands; (A.I.); (S.P.P.); (M.d.B.); (A.C.); (P.L.); (M.B.I.L.); (H.C.W.)
- Department of Internal Medicine, Division of Medical Oncology, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands
| | - Esther M. Heuts
- Department of Surgery, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands; (S.S.); (E.M.H.); (M.L.S.)
| | - Frans-Jan Hulsmans
- Department of Medical Imaging, Zuyderland Medical Center, P.O. Box 5500, 6130 MB Sittard-Geleen, The Netherlands;
| | - Avishek Chatterjee
- GROW-School for Oncology and Developmental Biology, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands; (A.I.); (S.P.P.); (M.d.B.); (A.C.); (P.L.); (M.B.I.L.); (H.C.W.)
- The D-Lab, Department of Precision Medicine, Maastricht University, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands
| | - Philippe Lambin
- GROW-School for Oncology and Developmental Biology, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands; (A.I.); (S.P.P.); (M.d.B.); (A.C.); (P.L.); (M.B.I.L.); (H.C.W.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands;
- The D-Lab, Department of Precision Medicine, Maastricht University, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands
| | - Marc B. I. Lobbes
- GROW-School for Oncology and Developmental Biology, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands; (A.I.); (S.P.P.); (M.d.B.); (A.C.); (P.L.); (M.B.I.L.); (H.C.W.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands;
- Department of Medical Imaging, Zuyderland Medical Center, P.O. Box 5500, 6130 MB Sittard-Geleen, The Netherlands;
| | - Henry C. Woodruff
- GROW-School for Oncology and Developmental Biology, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands; (A.I.); (S.P.P.); (M.d.B.); (A.C.); (P.L.); (M.B.I.L.); (H.C.W.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands;
- The D-Lab, Department of Precision Medicine, Maastricht University, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands
| | - Marjolein L. Smidt
- Department of Surgery, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands; (S.S.); (E.M.H.); (M.L.S.)
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands;
| |
Collapse
|