1
|
Thomas HMT, Wang HYC, Varghese AJ, Donovan EM, South CP, Saxby H, Nisbet A, Prakash V, Sasidharan BK, Pavamani SP, Devadhas D, Mathew M, Isiah RG, Evans PM. Reproducibility in Radiomics: A Comparison of Feature Extraction Methods and Two Independent Datasets. Appl Sci (Basel) 2024; 166:s00701-024-05977-4. [PMID: 38725869 PMCID: PMC7615943 DOI: 10.3390/app13127291] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 05/12/2024]
Abstract
Radiomics involves the extraction of information from medical images that are not visible to the human eye. There is evidence that these features can be used for treatment stratification and outcome prediction. However, there is much discussion about the reproducibility of results between different studies. This paper studies the reproducibility of CT texture features used in radiomics, comparing two feature extraction implementations, namely the MATLAB toolkit and Pyradiomics, when applied to independent datasets of CT scans of patients: (i) the open access RIDER dataset containing a set of repeat CT scans taken 15 min apart for 31 patients (RIDER Scan 1 and Scan 2, respectively) treated for lung cancer; and (ii) the open access HN1 dataset containing 137 patients treated for head and neck cancer. Gross tumor volume (GTV), manually outlined by an experienced observer available on both datasets, was used. The 43 common radiomics features available in MATLAB and Pyradiomics were calculated using two intensity-level quantization methods with and without an intensity threshold. Cases were ranked for each feature for all combinations of quantization parameters, and the Spearman's rank coefficient, rs, calculated. Reproducibility was defined when a highly correlated feature in the RIDER dataset also correlated highly in the HN1 dataset, and vice versa. A total of 29 out of the 43 reported stable features were found to be highly reproducible between MATLAB and Pyradiomics implementations, having a consistently high correlation in rank ordering for RIDER Scan 1 and RIDER Scan 2 (rs > 0.8). 18/43 reported features were common in the RIDER and HN1 datasets, suggesting they may be agnostic to disease site. Useful radiomics features should be selected based on reproducibility. This study identified a set of features that meet this requirement and validated the methodology for evaluating reproducibility between datasets.
Collapse
Affiliation(s)
- Hannah Mary T. Thomas
- Department of Radiation Oncology, Christian Medical College Vellore, Vellore 632004, Tamil Nadu, India
| | - Helen Y. C. Wang
- Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford GU2 7XH, UK
- Department of Medical Physics, Royal Surrey NHS Foundation Trust, Guildford GU2 7XX, UK
| | - Amal Joseph Varghese
- Department of Radiation Oncology, Christian Medical College Vellore, Vellore 632004, Tamil Nadu, India
| | - Ellen M. Donovan
- Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford GU2 7XH, UK
| | - Chris P. South
- Department of Medical Physics, Royal Surrey NHS Foundation Trust, Guildford GU2 7XX, UK
| | - Helen Saxby
- St Luke’s Cancer Centre, Royal Surrey NHS Foundation Trust, Guildford GU2 7XX, UK
| | - Andrew Nisbet
- Department of Medical Physics and Biomedical Engineering, University College London, London WC1E 6BT, UK
| | - Vineet Prakash
- St Luke’s Cancer Centre, Royal Surrey NHS Foundation Trust, Guildford GU2 7XX, UK
| | - Balu Krishna Sasidharan
- Department of Radiation Oncology, Christian Medical College Vellore, Vellore 632004, Tamil Nadu, India
| | - Simon Pradeep Pavamani
- Department of Radiation Oncology, Christian Medical College Vellore, Vellore 632004, Tamil Nadu, India
| | - Devakumar Devadhas
- Department of Nuclear Medicine, Christian Medical College Vellore, Vellore 632004, Tamil Nadu, India
| | - Manu Mathew
- Department of Radiation Oncology, Christian Medical College Vellore, Vellore 632004, Tamil Nadu, India
| | - Rajesh Gunasingam Isiah
- Department of Radiation Oncology, Christian Medical College Vellore, Vellore 632004, Tamil Nadu, India
| | - Philip M. Evans
- Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford GU2 7XH, UK
| |
Collapse
|
2
|
Varghese AJ, Gouthamchand V, Sasidharan BK, Wee L, Sidhique SK, Rao JP, Dekker A, Hoebers F, Devakumar D, Irodi A, Balasingh TP, Godson HF, Joel T, Mathew M, Gunasingam Isiah R, Pavamani SP, Thomas HMT. Multi-centre radiomics for prediction of recurrence following radical radiotherapy for head and neck cancers: Consequences of feature selection, machine learning classifiers and batch-effect harmonization. Phys Imaging Radiat Oncol 2023; 26:100450. [PMID: 37260438 PMCID: PMC10227455 DOI: 10.1016/j.phro.2023.100450] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 05/01/2023] [Accepted: 05/02/2023] [Indexed: 06/02/2023] Open
Abstract
Background and purpose Radiomics models trained with limited single institution data are often not reproducible and generalisable. We developed radiomics models that predict loco-regional recurrence within two years of radiotherapy with private and public datasets and their combinations, to simulate small and multi-institutional studies and study the responsiveness of the models to feature selection, machine learning algorithms, centre-effect harmonization and increased dataset sizes. Materials and methods 562 patients histologically confirmed and treated for locally advanced head-and-neck cancer (LA-HNC) from two public and two private datasets; one private dataset exclusively reserved for validation. Clinical contours of primary tumours were not recontoured and were used for Pyradiomics based feature extraction. ComBat harmonization was applied, and LASSO-Logistic Regression (LR) and Support Vector Machine (SVM) models were built. 95% confidence interval (CI) of 1000 bootstrapped area-under-the-Receiver-operating-curves (AUC) provided predictive performance. Responsiveness of the models' performance to the choice of feature selection methods, ComBat harmonization, machine learning classifier, single and pooled data was evaluated. Results LASSO and SelectKBest selected 14 and 16 features, respectively; three were overlapping. Without ComBat, the LR and SVM models for three institutional data showed AUCs (CI) of 0.513 (0.481-0.559) and 0.632 (0.586-0.665), respectively. Performances following ComBat revealed AUCs of 0.559 (0.536-0.590) and 0.662 (0.606-0.690), respectively. Compared to single cohort AUCs (0.562-0.629), SVM models from pooled data performed significantly better at AUC = 0.680. Conclusions Multi-institutional retrospective data accentuates the existing variabilities that affect radiomics. Carefully designed prospective, multi-institutional studies and data sharing are necessary for clinically relevant head-and-neck cancer prognostication models.
Collapse
Affiliation(s)
- Amal Joseph Varghese
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| | - Varsha Gouthamchand
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | | | - Leonard Wee
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Sharief K Sidhique
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| | | | - Andre Dekker
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Frank Hoebers
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Devadhas Devakumar
- Department of Nuclear Medicine, Christian Medical College, Vellore, Tamil Nadu, India
| | - Aparna Irodi
- Department of Radiology, Christian Medical College, Vellore, Tamil Nadu, India
| | | | - Henry Finlay Godson
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| | - T Joel
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| | - Manu Mathew
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| | | | | | - Hannah Mary T Thomas
- Department of Radiation Oncology, Christian Medical College, Vellore, Tamil Nadu, India
| |
Collapse
|