1
|
Russo L, Bottazzi S, Kocak B, Zormpas-Petridis K, Gui B, Stanzione A, Imbriaco M, Sala E, Cuocolo R, Ponsiglione A. Evaluating the quality of radiomics-based studies for endometrial cancer using RQS and METRICS tools. Eur Radiol 2024:10.1007/s00330-024-10947-6. [PMID: 39014086 DOI: 10.1007/s00330-024-10947-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2024] [Revised: 05/15/2024] [Accepted: 06/19/2024] [Indexed: 07/18/2024]
Abstract
OBJECTIVE To assess the methodological quality of radiomics-based models in endometrial cancer using the radiomics quality score (RQS) and METhodological radiomICs score (METRICS). METHODS We systematically reviewed studies published by October 30th, 2023. Inclusion criteria were original radiomics studies on endometrial cancer using CT, MRI, PET, or ultrasound. Articles underwent a quality assessment by novice and expert radiologists using RQS and METRICS. The inter-rater reliability for RQS and METRICS among radiologists with varying expertise was determined. Subgroup analyses were performed to assess whether scores varied according to study topic, imaging technique, publication year, and journal quartile. RESULTS Sixty-eight studies were analysed, with a median RQS of 11 (IQR, 9-14) and METRICS score of 67.6% (IQR, 58.8-76.0); two different articles reached maximum RQS of 19 and METRICS of 90.7%, respectively. Most studies utilised MRI (82.3%) and machine learning methods (88.2%). Characterisation and recurrence risk stratification were the most explored outcomes, featured in 35.3% and 19.1% of articles, respectively. High inter-rater reliability was observed for both RQS (ICC: 0.897; 95% CI: 0.821, 0.946) and METRICS (ICC: 0.959; 95% CI: 0.928, 0.979). Methodological limitations such as lack of external validation suggest areas for improvement. At subgroup analyses, no statistically significant difference was noted. CONCLUSIONS Whilst using RQS, the quality of endometrial cancer radiomics research was apparently unsatisfactory, METRICS depicts a good overall quality. Our study highlights the need for strict compliance with quality metrics. Adhering to these quality measures can increase the consistency of radiomics towards clinical application in the pre-operative management of endometrial cancer. CLINICAL RELEVANCE STATEMENT Both the RQS and METRICS can function as instrumental tools for identifying different methodological deficiencies in endometrial cancer radiomics research. However, METRICS also reflected a focus on the practical applicability and clarity of documentation. KEY POINTS The topic of radiomics currently lacks standardisation, limiting clinical implementation. METRICS scores were generally higher than the RQS, reflecting differences in the development process and methodological content. A positive trend in METRICS score may suggest growing attention to methodological aspects in radiomics research.
Collapse
Affiliation(s)
- Luca Russo
- Dipartimento di Scienze Radiologiche ed Ematologiche, Università Cattolica del Sacro Cuore, Rome, Italy
| | - Silvia Bottazzi
- Dipartimento di Scienze Radiologiche ed Ematologiche, Università Cattolica del Sacro Cuore, Rome, Italy
| | - Burak Kocak
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, Turkey
| | - Konstantinos Zormpas-Petridis
- Dipartimento Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy
| | - Benedetta Gui
- Dipartimento Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy
| | - Arnaldo Stanzione
- Department of Advanced Biomedical Sciences, University of Naples "Federico II", Naples, Italy
| | - Massimo Imbriaco
- Department of Advanced Biomedical Sciences, University of Naples "Federico II", Naples, Italy
| | - Evis Sala
- Dipartimento di Scienze Radiologiche ed Ematologiche, Università Cattolica del Sacro Cuore, Rome, Italy
- Dipartimento Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy
| | - Renato Cuocolo
- Department of Medicine, Surgery and Dentistry, University of Salerno, Baronissi, Italy.
| | - Andrea Ponsiglione
- Department of Advanced Biomedical Sciences, University of Naples "Federico II", Naples, Italy
| |
Collapse
|
2
|
Huang ML, Ren J, Jin ZY, Liu XY, Li Y, He YL, Xue HD. Application of magnetic resonance imaging radiomics in endometrial cancer: a systematic review and meta-analysis. LA RADIOLOGIA MEDICA 2024; 129:439-456. [PMID: 38349417 DOI: 10.1007/s11547-024-01765-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Accepted: 01/03/2024] [Indexed: 03/16/2024]
Abstract
PURPOSE We aimed to systematically assess the methodological quality and clinical potential application of published magnetic resonance imaging (MRI)-based radiomics studies about endometrial cancer (EC). METHODS Studies of EC radiomics analyses published between 1 January 2000 and 19 March 2023 were extracted, and their methodological quality was evaluated using the radiomics quality score (RQS) and Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). Pairwise correlation analyses and separate meta-analyses of studies exploring differential diagnoses and risk prediction were also performed. RESULTS Forty-five studies involving 3 aims were included. The mean RQS was 13.77 (range: 9-22.5); publication bias was observed in the areas of 'index test' and 'flow and timing'. A high RQS was significantly associated with therapy selection-aimed studies, low QUADAS-2 risk, recent publication year, and high-performance metrics. Raw data from 6 differential diagnosis and 34 risk prediction models were subjected to meta-analysis, revealing diagnostic odds ratios of 23.81 (95% confidence interval [CI] 8.48-66.83) and 18.23 (95% CI 13.68-24.29), respectively. CONCLUSION The methodological quality of radiomics studies involving patients with EC is unsatisfactory. However, MRI-based radiomics analyses showed promising utility in terms of differential diagnosis and risk prediction.
Collapse
Affiliation(s)
- Meng-Lin Huang
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shuai Fu Yuan 1#, Dongcheng Dist., Beijing, 100730, People's Republic of China
| | - Jing Ren
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shuai Fu Yuan 1#, Dongcheng Dist., Beijing, 100730, People's Republic of China
| | - Zheng-Yu Jin
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shuai Fu Yuan 1#, Dongcheng Dist., Beijing, 100730, People's Republic of China
| | - Xin-Yu Liu
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shuai Fu Yuan 1#, Dongcheng Dist., Beijing, 100730, People's Republic of China
| | - Yuan Li
- Department of Obstetrics and Gynecology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, National Clinical Research Center for Obstetric and Gynecologic Diseases, Shuai Fu Yuan 1#, Dongcheng Dist., Beijing, 100730, People's Republic of China.
| | - Yong-Lan He
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shuai Fu Yuan 1#, Dongcheng Dist., Beijing, 100730, People's Republic of China.
| | - Hua-Dan Xue
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shuai Fu Yuan 1#, Dongcheng Dist., Beijing, 100730, People's Republic of China.
| |
Collapse
|
3
|
Pasini G, Russo G, Mantarro C, Bini F, Richiusa S, Morgante L, Comelli A, Russo GI, Sabini MG, Cosentino S, Marinozzi F, Ippolito M, Stefano A. A Critical Analysis of the Robustness of Radiomics to Variations in Segmentation Methods in 18F-PSMA-1007 PET Images of Patients Affected by Prostate Cancer. Diagnostics (Basel) 2023; 13:3640. [PMID: 38132224 PMCID: PMC10743045 DOI: 10.3390/diagnostics13243640] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Revised: 11/29/2023] [Accepted: 12/08/2023] [Indexed: 12/23/2023] Open
Abstract
BACKGROUND Radiomics shows promising results in supporting the clinical decision process, and much effort has been put into its standardization, thus leading to the Imaging Biomarker Standardization Initiative (IBSI), that established how radiomics features should be computed. However, radiomics still lacks standardization and many factors, such as segmentation methods, limit study reproducibility and robustness. AIM We investigated the impact that three different segmentation methods (manual, thresholding and region growing) have on radiomics features extracted from 18F-PSMA-1007 Positron Emission Tomography (PET) images of 78 patients (43 Low Risk, 35 High Risk). Segmentation was repeated for each patient, thus leading to three datasets of segmentations. Then, feature extraction was performed for each dataset, and 1781 features (107 original, 930 Laplacian of Gaussian (LoG) features, 744 wavelet features) were extracted. Feature robustness and reproducibility were assessed through the intra class correlation coefficient (ICC) to measure agreement between the three segmentation methods. To assess the impact that the three methods had on machine learning models, feature selection was performed through a hybrid descriptive-inferential method, and selected features were given as input to three classifiers, K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Linear Discriminant Analysis (LDA), Random Forest (RF), AdaBoost and Neural Networks (NN), whose performance in discriminating between low-risk and high-risk patients have been validated through 30 times repeated five-fold cross validation. CONCLUSIONS Our study showed that segmentation methods influence radiomics features and that Shape features were the least reproducible (average ICC: 0.27), while GLCM features the most reproducible. Moreover, feature reproducibility changed depending on segmentation type, resulting in 51.18% of LoG features exhibiting excellent reproducibility (range average ICC: 0.68-0.87) and 47.85% of wavelet features exhibiting poor reproducibility that varied between wavelet sub-bands (range average ICC: 0.34-0.80) and resulted in the LLL band showing the highest average ICC (0.80). Finally, model performance showed that region growing led to the highest accuracy (74.49%), improved sensitivity (84.38%) and AUC (79.20%) in contrast with manual segmentation.
Collapse
Affiliation(s)
- Giovanni Pasini
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Eudossiana 18, 00184 Rome, Italy; (G.P.); (L.M.); (F.M.)
- Institute of Molecular Bioimaging and Physiology, National Research Council (IBFM-CNR), Contrada, Pietrapollastra-Pisciotto, 90015 Cefalù, Italy; (G.R.); (S.R.); (A.C.); (A.S.)
| | - Giorgio Russo
- Institute of Molecular Bioimaging and Physiology, National Research Council (IBFM-CNR), Contrada, Pietrapollastra-Pisciotto, 90015 Cefalù, Italy; (G.R.); (S.R.); (A.C.); (A.S.)
- National Laboratory of South, National Institute for Nuclear Physics (LNS-INFN), 95125 Catania, Italy
| | - Cristina Mantarro
- Nuclear Medicine Department, Cannizzaro Hospital, 95125 Catania, Italy; (C.M.); (S.C.); (M.I.)
| | - Fabiano Bini
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Eudossiana 18, 00184 Rome, Italy; (G.P.); (L.M.); (F.M.)
| | - Selene Richiusa
- Institute of Molecular Bioimaging and Physiology, National Research Council (IBFM-CNR), Contrada, Pietrapollastra-Pisciotto, 90015 Cefalù, Italy; (G.R.); (S.R.); (A.C.); (A.S.)
| | - Lucrezia Morgante
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Eudossiana 18, 00184 Rome, Italy; (G.P.); (L.M.); (F.M.)
| | - Albert Comelli
- Institute of Molecular Bioimaging and Physiology, National Research Council (IBFM-CNR), Contrada, Pietrapollastra-Pisciotto, 90015 Cefalù, Italy; (G.R.); (S.R.); (A.C.); (A.S.)
- Ri.MED Foundation, Via Bandiera 11, 90133 Palermo, Italy
| | - Giorgio Ivan Russo
- Department of Surgery, Urology Section, University of Catania, 95125 Catania, Italy;
| | | | - Sebastiano Cosentino
- Nuclear Medicine Department, Cannizzaro Hospital, 95125 Catania, Italy; (C.M.); (S.C.); (M.I.)
| | - Franco Marinozzi
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Eudossiana 18, 00184 Rome, Italy; (G.P.); (L.M.); (F.M.)
| | - Massimo Ippolito
- Nuclear Medicine Department, Cannizzaro Hospital, 95125 Catania, Italy; (C.M.); (S.C.); (M.I.)
| | - Alessandro Stefano
- Institute of Molecular Bioimaging and Physiology, National Research Council (IBFM-CNR), Contrada, Pietrapollastra-Pisciotto, 90015 Cefalù, Italy; (G.R.); (S.R.); (A.C.); (A.S.)
- National Laboratory of South, National Institute for Nuclear Physics (LNS-INFN), 95125 Catania, Italy
| |
Collapse
|