1
|
Kumar S, De Kock I, Blad W, Hare R, Pollok R, Taylor SA. Magnetic Resonance Enterography and Intestinal Ultrasound for the Assessment and Monitoring of Crohn's Disease. J Crohns Colitis 2024; 18:1450-1463. [PMID: 38554104 PMCID: PMC11369078 DOI: 10.1093/ecco-jcc/jjae042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 02/20/2024] [Indexed: 04/01/2024]
Abstract
Magnetic resonance enterography [MRE] and intestinal ultrasound [IUS] have developed rapidly in the past few decades, emerging as the primary non-invasive options for both diagnosing and monitoring Crohn's disease [CD]. In this review, we evaluate the pertinent data relating to the use of MRE and IUS in CD. We summarise the key imaging features of CD activity, highlight their increasing role in both the clinical and the research settings, and discuss how these modalities fit within the diagnostic pathway. We discuss how they can be used to assess disease activity and treatment responsiveness, including the emergence of activity scores for standardised reporting. Additionally, we address areas of controversy such as the use of contrast agents, the role of diffusion-weighted imaging, and point-of-care ultrasound. We also highlight exciting new developments, including the applications of artificial intelligence. Finally, we provide suggestions for future research priorities.
Collapse
Affiliation(s)
- Shankar Kumar
- Centre for Medical Imaging, University College London, London, UK
| | | | - William Blad
- Department of Gastroenterology, University College Hospitals NHS Foundation Trust, London, UK
| | - Richard Hare
- Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
| | - Richard Pollok
- Department of Gastroenterology, St George’s University Hospitals NHS Foundation Trust, London, UK
| | - Stuart A Taylor
- Centre for Medical Imaging, University College London, London, UK
| |
Collapse
|
2
|
Doniselli FM, Pascuzzo R, Mazzi F, Padelli F, Moscatelli M, Akinci D'Antonoli T, Cuocolo R, Aquino D, Cuccarini V, Sconfienza LM. Quality assessment of the MRI-radiomics studies for MGMT promoter methylation prediction in glioma: a systematic review and meta-analysis. Eur Radiol 2024; 34:5802-5815. [PMID: 38308012 PMCID: PMC11364578 DOI: 10.1007/s00330-024-10594-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2023] [Revised: 12/04/2023] [Accepted: 12/31/2023] [Indexed: 02/04/2024]
Abstract
OBJECTIVES To evaluate the methodological quality and diagnostic accuracy of MRI-based radiomic studies predicting O6-methylguanine-DNA methyltransferase (MGMT) promoter methylation status in gliomas. METHODS PubMed Medline, EMBASE, and Web of Science were searched to identify MRI-based radiomic studies on MGMT methylation in gliomas published until December 31, 2022. Three raters evaluated the study methodological quality with Radiomics Quality Score (RQS, 16 components) and Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis (TRIPOD, 22 items) scales. Risk of bias and applicability concerns were assessed with QUADAS-2 tool. A meta-analysis was performed to estimate the pooled area under the curve (AUC) and to assess inter-study heterogeneity. RESULTS We included 26 studies, published from 2016. The median RQS total score was 8 out of 36 (22%, range 8-44%). Thirteen studies performed external validation. All studies reported AUC or accuracy, but only 4 (15%) performed calibration and decision curve analysis. No studies performed phantom analysis, cost-effectiveness analysis, and prospective validation. The overall TRIPOD adherence score was between 50% and 70% in 16 studies and below 50% in 10 studies. The pooled AUC was 0.78 (95% CI, 0.73-0.83, I2 = 94.1%) with a high inter-study heterogeneity. Studies with external validation and including only WHO-grade IV gliomas had significantly lower AUC values (0.65; 95% CI, 0.57-0.73, p < 0.01). CONCLUSIONS Study RQS and adherence to TRIPOD guidelines was generally low. Radiomic prediction of MGMT methylation status showed great heterogeneity of results and lower performances in grade IV gliomas, which hinders its current implementation in clinical practice. CLINICAL RELEVANCE STATEMENT MGMT promoter methylation status appears to be variably correlated with MRI radiomic features; radiomic models are not sufficiently robust to be integrated into clinical practice to accurately predict MGMT promoter methylation status in patients with glioma before surgery. KEY POINTS • Adherence to the indications of TRIPOD guidelines was generally low, as was RQS total score. • MGMT promoter methylation status prediction with MRI radiomic features provided heterogeneous diagnostic accuracy results across studies. • Studies that included grade IV glioma only and performed external validation had significantly lower diagnostic accuracy than others.
Collapse
Affiliation(s)
- Fabio M Doniselli
- Neuroradiology Unit, Fondazione IRCCS Istituto Neurologico Carlo Besta, Via Giovanni Celoria 11, 20133, Milan, Italy
- Department of Biomedical Sciences for Health, Università degli Studi di Milano, Via Luigi Mangiagalli 31, 20133, Milan, Italy
| | - Riccardo Pascuzzo
- Neuroradiology Unit, Fondazione IRCCS Istituto Neurologico Carlo Besta, Via Giovanni Celoria 11, 20133, Milan, Italy.
| | - Federica Mazzi
- Neuroradiology Unit, Fondazione IRCCS Istituto Neurologico Carlo Besta, Via Giovanni Celoria 11, 20133, Milan, Italy
| | - Francesco Padelli
- Neuroradiology Unit, Fondazione IRCCS Istituto Neurologico Carlo Besta, Via Giovanni Celoria 11, 20133, Milan, Italy
| | - Marco Moscatelli
- Neuroradiology Unit, Fondazione IRCCS Istituto Neurologico Carlo Besta, Via Giovanni Celoria 11, 20133, Milan, Italy
| | - Tugba Akinci D'Antonoli
- Institute of Radiology and Nuclear Medicine, Cantonal Hospital Baselland, Rheinstrasse 26, 4410, Liestal, Switzerland
| | - Renato Cuocolo
- Department of Medicine, Surgery, and Dentistry, University of Salerno, Via Salvador Allende 43, Baronissi, 84081, Salerno, Italy
| | - Domenico Aquino
- Neuroradiology Unit, Fondazione IRCCS Istituto Neurologico Carlo Besta, Via Giovanni Celoria 11, 20133, Milan, Italy
| | - Valeria Cuccarini
- Neuroradiology Unit, Fondazione IRCCS Istituto Neurologico Carlo Besta, Via Giovanni Celoria 11, 20133, Milan, Italy
| | - Luca Maria Sconfienza
- Department of Biomedical Sciences for Health, Università degli Studi di Milano, Via Luigi Mangiagalli 31, 20133, Milan, Italy
- IRCCS Ospedale Galeazzi-Sant'Ambrogio, Via Cristina Belgioioso 173, 20157, Milan, Italy
| |
Collapse
|
3
|
Castellana R, Fanni SC, Roncella C, Romei C, Natrella M, Neri E. Radiomics and deep learning models for CT pre-operative lymph node staging in pancreatic ductal adenocarcinoma: A systematic review and meta-analysis. Eur J Radiol 2024; 176:111510. [PMID: 38781919 DOI: 10.1016/j.ejrad.2024.111510] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Revised: 04/23/2024] [Accepted: 05/13/2024] [Indexed: 05/25/2024]
Abstract
PURPOSE To evaluate the diagnostic accuracy of computed tomography (CT)-based radiomic algorithms and deep learning models to preoperatively identify lymph node metastasis (LNM) in patients with pancreatic ductal adenocarcinoma (PDAC). METHODS PubMed, CENTRAL, Scopus, Web of Science and IEEE databases were searched to identify relevant studies published up until February 11, 2024. Two reviewers screened all papers independently for eligibility. Studies reporting the accuracy of CT-based radiomics or deep learning models for detecting LNM in PDAC, using histopathology as the reference standard, were included. Quality was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2, the Radiomics Quality Score (RQS) and the the METhodological RadiomICs Score (METRICS). Overall sensitivity (SE), specificity (SP), diagnostic odds ratio (DOR), and the area under the curve (AUC) were calculated. RESULTS Four radiomics studies comprising 213 patients and four deep learning studies with 272 patients were included. The average RQS total score was 12.00 ± 3.89, corresponding to an RQS percentage of 33.33 ± 10.80, while the average METRICS score was 63.60 ± 10.88. A significant and strong positive correlation was found between RQS and METRICS (p = 0.016; r = 0.810). The pooled SE, SP, DOR, and AUC of all the studies were 0.83 (95 %CI = 0.77-0.88), 0.76 (95 %CI = 0.62-0.86), 15.70 (95 %CI = 8.12-27.50) and 0.85 (95 %CI = 0.77-0.88). Meta-regression analysis results indicated that neither the study type (radiomics vs deep learning) nor the dataset size of the studies had a significant effect on the DOR (p = 0.09 and p = 0.26, respectively). CONCLUSION Based on our meta-analysis findings, preoperative CT-based radiomics algorithms and deep learning models demonstrate favorable performance in predicting LNM in patients with PDAC, with a strong correlation between RQS and METRICS of the included studies.
Collapse
Affiliation(s)
- Roberto Castellana
- Diagnostic and Interventional Radiology, "Parini" Regional Hospital, Azienda USL della Valle d'Aosta, Viale Ginevra 3 11100, Aosta, Italy.
| | - Salvatore Claudio Fanni
- Department of Translational Research, Academic Radiology, University of Pisa, Via Paradisa 2, 56124 Pisa, Italy
| | - Claudia Roncella
- Radiology Unit, Apuane Hospital, Azienda USL Toscana Nord Ovest, Via Mattei 21, 54100, Massa, Italy
| | - Chiara Romei
- Department of Diagnostic Imaging, Diagnostic Radiology 2, Pisa University Hospital, Via Paradisa 2, 56124, Pisa, Italy
| | - Massimiliano Natrella
- Diagnostic and Interventional Radiology, "Parini" Regional Hospital, Azienda USL della Valle d'Aosta, Viale Ginevra 3 11100, Aosta, Italy
| | - Emanuele Neri
- Department of Translational Research, Academic Radiology, University of Pisa, Via Paradisa 2, 56124 Pisa, Italy
| |
Collapse
|
4
|
Gitto S, Cuocolo R, Giannetta V, Badalyan J, Di Luca F, Fusco S, Zantonelli G, Albano D, Messina C, Sconfienza LM. Effects of Interobserver Segmentation Variability and Intensity Discretization on MRI-Based Radiomic Feature Reproducibility of Lipoma and Atypical Lipomatous Tumor. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1187-1200. [PMID: 38332405 PMCID: PMC11169199 DOI: 10.1007/s10278-024-00999-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/16/2023] [Revised: 12/17/2023] [Accepted: 12/21/2023] [Indexed: 02/10/2024]
Abstract
Segmentation and image intensity discretization impact on radiomics workflow. The aim of this study is to investigate the influence of interobserver segmentation variability and intensity discretization methods on the reproducibility of MRI-based radiomic features in lipoma and atypical lipomatous tumor (ALT). Thirty patients with lipoma or ALT were retrospectively included. Three readers independently performed manual contour-focused segmentation on T1-weighted and T2-weighted sequences, including the whole tumor volume. Additionally, a marginal erosion was applied to segmentations to evaluate its influence on feature reproducibility. After image pre-processing, with included intensity discretization employing both fixed bin number and width approaches, 1106 radiomic features were extracted from each sequence. Intraclass correlation coefficient (ICC) 95% confidence interval lower bound ≥ 0.75 defined feature stability. In contour-focused vs. margin shrinkage segmentation, the rates of stable features extracted from T1-weighted and T2-weighted images ranged from 92.68 to 95.21% vs. 90.69 to 95.66% after fixed bin number discretization and from 95.75 to 97.65% vs. 95.39 to 96.47% after fixed bin width discretization, respectively, with no difference between the two segmentation approaches (p ≥ 0.175). Higher stable feature rates and higher feature ICC values were found when implementing discretization with fixed bin width compared to fixed bin number, regardless of the segmentation approach (p < 0.001). In conclusion, MRI radiomic features of lipoma and ALT are reproducible regardless of the segmentation approach and intensity discretization method, although a certain degree of interobserver variability highlights the need for a preliminary reliability analysis in future studies.
Collapse
Affiliation(s)
- Salvatore Gitto
- IRCCS Istituto Ortopedico Galeazzi, Via Cristina Belgioioso 173, 20157, Milan, Italy
- Dipartimento Di Scienze Biomediche Per La Salute, Università Degli Studi Di Milano, Milan, Italy
| | - Renato Cuocolo
- Department of Medicine, Surgery and Dentistry, University of Salerno, Baronissi, Italy
| | - Vincenzo Giannetta
- Diagnostic and Interventional Radiology Department, IRCCS Ospedale San Raffaele-Turro, Università Vita-Salute San Raffaele, Milan, Italy
| | - Julietta Badalyan
- Scuola Di Specializzazione in Statistica Sanitaria E Biometria, Università Degli Studi Di Milano, Milan, Italy
| | - Filippo Di Luca
- Scuola Di Specializzazione in Radiodiagnostica, Università Degli Studi Di Milano, Milan, Italy
| | - Stefano Fusco
- Dipartimento Di Scienze Biomediche Per La Salute, Università Degli Studi Di Milano, Milan, Italy
| | - Giulia Zantonelli
- Dipartimento Di Scienze Biomediche Per La Salute, Università Degli Studi Di Milano, Milan, Italy
| | - Domenico Albano
- IRCCS Istituto Ortopedico Galeazzi, Via Cristina Belgioioso 173, 20157, Milan, Italy
- Dipartimento Di Scienze Biomediche, Chirurgiche Ed Odontoiatriche, Università Degli Studi Di Milano, Milan, Italy
| | - Carmelo Messina
- IRCCS Istituto Ortopedico Galeazzi, Via Cristina Belgioioso 173, 20157, Milan, Italy
- Dipartimento Di Scienze Biomediche Per La Salute, Università Degli Studi Di Milano, Milan, Italy
| | - Luca Maria Sconfienza
- IRCCS Istituto Ortopedico Galeazzi, Via Cristina Belgioioso 173, 20157, Milan, Italy.
- Dipartimento Di Scienze Biomediche Per La Salute, Università Degli Studi Di Milano, Milan, Italy.
| |
Collapse
|
5
|
Deng L, Shuai P, Liu Y, Yong T, Liu Y, Li H, Zheng X. Diagnostic performance of radiomics for predicting osteoporosis in adults: a systematic review and meta-analysis. Osteoporos Int 2024:10.1007/s00198-024-07136-y. [PMID: 38802557 DOI: 10.1007/s00198-024-07136-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/07/2024] [Accepted: 05/16/2024] [Indexed: 05/29/2024]
Abstract
This study aimed to assess the diagnostic accuracy of radiomics for predicting osteoporosis and the quality of radiomic studies. The study protocol was prospectively registered on PROSPERO (CRD42023425058). We searched PubMed, EMBASE, Web of Science, and Cochrane Library databases from inception to June 1, 2023, for eligible articles that applied radiomic techniques to diagnosing osteoporosis or abnormal bone mass. Quality and risk of bias of the included studies were evaluated with radiomics quality score (RQS), METhodological RadiomICs Score (METRICS), and Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tools. The data analysis utilized the R program with mada, metafor, and meta packages. Ten retrospective studies with 5926 participants were included in the systematic review and meta-analysis. The overall risk of bias and applicability concerns for each domain of the studies were rated as low, except for one study which was considered to have a high risk of flow and time bias. The mean METRICS score was 70.1% (range 49.6-83.2%). There was moderate heterogeneity across studies and meta-regression identified sources of heterogeneity in the data, including imaging modality, feature selection method, and classifier. The pooled diagnostic odds ratio (DOR) under the bivariate random effects model across the studies was 57.22 (95% CI 27.62-118.52). The pooled sensitivity and specificity were 87% (95% CI 81-92%) and 87% (95% CI 77-93%), respectively. The area under the summary receiver operating characteristic curve (AUC) of the radiomic models was 0.94 (range 0.8 to 0.98). The results supported that the radiomic techniques had good accuracy in diagnosing osteoporosis or abnormal bone mass. The application of radiomics in osteoporosis diagnosis needs to be further confirmed by more prospective studies with rigorous adherence to existing guidelines and multicenter validation.
Collapse
Affiliation(s)
- Ling Deng
- Department of Health Management & Institute of Health Management, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
| | - Ping Shuai
- Department of Health Management & Institute of Health Management, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
| | - Youren Liu
- Department of Health Management & Institute of Health Management, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
| | - Tao Yong
- Department of Medical Information Center, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
| | - Yuping Liu
- Department of Health Management & Institute of Health Management, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
| | - Hang Li
- Department of Radiology, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China.
| | - Xiaoxia Zheng
- Department of Health Management & Institute of Health Management, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China.
| |
Collapse
|
6
|
Mirza-Aghazadeh-Attari M, Srinivas T, Kamireddy A, Kim A, Weiss CR. Radiomics Features Extracted From Pre- and Postprocedural Imaging in Early Prediction of Treatment Response in Patients Undergoing Transarterial Radioembolization of Hepatic Lesions: A Systematic Review, Meta-Analysis, and Quality Appraisal Study. J Am Coll Radiol 2024; 21:740-751. [PMID: 38220040 DOI: 10.1016/j.jacr.2023.12.029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Revised: 12/20/2023] [Accepted: 12/21/2023] [Indexed: 01/16/2024]
Abstract
INTRODUCTION Transarterial radioembolization (TARE) is one of the most promising therapeutic options for hepatic masses. Radiomics features, which are quantitative numeric features extracted from medical images, are considered to have potential in predicting treatment response in TARE. This article aims to provide meta-analytic evidence and critically appraise the methodology of radiomics studies published in this regard. METHODS A systematic search was performed on PubMed, Scopus, Embase, and Web of Science. All relevant articles were retrieved, and the characteristics of the studies were extracted. The Radiomics Quality Score and Checklist for Evaluation of Radiomics Research were used to assess the methodologic quality of the studies. Pooled sensitivity, specificity, and area under the receiver operating characteristic curve in predicting objective response were determined. RESULTS The systematic review included 15 studies. The average Radiomics Quality Score of these studies was 11.4 ± 2.1, and the average Checklist for Evaluation of Radiomics Research score was 33± 6.7. There was a notable correlation (correlation coefficient = 0.73) between the two metrics. Adherence to quality measures differed considerably among the studies and even within different components of the same studies. The pooled sensitivity and specificity of the radiomics models in predicting complete or partial response were 83.5% (95% confidence interval 76%-88.9%) and 86.7% (95% confidence interval 78%-92%), respectively. CONCLUSION Radiomics models show great potential in predicting treatment response in TARE of hepatic lesions. However, the heterogeneity seen between the methodologic quality of studies may limit the generalizability of the results. Future initiatives should aim to develop radiomics signatures using multiple external datasets and adhere to quality measures in radiomics methodology.
Collapse
Affiliation(s)
- Mohammad Mirza-Aghazadeh-Attari
- Division of Vascular and Interventional Radiology, Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins University, Baltimore, Maryland
| | - Tara Srinivas
- Division of Vascular and Interventional Radiology, Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins University, Baltimore, Maryland
| | - Arun Kamireddy
- Division of Vascular and Interventional Radiology, Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins University, Baltimore, Maryland
| | - Alan Kim
- Division of Vascular and Interventional Radiology, Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins University, Baltimore, Maryland
| | - Clifford R Weiss
- Division of Vascular and Interventional Radiology, Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins University, Baltimore, Maryland.
| |
Collapse
|
7
|
Aghakhanyan G, Filidei T, Febi M, Fanni SC, Marciano A, Francischello R, Caputo FP, Tumminello L, Cioni D, Neri E, Volterrani D. Advancing Pediatric Sarcomas through Radiomics: A Systematic Review and Prospective Assessment Using Radiomics Quality Score (RQS) and Methodological Radiomics Score (METRICS). Diagnostics (Basel) 2024; 14:832. [PMID: 38667477 PMCID: PMC11049622 DOI: 10.3390/diagnostics14080832] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2024] [Revised: 04/12/2024] [Accepted: 04/15/2024] [Indexed: 04/28/2024] Open
Abstract
Pediatric sarcomas, rare malignancies of mesenchymal origin, pose diagnostic and therapeutic challenges. In this review, we explore the role of radiomics in reshaping our understanding of pediatric sarcomas, emphasizing methodological considerations and applications such as diagnostics and predictive modeling. A systematic review conducted up to November 2023 identified 72 papers on radiomics analysis in pediatric sarcoma from PubMed/MEDLINE, Web of Knowledge, and Scopus. Following inclusion and exclusion criteria, 10 reports were included in this review. The studies, predominantly retrospective, focus on Ewing sarcoma and osteosarcoma, utilizing diverse imaging modalities, including CT, MRI, PET/CT, and PET/MRI. Manual segmentation is common, with a median of 35 features extracted. Radiomics Quality Score (RQS) and Methodological Radiomics Score (METRICS) assessments reveal a consistent emphasis on non-radiomic features, validation criteria, and improved methodological rigor in recent publications. Diagnostic applications dominate, with innovative studies exploring prognostic and treatment response aspects. Challenges include feature heterogeneity and sample size variations. The evolving landscape underscores the need for standardized methodologies. Despite challenges, the diagnostic and predictive potential of radiomics in pediatric oncology is evident, paving the way for precision medicine advancements.
Collapse
Affiliation(s)
- Gayane Aghakhanyan
- Department of Translational Research and of New Surgical and Medical Technology, University of Pisa, 56126 Pisa, Italy
| | - Tommaso Filidei
- Department of Translational Research and of New Surgical and Medical Technology, University of Pisa, 56126 Pisa, Italy
| | - Maria Febi
- Department of Translational Research and of New Surgical and Medical Technology, Academic Radiology, University of Pisa, 56126 Pisa, Italy (D.C.)
| | - Salvatore C. Fanni
- Department of Translational Research and of New Surgical and Medical Technology, Academic Radiology, University of Pisa, 56126 Pisa, Italy (D.C.)
| | - Andrea Marciano
- Department of Translational Research and of New Surgical and Medical Technology, University of Pisa, 56126 Pisa, Italy
| | - Roberto Francischello
- Department of Translational Research and of New Surgical and Medical Technology, Academic Radiology, University of Pisa, 56126 Pisa, Italy (D.C.)
| | - Francesca Pia Caputo
- Department of Translational Research and of New Surgical and Medical Technology, Academic Radiology, University of Pisa, 56126 Pisa, Italy (D.C.)
| | - Lorenzo Tumminello
- Department of Translational Research and of New Surgical and Medical Technology, Academic Radiology, University of Pisa, 56126 Pisa, Italy (D.C.)
| | - Dania Cioni
- Department of Translational Research and of New Surgical and Medical Technology, Academic Radiology, University of Pisa, 56126 Pisa, Italy (D.C.)
| | - Emanuele Neri
- Department of Translational Research and of New Surgical and Medical Technology, Academic Radiology, University of Pisa, 56126 Pisa, Italy (D.C.)
| | - Duccio Volterrani
- Department of Translational Research and of New Surgical and Medical Technology, University of Pisa, 56126 Pisa, Italy
- Regional Center of Nuclear Medicine, University Hospital of Pisa, 56126 Pisa, Italy
| |
Collapse
|
8
|
Tran K, Ginzburg D, Hong W, Attenberger U, Ko HS. Post-radiotherapy stage III/IV non-small cell lung cancer radiomics research: a systematic review and comparison of CLEAR and RQS frameworks. Eur Radiol 2024:10.1007/s00330-024-10736-1. [PMID: 38625613 DOI: 10.1007/s00330-024-10736-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 02/07/2024] [Accepted: 03/04/2024] [Indexed: 04/17/2024]
Abstract
BACKGROUND Lung cancer, the second most common cancer, presents persistently dismal prognoses. Radiomics, a promising field, aims to provide novel imaging biomarkers to improve outcomes. However, clinical translation faces reproducibility challenges, despite efforts to address them with quality scoring tools. OBJECTIVE This study had two objectives: 1) identify radiomics biomarkers in post-radiotherapy stage III/IV nonsmall cell lung cancer (NSCLC) patients, 2) evaluate research quality using the CLEAR (CheckList_for_EvaluAtion_of_Radiomics_research), RQS (Radiomics_Quality_Score) frameworks, and formulate an amalgamated CLEAR-RQS tool to enhance scientific rigor. MATERIALS AND METHODS A systematic literature review (Jun-Aug 2023, MEDLINE/PubMed/SCOPUS) was conducted concerning stage III/IV NSCLC, radiotherapy, and radiomic features (RF). Extracted data included study design particulars, such as sample size, radiotherapy/CT technique, selected RFs, and endpoints. CLEAR and RQS were merged into a CLEAR-RQS checklist. Three readers appraised articles utilizing CLEAR, RQS, and CLEAR-RQS metrics. RESULTS Out of 871 articles, 11 met the inclusion/exclusion criteria. The Median cohort size was 91 (range: 10-337) with 9 studies being single-center. No common RF were identified. The merged CLEAR-RQS checklist comprised 61 items. Most unreported items were within CLEAR's "methods" and "open-source," and within RQS's "phantom-calibration," "registry-enrolled prospective-trial-design," and "cost-effective-analysis" sections. No study scored above 50% on RQS. Median CLEAR scores were 55.74% (32.33/58 points), and for RQS, 17.59% (6.3/36 points). CLEAR-RQS article ranking fell between CLEAR and RQS and aligned with CLEAR. CONCLUSION Radiomics research in post-radiotherapy stage III/IV NSCLC exhibits variability and frequently low-quality reporting. The formulated CLEAR-RQS checklist may facilitate education and holds promise for enhancing radiomics research quality. CLINICAL RELEVANCE STATEMENT Current radiomics research in the field of stage III/IV postradiotherapy NSCLC is heterogenous, lacking reproducibility, with no identified imaging biomarker. Radiomics research quality assessment tools may enhance scientific rigor and thereby facilitate radiomics translation into clinical practice. KEY POINTS There is heterogenous and low radiomics research quality in postradiotherapy stage III/IV nonsmall cell lung cancer. Barriers to reproducibility are small cohort size, nonvalidated studies, missing technical parameters, and lack of data, code, and model sharing. CLEAR (CheckList_for_EvaluAtion_of_Radiomics_research), RQS (Radiomics_Quality_Score), and the amalgamated CLEAR-RQS tool are useful frameworks for assessing radiomics research quality and may provide a valuable resource for educational purposes in the field of radiomics.
Collapse
Affiliation(s)
- Kevin Tran
- Department of Cancer Imaging, Peter MacCallum Cancer Centre, 305 Grattan St, Melbourne, VIC 3000, Australia
- Faculty of Medicine, Dentistry & Health Sciences, University of Melbourne, Parkville, VIC 3052, Australia
| | - Daniel Ginzburg
- Department of Diagnostic and Interventional Radiology, Venusberg Campus 1, 53127, Bonn, Germany
| | - Wei Hong
- Personalised Oncology Division, The Walter and Eliza Hall Institute of Medical Research, Melbourne, VIC, Australia
| | - Ulrike Attenberger
- Department of Diagnostic and Interventional Radiology, Venusberg Campus 1, 53127, Bonn, Germany
| | - Hyun Soo Ko
- Department of Cancer Imaging, Peter MacCallum Cancer Centre, 305 Grattan St, Melbourne, VIC 3000, Australia.
- Department of Diagnostic and Interventional Radiology, Venusberg Campus 1, 53127, Bonn, Germany.
- The Sir Peter MacCallum Department of Oncology, University of Melbourne, 305 Grattan St, Melbourne, VIC 3000, Australia.
| |
Collapse
|
9
|
Kocak B, Akinci D'Antonoli T, Mercaldo N, Alberich-Bayarri A, Baessler B, Ambrosini I, Andreychenko AE, Bakas S, Beets-Tan RGH, Bressem K, Buvat I, Cannella R, Cappellini LA, Cavallo AU, Chepelev LL, Chu LCH, Demircioglu A, deSouza NM, Dietzel M, Fanni SC, Fedorov A, Fournier LS, Giannini V, Girometti R, Groot Lipman KBW, Kalarakis G, Kelly BS, Klontzas ME, Koh DM, Kotter E, Lee HY, Maas M, Marti-Bonmati L, Müller H, Obuchowski N, Orlhac F, Papanikolaou N, Petrash E, Pfaehler E, Pinto Dos Santos D, Ponsiglione A, Sabater S, Sardanelli F, Seeböck P, Sijtsema NM, Stanzione A, Traverso A, Ugga L, Vallières M, van Dijk LV, van Griethuysen JJM, van Hamersvelt RW, van Ooijen P, Vernuccio F, Wang A, Williams S, Witowski J, Zhang Z, Zwanenburg A, Cuocolo R. METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII. Insights Imaging 2024; 15:8. [PMID: 38228979 PMCID: PMC10792137 DOI: 10.1186/s13244-023-01572-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 11/20/2023] [Indexed: 01/18/2024] Open
Abstract
PURPOSE To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies. METHODS We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated. RESULT In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community. CONCLUSION In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers. CRITICAL RELEVANCE STATEMENT A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning. KEY POINTS • A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score ( https://metricsscore.github.io/metrics/METRICS.html ) and a repository created to collect feedback from the radiomics community ( https://github.com/metricsscore/metrics ).
Collapse
Affiliation(s)
- Burak Kocak
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, Turkey
| | - Tugba Akinci D'Antonoli
- Institute of Radiology and Nuclear Medicine, Cantonal Hospital Baselland, Liestal, Switzerland.
| | - Nathaniel Mercaldo
- Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | | | - Bettina Baessler
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Würzburg, Germany
| | - Ilaria Ambrosini
- Department of Translational Research, Academic Radiology, University of Pisa, Pisa, Italy
| | - Anna E Andreychenko
- Laboratory for Digital Public Health Technologies, ITMO University, St. Petersburg, Russian Federation
| | - Spyridon Bakas
- Division of Computational Pathology, Department of Pathology and Laboratory Medicine, School of Medicine, Indiana University, Indianapolis, IN, USA
- Center for Federated Learning in Precision Medicine, Indiana University, Indianapolis, IN, USA
| | - Regina G H Beets-Tan
- Department of Radiology, The Netherlands Cancer Institute, Amsterdam, the Netherlands
- GROW School for Oncology and Developmental Biology, Maastricht University Medical Center, Maastricht, the Netherlands
- Institute of Regional Health Research, University of Southern Denmark, Odense, Denmark
| | - Keno Bressem
- Department of Radiology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt- Universität zu Berlin, Berlin, Germany
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Irene Buvat
- Institut Curie, Inserm, PSL University, Laboratory of Translational Imaging in Oncology, Orsay, France
| | - Roberto Cannella
- Section of Radiology - Department of Biomedicine, Neuroscience and Advanced Diagnostics (BiND), University of Palermo, Palermo, Italy
| | | | - Armando Ugo Cavallo
- Division of Radiology, Istituto Dermopatico dell'Immacolata (IDI) IRCCS, Rome, Italy
| | - Leonid L Chepelev
- Joint Department of Medical Imaging, University Health Network, University of Toronto, Toronto, Canada
| | - Linda Chi Hang Chu
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, USA
| | - Aydin Demircioglu
- Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital , Essen, Germany
| | - Nandita M deSouza
- Division of Radiotherapy and Imaging, The Institute of Cancer Research, London, UK
- Department of Imaging, The Royal Marsden National Health Service (NHS) Foundation Trust, London, UK
| | - Matthias Dietzel
- Department of Radiology, University Hospital Erlangen, Erlangen, Germany
| | | | - Andrey Fedorov
- Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Laure S Fournier
- Department of Radiology, Université Paris Cité, AP-HP, Hôpital Européen Georges Pompidou, PARCC UMRS 970, INSERM, Paris, France
| | | | - Rossano Girometti
- Institute of Radiology, Department of Medicine, University of Udine, University Hospital S. Maria della Misericordia, Udine, Italy
| | - Kevin B W Groot Lipman
- Department of Radiology, The Netherlands Cancer Institute, Amsterdam, the Netherlands
- GROW School for Oncology and Developmental Biology, Maastricht University Medical Center, Maastricht, the Netherlands
- Department of Thoracic Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
| | - Georgios Kalarakis
- Department of Neuroradiology, Karolinska University Hospital, Stockholm, Sweden
- Department of Clinical Science, Division of Radiology, Intervention and Technology (CLINTEC), Karolinska Institutet, Stockholm, Sweden
- Department of Radiology, Medical School, University of Crete, Heraklion, Greece
| | - Brendan S Kelly
- Department of Radiology, St Vincent's University Hospital, Dublin, Ireland
- Insight Centre for Data Analytics, UCD, Dublin, Ireland
- School of Medicine, University College Dublin, Dublin, Ireland
| | - Michail E Klontzas
- Department of Medical Imaging, University Hospital of Heraklion, Crete, Greece
- Department of Radiology, School of Medicine, University of Crete, Heraklion, Crete, Greece
- Computational Biomedicine Laboratory, Institute of Computer Science, FORTH, Heraklion, Crete, Greece
| | - Dow-Mu Koh
- Department of Radiology, Royal Marsden Hospital, Sutton, UK
| | - Elmar Kotter
- Department of Diagnostic and Interventional Radiology, Faculty of Medicine and Medical Center-University of Freiburg, Freiburg, Germany
| | - Ho Yun Lee
- Department of Radiology and Center for Imaging Science, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea
- Department of Health Sciences and Technology, Samsung Advanced Institute for Health Science & Technology (SAIHST), Sungkyunkwan University, Seoul, South Korea
| | - Mario Maas
- Department of Radiology & Nuclear Medicine, Amsterdam UMC Location University of Amsterdam, Meibergdreef 9, Amsterdam, the Netherlands
| | - Luis Marti-Bonmati
- Medical Imaging Department and Biomedical Imaging Research Group, Hospital Universitario y Politécnico La Fe and Health Research Institute, Valencia, Spain
| | - Henning Müller
- University of Applied Sciences of Western Switzerland (HES-SO Valais), Sierra, Switzerland
- Department of Radiology and Medical Informatics, University of Geneva (UniGe), Geneva, Switzerland
| | - Nancy Obuchowski
- Quantitative Health Sciences, Lerner Research Institute, Cleveland Clinic, Cleveland, OH, USA
| | - Fanny Orlhac
- Institut Curie, Inserm, PSL University, Laboratory of Translational Imaging in Oncology, Orsay, France
| | - Nikolaos Papanikolaou
- Computational Clinical Imaging Group, Centre for the Unknown, Champalimaud Foundation, Lisbon, Portugal
- Department of Radiology, Royal Marsden Hospital and The Institute of Cancer Research, London, UK
| | - Ekaterina Petrash
- Radiology department, Research Institute of Pediatric Oncology and Hematology n. a. L.A. Durnov, National Medical Research Center of Oncology n. a. N.N. Blokhin Ministry of Health of Russian Federation, Moscow, Russia
- Medical Department IRA-Labs, Moscow, Russia
| | - Elisabeth Pfaehler
- Institute for advanced simulation (IAS-8): Machine learning and data analytics, Forschungszentrum Jülich, Jülich, Germany
| | - Daniel Pinto Dos Santos
- Department of Radiology, University Hospital of Cologne, Cologne, Germany
- Institute for Diagnostic and Interventional Radiology, Goethe-University Frankfurt Am Main, Frankfurt, Germany
| | - Andrea Ponsiglione
- Department of Advanced Biomedical Sciences, University of Naples Federico II, Naples, Italy
| | - Sebastià Sabater
- Department of Radiation Oncology, Complejo Hospitalario Universitario de Albacete, Albacete, Spain
| | - Francesco Sardanelli
- Department of Biomedical Sciences for Health, Università degli Studi di Milano, Milan, Italy
- Unit of Radiology, IRCCS Policlinico San Donato, San Donato Milanese, Milan, Italy
| | - Philipp Seeböck
- Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Nanna M Sijtsema
- Department of Radiation Oncology, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands
| | - Arnaldo Stanzione
- Department of Advanced Biomedical Sciences, University of Naples Federico II, Naples, Italy
| | - Alberto Traverso
- Department of Radiotherapy, Maastro Clinic, Maastricht, the Netherlands
- School of Medicine, Vita-Salute San Raffaele University, Milan, Italy
| | - Lorenzo Ugga
- Department of Advanced Biomedical Sciences, University of Naples Federico II, Naples, Italy
| | - Martin Vallières
- Department of Computer Science, Université de Sherbrooke, Sherbrooke, Canada
- Centre de recherche du Centre hospitalier universitaire de Sherbrooke, Sherbrooke, Canada
| | - Lisanne V van Dijk
- Department of Radiation Oncology, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands
| | | | - Robbert W van Hamersvelt
- Department of Radiology, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
| | - Peter van Ooijen
- Department of Radiotherapy, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| | - Federica Vernuccio
- Section of Radiology, Department of Biomedicine, Neuroscience and Advanced Diagnosis (Bi.N.D), University of Palermo, Palermo, 90127, Italy
| | - Alan Wang
- Centre for Medical Imaging & Centre for Brain Research, Faculty of Medical and Health Sciences, Auckland Bioengineering Institute, University of Auckland, Auckland, New Zealand
| | - Stuart Williams
- Department of Radiology, Norfolk & Norwich University Hospital, Colney Lane, Norwich, Norfolk, UK
| | - Jan Witowski
- Department of Radiology, New York University Grossman School of Medicine, New York, USA
| | - Zhongyi Zhang
- School of Information and Communication Technology, Griffith University, Nathan, Brisbane, Australia
| | - Alex Zwanenburg
- National Center for Tumor Diseases (NCT/UCC), Dresden, Germany
- OncoRay - National Center for Radiation Research in Oncology, Faculty of Medicine and University Hospital Carl Gustav Carus, Technische Universität Dresden, Helmholtz-Zentrum Dresden - Rossendorf, Dresden, Germany
- German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Renato Cuocolo
- Department of Medicine, Surgery and Dentistry, University of Salerno, Baronissi, Italy
| |
Collapse
|
10
|
Kong J, Zhang D. Current status and quality of radiomics studies for predicting outcome in acute ischemic stroke patients: a systematic review and meta-analysis. Front Neurol 2024; 14:1335851. [PMID: 38229595 PMCID: PMC10789857 DOI: 10.3389/fneur.2023.1335851] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 12/15/2023] [Indexed: 01/18/2024] Open
Abstract
Background Pre-treatment prediction of reperfusion and long-term prognosis in acute ischemic stroke (AIS) patients is crucial for effective treatment and decision-making. Recent studies have demonstrated that the inclusion of radiomics data can improve the performance of predictive models. This paper reviews published studies focused on radiomics-based prediction of reperfusion and long-term prognosis in AIS patients. Methods We systematically searched PubMed, Web of Science, and Cochrane databases up to September 9, 2023, for studies on radiomics-based prediction of AIS patient outcomes. The methodological quality of the included studies was evaluated using the phase classification criteria, the radiomics quality scoring (RQS) tool, and the Prediction model Risk Of Bias Assessment Tool (PROBAST). Two separate meta-analyses were performed of these studies that predict long-term prognosis and reperfusion in AIS patients. Results Sixteen studies with sample sizes ranging from 67 to 3,001 were identified. Ten studies were classified as phase II, and the remaining were categorized as phase 0 (n = 2), phase I (n = 1), and phase III (n = 3). The mean RQS score of all studies was 39.41%, ranging from 5.56 to 75%. Most studies (87.5%, 14/16) were at high risk of bias due to their retrospective design. The remaining two studies were categorized as low risk and unclear risk, respectively. The pooled area under the curve (AUC) was 0.88 [95% confidence interval (CI) 0.84-0.92] for predicting the long-term prognosis and 0.80 (95% CI 0.74-0.86) for predicting reperfusion in AIS. Conclusion Radiomics has the potential to predict immediate reperfusion and long-term outcomes in AIS patients. Further external validation and evaluation within the clinical workflow can facilitate personalized treatment for AIS patients. This systematic review provides valuable insights for optimizing radiomics prediction systems for both reperfusion and long-term outcomes in AIS patients. Systematic review registration https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42023461671, identifier CRD42023461671.
Collapse
Affiliation(s)
- Jinfen Kong
- Department of Radiology, Yuhuan Second People's Hospital, Yuhuan, Taizhou, Zhejiang, China
| | | |
Collapse
|