1
|
Yoshimura H, Kawahara D, Saito A, Ozawa S, Nagata Y. Prediction of prognosis in glioblastoma with radiomics features extracted by synthetic MRI images using cycle-consistent GAN. Phys Eng Sci Med 2024:10.1007/s13246-024-01443-8. [PMID: 38884673 DOI: 10.1007/s13246-024-01443-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Accepted: 05/14/2024] [Indexed: 06/18/2024]
Abstract
To propose a style transfer model for multi-contrast magnetic resonance imaging (MRI) images with a cycle-consistent generative adversarial network (CycleGAN) and evaluate the image quality and prognosis prediction performance for glioblastoma (GBM) patients from the extracted radiomics features. Style transfer models of T1 weighted MRI image (T1w) to T2 weighted MRI image (T2w) and T2w to T1w with CycleGAN were constructed using the BraTS dataset. The style transfer model was validated with the Cancer Genome Atlas Glioblastoma Multiforme (TCGA-GBM) dataset. Moreover, imaging features were extracted from real and synthesized images. These features were transformed to rad-scores by the least absolute shrinkage and selection operator (LASSO)-Cox regression. The prognosis performance was estimated by the Kaplan-Meier method. For the accuracy of the image quality of the real and synthesized MRI images, the MI, RMSE, PSNR, and SSIM were 0.991 ± 2.10 × 10 - 4 , 2.79 ± 0.16, 40.16 ± 0.38, and 0.995 ± 2.11 × 10 - 4 , for T2w, and .992 ± 2.63 × 10 - 4 , 2.49 ± 6.89 × 10 - 2 , 40.51 ± 0.22, and 0.993 ± 3.40 × 10 - 4 for T1w, respectively. The survival time had a significant difference between good and poor prognosis groups for both real and synthesized T2w (p < 0.05). However, the survival time had no significant difference between good and poor prognosis groups for both real and synthesized T1w. On the other hand, there was no significant difference between the real and synthesized T2w in both good and poor prognoses. The results of T1w were similar in the point that there was no significant difference between the real and synthesized T1w. It was found that the synthesized image could be used for prognosis prediction. The proposed prognostic model using CycleGAN could reduce the cost and time of image scanning, leading to a promotion to build the patient's outcome prediction with multi-contrast images.
Collapse
Affiliation(s)
- Hisanori Yoshimura
- Department of Radiation Oncology, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, 734-8551, Japan
- Department of Radiology, National Hospital Organization Kure Medical Center, Hiroshima, Japan
| | - Daisuke Kawahara
- Department of Radiation Oncology, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, 734-8551, Japan.
| | - Akito Saito
- Department of Radiation Oncology, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, 734-8551, Japan
| | - Shuichi Ozawa
- Department of Radiation Oncology, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, 734-8551, Japan
- Hiroshima High-Precision Radiotherapy Cancer Center, Hiroshima, 732-0057, Japan
| | - Yasushi Nagata
- Department of Radiation Oncology, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, 734-8551, Japan
- Hiroshima High-Precision Radiotherapy Cancer Center, Hiroshima, 732-0057, Japan
| |
Collapse
|
2
|
Wang H, Sang L, Xu J, Huang C, Huang Z. Multiparametric MRI-based radiomic nomogram for predicting HER-2 2+ status of breast cancer. Heliyon 2024; 10:e29875. [PMID: 38720718 PMCID: PMC11076642 DOI: 10.1016/j.heliyon.2024.e29875] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2023] [Revised: 04/16/2024] [Accepted: 04/16/2024] [Indexed: 05/12/2024] Open
Abstract
Objective To explore the application of multiparametric MRI-based radiomic nomogram for assessing HER-2 2+ status of breast cancer (BC). Methods Patients with pathology-proven HER-2 2+ invasive BC, who underwent preoperative MRI were divided into training (72 patients, 21 HER-2-positive and 51 HER-2-negative) and validation (32 patients, 9 HER-2-positive and 23 HER-2-negative) sets by randomization. All were classified as HER-2 2+ FISH-positive (HER-2-positive) or -negative (HER-2-negative) according to IHC and FISH. The 3D VOI was drawn on MR images by two radiologists. ADC, T2WI, and DCE images were analyzed separately to extract features (n = 1906). L1 regularization, F-test, and other methods were used to reduce dimensionality. Binary radiomics prediction models using features from single or combined imaging sequences were constructed using logistic regression (LR) classifier then and validated on a validation dataset. To build a radiomics nomogram, multivariate LR analysis was conducted to identify independent indicators. An evaluation of the model's predictive efficacy was made using AUC. Results On the basis of combined ADC, T2WI, and DCE images, ten radiomic features were extracted following feature dimensionality reduction. There was superior diagnostic efficiency of radiomic signature using all three sequences compared to either one or two sequences (AUC for training group: 0.883; AUC for validation group: 0.816). Based on multivariate LR analysis, radiomic signature and peritumoral edema were independent predictors for identifying HER-2 2 +. In both training and validation datasets, nomograms combining peritumoral edema and radiomics signature demonstrated an effective discrimination (AUCs were respectively 0.966 and 0. 884). Conclusion The nomogram that incorporated peritumoral edema and multiparametric MRI-based radiomic signature can be used to effectively predict the HER-2 2+ status of BC.
Collapse
Affiliation(s)
- Haili Wang
- Department of Radiology, Shandong Provincial Hospital Affliated to Shandong First Medical University, Jinan, 250021, Shandong, China
| | - Li Sang
- Department of Radiology, Shandong Provincial Hospital Affliated to Shandong First Medical University, Jinan, 250021, Shandong, China
| | - Jingxu Xu
- Department of Research Collaboration, R&D Center, Beijing Deepwise & League of, PHD Technology Co.Ltd, Beijing, China
| | - Chencui Huang
- Department of Research Collaboration, R&D Center, Beijing Deepwise & League of, PHD Technology Co.Ltd, Beijing, China
| | - Zhaoqin Huang
- Department of Radiology, Shandong Provincial Hospital Affliated to Shandong First Medical University, Jinan, 250021, Shandong, China
| |
Collapse
|
3
|
Park CJ, Kim S, Han K, Ahn SS, Kim D, Park YW, Chang JH, Kim SH, Lee SK. Diffusion- and Perfusion-Weighted MRI Radiomics for Survival Prediction in Patients with Lower-Grade Gliomas. Yonsei Med J 2024; 65:283-292. [PMID: 38653567 PMCID: PMC11045349 DOI: 10.3349/ymj.2023.0323] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 10/27/2023] [Accepted: 12/13/2023] [Indexed: 04/25/2024] Open
Abstract
PURPOSE Lower-grade gliomas of histologic grades 2 and 3 follow heterogenous clinical outcomes, which necessitates risk stratification. This study aimed to evaluate whether diffusion-weighted and perfusion-weighted MRI radiomics allow overall survival (OS) prediction in patients with lower-grade gliomas and investigate its prognostic value. MATERIALS AND METHODS In this retrospective study, radiomic features were extracted from apparent diffusion coefficient, relative cerebral blood volume map, and Ktrans map in patients with pathologically confirmed lower-grade gliomas (January 2012-February 2019). The radiomics risk score (RRS) calculated from selected features constituted a radiomics model. Multivariable Cox regression analysis, including clinical features and RRS, was performed. The models' integrated area under the receiver operating characteristic curves (iAUCs) were compared. The radiomics model combined with clinical features was presented as a nomogram. RESULTS The study included 129 patients (median age, 44 years; interquartile range, 37-57 years; 63 female): 90 patients for training set and 39 patients for test set. The RRS was an independent risk factor for OS with a hazard ratio of 6.01. The combined clinical and radiomics model achieved superior performance for OS prediction compared to the clinical model in both training (iAUC, 0.82 vs. 0.72, p=0.002) and test sets (0.88 vs. 0.76, p=0.04). The radiomics nomogram combined with clinical features exhibited good agreement between the actual and predicted OS with C-index of 0.83 and 0.87 in the training and test sets, respectively. CONCLUSION Adding diffusion- and perfusion-weighted MRI radiomics to clinical features improved survival prediction in lower-grade glioma.
Collapse
Affiliation(s)
- Chae Jung Park
- Department of Radiology, Research Institute of Radiological Science, Yongin Severance Hospital, Yonsei University College of Medicine, Seoul, Korea
| | - Sooyon Kim
- Department of Applied Statistics, Yonsei University, Seoul, Korea
| | - Kyunghwa Han
- Department of Radiology, Center for Clinical Imaging Data Science, Research Institute of Radiological Sciences, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea
| | - Sung Soo Ahn
- Department of Radiology, Center for Clinical Imaging Data Science, Research Institute of Radiological Sciences, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea.
| | - Dain Kim
- Graduate School of Artificial Intelligence, Pohang University of Science and Technology, Pohang, Korea
| | - Yae Won Park
- Department of Radiology, Center for Clinical Imaging Data Science, Research Institute of Radiological Sciences, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea
| | - Jong Hee Chang
- Department of Neurosurgery, Yonsei University College of Medicine, Seoul, Korea
| | - Se Hoon Kim
- Department of Pathology, Yonsei University College of Medicine, Seoul, Korea
| | - Seung-Koo Lee
- Department of Radiology, Center for Clinical Imaging Data Science, Research Institute of Radiological Sciences, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea
| |
Collapse
|
4
|
Lee JO, Ahn SS, Choi KS, Lee J, Jang J, Park JH, Hwang I, Park CK, Park SH, Chung JW, Choi SH. Added prognostic value of 3D deep learning-derived features from preoperative MRI for adult-type diffuse gliomas. Neuro Oncol 2024; 26:571-580. [PMID: 37855826 PMCID: PMC10912011 DOI: 10.1093/neuonc/noad202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Indexed: 10/20/2023] Open
Abstract
BACKGROUND To investigate the prognostic value of spatial features from whole-brain MRI using a three-dimensional (3D) convolutional neural network for adult-type diffuse gliomas. METHODS In a retrospective, multicenter study, 1925 diffuse glioma patients were enrolled from 5 datasets: SNUH (n = 708), UPenn (n = 425), UCSF (n = 500), TCGA (n = 160), and Severance (n = 132). The SNUH and Severance datasets served as external test sets. Precontrast and postcontrast 3D T1-weighted, T2-weighted, and T2-FLAIR images were processed as multichannel 3D images. A 3D-adapted SE-ResNeXt model was trained to predict overall survival. The prognostic value of the deep learning-based prognostic index (DPI), a spatial feature-derived quantitative score, and established prognostic markers were evaluated using Cox regression. Model evaluation was performed using the concordance index (C-index) and Brier score. RESULTS The MRI-only median DPI survival prediction model achieved C-indices of 0.709 and 0.677 (BS = 0.142 and 0.215) and survival differences (P < 0.001 and P = 0.002; log-rank test) for the SNUH and Severance datasets, respectively. Multivariate Cox analysis revealed DPI as a significant prognostic factor, independent of clinical and molecular genetic variables: hazard ratio = 0.032 and 0.036 (P < 0.001 and P = 0.004) for the SNUH and Severance datasets, respectively. Multimodal prediction models achieved higher C-indices than models using only clinical and molecular genetic variables: 0.783 vs. 0.774, P = 0.001, SNUH; 0.766 vs. 0.748, P = 0.023, Severance. CONCLUSIONS The global morphologic feature derived from 3D CNN models using whole-brain MRI has independent prognostic value for diffuse gliomas. Combining clinical, molecular genetic, and imaging data yields the best performance.
Collapse
Affiliation(s)
- Jung Oh Lee
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
- Artificial Intelligence Collaborative Network, Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Sung Soo Ahn
- Department of Radiology, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Kyu Sung Choi
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
- Artificial Intelligence Collaborative Network, Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Junhyeok Lee
- Interdisciplinary Programs in Cancer Biology Major, Seoul National University Graduate School, Seoul, Republic of Korea
| | - Joon Jang
- Department of Biomedical Sciences, Seoul National University, Seoul, Republic of Korea
| | - Jung Hyun Park
- Department of Radiology, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, South Korea
| | - Inpyeong Hwang
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
- Artificial Intelligence Collaborative Network, Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Chul-Kee Park
- Department of Neurosurgery, Seoul National University Hospital, Seoul, Republic of Korea
| | - Sung Hye Park
- Department of Pathology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Jin Wook Chung
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
- Artificial Intelligence Collaborative Network, Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
- Institute of Innovate Biomedical Technology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Seung Hong Choi
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
- Artificial Intelligence Collaborative Network, Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
- Center for Nanoparticle Research, Institute for Basic Science, Seoul, Republic of Korea
| |
Collapse
|
5
|
Yoon J, Baek N, Yoo RE, Choi SH, Kim TM, Park CK, Park SH, Won JK, Lee JH, Lee ST, Choi KS, Lee JY, Hwang I, Kang KM, Yun TJ. Added value of dynamic contrast-enhanced MR imaging in deep learning-based prediction of local recurrence in grade 4 adult-type diffuse gliomas patients. Sci Rep 2024; 14:2171. [PMID: 38273075 PMCID: PMC10810891 DOI: 10.1038/s41598-024-52841-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Accepted: 01/24/2024] [Indexed: 01/27/2024] Open
Abstract
Local recurrences in patients with grade 4 adult-type diffuse gliomas mostly occur within residual non-enhancing T2 hyperintensity areas after surgical resection. Unfortunately, it is challenging to distinguish non-enhancing tumors from edema in the non-enhancing T2 hyperintensity areas using conventional MRI alone. Quantitative DCE MRI parameters such as Ktrans and Ve convey permeability information of glioblastomas that cannot be provided by conventional MRI. We used the publicly available nnU-Net to train a deep learning model that incorporated both conventional and DCE MRI to detect the subtle difference in vessel leakiness due to neoangiogenesis between the non-recurrence area and the local recurrence area, which contains a higher proportion of high-grade glioma cells. We found that the addition of Ve doubled the sensitivity while nonsignificantly decreasing the specificity for prediction of local recurrence in glioblastomas, which implies that the combined model may result in fewer missed cases of local recurrence. The deep learning model predictive of local recurrence may enable risk-adapted radiotherapy planning in patients with grade 4 adult-type diffuse gliomas.
Collapse
Affiliation(s)
- Jungbin Yoon
- Department of Radiology, Seoul National University College of Medicine, 101, Daehangno, Jongno-gu, Seoul, 03080, Republic of Korea
| | - Nayeon Baek
- Department of Radiology, Seoul National University College of Medicine, 101, Daehangno, Jongno-gu, Seoul, 03080, Republic of Korea
| | - Roh-Eul Yoo
- Department of Radiology, Seoul National University College of Medicine, 101, Daehangno, Jongno-gu, Seoul, 03080, Republic of Korea.
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea.
| | - Seung Hong Choi
- Department of Radiology, Seoul National University College of Medicine, 101, Daehangno, Jongno-gu, Seoul, 03080, Republic of Korea.
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea.
- Center for Nanoparticle Research, Institute for Basic Science (IBS), Seoul, Republic of Korea.
- School of Chemical and Biological Engineering, Seoul National University, 1, Gwanak-ro, Gwanak-gu, Seoul, 302-909, Republic of Korea.
| | - Tae Min Kim
- Department of Internal Medicine, Cancer Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Chul-Kee Park
- Department of Neurosurgery, Biomedical Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Sung-Hye Park
- Department of Pathology, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Jae-Kyung Won
- Department of Pathology, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Joo Ho Lee
- Department of Radiation Oncology, Cancer Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Soon Tae Lee
- Department of Neurology, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Kyu Sung Choi
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Ji Ye Lee
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Inpyeong Hwang
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Koung Mi Kang
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Tae Jin Yun
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea
| |
Collapse
|
6
|
Zhang H, Zhang H, Zhang Y, Zhou B, Wu L, Yang W, Lei Y, Huang B. Multiparametric MRI-based fusion radiomics for predicting telomerase reverse transcriptase (TERT) promoter mutations and progression-free survival in glioblastoma: a multicentre study. Neuroradiology 2024; 66:81-92. [PMID: 37978079 DOI: 10.1007/s00234-023-03245-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 10/29/2023] [Indexed: 11/19/2023]
Abstract
PURPOSE This study evaluated the performance of multiparametric magnetic resonance imaging (MRI)-based fusion radiomics models (MMFRs) to predict telomerase reverse transcriptase (TERT) promoter mutation status and progression-free survival (PFS) in glioblastoma patients. METHODS We retrospectively analysed 208 glioblastoma patients from two hospitals. Quantitative imaging features were extracted from each patient's T1-weighted, T1-weighted contrast-enhanced, and T2-weighted preoperative images. Using a coarse-to-fine feature selection strategy, four radiomics signature models were constructed based on the three MRI sequences and their combination for TERT promoter mutation status and PFS; model performance was subsequently evaluated. Subgroup analyses were performed by the radiomics signature of TERT promoter mutation status and PFS to distinguish patients who could benefit from prolonged temozolomide chemotherapy cycles. RESULTS TERT promoter mutation status was best predicted by MMFR, with an area under the curve (AUC) of 0.816 and 0.812 for the training and internal validation sets, respectively. The external test set also achieved stable and optimal prediction results (AUC, 0.823). MMFR better predicted patient PFS compared with the single-sequence radiomics signature in the test set (C-index, 0.643 vs 0.561 vs 0.620 vs 0.628). Subgroup analyses showed that more than six cycles of postoperative temozolomide chemotherapy were associated with improved PFS for patients in class 2 (high TERT promoter mutation and high survival rates; HR, 0.222; 95% CI, 0.054 - 0.923; p = 0.025). CONCLUSION MMFR is an effective method to predict TERT promoter mutations and PFS in patients with glioblastoma. Moreover, subgroup analysis could differentiate patients who may benefit from prolonged TMZ chemotherapy cycles.
Collapse
Affiliation(s)
- Hongbo Zhang
- The Second School of Clinical Medicine, Southern Medical University, Guangzhou, 510515, China
- Department of Radiology, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, #106 Zhongshan 2Nd Road, Guangzhou, 510080, China
- Department of Radiology, The Seventh Affiliated Hospital, Sun Yat-Sen University, Shenzhen, 517108, China
| | - Hanwen Zhang
- Department of Radiology, The First Affiliated Hospital of Shenzhen University, Health Science Center, Shenzhen Second People's Hospital, #3002 SunGangXi Road, Shenzhen, 518035, China
| | - Yuze Zhang
- The Second School of Clinical Medicine, Southern Medical University, Guangzhou, 510515, China
- Department of Radiology, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, #106 Zhongshan 2Nd Road, Guangzhou, 510080, China
| | - Beibei Zhou
- Department of Radiology, The Seventh Affiliated Hospital, Sun Yat-Sen University, Shenzhen, 517108, China
| | - Lei Wu
- Department of Radiology, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, #106 Zhongshan 2Nd Road, Guangzhou, 510080, China
| | - Wanqun Yang
- Department of Radiology, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, #106 Zhongshan 2Nd Road, Guangzhou, 510080, China
| | - Yi Lei
- Department of Radiology, The First Affiliated Hospital of Shenzhen University, Health Science Center, Shenzhen Second People's Hospital, #3002 SunGangXi Road, Shenzhen, 518035, China.
| | - Biao Huang
- The Second School of Clinical Medicine, Southern Medical University, Guangzhou, 510515, China.
- Department of Radiology, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, #106 Zhongshan 2Nd Road, Guangzhou, 510080, China.
| |
Collapse
|
7
|
Choi Y, Jang J, Kim BS, Ahn KJ. Pretreatment MR-based radiomics in patients with glioblastoma: A systematic review and meta-analysis of prognostic endpoints. Eur J Radiol 2023; 168:111130. [PMID: 37827087 DOI: 10.1016/j.ejrad.2023.111130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 09/23/2023] [Accepted: 09/30/2023] [Indexed: 10/14/2023]
Abstract
PURPOSE Recent studies have shown promise of MR-based radiomics in predicting the survival of patients with untreated glioblastoma. This study aimed to comprehensively collate evidence to assess the prognostic value of radiomics in glioblastoma. METHODS PubMed-MEDLINE, Embase, and Web of Science were searched to find original articles investigating the prognostic value of MR-based radiomics in glioblastoma published up to July 14, 2023. Concordance indexes (C-indexes) and Cox proportional hazards ratios (HRs) of overall survival (OS) and progression-free survival (PFS) were pooled via random-effects modeling. For studies aimed at classifying long-term and short-term PFS, a hierarchical regression model was used to calculate pooled sensitivity and specificity. Between-study heterogeneity was assessed using the Higgin inconsistency index (I2). Subgroup regression analysis was performed to find potential factors contributing to heterogeneity. Publication bias was assessed via funnel plots and the Egger test. RESULTS Among 1371 abstracts, 18 and 17 studies were included for qualitative and quantitative data synthesis, respectively. Respective pooled C-indexes and HRs for OS were 0.65 (95 % confidence interval [CI], 0.58-0.72) and 2.88 (95 % CI, 2.28-3.64), whereas those for PFS were 0.61 (95 % CI, 0.55-0.66) and 2.78 (95 % CI, 1.91-4.03). Among 4 studies that predicted short-term PFS, the pooled sensitivity and specificity were 0.77 (95 % CI, 0.58-0.89) and 0.60 (95 % CI, 0.45-0.73), respectively. There was a substantial between-study heterogeneity among studies with the survival endpoint of OS C-index (n = 9, I2 = 83.8 %). Publication bias was not observed overall. CONCLUSION Pretreatment MR-based radiomics provided modest prognostic value in both OS and PFS in patients with glioblastoma.
Collapse
Affiliation(s)
- Yangsean Choi
- Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Republic of Korea; Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Centre, Seoul, Republic of Korea
| | - Jinhee Jang
- Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Republic of Korea
| | - Bum-Soo Kim
- Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Republic of Korea
| | - Kook-Jin Ahn
- Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Republic of Korea.
| |
Collapse
|
8
|
Zhang J, Teng X, Zhang X, Lam SK, Lin Z, Liang Y, Yu H, Siu SWK, Chang ATY, Zhang H, Kong FM, Yang R, Cai J. Comparing effectiveness of image perturbation and test retest imaging in improving radiomic model reliability. Sci Rep 2023; 13:18263. [PMID: 37880324 PMCID: PMC10600245 DOI: 10.1038/s41598-023-45477-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2023] [Accepted: 10/19/2023] [Indexed: 10/27/2023] Open
Abstract
Image perturbation is a promising technique to assess radiomic feature repeatability, but whether it can achieve the same effect as test-retest imaging on model reliability is unknown. This study aimed to compare radiomic model reliability based on repeatable features determined by the two methods using four different classifiers. A 191-patient public breast cancer dataset with 71 test-retest scans was used with pre-determined 117 training and 74 testing samples. We collected apparent diffusion coefficient images and manual tumor segmentations for radiomic feature extraction. Random translations, rotations, and contour randomizations were performed on the training images, and intra-class correlation coefficient (ICC) was used to filter high repeatable features. We evaluated model reliability in both internal generalizability and robustness, which were quantified by training and testing AUC and prediction ICC. Higher testing performance was found at higher feature ICC thresholds, but it dropped significantly at ICC = 0.95 for the test-retest model. Similar optimal reliability can be achieved with testing AUC = 0.7-0.8 and prediction ICC > 0.9 at the ICC threshold of 0.9. It is recommended to include feature repeatability analysis using image perturbation in any radiomic study when test-retest is not feasible, but care should be taken when deciding the optimal feature repeatability criteria.
Collapse
Affiliation(s)
- Jiang Zhang
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Y920, Lee Shau Kee Building, Hung Hom, Kowloon, Hong Kong, China
| | - Xinzhi Teng
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Y920, Lee Shau Kee Building, Hung Hom, Kowloon, Hong Kong, China
| | - Xinyu Zhang
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Y920, Lee Shau Kee Building, Hung Hom, Kowloon, Hong Kong, China
| | - Sai-Kit Lam
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong, China
- Research Institute for Smart Ageing, The Hong Kong Polytechnic University, Hong Kong, China
| | - Zhongshi Lin
- Shenzhen Institute for Drug Control (Shenzhen Testing Center of Medical Devices), Shenzhen, China
| | - Yongyi Liang
- Shenzhen Institute for Drug Control (Shenzhen Testing Center of Medical Devices), Shenzhen, China
| | - Hao Yu
- Institute of Biomedical and Health Engineering, Chinese Academy of Sciences Shenzhen Institutes of Advanced Technology, Shenzhen, China
| | - Steven Wai Kwan Siu
- Department of Clinical Oncology, The University of Hong Kong, Hong Kong, China
| | - Amy Tien Yee Chang
- Department of Clinical Oncology, The University of Hong Kong, Hong Kong, China
| | - Hua Zhang
- Beijing Linking Medical Technology Co., Ltd., Beijing, China
| | - Feng-Ming Kong
- Department of Clinical Oncology, The University of Hong Kong, Hong Kong, China
- Department of Clinical Oncology, The University of Hong Kong-Shenzhen Hospital, Shenzhen, China
| | - Ruijie Yang
- Department of Radiation Oncology, Cancer Center, Peking University Third Hospital, Beijing, China
| | - Jing Cai
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Y920, Lee Shau Kee Building, Hung Hom, Kowloon, Hong Kong, China.
- Research Institute for Smart Ageing, The Hong Kong Polytechnic University, Hong Kong, China.
- The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen, China.
| |
Collapse
|
9
|
Le VH, Minh TNT, Kha QH, Le NQK. A transfer learning approach on MRI-based radiomics signature for overall survival prediction of low-grade and high-grade gliomas. Med Biol Eng Comput 2023; 61:2699-2712. [PMID: 37432527 DOI: 10.1007/s11517-023-02875-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 06/20/2023] [Indexed: 07/12/2023]
Abstract
Lower-grade gliomas (LGG) can eventually progress to glioblastoma (GBM) and death. In the context of the transfer learning approach, we aimed to train and test an MRI-based radiomics model for predicting survival in GBM patients and validate it in LGG patients. From each patient's 704 MRI-based radiomics features, we chose seventeen optimal radiomics signatures in the GBM training set (n = 71) and used these features in both the GBM testing set (n = 31) and LGG validation set (n = 107) for further analysis. Each patient's risk score, calculated based on those optimal radiomics signatures, was chosen to represent the radiomics model. We compared the radiomics model with clinical, gene status models, and combined model integrating radiomics, clinical, and gene status in predicting survival. The average iAUCs of combined models in training, testing, and validation sets were respectively 0.804, 0.878, and 0.802, and those of radiomics models were 0.798, 0.867, and 0.717. The average iAUCs of gene status and clinical models ranged from 0.522 to 0.735 in all three sets. The radiomics model trained in GBM patients can effectively predict the overall survival of GBM and LGG patients, and the combined model improved this ability.
Collapse
Affiliation(s)
- Viet Huan Le
- International Ph.D. Program in Medicine, College of Medicine, Taipei Medical University, Taipei, 110, Taiwan
- Department of Thoracic Surgery, Khanh Hoa General Hospital, Nha Trang City, 65000, Vietnam
| | - Tran Nguyen Tuan Minh
- International Ph.D. Program in Medicine, College of Medicine, Taipei Medical University, Taipei, 110, Taiwan
| | - Quang Hien Kha
- International Ph.D. Program in Medicine, College of Medicine, Taipei Medical University, Taipei, 110, Taiwan
| | - Nguyen Quoc Khanh Le
- Professional Master Program in Artificial Intelligence in Medicine, College of Medicine, Taipei Medical University, Taipei, 110, Taiwan.
- Research Center for Artificial Intelligence in Medicine, Taipei Medical University, Taipei, 110, Taiwan.
- AIBioMed Research Group, Taipei Medical University, Taipei, 110, Taiwan.
- Translational Imaging Research Center, Taipei Medical University Hospital, Taipei, 110, Taiwan.
| |
Collapse
|
10
|
Sohn B, Park K, Ahn SS, Park YW, Choi SH, Kang SG, Kim SH, Chang JH, Lee SK. Dynamic contrast-enhanced MRI radiomics model predicts epidermal growth factor receptor amplification in glioblastoma, IDH-wildtype. J Neurooncol 2023; 164:341-351. [PMID: 37689596 DOI: 10.1007/s11060-023-04435-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Accepted: 08/23/2023] [Indexed: 09/11/2023]
Abstract
PURPOSE To develop and validate a dynamic contrast-enhanced (DCE) MRI-based radiomics model to predict epidermal growth factor receptor (EGFR) amplification in patients with glioblastoma, isocitrate dehydrogenase (IDH) wildtype. METHODS Patients with pathologically confirmed glioblastoma, IDH wildtype, from January 2015 to December 2020, with an EGFR amplification status, were included. Patients who did not undergo DCE or conventional brain MRI were excluded. Patients were categorized into training and test sets by a ratio of 7:3. DCE MRI data were used to generate volume transfer constant (Ktrans) and extracellular volume fraction (Ve) maps. Ktrans, Ve, and conventional MRI were then used to extract the radiomics features, from which the prediction models for EGFR amplification status were developed and validated. RESULTS A total of 190 patients (mean age, 59.9; male, 55.3%), divided into training (n = 133) and test (n = 57) sets, were enrolled. In the test set, the radiomics model using the Ktrans map exhibited the highest area under the receiver operating characteristic curve (AUROC), 0.80 (95% confidence interval [CI], 0.65-0.95). The AUROC for the Ve map-based and conventional MRI-based models were 0.74 (95% CI, 0.58-0.90) and 0.76 (95% CI, 0.61-0.91). CONCLUSION The DCE MRI-based radiomics model that predicts EGFR amplification in glioblastoma, IDH wildtype, was developed and validated. The MRI-based radiomics model using the Ktrans map has higher AUROC than conventional MRI.
Collapse
Affiliation(s)
- Beomseok Sohn
- Department of Radiology and Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, South Korea
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea
| | - Kisung Park
- Department of Radiology and Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, South Korea
- Department of Mechanical Engineering, Pohang University of Science and Technology, Pohang, South Korea
| | - Sung Soo Ahn
- Department of Radiology and Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, South Korea.
| | - Yae Won Park
- Department of Radiology and Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, South Korea
| | - Seung Hong Choi
- Department of Radiology, Seoul National University Hospital, Seoul, South Korea
| | - Seok-Gu Kang
- Department of Neurosurgery, Yonsei University College of Medicine, Seoul, South Korea
| | - Se Hoon Kim
- Department of Pathology, Yonsei University College of Medicine, Seoul, South Korea
| | - Jong Hee Chang
- Department of Neurosurgery, Yonsei University College of Medicine, Seoul, South Korea
| | - Seung-Koo Lee
- Department of Radiology and Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, South Korea
| |
Collapse
|
11
|
Heo D, Lee J, Yoo RE, Choi SH, Kim TM, Park CK, Park SH, Won JK, Lee JH, Lee ST, Choi KS, Lee JY, Hwang I, Kang KM, Yun TJ. Deep learning based on dynamic susceptibility contrast MR imaging for prediction of local progression in adult-type diffuse glioma (grade 4). Sci Rep 2023; 13:13864. [PMID: 37620555 PMCID: PMC10449894 DOI: 10.1038/s41598-023-41171-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Accepted: 08/23/2023] [Indexed: 08/26/2023] Open
Abstract
Adult-type diffuse glioma (grade 4) has infiltrating nature, and therefore local progression is likely to occur within surrounding non-enhancing T2 hyperintense areas even after gross total resection of contrast-enhancing lesions. Cerebral blood volume (CBV) obtained from dynamic susceptibility contrast perfusion-weighted imaging (DSC-PWI) is a parameter that is well-known to be a surrogate marker of both histologic and angiographic vascularity in tumors. We built two nnU-Net deep learning models for prediction of early local progression in adult-type diffuse glioma (grade 4), one using conventional MRI alone and one using multiparametric MRI, including conventional MRI and DSC-PWI. Local progression areas were annotated in a non-enhancing T2 hyperintense lesion on preoperative T2 FLAIR images, using the follow-up contrast-enhanced (CE) T1-weighted (T1W) images as the reference standard. The sensitivity was doubled with the addition of nCBV (80% vs. 40%, P = 0.02) while the specificity was decreased nonsignificantly (29% vs. 48%, P = 0.39), suggesting that fewer cases of early local progression would be missed with the addition of nCBV. While the diagnostic performance of CBV model is still poor and needs improving, the multiparametric deep learning model, which presumably learned from the subtle difference in vascularity between early local progression and non-progression voxels within perilesional T2 hyperintensity, may facilitate risk-adapted radiotherapy planning in adult-type diffuse glioma (grade 4) patients.
Collapse
Affiliation(s)
- Donggeon Heo
- Department of Radiology, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Jisoo Lee
- Department of Radiology, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Roh-Eul Yoo
- Department of Radiology, Seoul National University College of Medicine, Seoul, Republic of Korea.
- Department of Radiology, Seoul National University Hospital, 101, Daehangno, Jongno-Gu, Seoul, 03080, Republic of Korea.
| | - Seung Hong Choi
- Department of Radiology, Seoul National University College of Medicine, Seoul, Republic of Korea.
- Department of Radiology, Seoul National University Hospital, 101, Daehangno, Jongno-Gu, Seoul, 03080, Republic of Korea.
- Center for Nanoparticle Research, Institute for Basic Science (IBS), Seoul, Republic of Korea.
- School of Chemical and Biological Engineering, Seoul National University, 1, Gwanak-Ro, Gwanak-Gu, Seoul, 302-909, Republic of Korea.
| | - Tae Min Kim
- Department of Internal Medicine, Cancer Research Institute, Seoul National University College of Medicine, Seoul, Korea
| | - Chul-Kee Park
- Department of Neurosurgery, Biomedical Research Institute, Seoul National University College of Medicine, Seoul, Korea
| | - Sung-Hye Park
- Department of Pathology, Seoul National University College of Medicine, Seoul, Korea
| | - Jae-Kyung Won
- Department of Pathology, Seoul National University College of Medicine, Seoul, Korea
| | - Joo Ho Lee
- Department of Radiation Oncology, Cancer Research Institute, Seoul National University College of Medicine, Seoul, Korea
| | - Soon Tae Lee
- Department of Neurology, Seoul National University College of Medicine, Seoul, Korea
| | - Kyu Sung Choi
- Department of Radiology, Seoul National University Hospital, 101, Daehangno, Jongno-Gu, Seoul, 03080, Republic of Korea
| | - Ji Ye Lee
- Department of Radiology, Seoul National University Hospital, 101, Daehangno, Jongno-Gu, Seoul, 03080, Republic of Korea
| | - Inpyeong Hwang
- Department of Radiology, Seoul National University Hospital, 101, Daehangno, Jongno-Gu, Seoul, 03080, Republic of Korea
| | - Koung Mi Kang
- Department of Radiology, Seoul National University Hospital, 101, Daehangno, Jongno-Gu, Seoul, 03080, Republic of Korea
| | - Tae Jin Yun
- Department of Radiology, Seoul National University Hospital, 101, Daehangno, Jongno-Gu, Seoul, 03080, Republic of Korea
| |
Collapse
|
12
|
Bagher-Ebadian H, Brown SL, Ghassemi MM, Nagaraja TN, Movsas B, Ewing JR, Chetty IJ. Radiomics characterization of tissues in an animal brain tumor model imaged using dynamic contrast enhanced (DCE) MRI. Sci Rep 2023; 13:10693. [PMID: 37394559 DOI: 10.1038/s41598-023-37723-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Accepted: 06/27/2023] [Indexed: 07/04/2023] Open
Abstract
Here, we investigate radiomics-based characterization of tumor vascular and microenvironmental properties in an orthotopic rat brain tumor model measured using dynamic-contrast-enhanced (DCE) MRI. Thirty-two immune compromised-RNU rats implanted with human U-251N cancer cells were imaged using DCE-MRI (7Tesla, Dual-Gradient-Echo). The aim was to perform pharmacokinetic analysis using a nested model (NM) selection technique to classify brain regions according to vasculature properties considered as the source of truth. A two-dimensional convolutional-based radiomics analysis was performed on the raw-DCE-MRI of the rat brains to generate dynamic radiomics maps. The raw-DCE-MRI and respective radiomics maps were used to build 28 unsupervised Kohonen self-organizing-maps (K-SOMs). A Silhouette-Coefficient (SC), k-fold Nested-Cross-Validation (k-fold-NCV), and feature engineering analyses were performed on the K-SOMs' feature spaces to quantify the distinction power of radiomics features compared to raw-DCE-MRI for classification of different Nested Models. Results showed that eight radiomics features outperformed respective raw-DCE-MRI in prediction of the three nested models. The average percent difference in SCs between radiomics features and raw-DCE-MRI was: 29.875% ± 12.922%, p < 0.001. This work establishes an important first step toward spatiotemporal characterization of brain regions using radiomics signatures, which is fundamental toward staging of tumors and evaluation of tumor response to different treatments.
Collapse
Affiliation(s)
- Hassan Bagher-Ebadian
- Department of Radiation Oncology, Henry Ford Health, Detroit, MI, 48202, USA.
- Department of Radiology, Michigan State University, East Lansing, MI, 48824, USA.
- Department of Osteopathic Medicine, Michigan State University, East Lansing, MI, 48824, USA.
- Department of Physics, Oakland University, Rochester, MI, 48309, USA.
| | - Stephen L Brown
- Department of Radiation Oncology, Henry Ford Health, Detroit, MI, 48202, USA
- Department of Radiology, Michigan State University, East Lansing, MI, 48824, USA
- Department of Radiation Oncology, Wayne State University, Detroit, MI, 48202, USA
| | - Mohammad M Ghassemi
- Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, 48824, USA
| | - Tavarekere N Nagaraja
- Department of Radiology, Michigan State University, East Lansing, MI, 48824, USA
- Department of Neurosurgery, Henry Ford Health, Detroit, MI, 48202, USA
| | - Benjamin Movsas
- Department of Radiation Oncology, Henry Ford Health, Detroit, MI, 48202, USA
- Department of Radiology, Michigan State University, East Lansing, MI, 48824, USA
- Department of Radiation Oncology, Wayne State University, Detroit, MI, 48202, USA
| | - James R Ewing
- Department of Radiology, Michigan State University, East Lansing, MI, 48824, USA
- Department of Physics, Oakland University, Rochester, MI, 48309, USA
- Department of Neurosurgery, Henry Ford Health, Detroit, MI, 48202, USA
- Department of Neurology, Henry Ford Health, Detroit, MI, 48202, USA
- Department of Neurology, Wayne State University, Detroit, MI, 48202, USA
| | - Indrin J Chetty
- Department of Radiation Oncology, Henry Ford Health, Detroit, MI, 48202, USA
- Department of Physics, Oakland University, Rochester, MI, 48309, USA
- Department of Radiation Oncology, Wayne State University, Detroit, MI, 48202, USA
| |
Collapse
|
13
|
Yao Y, Tan X, Yin W, Kou Y, Wang X, Jiang X, Chen S, Liu Y, Dang J, Yin J, Cheng Z. Performance of 18 F-FAPI PET/CT in assessing glioblastoma before radiotherapy: a pilot study. BMC Med Imaging 2022; 22:226. [PMID: 36566187 PMCID: PMC9789562 DOI: 10.1186/s12880-022-00952-w] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2022] [Accepted: 12/14/2022] [Indexed: 12/25/2022] Open
Abstract
BACKGROUND We aimed to determine the performance of 18 F-FAPI PET/CT used for preprocedural assessment of glioblastoma before radiotherapy. METHODS Twelve glioblastoma patients having undergone incomplete surgical resection or biopsy were examined with 18 F-FAPI PET/CT and MRI scanning before radiotherapy. All patients had confirmed tumor residues according to findings of histopathological and/or long-term clinical and radiological follow-ups. Lesion characterization data, including SUVmax and tumor-to-background ratio (TBR) on PET/CT were attained. PET/CT and MRI findings were compared in terms of number of lesions. The correlation between immunohistochemistry, molecular expression, and PET/CT parameters was also evaluated. RESULTS 18 F-FAPI PET/CT detected 16 FAPI-avid out of 23 lesions in 12 patients described on MRI. MRI was statistically different from 18 F-FAPI PET/CT for lesion detection according to the exact McNemar statistical test (P = 0.0156). The SUVmax and TBR of the glioblastomas was 7.08 ± 3.55 and 19.95 ± 13.22, respectively. The sensitivity and positive predictive value (PPV) of 18 F-FAPI PET were 69.6% and 100%, respectively. Neither the Ki-67 index nor the molecular expression was correlated with the FAPI-PET/CT parameters. CONCLUSION 18 F-FAPI PET/CT detects glioblastomas at a lower rate than MRI. However, the 100% PPV of the examination may make it useful for differentiating controversial lesions detected on MRI. The 18 F-FAPI-avid lesions are displayed more clearly probably due to a higher TBR. 18 F-FAPI PET/CT imaging might find application in glioblastoma biopsy and radiotherapy planning.
Collapse
Affiliation(s)
- Yutang Yao
- grid.54549.390000 0004 0369 4060Department of Nuclear Medicine, Sichuan Cancer Hospital&Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No.55, Section 4, South People’s Road, Sichuan 610041 Chengdu, China
| | - Xiaofei Tan
- grid.54549.390000 0004 0369 4060Department of Nuclear Medicine, Sichuan Cancer Hospital&Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No.55, Section 4, South People’s Road, Sichuan 610041 Chengdu, China
| | - Wenya Yin
- grid.54549.390000 0004 0369 4060Department of Radiation Oncology, Sichuan Cancer Hospital&Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No.55, Section 4, South People’s Road, 610041 Chengdu, China
| | - Ying Kou
- grid.54549.390000 0004 0369 4060Department of Nuclear Medicine, Sichuan Cancer Hospital&Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No.55, Section 4, South People’s Road, Sichuan 610041 Chengdu, China
| | - Xiaoxiong Wang
- grid.54549.390000 0004 0369 4060Department of Nuclear Medicine, Sichuan Cancer Hospital&Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No.55, Section 4, South People’s Road, Sichuan 610041 Chengdu, China
| | - Xiao Jiang
- grid.54549.390000 0004 0369 4060Department of Nuclear Medicine, Sichuan Cancer Hospital&Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No.55, Section 4, South People’s Road, Sichuan 610041 Chengdu, China ,grid.410655.30000 0001 0157 8259Institute of Isotope, China Institute of Atomic Energy, 102413 Beijing, China
| | - Shirong Chen
- grid.54549.390000 0004 0369 4060Department of Nuclear Medicine, Sichuan Cancer Hospital&Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No.55, Section 4, South People’s Road, Sichuan 610041 Chengdu, China
| | - Yongli Liu
- grid.54549.390000 0004 0369 4060Department of Nuclear Medicine, Sichuan Cancer Hospital&Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No.55, Section 4, South People’s Road, Sichuan 610041 Chengdu, China
| | - Jun Dang
- grid.54549.390000 0004 0369 4060Department of Nuclear Medicine, Sichuan Cancer Hospital&Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No.55, Section 4, South People’s Road, Sichuan 610041 Chengdu, China
| | - Jun Yin
- grid.54549.390000 0004 0369 4060Department of Radiation Oncology, Sichuan Cancer Hospital&Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No.55, Section 4, South People’s Road, 610041 Chengdu, China
| | - Zhuzhong Cheng
- grid.54549.390000 0004 0369 4060Department of Nuclear Medicine, Sichuan Cancer Hospital&Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No.55, Section 4, South People’s Road, Sichuan 610041 Chengdu, China
| |
Collapse
|
14
|
Radiomics for Discriminating Benign and Malignant Salivary Gland Tumors; Which Radiomic Feature Categories and MRI Sequences Should Be Used? Cancers (Basel) 2022; 14:cancers14235804. [PMID: 36497285 PMCID: PMC9740105 DOI: 10.3390/cancers14235804] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2022] [Revised: 11/12/2022] [Accepted: 11/22/2022] [Indexed: 11/26/2022] Open
Abstract
The lack of a consistent MRI radiomic signature, partly due to the multitude of initial feature analyses, limits the widespread clinical application of radiomics for the discrimination of salivary gland tumors (SGTs). This study aimed to identify the optimal radiomics feature category and MRI sequence for characterizing SGTs, which could serve as a step towards obtaining a consensus on a radiomics signature. Preliminary radiomics models were built to discriminate malignant SGTs (n = 34) from benign SGTs (n = 57) on T1-weighted (T1WI), fat-suppressed (FS)-T2WI and contrast-enhanced (CE)-T1WI images using six feature categories. The discrimination performances of these preliminary models were evaluated using 5-fold-cross-validation with 100 repetitions and the area under the receiver operating characteristic curve (AUC). The differences between models’ performances were identified using one-way ANOVA. Results show that the best feature categories were logarithm for T1WI and CE-T1WI and exponential for FS-T2WI, with AUCs of 0.828, 0.754 and 0.819, respectively. These AUCs were higher than the AUCs obtained using all feature categories combined, which were 0.750, 0.707 and 0.774, respectively (p < 0.001). The highest AUC (0.846) was obtained using a combination of T1WI + logarithm and FS-T2WI + exponential features, which reduced the initial features by 94.0% (from 1015 × 3 to 91 × 2). CE-T1WI did not improve performance. Using one feature category rather than all feature categories combined reduced the number of initial features without compromising radiomic performance.
Collapse
|
15
|
Li WZ, Wu G, Li TS, Dai GM, Liao YT, Yang QY, Chen F, Huang WY. Dynamic contrast-enhanced magnetic resonance imaging-based radiomics for the prediction of progression-free survival in advanced nasopharyngeal carcinoma. Front Oncol 2022; 12:955866. [PMID: 36338711 PMCID: PMC9627984 DOI: 10.3389/fonc.2022.955866] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2022] [Accepted: 09/05/2022] [Indexed: 08/30/2023] Open
Abstract
To establish a multidimensional nomogram model for predicting progression-free survival (PFS) and risk stratification in patients with advanced nasopharyngeal carcinoma (NPC). This retrospective cross-sectional study included 156 patients with advanced NPC who underwent dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). Radiomic features were extracted from the efflux rate constant (Ktrans ) and extracellular extravascular volume (Ve ) mapping derived from DCE-MRI. Least absolute shrinkage and selection operator (LASSO) Cox regression analysis was applied for feature selection. The Radscore was constructed using the selected features with their respective weights in the LASSO Cox regression analysis. A nomogram model combining the Radscore and clinical factors was built using multivariate Cox regression analysis. The C-index was used to assess the discrimination power of the Radscore and nomogram. The Kaplan-Meier method was used for survival analysis. Of the 360 radiomic features, 28 were selected (7, 6, and 15 features extracted from Ktrans , Ve, and Ktrans +Ve images, respectively). The combined Radscore k trans +Ve (C-index, 0.703, 95% confidence interval [CI]: 0.571-0.836) showed higher efficacy in predicting the prognosis of advanced NPC than Radscore k trans (C-index, 0.693; 95% CI, 0.560-0.826) and Radscore Ve (C-index, 0.614; 95% CI, 0.481-0.746) did. Multivariable Cox regression analysis revealed clinical stage, T stage, and treatment with nimotuzumab as risk factors for PFS. The nomogram established by Radscore k trans +Ve and risk factors (C-index, 0.732; 95% CI: 0.599-0.864) was better than Radscore k trans +Ve in predicting PFS in patients with advanced NPC. A lower Radscore k trans +Ve (HR 3.5584, 95% CI 2.1341-5.933), lower clinical stage (hazard ratio [HR] 1.5982, 95% CI 0.5262-4.854), lower T stage (HR 1.4365, 95% CI 0.6745-3.060), and nimotuzumab (NTZ) treatment (HR 0.7879, 95% CI 0.4899-1.267) were associated with longer PFS. Kaplan-Meier analysis showed a lower PFS in the high-risk group than in the low-risk group (p<0.0001). The nomogram based on combined pretreatment DCE-MRI radiomics features, NTZ, and clinicopathological risk factors may be considered as a noninvasive imaging marker for predicting individual PFS in patients with advanced NPC.
Collapse
Affiliation(s)
- Wen-zhu Li
- Department of Radiology, Hainan General Hospital, Hainan Affiliated Hospital of Hainan Medical University, Haikou, China
| | - Gang Wu
- Department of Radiotherapy, Hainan General Hospital, Hainan Affiliated Hospital of Hainan Medical University, Haikou, China
| | - Tian-sheng Li
- Department of Radiology, Hainan General Hospital, Hainan Affiliated Hospital of Hainan Medical University, Haikou, China
| | - Gan-mian Dai
- Department of Radiology, Hainan General Hospital, Hainan Affiliated Hospital of Hainan Medical University, Haikou, China
| | - Yu-ting Liao
- Department of Pharmaceutical Diagnostics, GE Healthcare, Guangzhou, China
| | - Qian-yu Yang
- Department of Radiology, Hainan General Hospital, Hainan Affiliated Hospital of Hainan Medical University, Haikou, China
| | - Feng Chen
- Department of Radiology, Hainan General Hospital, Hainan Affiliated Hospital of Hainan Medical University, Haikou, China
| | - Wei-yuan Huang
- Department of Radiology, Hainan General Hospital, Hainan Affiliated Hospital of Hainan Medical University, Haikou, China
| |
Collapse
|
16
|
Validation of MRI-Based Models to Predict MGMT Promoter Methylation in Gliomas: BraTS 2021 Radiogenomics Challenge. Cancers (Basel) 2022; 14:cancers14194827. [PMID: 36230750 PMCID: PMC9562637 DOI: 10.3390/cancers14194827] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2022] [Revised: 09/19/2022] [Accepted: 09/30/2022] [Indexed: 12/05/2022] Open
Abstract
Simple Summary O6-methylguanine-DNA methyl transferase (MGMT) methylation in glioblastoma is an important prognostic and predictive factor that requires an invasive surgical procedure for identification. In several recent studies, MGMT methylation prediction models were developed using MR images, and good diagnostic performance was achieved, which seems to indicate a promising future for radiogenomics. However, the diagnostic performance was not reproducible for numerous research teams when using a larger dataset in the RSNA-MICCAI Brain Tumor Radiogenomic Classification 2021 challenge. To our knowledge, there has been no study regarding the external validation of MGMT prediction models using large-scale multicenter datasets. We tested recent CNN architectures via extensive experiments to investigate whether MGMT methylation in gliomas can be predicted using MRI. With unexpected negative results, approximately 80% of the developed models showed no significant difference with the chance level of 50% in terms of external validation accuracy. In conclusion, MGMT methylation status of gliomas may not be predictable with preoperative MRI, even using deep learning. Abstract O6-methylguanine-DNA methyl transferase (MGMT) methylation prediction models were developed using only small datasets without proper external validation and achieved good diagnostic performance, which seems to indicate a promising future for radiogenomics. However, the diagnostic performance was not reproducible for numerous research teams when using a larger dataset in the RSNA-MICCAI Brain Tumor Radiogenomic Classification 2021 challenge. To our knowledge, there has been no study regarding the external validation of MGMT prediction models using large-scale multicenter datasets. We tested recent CNN architectures via extensive experiments to investigate whether MGMT methylation in gliomas can be predicted using MR images. Specifically, prediction models were developed and validated with different training datasets: (1) the merged (SNUH + BraTS) (n = 985); (2) SNUH (n = 400); and (3) BraTS datasets (n = 585). A total of 420 training and validation experiments were performed on combinations of datasets, convolutional neural network (CNN) architectures, MRI sequences, and random seed numbers. The first-place solution of the RSNA-MICCAI radiogenomic challenge was also validated using the external test set (SNUH). For model evaluation, the area under the receiver operating characteristic curve (AUROC), accuracy, precision, and recall were obtained. With unexpected negative results, 80.2% (337/420) and 60.0% (252/420) of the 420 developed models showed no significant difference with a chance level of 50% in terms of test accuracy and test AUROC, respectively. The test AUROC and accuracy of the first-place solution of the BraTS 2021 challenge were 56.2% and 54.8%, respectively, as validated on the SNUH dataset. In conclusion, MGMT methylation status of gliomas may not be predictable with preoperative MR images even using deep learning.
Collapse
|
17
|
Park SH, Han K. How to Clearly and Accurately Report Odds Ratio and Hazard Ratio in Diagnostic Research Studies? Korean J Radiol 2022; 23:777-784. [PMID: 35695319 PMCID: PMC9340231 DOI: 10.3348/kjr.2022.0249] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Accepted: 04/11/2022] [Indexed: 01/17/2023] Open
Affiliation(s)
- Seong Ho Park
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea.
| | - Kyunghwa Han
- Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, Korea
| |
Collapse
|
18
|
Stumpo V, Guida L, Bellomo J, Van Niftrik CHB, Sebök M, Berhouma M, Bink A, Weller M, Kulcsar Z, Regli L, Fierstra J. Hemodynamic Imaging in Cerebral Diffuse Glioma—Part B: Molecular Correlates, Treatment Effect Monitoring, Prognosis, and Future Directions. Cancers (Basel) 2022; 14:cancers14051342. [PMID: 35267650 PMCID: PMC8909110 DOI: 10.3390/cancers14051342] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2022] [Revised: 03/01/2022] [Accepted: 03/02/2022] [Indexed: 02/05/2023] Open
Abstract
Gliomas, and glioblastoma in particular, exhibit an extensive intra- and inter-tumoral molecular heterogeneity which represents complex biological features correlating to the efficacy of treatment response and survival. From a neuroimaging point of view, these specific molecular and histopathological features may be used to yield imaging biomarkers as surrogates for distinct tumor genotypes and phenotypes. The development of comprehensive glioma imaging markers has potential for improved glioma characterization that would assist in the clinical work-up of preoperative treatment planning and treatment effect monitoring. In particular, the differentiation of tumor recurrence or true progression from pseudoprogression, pseudoresponse, and radiation-induced necrosis can still not reliably be made through standard neuroimaging only. Given the abundant vascular and hemodynamic alterations present in diffuse glioma, advanced hemodynamic imaging approaches constitute an attractive area of clinical imaging development. In this context, the inclusion of objective measurable glioma imaging features may have the potential to enhance the individualized care of diffuse glioma patients, better informing of standard-of-care treatment efficacy and of novel therapies, such as the immunotherapies that are currently increasingly investigated. In Part B of this two-review series, we assess the available evidence pertaining to hemodynamic imaging for molecular feature prediction, in particular focusing on isocitrate dehydrogenase (IDH) mutation status, MGMT promoter methylation, 1p19q codeletion, and EGFR alterations. The results for the differentiation of tumor progression/recurrence from treatment effects have also been the focus of active research and are presented together with the prognostic correlations identified by advanced hemodynamic imaging studies. Finally, the state-of-the-art concepts and advancements of hemodynamic imaging modalities are reviewed together with the advantages derived from the implementation of radiomics and machine learning analyses pipelines.
Collapse
Affiliation(s)
- Vittorio Stumpo
- Department of Neurosurgery, University Hospital Zurich, 8091 Zurich, Switzerland; (L.G.); (J.B.); (C.H.B.V.N.); (M.S.); (L.R.); (J.F.)
- Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, 8057 Zurich, Switzerland; (A.B.); (M.W.); (Z.K.)
- Correspondence:
| | - Lelio Guida
- Department of Neurosurgery, University Hospital Zurich, 8091 Zurich, Switzerland; (L.G.); (J.B.); (C.H.B.V.N.); (M.S.); (L.R.); (J.F.)
- Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, 8057 Zurich, Switzerland; (A.B.); (M.W.); (Z.K.)
| | - Jacopo Bellomo
- Department of Neurosurgery, University Hospital Zurich, 8091 Zurich, Switzerland; (L.G.); (J.B.); (C.H.B.V.N.); (M.S.); (L.R.); (J.F.)
- Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, 8057 Zurich, Switzerland; (A.B.); (M.W.); (Z.K.)
| | - Christiaan Hendrik Bas Van Niftrik
- Department of Neurosurgery, University Hospital Zurich, 8091 Zurich, Switzerland; (L.G.); (J.B.); (C.H.B.V.N.); (M.S.); (L.R.); (J.F.)
- Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, 8057 Zurich, Switzerland; (A.B.); (M.W.); (Z.K.)
| | - Martina Sebök
- Department of Neurosurgery, University Hospital Zurich, 8091 Zurich, Switzerland; (L.G.); (J.B.); (C.H.B.V.N.); (M.S.); (L.R.); (J.F.)
- Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, 8057 Zurich, Switzerland; (A.B.); (M.W.); (Z.K.)
| | - Moncef Berhouma
- Department of Neurosurgical Oncology and Vascular Neurosurgery, Pierre Wertheimer Neurological and Neurosurgical Hospital, Hospices Civils de Lyon, 69500 Lyon, France;
| | - Andrea Bink
- Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, 8057 Zurich, Switzerland; (A.B.); (M.W.); (Z.K.)
- Department of Neuroradiology, University Hospital Zurich, 8091 Zurich, Switzerland
| | - Michael Weller
- Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, 8057 Zurich, Switzerland; (A.B.); (M.W.); (Z.K.)
- Department of Neurology, University Hospital Zurich, 8091 Zurich, Switzerland
| | - Zsolt Kulcsar
- Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, 8057 Zurich, Switzerland; (A.B.); (M.W.); (Z.K.)
- Department of Neuroradiology, University Hospital Zurich, 8091 Zurich, Switzerland
| | - Luca Regli
- Department of Neurosurgery, University Hospital Zurich, 8091 Zurich, Switzerland; (L.G.); (J.B.); (C.H.B.V.N.); (M.S.); (L.R.); (J.F.)
- Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, 8057 Zurich, Switzerland; (A.B.); (M.W.); (Z.K.)
| | - Jorn Fierstra
- Department of Neurosurgery, University Hospital Zurich, 8091 Zurich, Switzerland; (L.G.); (J.B.); (C.H.B.V.N.); (M.S.); (L.R.); (J.F.)
- Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, 8057 Zurich, Switzerland; (A.B.); (M.W.); (Z.K.)
| |
Collapse
|
19
|
Ammari S, Sallé de Chou R, Balleyguier C, Chouzenoux E, Touat M, Quillent A, Dumont S, Bockel S, Garcia GCTE, Elhaik M, Francois B, Borget V, Lassau N, Khettab M, Assi T. A Predictive Clinical-Radiomics Nomogram for Survival Prediction of Glioblastoma Using MRI. Diagnostics (Basel) 2021; 11:diagnostics11112043. [PMID: 34829395 PMCID: PMC8624566 DOI: 10.3390/diagnostics11112043] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2021] [Revised: 10/20/2021] [Accepted: 10/21/2021] [Indexed: 01/01/2023] Open
Abstract
Glioblastoma (GBM) is the most common and aggressive primary brain tumor in adult patients with a median survival of around one year. Prediction of survival outcomes in GBM patients could represent a huge step in treatment personalization. The objective of this study was to develop machine learning (ML) algorithms for survival prediction of GBM patient. We identified a radiomic signature on a training-set composed of data from the 2019 BraTS challenge (210 patients) from MRI retrieved at diagnosis. Then, using this signature along with the age of the patients for training classification models, we obtained on test-sets AUCs of 0.85, 0.74 and 0.58 (0.92, 0.88 and 0.75 on the training-sets) for survival at 9-, 12- and 15-months, respectively. This signature was then validated on an independent cohort of 116 GBM patients with confirmed disease relapse for the prediction of patients surviving less or more than the median OS of 22 months. Our model insured an AUC of 0.71 (0.65 on train). The Kaplan–Meier method showed significant OS difference between groups (log-rank p = 0.05). These results suggest that radiomic signatures may improve survival outcome predictions in GBM thus creating a solid clinical tool for tailoring therapy in this population.
Collapse
Affiliation(s)
- Samy Ammari
- Biomaps, UMR1281 INSERM, CEA, CNRS, Université Paris-Saclay, 94805 Villejuif, France; (S.A.); (C.B.); (M.E.); (B.F.); (V.B.); (N.L.)
- Department of Imaging, Gustave Roussy, Université Paris Saclay, 94805 Villejuif, France;
| | - Raoul Sallé de Chou
- Biomaps, UMR1281 INSERM, CEA, CNRS, Université Paris-Saclay, 94805 Villejuif, France; (S.A.); (C.B.); (M.E.); (B.F.); (V.B.); (N.L.)
- Correspondence:
| | - Corinne Balleyguier
- Biomaps, UMR1281 INSERM, CEA, CNRS, Université Paris-Saclay, 94805 Villejuif, France; (S.A.); (C.B.); (M.E.); (B.F.); (V.B.); (N.L.)
- Department of Imaging, Gustave Roussy, Université Paris Saclay, 94805 Villejuif, France;
| | - Emilie Chouzenoux
- Centre de Vision Numérique, OPIS, CentraleSupélec, Inria, Université Paris-Saclay, 91190 Gif-sur-Yvette, France; (E.C.); (A.Q.)
| | - Mehdi Touat
- Service de Neurologie 2-Mazarin, AP-HP Hôpitaux Universitaires La Pitié Salpêtrière-Charles Foix, 75013 Paris, France;
- Institut du Cerveau et de la Moelle Epinière, CNRS, UMR S 1127, Inserm, Sorbonne Université, 75013 Paris, France
| | - Arnaud Quillent
- Centre de Vision Numérique, OPIS, CentraleSupélec, Inria, Université Paris-Saclay, 91190 Gif-sur-Yvette, France; (E.C.); (A.Q.)
| | - Sarah Dumont
- Department of oncology, Gustave Roussy, Université Paris Saclay, 94805 Villejuif, France; (S.D.); (T.A.)
| | - Sophie Bockel
- Department of Radiation Oncology, Gustave Roussy Cancer Campus, 94800 Villejuif, France;
| | | | - Mickael Elhaik
- Biomaps, UMR1281 INSERM, CEA, CNRS, Université Paris-Saclay, 94805 Villejuif, France; (S.A.); (C.B.); (M.E.); (B.F.); (V.B.); (N.L.)
| | - Bidault Francois
- Biomaps, UMR1281 INSERM, CEA, CNRS, Université Paris-Saclay, 94805 Villejuif, France; (S.A.); (C.B.); (M.E.); (B.F.); (V.B.); (N.L.)
- Department of Imaging, Gustave Roussy, Université Paris Saclay, 94805 Villejuif, France;
| | - Valentin Borget
- Biomaps, UMR1281 INSERM, CEA, CNRS, Université Paris-Saclay, 94805 Villejuif, France; (S.A.); (C.B.); (M.E.); (B.F.); (V.B.); (N.L.)
| | - Nathalie Lassau
- Biomaps, UMR1281 INSERM, CEA, CNRS, Université Paris-Saclay, 94805 Villejuif, France; (S.A.); (C.B.); (M.E.); (B.F.); (V.B.); (N.L.)
- Department of Imaging, Gustave Roussy, Université Paris Saclay, 94805 Villejuif, France;
| | - Mohamed Khettab
- Medical Oncology Unit, CHU de La Réunion, Université de La Réunion, 97410 Saint Pierre, France;
| | - Tarek Assi
- Department of oncology, Gustave Roussy, Université Paris Saclay, 94805 Villejuif, France; (S.D.); (T.A.)
| |
Collapse
|