1
|
Watzenboeck ML, Beer L, Kifjak D, Röhrich S, Heidinger BH, Prayer F, Milos RI, Apfaltrer P, Langs G, Baltzer PAT, Prosch H. Contrast Agent Dynamics Determine Radiomics Profiles in Oncologic Imaging. Cancers (Basel) 2024; 16:1519. [PMID: 38672601 PMCID: PMC11049400 DOI: 10.3390/cancers16081519] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2024] [Revised: 04/11/2024] [Accepted: 04/15/2024] [Indexed: 04/28/2024] Open
Abstract
BACKGROUND The reproducibility of radiomics features extracted from CT and MRI examinations depends on several physiological and technical factors. The aim was to evaluate the impact of contrast agent timing on the stability of radiomics features using dynamic contrast-enhanced perfusion CT (dceCT) or MRI (dceMRI) in prostate and lung cancers. METHODS Radiomics features were extracted from dceCT or dceMRI images in patients with biopsy-proven peripheral prostate cancer (pzPC) or biopsy-proven non-small cell lung cancer (NSCLC), respectively. Features that showed significant differences between contrast phases were identified using linear mixed models. An L2-penalized logistic regression classifier was used to predict class labels for pzPC and unaffected prostate regions-of-interest (ROIs). RESULTS Nine pzPC and 28 NSCLC patients, who were imaged with dceCT and/or dceMRI, were included in this study. After normalizing for individual enhancement patterns by defining seven individual phases based on a reference vessel, 19, 467 and 128 out of 1204 CT features showed significant temporal dynamics in healthy prostate parenchyma, prostate tumors and lung tumors, respectively. CT radiomics-based classification accuracy of healthy and tumor ROIs was highly dependent on contrast agent phase. For dceMRI, 899 and 1027 out of 1118 features were significantly dependent on time after contrast agent injection for prostate and lung tumors. CONCLUSIONS CT and MRI radiomics features in both prostate and lung tumors are significantly affected by interindividual differences in contrast agent dynamics.
Collapse
Affiliation(s)
- Martin L. Watzenboeck
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria (G.L.); (P.A.T.B.); (H.P.)
- Computational Imaging Research Lab, Christian Doppler Laboratory for Machine Learning Driven Precision Medicine, Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria
| | - Lucian Beer
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria (G.L.); (P.A.T.B.); (H.P.)
- Computational Imaging Research Lab, Christian Doppler Laboratory for Machine Learning Driven Precision Medicine, Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria
| | - Daria Kifjak
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria (G.L.); (P.A.T.B.); (H.P.)
- Computational Imaging Research Lab, Christian Doppler Laboratory for Machine Learning Driven Precision Medicine, Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria
| | - Sebastian Röhrich
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria (G.L.); (P.A.T.B.); (H.P.)
- Computational Imaging Research Lab, Christian Doppler Laboratory for Machine Learning Driven Precision Medicine, Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria
| | - Benedikt H. Heidinger
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria (G.L.); (P.A.T.B.); (H.P.)
- Computational Imaging Research Lab, Christian Doppler Laboratory for Machine Learning Driven Precision Medicine, Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria
| | - Florian Prayer
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria (G.L.); (P.A.T.B.); (H.P.)
- Computational Imaging Research Lab, Christian Doppler Laboratory for Machine Learning Driven Precision Medicine, Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria
| | - Ruxandra-Iulia Milos
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria (G.L.); (P.A.T.B.); (H.P.)
- Computational Imaging Research Lab, Christian Doppler Laboratory for Machine Learning Driven Precision Medicine, Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria
| | - Paul Apfaltrer
- Zentralröntgeninstitut für Diagnostik, Interventionelle Radiologie und Nuklearmedizin, Landesklinikum Wiener Neustadt, 2700 Wiener Neustadt, Austria
| | - Georg Langs
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria (G.L.); (P.A.T.B.); (H.P.)
- Computational Imaging Research Lab, Christian Doppler Laboratory for Machine Learning Driven Precision Medicine, Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria
| | - Pascal A. T. Baltzer
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria (G.L.); (P.A.T.B.); (H.P.)
| | - Helmut Prosch
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria (G.L.); (P.A.T.B.); (H.P.)
- Computational Imaging Research Lab, Christian Doppler Laboratory for Machine Learning Driven Precision Medicine, Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, 1090 Vienna, Austria
| |
Collapse
|
2
|
Suman G, Koo CW. Recent Advancements in Computed Tomography Assessment of Fibrotic Interstitial Lung Diseases. J Thorac Imaging 2023; 38:S7-S18. [PMID: 37015833 DOI: 10.1097/rti.0000000000000705] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/06/2023]
Abstract
Interstitial lung disease (ILD) is a heterogeneous group of disorders with complex and varied imaging manifestations and prognosis. High-resolution computed tomography (HRCT) is the current standard-of-care imaging tool for ILD assessment. However, visual evaluation of HRCT is limited by interobserver variation and poor sensitivity for subtle changes. Such challenges have led to tremendous recent research interest in objective and reproducible methods to examine ILDs. Computer-aided CT analysis to include texture analysis and machine learning methods have recently been shown to be viable supplements to traditional visual assessment through improved characterization and quantification of ILDs. These quantitative tools have not only been shown to correlate well with pulmonary function tests and patient outcomes but are also useful in disease diagnosis, surveillance and management. In this review, we provide an overview of recent computer-aided tools in diagnosis, prognosis, and longitudinal evaluation of fibrotic ILDs, while outlining some of the pitfalls and challenges that have precluded further advancement of these tools as well as potential solutions and further endeavors.
Collapse
Affiliation(s)
- Garima Suman
- Division of Thoracic Imaging, Mayo Clinic, Rochester, MN
| | | |
Collapse
|
3
|
Zhang YP, Zhang XY, Cheng YT, Li B, Teng XZ, Zhang J, Lam S, Zhou T, Ma ZR, Sheng JB, Tam VCW, Lee SWY, Ge H, Cai J. Artificial intelligence-driven radiomics study in cancer: the role of feature engineering and modeling. Mil Med Res 2023; 10:22. [PMID: 37189155 DOI: 10.1186/s40779-023-00458-8] [Citation(s) in RCA: 17] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Accepted: 05/04/2023] [Indexed: 05/17/2023] Open
Abstract
Modern medicine is reliant on various medical imaging technologies for non-invasively observing patients' anatomy. However, the interpretation of medical images can be highly subjective and dependent on the expertise of clinicians. Moreover, some potentially useful quantitative information in medical images, especially that which is not visible to the naked eye, is often ignored during clinical practice. In contrast, radiomics performs high-throughput feature extraction from medical images, which enables quantitative analysis of medical images and prediction of various clinical endpoints. Studies have reported that radiomics exhibits promising performance in diagnosis and predicting treatment responses and prognosis, demonstrating its potential to be a non-invasive auxiliary tool for personalized medicine. However, radiomics remains in a developmental phase as numerous technical challenges have yet to be solved, especially in feature engineering and statistical modeling. In this review, we introduce the current utility of radiomics by summarizing research on its application in the diagnosis, prognosis, and prediction of treatment responses in patients with cancer. We focus on machine learning approaches, for feature extraction and selection during feature engineering and for imbalanced datasets and multi-modality fusion during statistical modeling. Furthermore, we introduce the stability, reproducibility, and interpretability of features, and the generalizability and interpretability of models. Finally, we offer possible solutions to current challenges in radiomics research.
Collapse
Affiliation(s)
- Yuan-Peng Zhang
- Department of Medical Informatics, Nantong University, Nantong, 226001, Jiangsu, China
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
- The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen, 518000, Guangdong, China
| | - Xin-Yun Zhang
- Department of Medical Informatics, Nantong University, Nantong, 226001, Jiangsu, China
| | - Yu-Ting Cheng
- Department of Medical Informatics, Nantong University, Nantong, 226001, Jiangsu, China
| | - Bing Li
- Department of Radiation Oncology, the Affiliated Cancer Hospital of Zhengzhou University and Henan Cancer Hospital, Zhengzhou, 450008, Henan, China
| | - Xin-Zhi Teng
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Jiang Zhang
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Saikit Lam
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Ta Zhou
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Zong-Rui Ma
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Jia-Bao Sheng
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Victor C W Tam
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Shara W Y Lee
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China
| | - Hong Ge
- Department of Radiation Oncology, the Affiliated Cancer Hospital of Zhengzhou University and Henan Cancer Hospital, Zhengzhou, 450008, Henan, China
| | - Jing Cai
- Department of Health Technology and Informatics, the Hong Kong Polytechnic University, Hong Kong, 999077, China.
- The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen, 518000, Guangdong, China.
| |
Collapse
|
4
|
Watzenboeck ML, Heidinger BH, Rainer J, Schmidbauer V, Ulm B, Rubesova E, Prayer D, Kasprian G, Prayer F. Reproducibility of 2D versus 3D radiomics for quantitative assessment of fetal lung development: a retrospective fetal MRI study. Insights Imaging 2023; 14:31. [PMID: 36752863 PMCID: PMC9908803 DOI: 10.1186/s13244-023-01376-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2022] [Accepted: 01/16/2023] [Indexed: 02/09/2023] Open
Abstract
PURPOSE To investigate the reproducibility of radiomics features extracted from two-dimensional regions of interest (2D ROIs) versus whole lung (3D) ROIs in repeated in-vivo fetal magnetic resonance imaging (MRI) acquisitions. METHODS Thirty fetal MRI scans including two axial T2-weighted acquisitions of the lungs were analysed. 2D (lung at the level of the carina) and 3D (whole lung) ROIs were manually segmented using ITK-Snap. Ninety-five radiomics features were extracted from 2 and 3D ROIs in initial and repeat acquisitions using Pyradiomics. Radiomics feature intra-class correlation coefficients (ICC) were calculated between 2 and 3D ROIs in the initial acquisition, and between 2 and 3D ROIs in repeated acquisitions, respectively. RESULTS MRI data of 11 (36.7%) female and 19 (63.3%) male fetuses acquired at a median 25 + 0 gestational weeks plus days (GW) (interquartile range [IQR] 23 + 4 - 27 + 0 GW) were assessed. Median radiomics feature ICC between 2 and 3D ROIs in the initial MRI acquisition was 0.733 (IQR 0.313-0.814, range 0.018-0.970). ICCs between radiomics features extracted using 3D ROIs in initial and repeat acquisitions (median 0.908 [IQR 0.824-0.929, range 0.335-0.996]) were significantly higher compared to 2D ROIs (0.771 [0.699-0.835, 0.048-0.965]) (p < 0.001). CONCLUSION Fetal MRI radiomics features extracted from 3D whole lung segmentation masks showed significantly higher reproducibility across repeat acquisitions compared to 2D ROIs. Therefore, fetal MRI whole lung radiomics features are robust diagnostic and potentially prognostic tools in the image-based in-vivo quantitative assessment of lung development.
Collapse
Affiliation(s)
- Martin L. Watzenboeck
- grid.22937.3d0000 0000 9259 8492Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Währinger Gürtel 18-20, 1090 Vienna, Austria
| | - Benedikt H. Heidinger
- grid.22937.3d0000 0000 9259 8492Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Währinger Gürtel 18-20, 1090 Vienna, Austria
| | - Julian Rainer
- grid.22937.3d0000 0000 9259 8492Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Währinger Gürtel 18-20, 1090 Vienna, Austria
| | - Victor Schmidbauer
- grid.22937.3d0000 0000 9259 8492Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Währinger Gürtel 18-20, 1090 Vienna, Austria
| | - Barbara Ulm
- grid.22937.3d0000 0000 9259 8492Department of Obstetrics and Gynecology, Medical University of Vienna, Spitalgasse 23, Währinger Gürtel 18-20, 1090 Vienna, Austria
| | - Erika Rubesova
- grid.168010.e0000000419368956Department of Pediatric Radiology, Lucile Packard Children’s Hospital at Stanford, Stanford University, 725 Welch Road, Stanford, CA 94305 USA
| | - Daniela Prayer
- Imaging Bellaria, Bellariastrasse 3, 1010 Vienna, Austria
| | - Gregor Kasprian
- grid.22937.3d0000 0000 9259 8492Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Währinger Gürtel 18-20, 1090 Vienna, Austria
| | - Florian Prayer
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Währinger Gürtel 18-20, 1090, Vienna, Austria.
| |
Collapse
|
5
|
Prayer F, Watzenböck ML, Heidinger BH, Rainer J, Schmidbauer V, Prosch H, Ulm B, Rubesova E, Prayer D, Kasprian G. Fetal MRI radiomics: non-invasive and reproducible quantification of human lung maturity. Eur Radiol 2023; 33:4205-4213. [PMID: 36604329 PMCID: PMC10182107 DOI: 10.1007/s00330-022-09367-1] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Revised: 10/26/2022] [Accepted: 12/05/2022] [Indexed: 01/07/2023]
Abstract
OBJECTIVES To assess the reproducibility of radiomics features extracted from the developing lung in repeated in-vivo fetal MRI acquisitions. METHODS In-vivo MRI (1.5 Tesla) scans of 30 fetuses, each including two axial and one coronal T2-weighted sequences of the whole lung with all other acquisition parameters kept constant, were retrospectively identified. Manual segmentation of the lungs was performed using ITK-Snap. One hundred radiomics features were extracted from fetal lung MRI data using Pyradiomics, resulting in 90 datasets. Intra-class correlation coefficients (ICC) of radiomics features were calculated between baseline and repeat axial acquisitions and between baseline axial and coronal acquisitions. RESULTS MRI data of 30 fetuses (12 [40%] females, 18 [60%] males) at a median gestational age of 24 + 5 gestational weeks plus days (GW) (interquartile range [IQR] 3 + 3 GW, range 21 + 1 to 32 + 6 GW) were included. Median ICC of radiomics features between baseline and repeat axial MR acquisitions was 0.92 (IQR 0.13, range 0.33 to 1), with 60 features exhibiting excellent (ICC > 0.9), 27 good (> 0.75-0.9), twelve moderate (0.5-0.75), and one poor (ICC < 0.5) reproducibility. Median ICC of radiomics features between baseline axial and coronal MR acquisitions was 0.79 (IQR 0.15, range 0.2 to 1), with 20 features exhibiting excellent, 47 good, 29 moderate, and four poor reproducibility. CONCLUSION Standardized in-vivo fetal MRI allows reproducible extraction of lung radiomics features. In the future, radiomics analysis may improve diagnostic and prognostic yield of fetal MRI in normal and pathologic lung development. KEY POINTS • Non-invasive fetal MRI acquired using a standardized protocol allows reproducible extraction of radiomics features from the developing lung for objective tissue characterization. • Alteration of imaging plane between fetal MRI acquisitions has a negative impact on lung radiomics feature reproducibility. • Fetal MRI radiomics features reflecting the microstructure and shape of the fetal lung could complement observed-to-expected lung volume in the prediction of postnatal outcome and optimal treatment of fetuses with abnormal lung development in the future.
Collapse
Affiliation(s)
- Florian Prayer
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Spitalgasse 23, 1090, Vienna, Austria
| | - Martin L Watzenböck
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Spitalgasse 23, 1090, Vienna, Austria
| | - Benedikt H Heidinger
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Spitalgasse 23, 1090, Vienna, Austria
| | - Julian Rainer
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Spitalgasse 23, 1090, Vienna, Austria
| | - Victor Schmidbauer
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Spitalgasse 23, 1090, Vienna, Austria
| | - Helmut Prosch
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Spitalgasse 23, 1090, Vienna, Austria
| | - Barbara Ulm
- Department of Obstetrics and Gynecology, Medical University of Vienna, Spitalgasse 23, 1090, Vienna, Vienna, Austria
| | - Erika Rubesova
- Department of Pediatric Radiology, Lucile Packard Children's Hospital at Stanford, Stanford University, 725 Welch Road, Stanford, CA, 94305, USA
| | - Daniela Prayer
- Imaging Bellaria, Bellariastrasse 3, 1010, Vienna, Austria
| | - Gregor Kasprian
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Spitalgasse 23, 1090, Vienna, Austria.
| |
Collapse
|
6
|
Cui Y, Yin FF. Impact of image quality on radiomics applications. Phys Med Biol 2022; 67. [DOI: 10.1088/1361-6560/ac7fd7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2022] [Accepted: 07/08/2022] [Indexed: 11/12/2022]
Abstract
Abstract
Radiomics features extracted from medical images have been widely reported to be useful in the patient specific outcome modeling for variety of assessment and prediction purposes. Successful application of radiomics features as imaging biomarkers, however, is dependent on the robustness of the approach to the variation in each step of the modeling workflow. Variation in the input image quality is one of the main sources that impacts the reproducibility of radiomics analysis when a model is applied to broader range of medical imaging data. The quality of medical image is generally affected by both the scanner related factors such as image acquisition/reconstruction settings and the patient related factors such as patient motion. This article aimed to review the published literatures in this field that reported the impact of various imaging factors on the radiomics features through the change in image quality. The literatures were categorized by different imaging modalities and also tabulated based on the imaging parameters and the class of radiomics features included in the study. Strategies for image quality standardization were discussed based on the relevant literatures and recommendations for reducing the impact of image quality variation on the radiomics in multi-institutional clinical trial were summarized at the end of this article.
Collapse
|
7
|
The Potential and Emerging Role of Quantitative Imaging Biomarkers for Cancer Characterization. Cancers (Basel) 2022; 14:cancers14143349. [PMID: 35884409 PMCID: PMC9321521 DOI: 10.3390/cancers14143349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2022] [Revised: 07/07/2022] [Accepted: 07/08/2022] [Indexed: 12/10/2022] Open
Abstract
Simple Summary Modern, personalized therapy approaches are increasingly changing advanced cancer into a chronic disease. Compared to imaging, novel omics methodologies in molecular biology have already achieved an individual characterization of cancerous lesions. With quantitative imaging biomarkers, analyzed by radiomics or deep learning, an imaging-based assessment of tumoral biology can be brought into clinical practice. Combining these with other non-invasive methods, e.g., liquid profiling, could allow for more individual decision making regarding therapies and applications. Abstract Similar to the transformation towards personalized oncology treatment, emerging techniques for evaluating oncologic imaging are fostering a transition from traditional response assessment towards more comprehensive cancer characterization via imaging. This development can be seen as key to the achievement of truly personalized and optimized cancer diagnosis and treatment. This review gives a methodological introduction for clinicians interested in the potential of quantitative imaging biomarkers, treating of radiomics models, texture visualization, convolutional neural networks and automated segmentation, in particular. Based on an introduction to these methods, clinical evidence for the corresponding imaging biomarkers—(i) dignity and etiology assessment; (ii) tumoral heterogeneity; (iii) aggressiveness and response; and (iv) targeting for biopsy and therapy—is summarized. Further requirements for the clinical implementation of these imaging biomarkers and the synergistic potential of personalized molecular cancer diagnostics and liquid profiling are discussed.
Collapse
|
8
|
Peng X, Yang S, Zhou L, Mei Y, Shi L, Zhang R, Shan F, Liu L. Repeatability and Reproducibility of Computed Tomography Radiomics for Pulmonary Nodules: A Multicenter Phantom Study. Invest Radiol 2022; 57:242-253. [PMID: 34743134 PMCID: PMC8903219 DOI: 10.1097/rli.0000000000000834] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2021] [Accepted: 08/31/2021] [Indexed: 11/25/2022]
Abstract
BACKGROUND Radiomics can yield minable information from medical images, which can facilitate computer-aided diagnosis. However, the lack of repeatability and reproducibility of radiomic features (RFs) may hinder their generalizability in clinical applications. OBJECTIVES The aims of this study were to explore 3 main sources of variability in RFs, investigate their influencing magnitudes and patterns, and identify a subset of robust RFs for further studies. MATERIALS AND METHODS A chest phantom with nodules was scanned with different computed tomography (CT) scanners repeatedly with varying acquisition and reconstruction parameters (April-May 2019) to evaluate 3 sources of variability: test-retest, inter-CT, and intra-CT protocol variability. The robustness of the RFs was measured using the concordance correlation coefficient, dynamic range, and intraclass correlation coefficient (ICC). The influencing magnitudes and patterns were analyzed using the Friedman test and Spearman rank correlation coefficient. Stable and informative RFs were selected, and their redundancy was eliminated using hierarchical clustering. Clinical validation was also performed to verify the clinical effectiveness and potential enhancement of the generalizability of radiomics research. RESULTS A total of 1295 RFs that showed all 3 sources of variability were included. The reconstruction kernel and the iteration level showed the greatest (ICC, 0.35 ± 0.31) and the least (ICC, 0.63 ± 0.27) influence on magnitudes. The different sources of variability showed relatively consistent patterns of influence (false discovery rate <0.001). Finally, we obtained a subset of 19 stable, informative, and nonredundant RFs under all 3 sources of variability. These RFs exhibited clinical effectiveness and showed better prediction performance than unstable RFs in the validation dataset (P = 0.017, Delong test). CONCLUSIONS The stability of RFs was affected to different degrees by test-retest and differences in CT manufacturers and models and CT acquisition and reconstruction parameters, but the influences of these factors showed relatively consistent patterns. We also obtained a subset of 19 stable, informative, and nonredundant RFs that should be preferably used to enhance the generalizability of further radiomics research.
Collapse
Affiliation(s)
- Xueqing Peng
- From the Institutes of Biomedical Sciences, Fudan University, Shanghai
| | - Shuyi Yang
- Department of Radiology, Zhongshan Hospital, Fudan University, Shanghai
- Shanghai Institute of Medical Imaging, Shanghai
- Department of Medical Imaging, Shanghai Medical College, Fudan University, Shanghai
| | - Lingxiao Zhou
- Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen, Guangdong Province
| | - Yu Mei
- Shanghai Mental Health Center, Shanghai
| | - Lili Shi
- From the Institutes of Biomedical Sciences, Fudan University, Shanghai
| | - Rengyin Zhang
- Department of Radiology, Shanghai Public Health Clinical Center, Fudan University, Shanghai
| | - Fei Shan
- Department of Radiology, Shanghai Public Health Clinical Center, Fudan University, Shanghai
| | - Lei Liu
- From the Institutes of Biomedical Sciences, Fudan University, Shanghai
- School of Basic Medical Sciences, Fudan University, Shanghai, China
| |
Collapse
|
9
|
CT Reconstruction Kernels and the Effect of Pre- and Post-Processing on the Reproducibility of Handcrafted Radiomic Features. J Pers Med 2022; 12:jpm12040553. [PMID: 35455668 PMCID: PMC9030848 DOI: 10.3390/jpm12040553] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2022] [Revised: 03/27/2022] [Accepted: 03/28/2022] [Indexed: 02/06/2023] Open
Abstract
Handcrafted radiomics features (HRFs) are quantitative features extracted from medical images to decode biological information to improve clinical decision making. Despite the potential of the field, limitations have been identified. The most important identified limitation, currently, is the sensitivity of HRF to variations in image acquisition and reconstruction parameters. In this study, we investigated the use of Reconstruction Kernel Normalization (RKN) and ComBat harmonization to improve the reproducibility of HRFs across scans acquired with different reconstruction kernels. A set of phantom scans (n = 28) acquired on five different scanner models was analyzed. HRFs were extracted from the original scans, and scans were harmonized using the RKN method. ComBat harmonization was applied on both sets of HRFs. The reproducibility of HRFs was assessed using the concordance correlation coefficient. The difference in the number of reproducible HRFs in each scenario was assessed using McNemar’s test. The majority of HRFs were found to be sensitive to variations in the reconstruction kernels, and only six HRFs were found to be robust with respect to variations in reconstruction kernels. The use of RKN resulted in a significant increment in the number of reproducible HRFs in 19 out of the 67 investigated scenarios (28.4%), while the ComBat technique resulted in a significant increment in 36 (53.7%) scenarios. The combination of methods resulted in a significant increment in 53 (79.1%) scenarios compared to the HRFs extracted from original images. Since the benefit of applying the harmonization methods depended on the data being harmonized, reproducibility analysis is recommended before performing radiomics analysis. For future radiomics studies incorporating images acquired with similar image acquisition and reconstruction parameters, except for the reconstruction kernels, we recommend the systematic use of the pre- and post-processing approaches (respectively, RKN and ComBat).
Collapse
|
10
|
Assessing radiomics feature stability with simulated CT acquisitions. Sci Rep 2022; 12:4732. [PMID: 35304508 PMCID: PMC8933485 DOI: 10.1038/s41598-022-08301-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2021] [Accepted: 03/03/2022] [Indexed: 11/29/2022] Open
Abstract
Medical imaging quantitative features had once disputable usefulness in clinical studies. Nowadays, advancements in analysis techniques, for instance through machine learning, have enabled quantitative features to be progressively useful in diagnosis and research. Tissue characterisation is improved via the “radiomics” features, whose extraction can be automated. Despite the advances, stability of quantitative features remains an important open problem. As features can be highly sensitive to variations of acquisition details, it is not trivial to quantify stability and efficiently select stable features. In this work, we develop and validate a Computed Tomography (CT) simulator environment based on the publicly available ASTRA toolbox (www.astra-toolbox.com). We show that the variability, stability and discriminative power of the radiomics features extracted from the virtual phantom images generated by the simulator are similar to those observed in a tandem phantom study. Additionally, we show that the variability is matched between a multi-center phantom study and simulated results. Consequently, we demonstrate that the simulator can be utilised to assess radiomics features’ stability and discriminative power.
Collapse
|
11
|
Jimenez-Del-Toro O, Aberle C, Bach M, Schaer R, Obmann MM, Flouris K, Konukoglu E, Stieltjes B, Müller H, Depeursinge A. The Discriminative Power and Stability of Radiomics Features With Computed Tomography Variations: Task-Based Analysis in an Anthropomorphic 3D-Printed CT Phantom. Invest Radiol 2021; 56:820-825. [PMID: 34038065 DOI: 10.1097/rli.0000000000000795] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
OBJECTIVES The aims of this study were to determine the stability of radiomics features against computed tomography (CT) parameter variations and to study their discriminative power concerning tissue classification using a 3D-printed CT phantom based on real patient data. MATERIALS AND METHODS A radiopaque 3D phantom was developed using real patient data and a potassium iodide solution paper-printing technique. Normal liver tissue and 3 lesion types (benign cyst, hemangioma, and metastasis) were manually annotated in the phantom. The stability and discriminative power of 86 radiomics features were assessed in measurements taken from 240 CT series with 8 parameter variations of reconstruction algorithms, reconstruction kernels, slice thickness, and slice spacing. Pairwise parameter group and pairwise tissue class comparisons were performed using Wilcoxon signed rank tests. RESULTS In total, 19,264 feature stability tests and 8256 discriminative power tests were performed. The 8 CT parameter variation pairwise group comparisons had statistically significant differences on average in 78/86 radiomics features. On the other hand, 84% of the univariate radiomics feature tests had a successful and statistically significant differentiation of the 4 classes of liver tissue. The 86 radiomics features were ranked according to the cumulative sum of successful stability and discriminative power tests. CONCLUSIONS The differences in radiomics feature values obtained from different types of liver tissue are generally greater than the intraclass differences resulting from CT parameter variations.
Collapse
Affiliation(s)
| | - Christoph Aberle
- Department of Radiology, University Hospital Basel, University of Basel, Basel
| | - Michael Bach
- Department of Radiology, University Hospital Basel, University of Basel, Basel
| | - Roger Schaer
- From the University of Applied Sciences Western Switzerland (HES-SO) Valais, Sierre
| | - Markus M Obmann
- Department of Radiology, University Hospital Basel, University of Basel, Basel
| | | | | | - Bram Stieltjes
- Department of Radiology, University Hospital Basel, University of Basel, Basel
| | | | | |
Collapse
|
12
|
Xue C, Yuan J, Lo GG, Chang ATY, Poon DMC, Wong OL, Zhou Y, Chu WCW. Radiomics feature reliability assessed by intraclass correlation coefficient: a systematic review. Quant Imaging Med Surg 2021; 11:4431-4460. [PMID: 34603997 DOI: 10.21037/qims-21-86] [Citation(s) in RCA: 63] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2021] [Accepted: 05/17/2021] [Indexed: 12/13/2022]
Abstract
Radiomics research is rapidly growing in recent years, but more concerns on radiomics reliability are also raised. This review attempts to update and overview the current status of radiomics reliability research in the ever expanding medical literature from the perspective of a single reliability metric of intraclass correlation coefficient (ICC). To conduct this systematic review, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. After literature search and selection, a total of 481 radiomics studies using CT, PET, or MRI, covering a wide range of subject and disease types, were included for review. In these highly heterogeneous studies, feature reliability to image segmentation was much more investigated than reliability to other factors, such as image acquisition, reconstruction, post-processing, and feature quantification. The reported ICCs also suggested high radiomics feature reliability to image segmentation. Image acquisition was found to introduce much more feature variability than image segmentation, in particular for MRI, based on the reported ICC values. Image post-processing and feature quantification yielded different levels of radiomics reliability and might be used to mitigate image acquisition-induced variability. Some common flaws and pitfalls in ICC use were identified, and suggestions on better ICC use were given. Due to the extremely high study heterogeneities and possible risks of bias, the degree of radiomics feature reliability that has been achieved could not yet be safely synthesized or derived in this review. More future researches on radiomics reliability are warranted.
Collapse
Affiliation(s)
- Cindy Xue
- Medical Physics and Research Department, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China.,Department of Imaging and Interventional Radiology, Faculty of Medicine, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong, China
| | - Jing Yuan
- Medical Physics and Research Department, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Gladys G Lo
- Department of Diagnostic & Interventional Radiology, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Amy T Y Chang
- Comprehensive Oncology Centre, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Darren M C Poon
- Comprehensive Oncology Centre, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Oi Lei Wong
- Medical Physics and Research Department, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Yihang Zhou
- Medical Physics and Research Department, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Winnie C W Chu
- Department of Imaging and Interventional Radiology, Faculty of Medicine, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong, China
| |
Collapse
|
13
|
Perkonigg M, Hofmanninger J, Herold CJ, Brink JA, Pianykh O, Prosch H, Langs G. Dynamic memory to alleviate catastrophic forgetting in continual learning with medical imaging. Nat Commun 2021; 12:5678. [PMID: 34584080 PMCID: PMC8479083 DOI: 10.1038/s41467-021-25858-z] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Accepted: 08/19/2021] [Indexed: 11/08/2022] Open
Abstract
Medical imaging is a central part of clinical diagnosis and treatment guidance. Machine learning has increasingly gained relevance because it captures features of disease and treatment response that are relevant for therapeutic decision-making. In clinical practice, the continuous progress of image acquisition technology or diagnostic procedures, the diversity of scanners, and evolving imaging protocols hamper the utility of machine learning, as prediction accuracy on new data deteriorates, or models become outdated due to these domain shifts. We propose a continual learning approach to deal with such domain shifts occurring at unknown time points. We adapt models to emerging variations in a continuous data stream while counteracting catastrophic forgetting. A dynamic memory enables rehearsal on a subset of diverse training data to mitigate forgetting while enabling models to expand to new domains. The technique balances memory by detecting pseudo-domains, representing different style clusters within the data stream. Evaluation of two different tasks, cardiac segmentation in magnetic resonance imaging and lung nodule detection in computed tomography, demonstrate a consistent advantage of the method.
Collapse
Affiliation(s)
- Matthias Perkonigg
- Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Johannes Hofmanninger
- Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Christian J Herold
- Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - James A Brink
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Oleg Pianykh
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Helmut Prosch
- Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Georg Langs
- Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria.
| |
Collapse
|
14
|
Mali SA, Ibrahim A, Woodruff HC, Andrearczyk V, Müller H, Primakov S, Salahuddin Z, Chatterjee A, Lambin P. Making Radiomics More Reproducible across Scanner and Imaging Protocol Variations: A Review of Harmonization Methods. J Pers Med 2021; 11:842. [PMID: 34575619 PMCID: PMC8472571 DOI: 10.3390/jpm11090842] [Citation(s) in RCA: 68] [Impact Index Per Article: 22.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2021] [Revised: 08/21/2021] [Accepted: 08/24/2021] [Indexed: 12/13/2022] Open
Abstract
Radiomics converts medical images into mineable data via a high-throughput extraction of quantitative features used for clinical decision support. However, these radiomic features are susceptible to variation across scanners, acquisition protocols, and reconstruction settings. Various investigations have assessed the reproducibility and validation of radiomic features across these discrepancies. In this narrative review, we combine systematic keyword searches with prior domain knowledge to discuss various harmonization solutions to make the radiomic features more reproducible across various scanners and protocol settings. Different harmonization solutions are discussed and divided into two main categories: image domain and feature domain. The image domain category comprises methods such as the standardization of image acquisition, post-processing of raw sensor-level image data, data augmentation techniques, and style transfer. The feature domain category consists of methods such as the identification of reproducible features and normalization techniques such as statistical normalization, intensity harmonization, ComBat and its derivatives, and normalization using deep learning. We also reflect upon the importance of deep learning solutions for addressing variability across multi-centric radiomic studies especially using generative adversarial networks (GANs), neural style transfer (NST) techniques, or a combination of both. We cover a broader range of methods especially GANs and NST methods in more detail than previous reviews.
Collapse
Affiliation(s)
- Shruti Atul Mali
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
| | - Abdalla Ibrahim
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
- Department of Radiology and Nuclear Medicine, GROW—School for Oncology, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands
- Department of Medical Physics, Division of Nuclear Medicine and Oncological Imaging, Hospital Center Universitaire de Liege, 4000 Liege, Belgium
- Department of Nuclear Medicine and Comprehensive Diagnostic Center Aachen (CDCA), University Hospital RWTH Aachen University, 52074 Aachen, Germany
| | - Henry C. Woodruff
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
- Department of Radiology and Nuclear Medicine, GROW—School for Oncology, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands
| | - Vincent Andrearczyk
- Institute of Information Systems, University of Applied Sciences and Arts Western Switzerland (HES-SO), rue du Technopole 3, 3960 Sierre, Switzerland; (V.A.); (H.M.)
| | - Henning Müller
- Institute of Information Systems, University of Applied Sciences and Arts Western Switzerland (HES-SO), rue du Technopole 3, 3960 Sierre, Switzerland; (V.A.); (H.M.)
| | - Sergey Primakov
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
| | - Zohaib Salahuddin
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
| | - Avishek Chatterjee
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
| | - Philippe Lambin
- The D-Lab, Department of Precision Medicine, GROW—School for Oncology, Maastricht University, Maastricht, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands; (A.I.); (H.C.W.); (S.P.); (Z.S.); (A.C.); (P.L.)
- Department of Radiology and Nuclear Medicine, GROW—School for Oncology, Maastricht University Medical Center+, P.O. Box 5800, 6202 AZ Maastricht, The Netherlands
| |
Collapse
|
15
|
Ibrahim A, Refaee T, Leijenaar RTH, Primakov S, Hustinx R, Mottaghy FM, Woodruff HC, Maidment ADA, Lambin P. The application of a workflow integrating the variable reproducibility and harmonizability of radiomic features on a phantom dataset. PLoS One 2021; 16:e0251147. [PMID: 33961646 PMCID: PMC8104396 DOI: 10.1371/journal.pone.0251147] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2020] [Accepted: 04/20/2021] [Indexed: 12/22/2022] Open
Abstract
Radiomics–the high throughput extraction of quantitative features from medical images and their correlation with clinical and biological endpoints- is the subject of active and extensive research. Although the field shows promise, the generalizability of radiomic signatures is affected significantly by differences in scan acquisition and reconstruction settings. Previous studies reported on the sensitivity of radiomic features (RFs) to test-retest variability, inter-observer segmentation variability, and intra-scanner variability. A framework involving robust radiomics analysis and the application of a post-reconstruction feature harmonization method using ComBat was recently proposed to address these challenges. In this study, we investigated the reproducibility of RFs across different scanners and scanning parameters using this framework. We analysed thirteen scans of a ten-layer phantom that were acquired differently. Each layer was subdivided into sixteen regions of interest (ROIs), and the scans were compared in a pairwise manner, resulting in seventy-eight different scenarios. Ninety-one RFs were extracted from each ROI. As hypothesized, we demonstrate that the reproducibility of a given RF is not a constant but is dependent on the heterogeneity found in the data under analysis. The number (%) of reproducible RFs varied across the pairwise scenarios investigated, having a wide range between 8 (8.8%) and 78 (85.7%) RFs. Furthermore, in contrast to what has been previously reported, and as hypothesized in the robust radiomics analysis framework, our results demonstrate that ComBat cannot be applied to all RFs but rather on a percentage of those–the “ComBatable” RFs–which differed depending on the data being harmonized. The number (%) of reproducible RFs following ComBat harmonization varied across the pairwise scenarios investigated, ranging from 14 (15.4%) to 80 (87.9%) RFs, and was found to depend on the heterogeneity in the data. We conclude that the standardization of image acquisition protocols remains the cornerstone for improving the reproducibility of RFs, and the generalizability of the signatures developed. Our proposed approach helps identify the reproducible RFs across different datasets.
Collapse
Affiliation(s)
- Abdalla Ibrahim
- The D-Lab, Department of Precision Medicine, GROW- School for Oncology, Maastricht University, Maastricht, The Netherlands
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands
- Division of Nuclear Medicine and Oncological Imaging, Department of Medical Physics, University Hospital of Liège and GIGA CRC-in vivo imaging, University of Liège, Liege, Belgium
- Department of Nuclear Medicine and Comprehensive Diagnostic Centre Aachen (CDCA), University Hospital RWTH Aachen University, Aachen, Germany
- * E-mail:
| | - Turkey Refaee
- The D-Lab, Department of Precision Medicine, GROW- School for Oncology, Maastricht University, Maastricht, The Netherlands
- Faculty of Applied Medical Sciences, Department of Diagnostic Radiology, Jazan University, Jazan, Saudi Arabia
| | | | - Sergey Primakov
- The D-Lab, Department of Precision Medicine, GROW- School for Oncology, Maastricht University, Maastricht, The Netherlands
- Department of Nuclear Medicine and Comprehensive Diagnostic Centre Aachen (CDCA), University Hospital RWTH Aachen University, Aachen, Germany
| | - Roland Hustinx
- Division of Nuclear Medicine and Oncological Imaging, Department of Medical Physics, University Hospital of Liège and GIGA CRC-in vivo imaging, University of Liège, Liege, Belgium
| | - Felix M. Mottaghy
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands
- Department of Nuclear Medicine and Comprehensive Diagnostic Centre Aachen (CDCA), University Hospital RWTH Aachen University, Aachen, Germany
| | - Henry C. Woodruff
- The D-Lab, Department of Precision Medicine, GROW- School for Oncology, Maastricht University, Maastricht, The Netherlands
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Andrew D. A. Maidment
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Philippe Lambin
- The D-Lab, Department of Precision Medicine, GROW- School for Oncology, Maastricht University, Maastricht, The Netherlands
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, Maastricht, The Netherlands
| |
Collapse
|