"Real-world" radiomics from multi-vendor MRI: an original retrospective study on the prediction of nodal status and disease survival in breast cancer, as an exemplar to promote discussion of the wider issues

Curation of myeloma observational study MALIMAR using XNAT: solving the challenges posed by real-world data

OBJECTIVES

MAchine Learning In MyelomA Response (MALIMAR) is an observational clinical study combining "real-world" and clinical trial data, both retrospective and prospective. Images were acquired on three MRI scanners over a 10-year window at two institutions, leading to a need for extensive curation.

METHODS

Curation involved image aggregation, pseudonymisation, allocation between project phases, data cleaning, upload to an XNAT repository visible from multiple sites, annotation, incorporation of machine learning research outputs and quality assurance using programmatic methods.

RESULTS

A total of 796 whole-body MR imaging sessions from 462 subjects were curated. A major change in scan protocol part way through the retrospective window meant that approximately 30% of available imaging sessions had properties that differed significantly from the remainder of the data. Issues were found with a vendor-supplied clinical algorithm for "composing" whole-body images from multiple imaging stations. Historic weaknesses in a digital video disk (DVD) research archive (already addressed by the mid-2010s) were highlighted by incomplete datasets, some of which could not be completely recovered. The final dataset contained 736 imaging sessions for 432 subjects. Software was written to clean and harmonise data. Implications for the subsequent machine learning activity are considered.

CONCLUSIONS

MALIMAR exemplifies the vital role that curation plays in machine learning studies that use real-world data. A research repository such as XNAT facilitates day-to-day management, ensures robustness and consistency and enhances the value of the final dataset. The types of process described here will be vital for future large-scale multi-institutional and multi-national imaging projects.

CRITICAL RELEVANCE STATEMENT

This article showcases innovative data curation methods using a state-of-the-art image repository platform; such tools will be vital for managing the large multi-institutional datasets required to train and validate generalisable ML algorithms and future foundation models in medical imaging.

KEY POINTS

• Heterogeneous data in the MALIMAR study required the development of novel curation strategies. • Correction of multiple problems affecting the real-world data was successful, but implications for machine learning are still being evaluated. • Modern image repositories have rich application programming interfaces enabling data enrichment and programmatic QA, making them much more than simple "image marts".

Clinical application of machine learning models in patients with prostate cancer before prostatectomy

BACKGROUND

To build machine learning predictive models for surgical risk assessment of extracapsular extension (ECE) in patients with prostate cancer (PCa) before radical prostatectomy; and to compare the use of decision curve analysis (DCA) and receiver operating characteristic (ROC) metrics for selecting input feature combinations in models.

METHODS

This retrospective observational study included two independent data sets: 139 participants from a single institution (training), and 55 from 15 other institutions (external validation), both treated with Robotic Assisted Radical Prostatectomy (RARP). Five ML models, based on different combinations of clinical, semantic (interpreted by a radiologist) and radiomics features computed from T2W-MRI images, were built to predict extracapsular extension in the prostatectomy specimen (pECE+). DCA plots were used to rank the models' net benefit when assigning patients to prostatectomy with non-nerve-sparing surgery (NNSS) or nerve-sparing surgery (NSS), depending on the predicted ECE status. DCA model rankings were compared with those drived from ROC area under the curve (AUC).

RESULTS

In the training data, the model using clinical, semantic, and radiomics features gave the highest net benefit values across relevant threshold probabilities, and similar decision curve was observed in the external validation data. The model ranking using the AUC was different in the discovery group and favoured the model using clinical + semantic features only.

CONCLUSIONS

The combined model based on clinical, semantic and radiomic features may be used to predict pECE + in patients with PCa and results in a positive net benefit when used to choose between prostatectomy with NNS or NNSS.

Successes and challenges in extracting information from DICOM image databases for audit and research

In radiography, much valuable associated data (metadata) is generated during image acquisition. The current setup of picture archive and communication systems (PACS) can make extraction of this metadata difficult, especially as it is typically stored with the image. The aim of this work is to examine the current challenges in extracting image metadata and to discuss the potential benefits of using this rich information. This work focuses on breast screening, though the conclusions are applicable to other modalities.The data stored in PACS contain information, currently underutilised, and is of great benefit for auditing and improving imaging and radiographic practice. From the literature, we present examples of the potential clinical benefit such as audits of dose, and radiographic practice, as well as more advanced research highlighting the effects of radiographic practice, e.g. cancer detection rates affected by imaging technology.This review considers the challenges in extracting data, namely,• The search tools for data on most PACS are inadequate being both time-consuming and limited in elements that can be searched.• Security and information governance considerations• Anonymisation of data if required• Data curationThe review describes some solutions that have been successfully implemented.• Retrospective extraction: direct query on PACS• Extracting data prospectively• Use of structured reports• Use of trusted research environmentsUltimately, the data access process will be made easier by inclusion during PACS procurement. Auditing data from PACS can be used to improve quality of imaging and workflow, all of which will be a clinical benefit to patients.

Characterization of Breast Tumors from MR Images Using Radiomics and Machine Learning Approaches

Determining histological subtypes, such as invasive ductal and invasive lobular carcinomas (IDCs and ILCs) and immunohistochemical markers, such as estrogen response (ER), progesterone response (PR), and the HER2 protein status is important in planning breast cancer treatment. MRI-based radiomic analysis is emerging as a non-invasive substitute for biopsy to determine these signatures. We explore the effectiveness of radiomics-based and CNN (convolutional neural network)-based classification models to this end. T1-weighted dynamic contrast-enhanced, contrast-subtracted T1, and T2-weighted MR images of 429 breast cancer tumors from 323 patients are used. Various combinations of input data and classification schemes are applied for ER⁺ vs. ER^-, PR⁺ vs. PR^-, HER2⁺ vs. HER2^-, and IDC vs. ILC classification tasks. The best results were obtained for the ER⁺ vs. ER^- and IDC vs. ILC classification tasks, with their respective AUCs reaching 0.78 and 0.73 on test data. The results with multi-contrast input data were generally better than the mono-contrast alone. The radiomics and CNN-based approaches generally exhibited comparable results. ER and IDC/ILC classification results were promising. PR and HER2 classifications need further investigation through a larger dataset. Better results by using multi-contrast data might indicate that multi-parametric quantitative MRI could be used to achieve more reliable classifiers.

Phantom Study on the Robustness of MR Radiomics Features: Comparing the Applicability of 3D Printed and Biological Phantoms

The objectives of our study were to (a) evaluate the feasibility of using 3D printed phantoms in magnetic resonance imaging (MR) in assessing the robustness and repeatability of radiomic parameters and (b) to compare the results obtained from the 3D printed phantoms to metrics obtained in biological phantoms. To this end, three different 3D phantoms were printed: a Hilbert cube (5 × 5 × 5 cm³) and two cubic quick response (QR) code phantoms (a large phantom (large QR) (5 × 5 × 4 cm³) and a small phantom (small QR) (4 × 4 × 3 cm³)). All 3D printed and biological phantoms (kiwis, tomatoes, and onions) were scanned thrice on clinical 1.5 T and 3 T MR with 1 mm and 2 mm isotropic resolution. Subsequent analyses included analyses of several radiomics indices (RI), their repeatability and reliability were calculated using the coefficient of variation (CV), the relative percentage difference (RPD), and the interclass coefficient (ICC) parameters. Additionally, the readability of QR codes obtained from the MR images was examined with several mobile phones and algorithms. The best repeatability (CV ≤ 10%) is reported for the acquisition protocols with the highest spatial resolution. In general, the repeatability and reliability of RI were better in data obtained at 1.5 T (CV = 1.9) than at 3 T (CV = 2.11). Furthermore, we report good agreements between results obtained for the 3D phantoms and biological phantoms. Finally, analyses of the read-out rate of the QR code revealed better texture analyses for images with a spatial resolution of 1 mm than 2 mm. In conclusion, 3D printing techniques offer a unique solution to create textures for analyzing the reliability of radiomic data from MR scans.

Prediction of Incomplete Response of Primary Tumour Based on Clinical and Radiomics Features in Inoperable Head and Neck Cancers after Definitive Treatment

Radical treatment of patients diagnosed with inoperable and locally advanced head and neck cancers (LAHNC) is still a challenge for clinicians. Prediction of incomplete response (IR) of primary tumour would be of value to the treatment optimization for patients with LAHNC. Aim of this study was to develop and evaluate models based on clinical and radiomics features for prediction of IR in patients diagnosed with LAHNC and treated with definitive chemoradiation or radiotherapy. Clinical and imaging data of 290 patients were included into this retrospective study. Clinical model was built based on tumour and patient related features. Radiomics features were extracted based on imaging data, consisting of contrast- and non-contrast-enhanced pre-treatment CT images, obtained in process of diagnosis and radiotherapy planning. Performance of clinical and combined models were evaluated with area under the ROC curve (AUROC). Classification performance was evaluated using 5-fold cross validation. Model based on selected clinical features including ECOG performance, tumour stage T3/4, primary site: oral cavity and tumour volume were significantly predictive for IR, with AUROC of 0.78. Combining clinical and radiomics features did not improve model’s performance, achieving AUROC 0.77 and 0.68 for non-contrast enhanced and contrast-enhanced images respectively. The model based on clinical features showed good performance in IR prediction. Combined model performance suggests that real-world imaging data might not yet be ready for use in predictive models.

3D DCE-MRI Radiomic Analysis for Malignant Lesion Prediction in Breast Cancer Patients

RATIONALE AND OBJECTIVES

To develop and validate a radiomic model, with radiomic features extracted from breast Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE-MRI) from a 1.5T scanner, for predicting the malignancy of masses with enhancement. Images were acquired using an 8-channel breast coil in the axial plane. The rationale behind this study is to show the feasibility of a radiomics-powered model that could be integrated into the clinical practice by exploiting only standard-of-care DCE-MRI with the goal of reducing the required image pre-processing (ie, normalization and quantitative imaging map generation).

MATERIALS AND METHODS

107 radiomic features were extracted from a manually annotated dataset of 111 patients, which was split into discovery and test sets. A feature calibration and pre-processing step was performed to find only robust non-redundant features. An in-depth discovery analysis was performed to define a predictive model: for this purpose, a Support Vector Machine (SVM) was trained in a nested 5-fold cross-validation scheme, by exploiting several unsupervised feature selection methods. The predictive model performance was evaluated in terms of Area Under the Receiver Operating Characteristic (AUROC), specificity, sensitivity, PPV and NPV. The test was performed on unseen held-out data.

RESULTS

The model combining Unsupervised Discriminative Feature Selection (UDFS) and SVMs on average achieved the best performance on the blinded test set: AUROC = 0.725±0.091, sensitivity = 0.709±0.176, specificity = 0.741±0.114, PPV = 0.72±0.093, and NPV = 0.75±0.114.

CONCLUSION

In this study, we built a radiomic predictive model based on breast DCE-MRI, using only the strongest enhancement phase, with promising results in terms of accuracy and specificity in the differentiation of malignant from benign breast lesions.

Quantifying lung cancer heterogeneity using novel CT features: a cross-institute study

Background

Radiomics-based image metrics are not used in the clinic despite the rapidly growing literature. We selected eight promising radiomic features and validated their value in decoding lung cancer heterogeneity.

Methods

CT images of 236 lung cancer patients were obtained from three different institutes, whereupon radiomic features were extracted according to a standardized procedure. The predictive value for patient long-term prognosis and association with routinely used semantic, genetic (e.g., epidermal growth factor receptor (EGFR)), and histopathological cancer profiles were validated. Feature measurement reproducibility was assessed.

Results

All eight selected features were robust across repeat scans (intraclass coefficient range: 0.81–0.99), and were associated with at least one of the cancer profiles: prognostic, semantic, genetic, and histopathological. For instance, “kurtosis” had a high predictive value of early death (AUC at first year: 0.70–0.75 in two independent cohorts), negative association with histopathological grade (Spearman’s r: − 0.30), and altered expression levels regarding EGFR mutation and semantic characteristics (solid intensity, spiculated shape, juxtapleural location, and pleura tag; all p < 0.05). Combined as a radiomic score, the features had a higher area under curve for predicting 5-year survival (train: 0.855, test: 0.780, external validation: 0.760) than routine characteristics (0.733, 0.622, 0.613, respectively), and a better capability in patient death risk stratification (hazard ratio: 5.828, 95% confidence interval: 2.915–11.561) than histopathological staging and grading.

Conclusions

We highlighted the clinical value of radiomic features. Following confirmation, these features may change the way in which we approach CT imaging and improve the individualized care of lung cancer patients.

Radiomics in Oncology: A Practical Guide

Radiomics refers to the extraction of mineable data from medical imaging and has been applied within oncology to improve diagnosis, prognostication, and clinical decision support, with the goal of delivering precision medicine. The authors provide a practical approach for successfully implementing a radiomic workflow from planning and conceptualization through manuscript writing. Applications in oncology typically are either classification tasks that involve computing the probability of a sample belonging to a category, such as benign versus malignant, or prediction of clinical events with a time-to-event analysis, such as overall survival. The radiomic workflow is multidisciplinary, involving radiologists and data and imaging scientists, and follows a stepwise process involving tumor segmentation, image preprocessing, feature extraction, model development, and validation. Images are curated and processed before segmentation, which can be performed on tumors, tumor subregions, or peritumoral zones. Extracted features typically describe the distribution of signal intensities and spatial relationship of pixels within a region of interest. To improve model performance and reduce overfitting, redundant and nonreproducible features are removed. Validation is essential to estimate model performance in new data and can be performed iteratively on samples of the dataset (cross-validation) or on a separate hold-out dataset by using internal or external data. A variety of noncommercial and commercial radiomic software applications can be used. Guidelines and artificial intelligence checklists are useful when planning and writing up radiomic studies. Although interest in the field continues to grow, radiologists should be familiar with potential pitfalls to ensure that meaningful conclusions can be drawn.

Online supplemental material is available for this article.

Published under a CC BY 4.0 license.

A Low-Dose CT-Based Radiomic Model to Improve Characterization and Screening Recall Intervals of Indeterminate Prevalent Pulmonary Nodules

Lung cancer (LC) is currently one of the main causes of cancer-related deaths worldwide. Low-dose computed tomography (LDCT) of the chest has been proven effective in secondary prevention (i.e., early detection) of LC by several trials. In this work, we investigated the potential impact of radiomics on indeterminate prevalent pulmonary nodule (PN) characterization and risk stratification in subjects undergoing LDCT-based LC screening. As a proof-of-concept for radiomic analyses, the first aim of our study was to assess whether indeterminate PNs could be automatically classified by an LDCT radiomic classifier as solid or sub-solid (first-level classification), and in particular for sub-solid lesions, as non-solid versus part-solid (second-level classification). The second aim of the study was to assess whether an LCDT radiomic classifier could automatically predict PN risk of malignancy, and thus optimize LDCT recall timing in screening programs. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC), accuracy, positive predictive value, negative predictive value, sensitivity, and specificity. The experimental results showed that an LDCT radiomic machine learning classifier can achieve excellent performance for characterization of screen-detected PNs (mean AUC of 0.89 ± 0.02 and 0.80 ± 0.18 on the blinded test dataset for the first-level and second-level classifiers, respectively), providing quantitative information to support clinical management. Our study showed that a radiomic classifier could be used to optimize LDCT recall for indeterminate PNs. According to the performance of such a classifier on the blinded test dataset, within the first 6 months, 46% of the malignant PNs and 38% of the benign ones were identified, improving early detection of LC by doubling the current detection rate of malignant nodules from 23% to 46% at a low cost of false positives. In conclusion, we showed the high potential of LDCT-based radiomics for improving the characterization and optimizing screening recall intervals of indeterminate PNs.