Prediction of clinical phenotypes in invasive breast carcinomas from the integration of radiomics and genomics data

Impact of MRI radiomic feature normalization for prognostic modelling in uterine endometrial and cervical cancers

Widespread clinical use of MRI radiomic tumor profiling for prognostication and treatment planning in cancers faces major obstacles due to limitations in standardization of radiomic features. The purpose of the current work was to assess the impact of different MRI scanning- and normalization protocols for the statistical analyses of tumor radiomic data in two patient cohorts with uterine endometrial-(EC) (n = 136) and cervical (CC) (n = 132) cancer. 1.5 T and 3 T, T1-weighted MRI 2 min post-contrast injection, T2-weighted turbo spin echo imaging, and diffusion-weighted imaging were acquired. Radiomic features were extracted from within manually segmented tumors in 3D and normalized either using z-score normalization or a linear regression model (LRM) accounting for linear dependencies with MRI acquisition parameters. Patients were clustered into two groups based on radiomic profile. Impact of MRI scanning parameters on cluster composition and prognostication were analyzed using Kruskal-Wallis tests, Kaplan-Meier plots, log-rank test, random survival forests and LASSO Cox regression with time-dependent area under curve (tdAUC) (α = 0.05). A large proportion of the radiomic features was statistically associated with MRI scanning protocol in both cohorts (EC: 162/385 [42%]; CC: 180/292 [62%]). A substantial number of EC (49/136 [36%]) and CC (50/132 [38%]) patients changed cluster when clustering was performed after z-score-versus LRM normalization. Prognostic modeling based on cluster groups yielded similar outputs for the two normalization methods in the EC/CC cohorts (log-rank test; z-score: p = 0.02/0.33; LRM: p = 0.01/0.45). Mean tdAUC for prognostic modeling of disease-specific survival (DSS) by the radiomic features in EC/CC was similar for the two normalization methods (random survival forests; z-score: mean tdAUC = 0.77/0.78; LRM: mean tdAUC = 0.80/0.75; LASSO Cox; z-score: mean tdAUC = 0.64/0.76; LRM: mean tdAUC = 0.76/0.75). Severe biases in tumor radiomics data due to MRI scanning parameters exist. Z-score normalization does not eliminate these biases, whereas LRM normalization effectively does. Still, radiomic cluster groups after z-score- and LRM normalization were similarly associated with DSS in EC and CC patients.

Deep-learning models for image-based gynecological cancer diagnosis: a systematic review and meta- analysis

Introduction

Gynecological cancers pose a significant threat to women worldwide, especially those in resource-limited settings. Human analysis of images remains the primary method of diagnosis, but it can be inconsistent and inaccurate. Deep learning (DL) can potentially enhance image-based diagnosis by providing objective and accurate results. This systematic review and meta-analysis aimed to summarize the recent advances of deep learning (DL) techniques for gynecological cancer diagnosis using various images and explore their future implications.

Methods

The study followed the PRISMA-2 guidelines, and the protocol was registered in PROSPERO. Five databases were searched for articles published from January 2018 to December 2022. Articles that focused on five types of gynecological cancer and used DL for diagnosis were selected. Two reviewers assessed the articles for eligibility and quality using the QUADAS-2 tool. Data was extracted from each study, and the performance of DL techniques for gynecological cancer classification was estimated by pooling and transforming sensitivity and specificity values using a random-effects model.

Results

The review included 48 studies, and the meta-analysis included 24 studies. The studies used different images and models to diagnose different gynecological cancers. The most popular models were ResNet, VGGNet, and UNet. DL algorithms showed more sensitivity but less specificity compared to machine learning (ML) methods. The AUC of the summary receiver operating characteristic plot was higher for DL algorithms than for ML methods. Of the 48 studies included, 41 were at low risk of bias.

Conclusion

This review highlights the potential of DL in improving the screening and diagnosis of gynecological cancer, particularly in resource-limited settings. However, the high heterogeneity and quality of the studies could affect the validity of the results. Further research is necessary to validate the findings of this study and to explore the potential of DL in improving gynecological cancer diagnosis.

The use of radiomics in magnetic resonance imaging for the pre-treatment characterisation of breast cancers: A scoping review

Radiomics is an emerging field that aims to extract and analyse a comprehensive set of quantitative features from medical images. This scoping review is focused on MRI-based radiomic features for the molecular profiling of breast tumours and the implications of this work for predicting patient outcomes. A thorough systematic literature search and outcome extraction were performed to identify relevant studies published in MEDLINE/PubMed (National Centre for Biotechnology Information), EMBASE and Scopus from 2015 onwards. The following information was retrieved from each article: study purpose, study design, extracted radiomic features, machine learning technique(s), sample size/characteristics, statistical result(s) and implications on patient outcomes. Based on the study purpose, four key themes were identified in the included 63 studies: tumour subtype classification (n = 35), pathologically complete response (pCR) prediction (n = 15), lymph node metastasis (LNM) detection (n = 7) and recurrence rate prediction (n = 6). In all four themes, reported accuracies widely varied among the studies, for example, area under receiver characteristics curve (AUC) for detecting LNM ranged from 0.72 to 0.91 and the AUC for predicting pCR ranged from 0.71 to 0.99. In all four themes, combining radiomic features with clinical data improved the predictive models. Preliminary results of this study showed radiomics potential to characterise the whole tumour heterogeneity, with clear implications for individual-targeted treatment. However, radiomics is still in the pre-clinical phase, currently with an insufficient number of large multicentre studies and those existing studies are often limited by insufficient methodological transparency and standardised workflow. Consequently, the clinical translation of existing studies is currently limited.

Longitudinal dynamic contrast-enhanced MRI radiomic models for early prediction of response to neoadjuvant systemic therapy in triple-negative breast cancer

Early prediction of neoadjuvant systemic therapy (NAST) response for triple-negative breast cancer (TNBC) patients could help oncologists select individualized treatment and avoid toxic effects associated with ineffective therapy in patients unlikely to achieve pathologic complete response (pCR). The objective of this study is to evaluate the performance of radiomic features of the peritumoral and tumoral regions from dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) acquired at different time points of NAST for early treatment response prediction in TNBC. This study included 163 Stage I-III patients with TNBC undergoing NAST as part of a prospective clinical trial (NCT02276443). Peritumoral and tumoral regions of interest were segmented on DCE images at baseline (BL) and after two (C2) and four (C4) cycles of NAST. Ten first-order (FO) radiomic features and 300 gray-level-co-occurrence matrix (GLCM) features were calculated. Area under the receiver operating characteristic curve (AUC) and Wilcoxon rank sum test were used to determine the most predictive features. Multivariate logistic regression models were used for performance assessment. Pearson correlation was used to assess intrareader and interreader variability. Seventy-eight patients (48%) had pCR (52 training, 26 testing), and 85 (52%) had non-pCR (57 training, 28 testing). Forty-six radiomic features had AUC at least 0.70, and 13 multivariate models had AUC at least 0.75 for training and testing sets. The Pearson correlation showed significant correlation between readers. In conclusion, Radiomic features from DCE-MRI are useful for differentiating pCR and non-pCR. Similarly, predictive radiomic models based on these features can improve early noninvasive treatment response prediction in TNBC patients undergoing NAST.

Patient Graph Deep Learning to Predict Breast Cancer Molecular Subtype

Breast cancer is a heterogeneous disease consisting of a diverse set of genomic mutations and clinical characteristics. The molecular subtypes of breast cancer are closely tied to prognosis and therapeutic treatment options. We investigate using deep graph learning on a collection of patient factors from multiple diagnostic disciplines to better represent breast cancer patient information and predict molecular subtype. Our method models breast cancer patient data into a multi-relational directed graph with extracted feature embeddings to directly represent patient information and diagnostic test results. We develop a radiographic image feature extraction pipeline to produce vector representation of breast cancer tumors in DCE-MRI and an autoencoder-based genomic variant embedding method to map variant assay results to a low-dimensional latent space. We leverage related-domain transfer learning to train and evaluate a Relational Graph Convolutional Network to predict the probabilities of molecular subtypes for individual breast cancer patient graphs. Our work found that utilizing information from multiple multimodal diagnostic disciplines improved the model's prediction results and produced more distinct learned feature representations for breast cancer patients. This research demonstrates the capabilities of graph neural networks and deep learning feature representation to perform multimodal data fusion and representation in the breast cancer domain.

A Radiomics Model Based on Synthetic MRI Acquisition for Predicting Neoadjuvant Systemic Treatment Response in Triple-Negative Breast Cancer

Purpose To determine if a radiomics model based on quantitative maps acquired with synthetic MRI (SyMRI) is useful for predicting neoadjuvant systemic therapy (NAST) response in triple-negative breast cancer (TNBC). Materials and Methods In this prospective study, 181 women diagnosed with stage I-III TNBC were scanned with a SyMRI sequence at baseline and at midtreatment (after four cycles of NAST), producing T1, T2, and proton density (PD) maps. Histopathologic analysis at surgery was used to determine pathologic complete response (pCR) or non-pCR status. From three-dimensional tumor contours drawn on the three maps, 310 histogram and textural features were extracted, resulting in 930 features per scan. Radiomic features were compared between pCR and non-pCR groups by using Wilcoxon rank sum test. To build a multivariable predictive model, logistic regression with elastic net regularization and cross-validation was performed for texture feature selection using 119 participants (median age, 52 years [range, 26-77 years]). An independent testing cohort of 62 participants (median age, 48 years [range, 23-74 years]) was used to evaluate and compare the models by area under the receiver operating characteristic curve (AUC). Results Univariable analysis identified 15 T1, 10 T2, and 12 PD radiomic features at midtreatment that predicted pCR with an AUC greater than 0.70 in both the training and testing cohorts. Multivariable radiomics models of maps acquired at midtreatment demonstrated superior performance over those acquired at baseline, achieving AUCs as high as 0.78 and 0.72 in the training and testing cohorts, respectively. Conclusion SyMRI-based radiomic features acquired at midtreatment are potentially useful for identifying early NAST responders in TNBC. Keywords: MR Imaging, Breast, Outcomes Analysis ClinicalTrials.gov registration no. NCT02276443 Supplemental material is available for this article. © RSNA, 2023 See also the commentary by Houser and Rapelyea in this issue.

Machine learning with multimodal data for COVID-19

In response to the unprecedented global healthcare crisis of the COVID-19 pandemic, the scientific community has joined forces to tackle the challenges and prepare for future pandemics. Multiple modalities of data have been investigated to understand the nature of COVID-19. In this paper, MIDRC investigators present an overview of the state-of-the-art development of multimodal machine learning for COVID-19 and model assessment considerations for future studies. We begin with a discussion of the lessons learned from radiogenomic studies for cancer diagnosis. We then summarize the multi-modality COVID-19 data investigated in the literature including symptoms and other clinical data, laboratory tests, imaging, pathology, physiology, and other omics data. Publicly available multimodal COVID-19 data provided by MIDRC and other sources are summarized. After an overview of machine learning developments using multimodal data for COVID-19, we present our perspectives on the future development of multimodal machine learning models for COVID-19.

Characterization of Breast Tumors from MR Images Using Radiomics and Machine Learning Approaches

Determining histological subtypes, such as invasive ductal and invasive lobular carcinomas (IDCs and ILCs) and immunohistochemical markers, such as estrogen response (ER), progesterone response (PR), and the HER2 protein status is important in planning breast cancer treatment. MRI-based radiomic analysis is emerging as a non-invasive substitute for biopsy to determine these signatures. We explore the effectiveness of radiomics-based and CNN (convolutional neural network)-based classification models to this end. T1-weighted dynamic contrast-enhanced, contrast-subtracted T1, and T2-weighted MR images of 429 breast cancer tumors from 323 patients are used. Various combinations of input data and classification schemes are applied for ER⁺ vs. ER^-, PR⁺ vs. PR^-, HER2⁺ vs. HER2^-, and IDC vs. ILC classification tasks. The best results were obtained for the ER⁺ vs. ER^- and IDC vs. ILC classification tasks, with their respective AUCs reaching 0.78 and 0.73 on test data. The results with multi-contrast input data were generally better than the mono-contrast alone. The radiomics and CNN-based approaches generally exhibited comparable results. ER and IDC/ILC classification results were promising. PR and HER2 classifications need further investigation through a larger dataset. Better results by using multi-contrast data might indicate that multi-parametric quantitative MRI could be used to achieve more reliable classifiers.

Preoperative Breast MRI: Current Evidence and Patient Selection

Breast MRI is the most sensitive imaging modality for the assessment of newly diagnosed breast cancer extent and can detect additional mammographically and clinically occult breast cancers in the ipsilateral and contralateral breasts. Nonetheless, appropriate use of breast MRI in the setting of newly diagnosed breast cancer remains debated. Though highly sensitive, MRI is less specific and may result in false positives and overestimation of disease when MRI findings are not biopsied prior to surgical excision. Furthermore, improved anatomic depiction of breast cancer on MRI has not consistently translated to improved clinical outcomes, such as lower rates of re-excision or breast cancer recurrence, though there is a paucity of well-designed studies examining these issues. In addition, current treatment paradigms have been developed in the absence of this more accurate depiction of disease span, which likely has limited the value of MRI. These issues have led to inconsistent and variable utilization of preoperative MRI across practice settings and providers. In this review, we discuss the history of breast MRI and its current use and recommendations with a focus on the preoperative setting. We review the evidence surrounding the use of preoperative MRI in the evaluation of breast malignancies and discuss the data on breast MRI in the setting of specific patient factors often used to determine breast MRI eligibility, such as age, index tumor phenotype, and breast density. Finally, we review the impact of breast MRI on surgical outcomes (re-excision and mastectomy rates) and long-term breast recurrence and survival outcomes.

Predicting transient ischemic attack risk in patients with mild carotid stenosis using machine learning and CT radiomics

Objective

This study aims to establish a radiomics-based machine learning model that predicts the risk of transient ischemic attack in patients with mild carotid stenosis (30-50% North American Symptomatic Carotid Endarterectomy Trial) using extracted computed tomography radiomics features and clinical information.

Methods

A total of 179 patients underwent carotid computed tomography angiography (CTA), and 219 carotid arteries with a plaque at the carotid bifurcation or proximal to the internal carotid artery were selected. The patients were divided into two groups; patients with symptoms of transient ischemic attack after CTA and patients without symptoms of transient ischemic attack after CTA. Then we performed random sampling methods stratified by the predictive outcome to obtain the training set (N = 165) and testing set (N = 66). 3D Slicer was employed to select the site of plaque on the computed tomography image as the volume of interest. An open-source package PyRadiomics in Python was used to extract radiomics features from the volume of interests. The random forest and logistic regression models were used to screen feature variables, and five classification algorithms were used, including random forest, eXtreme Gradient Boosting, logistic regression, support vector machine, and k-nearest neighbors. Data on radiomic feature information, clinical information, and the combination of these pieces of information were used to generate the model that predicts the risk of transient ischemic attack in patients with mild carotid artery stenosis (30-50% North American Symptomatic Carotid Endarterectomy Trial).

Results

The random forest model that was built based on the radiomics and clinical feature information had the highest accuracy (area under curve = 0.879; 95% confidence interval, 0.787-0.979). The combined model outperformed the clinical model, whereas the combined model showed no significant difference from the radiomics model.

Conclusion

The random forest model constructed with both radiomics and clinical information can accurately predict and improve discriminative power of computed tomography angiography in identifying ischemic symptoms in patients with carotid atherosclerosis. This model can aid in guiding the follow-up treatment of patients at high risk.

A Two-Step Feature Selection Radiomic Approach to Predict Molecular Outcomes in Breast Cancer

Breast Cancer (BC) is the most common cancer among women worldwide and is characterized by intra- and inter-tumor heterogeneity that strongly contributes towards its poor prognosis. The Estrogen Receptor (ER), Progesterone Receptor (PR), Human Epidermal Growth Factor Receptor 2 (HER2), and Ki67 antigen are the most examined markers depicting BC heterogeneity and have been shown to have a strong impact on BC prognosis. Radiomics can noninvasively predict BC heterogeneity through the quantitative evaluation of medical images, such as Magnetic Resonance Imaging (MRI), which has become increasingly important in the detection and characterization of BC. However, the lack of comprehensive BC datasets in terms of molecular outcomes and MRI modalities, and the absence of a general methodology to build and compare feature selection approaches and predictive models, limit the routine use of radiomics in the BC clinical practice. In this work, a new radiomic approach based on a two-step feature selection process was proposed to build predictors for ER, PR, HER2, and Ki67 markers. An in-house dataset was used, containing 92 multiparametric MRIs of patients with histologically proven BC and all four relevant biomarkers available. Thousands of radiomic features were extracted from post-contrast and subtracted Dynamic Contrast-Enanched (DCE) MRI images, Apparent Diffusion Coefficient (ADC) maps, and T2-weighted (T2) images. The two-step feature selection approach was used to identify significant radiomic features properly and then to build the final prediction models. They showed remarkable results in terms of F1-score for all the biomarkers: 84%, 63%, 90%, and 72% for ER, HER2, Ki67, and PR, respectively. When possible, the models were validated on the TCGA/TCIA Breast Cancer dataset, returning promising results (F1-score = 88% for the ER+/ER- classification task). The developed approach efficiently characterized BC heterogeneity according to the examined molecular biomarkers.

Radiogenomic association of deep MR imaging features with genomic profiles and clinical characteristics in breast cancer

BACKGROUND

It has been believed that traditional handcrafted radiomic features extracted from magnetic resonance imaging (MRI) of tumors are normally shallow and low-ordered. Recent advancement in deep learning technology shows that the high-order deep radiomic features extracted automatically from tumor images can capture tumor heterogeneity in a more efficient way. We hypothesize that MRI-based deep radiomic phenotypes have significant associations with molecular profiles of breast cancer tumors. We aim to identify deep radiomic features (DRFs) from MRI, evaluate their significance in predicting breast cancer (BC) clinical characteristics and explore their associations with multi-level genomic factors.

METHODS

A denoising autoencoder was built to retrospectively extract 4,096 DRFs from 110 BC patients' MRI. Visualization and clustering were applied to these DRFs. Linear Mixed Effect models were used to test their associations with multi-level genomic features (GFs) (risk genes, gene signatures, and biological pathway activities) extracted from the same patients' mRNA expression profile. A Least Absolute Shrinkage and Selection Operator model was used to identify the most predictive DRFs for each clinical characteristic (tumor size (T), lymph node metastasis (N), estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2) status).

RESULTS

Thirty-six conventional radiomic features (CRFs) for 87 of the 110 BC patients provided by a previous study were used for comparison. More than 1,000 DRFs were associated with the risk genes, gene signatures, and biological pathways activities (adjusted P-value < 0.05). DRFs produced better performance in predicting T, N, ER, PR, and HER2 status (AUC > 0.9) using DRFs. These DRFs showed significant powers of stratifying patients, linking to relevant biological and clinical characteristics. As a contrast, only eight risk genes were associated with CRFs. The RFs performed worse in predicting clinical characteristics than DRFs.

CONCLUSIONS

The deep learning-based auto MRI features perform better in predicting BC clinical characteristics, which are more significantly associated with GFs than traditional semi-auto MRI features. Our radiogenomic approach for identifying MRI-based imaging signatures may pave potential pathways for the discovery of genetic mechanisms regulating specific tumor phenotypes and may enable a more rapid innovation of novel imaging modalities, hence accelerating their translation to personalized medicine.

Development and validation of an MRI-radiomics nomogram for the prognosis of pancreatic ductal adenocarcinoma

Objective

To develop and validate an MRI-radiomics nomogram for the prognosis of pancreatic ductal adenocarcinoma (PDAC).

Background

"Radiomics" enables the investigation of huge amounts of radiological features in parallel by extracting high-throughput imaging data. MRI provides better tissue contrast with no ionizing radiation for PDAC.

Methods

There were 78 PDAC patients enrolled in this study. In total, there were 386 radiomics features extracted from MRI scan, which were screened by the least absolute shrinkage and selection operator algorithm to develop a risk score. Cox multivariate regression analysis was applied to develop the radiomics-based nomogram. The performance was assessed by discrimination and calibration.

Results

The radiomics-based risk-score was significantly associated with PDAC overall survival (OS) (P < 0.05). With respect to survival prediction, integrating the risk score, clinical data and TNM information into the nomogram exhibited better performance than the TNM staging system, radiomics model and clinical model. In addition, the nomogram showed fine discrimination and calibration.

Conclusions

The radiomics nomogram incorporating the radiomics data, clinical data and TNM information exhibited precise survival prediction for PDAC, which may help accelerate personalized precision treatment.

Clinical trial registration

clinicaltrials.gov, identifier NCT05313854.

A Review of Fusion Methods for Omics and Imaging Data

The development of omics data and biomedical images has greatly advanced the progress of precision medicine in diagnosis, treatment, and prognosis. The fusion of omics and imaging data, i.e., omics-imaging fusion, offers a new strategy for understanding complex diseases. However, due to a variety of issues such as the limited number of samples, high dimensionality of features, and heterogeneity of different data types, efficiently learning complementary or associated discriminative fusion information from omics and imaging data remains a challenge. Recently, numerous machine learning methods have been proposed to alleviate these problems. In this review, from the perspective of fusion levels and fusion methods, we first provide an overview of preprocessing and feature extraction methods for omics and imaging data, and comprehensively analyze and summarize the basic forms and variations of commonly used and newly emerging fusion methods, along with their advantages, disadvantages and the applicable scope. We then describe public datasets and compare experimental results of various fusion methods on the ADNI and TCGA datasets. Finally, we discuss future prospects and highlight remaining challenges in the field.

Development and validation of a radiogenomics model to predict axillary lymph node metastasis in breast cancer integrating MRI with transcriptome data: A multicohort study

Introduction

To develop and validate a radiogenomics model for predicting axillary lymph node metastasis (ALNM) in breast cancer compared to a genomics and radiomics model.

Methods

This retrospective study integrated transcriptomic data from The Cancer Genome Atlas with matched MRI data from The Cancer Imaging Archive for the same set of 111 patients with breast cancer, which were used as the training and testing groups. Fifteen patients from one hospital were enrolled as the external validation group. Radiomics features were extracted from dynamic contrast-enhanced (DCE)-MRI of breast cancer, and genomics features were derived from differentially expressed gene analysis of transcriptome data. Boruta was used for genomics and radiomics data dimension reduction and feature selection. Logistic regression was applied to develop genomics, radiomics, and radiogenomics models to predict ALNM. The performance of the three models was assessed by receiver operating characteristic curves and compared by the Delong test.

Results

The genomics model was established by nine genomics features, and the radiomics model was established by three radiomics features. The two models showed good discrimination performance in predicting ALNM in breast cancer, with areas under the curves (AUCs) of 0.80, 0.67, and 0.52 for the genomics model and 0.72, 0.68, and 0.71 for the radiomics model in the training, testing and external validation groups, respectively. The radiogenomics model integrated with five genomics features and three radiomics features had a better performance, with AUCs of 0.84, 0.75, and 0.82 in the three groups, respectively, which was higher than the AUC of the radiomics model in the training group and the genomics model in the external validation group (both P < 0.05).

Conclusion

The radiogenomics model combining radiomics features and genomics features improved the performance to predict ALNM in breast cancer.

Quantification of lung function on CT images based on pulmonary radiomic filtering

PURPOSE

To develop a radiomics filtering technique for characterizing spatial-encoded regional pulmonary ventilation information on lung computed tomography (CT).

METHODS

The lung volume was segmented on 46 CT images, and a 3D sliding window kernel was implemented across the lung volume to capture the spatial-encoded image information. Fifty-three radiomic features were extracted within the kernel, resulting in a fourth-order tensor object. As such, each voxel coordinate of the original lung was represented as a 53-dimensional feature vector, such that radiomic features could be viewed as feature maps within the lungs. To test the technique as a potential pulmonary ventilation biomarker, the radiomic feature maps were compared to paired functional images (Galligas PET or DTPA-SPECT) based on the Spearman correlation (ρ) analysis.

RESULTS

The radiomic feature maps GLRLM-based Run-Length Non-Uniformity and GLCOM-based Sum Average are found to be highly correlated with the functional imaging. The achieved ρ (median [range]) for the two features are 0.46 [0.05, 0.67] and 0.45 [0.21, 0.65] across 46 patients and 2 functional imaging modalities, respectively.

CONCLUSIONS

The results provide evidence that local regions of sparsely encoded heterogeneous lung parenchyma on CT are associated with diminished radiotracer uptake and measured lung ventilation defects on PET/SPECT imaging. These findings demonstrate the potential of radiomics to serve as a complementary tool to the current lung quantification techniques and provide hypothesis-generating data for future studies.

Machine learning for autism spectrum disorder diagnosis using structural magnetic resonance imaging: Promising but challenging

Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder that affects approximately 1% of the population and causes significant burdens. ASD's pathogenesis remains elusive; hence, diagnosis is based on a constellation of behaviors. Structural magnetic resonance imaging (sMRI) studies have shown several abnormalities in volumetric and geometric features of the autistic brain. However, inconsistent findings prevented most contributions from being translated into clinical practice. Establishing reliable biomarkers for ASD using sMRI is crucial for the correct diagnosis and treatment. In recent years, machine learning (ML) and specifically deep learning (DL) have quickly extended to almost every sector, notably in disease diagnosis. Thus, this has led to a shift and improvement in ASD diagnostic methods, fulfilling most clinical diagnostic requirements. However, ASD discovery remains difficult. This review examines the ML-based ASD diagnosis literature over the past 5 years. A literature-based taxonomy of the research landscape has been mapped, and the major aspects of this topic have been covered. First, we provide an overview of ML's general classification pipeline and the features of sMRI. Next, representative studies are highlighted and discussed in detail with respect to methods, and biomarkers. Finally, we highlight many common challenges and make recommendations for future directions. In short, the limited sample size was the main obstacle; Thus, comprehensive data sets and rigorous methods are necessary to check the generalizability of the results. ML technologies are expected to advance significantly in the coming years, contributing to the diagnosis of ASD and helping clinicians soon.

Development and Validation of a DeepSurv Nomogram to Predict Survival Outcomes and Guide Personalized Adjuvant Chemotherapy in Non-Small Cell Lung Cancer

Objective

To develop and validate a DeepSurv nomogram based on radiomic features extracted from computed tomography images and clinicopathological factors, to predict the overall survival and guide individualized adjuvant chemotherapy in patients with non-small cell lung cancer (NSCLC).

Patients and Methods

This retrospective study involved 976 consecutive patients with NSCLC (training cohort, n=683; validation cohort, n=293). DeepSurv was constructed based on 1,227 radiomic features, and the risk score was calculated for each patient as the output. A clinical multivariate Cox regression model was built with clinicopathological factors to determine the independent risk factors. Finally, a DeepSurv nomogram was constructed by integrating the risk score and independent clinicopathological factors. The discrimination capability, calibration, and clinical usefulness of the nomogram performance were assessed using concordance index evaluation, the Greenwood-Nam-D’Agostino test, and decision curve analysis, respectively. The treatment strategy was analyzed using a Kaplan–Meier curve and log-rank test for the high- and low-risk groups.

Results

The DeepSurv nomogram yielded a significantly better concordance index (training cohort, 0.821; validation cohort 0.768) with goodness-of-fit (P<0.05). The risk score, age, thyroid transcription factor-1, Ki-67, and disease stage were the independent risk factors for NSCLC.The Greenwood-Nam-D’Agostino test showed good calibration performance (P=0.39). Both high- and low-risk patients did not benefit from adjuvant chemotherapy, and chemotherapy in low-risk groups may lead to a poorer prognosis.

Conclusions

The DeepSurv nomogram, which is based on the risk score and independent risk factors, had good predictive performance for survival outcome. Further, it could be used to guide personalized adjuvant chemotherapy in patients with NSCLC.

Can artificial intelligence improve cancer treatments?

Artificial intelligence (AI) powered by the accumulating clinical and molecular data about cancer has fueled the expectation that a transformation in cancer treatments towards significant improvement of patient outcomes is at hand. However, such transformation has been so far elusive. The opacity of AI algorithms and the lack of quality annotated data being available at population scale are among the challenges to the application of AI in oncology. Fundamentally however, the heterogeneity of cancer and its evolutionary dynamics make every tumor response to therapy sufficiently different from the population, machine-learned statistical models, challenging hence the capacity of these models to yield reliable inferences about treatment recommendations that can improve patient outcomes. This article reviews the nominal elements of clinical decision-making for precision oncology and frames the utility of AI to cancer treatment improvements in light of cancer unique challenges.

Mining intrinsic information of convalescent patients after suffering coronavirus disease 2019 in Wuhan

OBJECTIVE

To summarize the potential characteristics of convalescent patients with coronavirus disease 2019 (COVID-19) in China based on emerging clinical tongue data and guide the treatment and recovery of COVID-19 patients from the perspective of Traditional Chinese Medicine tongue diagnosis.

METHODS

In this study, we developed and validated radiomics-based and lab-based methods as a novel approach to provide individualized pretreatment evaluation by analyzing different features to mine the orderliness behind tongue data of convalescent patients. In addition, this study analyzed the tongue features of convalescent patients from clinical tongue qualitative values, including thick and thin, fur, peeling, fat and lean, tooth marks and cracked, and greasy and putrid fur.

RESULTS

We included 2164 tongue images in total (34% from day 0, 35.4% from day 14 and 30.6% from day 28) from convalescent patients. The significance results are shown as follows. Firstly, as the recovery time prolongs, the L average values of tongue and coat decrease from 60.21 to 57.18 and from 60.06 to 57.03 respectively. Secondly, the decrease of abnormality rate of tongue coat, included greasy tongue fur, putrid fur, teeth-mark, thick-thin fur, are of significant statistical difference ( < 0.05). Thirdly, the average value of gray-level co-occurrence matrices increases from 0.173 to 0.194, the average value of entropy increases from 0.606 to 0.665, the average value of inverse difference normalized decrease from 0.981 to 0.979, and the average value of dissimilarity decrease from 0.1576 to 0.1828. The details of other radiomics features are describe in results section.

CONCLUSIONS

Our experiment shows that patients in different recovery periods have a relationship with quantitative values of tongue images, including L color space of the tongue and coat radiomics features analysis. This relationship can help clinical doctors master the recovery and health of patients as soon as possible and improve their understanding of the potential mechanisms underlying the dynamic changes and mechanisms underlying COVID-19.

Associating Preoperative MRI Features and Gene Expression Signatures of Early-stage Hepatocellular Carcinoma Patients using Machine Learning

BACKGROUND AND AIMS

The relationship between quantitative magnetic resonance imaging (MRI) imaging features and gene-expression signatures associated with the recurrence of hepatocellular carcinoma (HCC) is not well studied.

METHODS

In this study, we generated multivariable regression models to explore the correlation between the preoperative MRI features and Golgi membrane protein 1 (GOLM1), SET domain containing 7 (SETD7), and Rho family GTPase 1 (RND1) gene expression levels in a cohort study including 92 early-stage HCC patients. A total of 307 imaging features of tumor texture and shape were computed from T2-weighted MRI. The key MRI features were identified by performing a multi-step feature selection procedure including the correlation analysis and the application of RELIEFF algorithm. Afterward, regression models were generated using kernel-based support vector machines with 5-fold cross-validation.

RESULTS

The features computed from higher specificity MRI better described GOLM1 and RND1 gene-expression levels, while imaging features computed from lower specificity MRI data were more descriptive for the SETD7 gene. The GOLM1 regression model generated with three features demonstrated a moderate positive correlation (p<0.001), and the RND1 model developed with five variables was positively associated (p<0.001) with gene expression levels. Moreover, RND1 regression model integrating four features was moderately correlated with expressed RND1 levels (p<0.001).

CONCLUSIONS

The results demonstrated that MRI radiomics features could help quantify GOLM1, SETD7, and RND1 expression levels noninvasively and predict the recurrence risk for early-stage HCC patients.

A holistic overview of deep learning approach in medical imaging

Medical images are a rich source of invaluable necessary information used by clinicians. Recent technologies have introduced many advancements for exploiting the most of this information and use it to generate better analysis. Deep learning (DL) techniques have been empowered in medical images analysis using computer-assisted imaging contexts and presenting a lot of solutions and improvements while analyzing these images by radiologists and other specialists. In this paper, we present a survey of DL techniques used for variety of tasks along with the different medical image's modalities to provide critical review of the recent developments in this direction. We have organized our paper to provide significant contribution of deep leaning traits and learn its concepts, which is in turn helpful for non-expert in medical society. Then, we present several applications of deep learning (e.g., segmentation, classification, detection, etc.) which are commonly used for clinical purposes for different anatomical site, and we also present the main key terms for DL attributes like basic architecture, data augmentation, transfer learning, and feature selection methods. Medical images as inputs to deep learning architectures will be the mainstream in the coming years, and novel DL techniques are predicted to be the core of medical images analysis. We conclude our paper by addressing some research challenges and the suggested solutions for them found in literature, and also future promises and directions for further developments.

Assessing PD-L1 Expression Status Using Radiomic Features from Contrast-Enhanced Breast MRI in Breast Cancer Patients: Initial Results

Simple Summary

To our knowledge, this is the first study assessing radiomics coupled with machine learning from MRI-derived features to predict PD-L1 expression status in biopsy-proven triple negative breast cancers and comparing the performance of this approach with the performance of qualitative assessment by two radiologists. This pilot study shows that radiomics analysis coupled with machine learning of DCE-MRI is a promising approach to derive prognostic and predictive information and to select patients who could benefit from anti-PD-1/PD-L1 treatment. This technique could also be used to monitor PD-L1 expression, as it can vary over time and between different regions of the tumor, thus avoiding repeated biopsies.

Abstract

The purpose of this retrospective study was to assess whether radiomics analysis coupled with machine learning (ML) based on standard-of-care dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) can predict PD-L1 expression status in patients with triple negative breast cancer, and to compare the performance of this approach with radiologist review. Patients with biopsy-proven triple negative breast cancer who underwent pre-treatment breast MRI and whose PD-L1 status was available were included. Following 3D tumor segmentation and extraction of radiomic features, radiomic features with significant differences between PD-L1+ and PD-L1− patients were determined, and a final predictive model to predict PD-L1 status was developed using a coarse decision tree and five-fold cross-validation. Separately, all lesions were qualitatively assessed by two radiologists independently according to the BI-RADS lexicon. Of 62 women (mean age 47, range 31–81), 27 had PD-L1− tumors and 35 had PD-L1+ tumors. The final radiomics model to predict PD-L1 status utilized three MRI parameters, i.e., variance (FO), run length variance (RLM), and large zone low grey level emphasis (LZLGLE), for a sensitivity of 90.7%, specificity of 85.1%, and diagnostic accuracy of 88.2%. There were no significant associations between qualitative assessed DCE-MRI imaging features and PD-L1 status. Thus, radiomics analysis coupled with ML based on standard-of-care DCE-MRI is a promising approach to derive prognostic and predictive information and to select patients who could benefit from anti-PD-1/PD-L1 treatment.

How does DICOM support big data management? Investigating its use in medical imaging community

The diagnostic imaging field is experiencing considerable growth, followed by increasing production of massive amounts of data. The lack of standardization and privacy concerns are considered the main barriers to big data capitalization. This work aims to verify whether the advanced features of the DICOM standard, beyond imaging data storage, are effectively used in research practice. This issue will be analyzed by investigating the publicly shared medical imaging databases and assessing how much the most common medical imaging software tools support DICOM in all its potential. Therefore, 100 public databases and ten medical imaging software tools were selected and examined using a systematic approach. In particular, the DICOM fields related to privacy, segmentation and reporting have been assessed in the selected database; software tools have been evaluated for reading and writing the same DICOM fields. From our analysis, less than a third of the databases examined use the DICOM format to record meaningful information to manage the images. Regarding software, the vast majority does not allow the management, reading and writing of some or all the DICOM fields. Surprisingly, if we observe chest computed tomography data sharing to address the COVID-19 emergency, there are only two datasets out of 12 released in DICOM format. Our work shows how the DICOM can potentially fully support big data management; however, further efforts are still needed from the scientific and technological community to promote the use of the existing standard, encouraging data sharing and interoperability for a concrete development of big data analytics.

Radiomics as a New Frontier of Imaging for Cancer Prognosis: A Narrative Review

The evaluation of the efficacy of different therapies is of paramount importance for the patients and the clinicians in oncology, and it is usually possible by performing imaging investigations that are interpreted, taking in consideration different response evaluation criteria. In the last decade, texture analysis (TA) has been developed in order to help the radiologist to quantify and identify parameters related to tumor heterogeneity, which cannot be appreciated by the naked eye, that can be correlated with different endpoints, including cancer prognosis. The aim of this work is to analyze the impact of texture in the prediction of response and in prognosis stratification in oncology, taking into consideration different pathologies (lung cancer, breast cancer, gastric cancer, hepatic cancer, rectal cancer). Key references were derived from a PubMed query. Hand searching and clinicaltrials.gov were also used. This paper contains a narrative report and a critical discussion of radiomics approaches related to cancer prognosis in different fields of diseases.

The Impact of Tumor Edema on T2-Weighted 3T-MRI Invasive Breast Cancer Histological Characterization: A Pilot Radiomics Study

Simple Summary

Breast cancer is the most common cancer in women worldwide. Currently the use of MR is mandatory in staging phase. The standard protocol includes T2-weighted sequences for morphology and signal analysis, T1-weighted images for adding information (i.e., ematic or adipous components), diffusion-weighted sequences which provide information on tissue cellularity, and dynamic post-contrast sequences useful for detecting and locating lesions. Although not considered among the main prognostic factors in current guidelines, tumor-associated edema provides useful information on tumor aggressiveness, and has been shown to be associated with the main histological tumor characteristics. With this work, entitled “The Impact of Tumor Edema on T2-weighted 3T-MRI Invasive Breast Cancer Histological Characterization: a Pilot Radiomics Study”, we want to demonstrate that radiomics edema, based on algorithms that allow the extraction of imaging features not visible to the human eye, can further increase the accuracy in the prediction of histological factors compared to the use of traditional information only.

Abstract

Background: to evaluate the contribution of edema associated with histological features to the prediction of breast cancer (BC) prognosis using T2-weighted MRI radiomics. Methods: 160 patients who underwent staging 3T-MRI from January 2015 to January 2019, with 164 histologically proven invasive BC lesions, were retrospectively reviewed. Patient data (age, menopausal status, family history, hormone therapy), tumor MRI-features (location, margins, enhancement) and histological features (histological type, grading, ER, PgR, HER2, Ki-67 index) were collected. Of the 160 MRI exams, 120 were considered eligible, corresponding to 127 lesions. T2-MRI were used to identify edema, which was classified in four groups: peritumoral, pre-pectoral, subcutaneous, or diffuse. A semi-automatic segmentation of the edema was performed for each lesion, using 3D Slicer open-source software. Main radiomics features were extracted and selected using a wrapper selection method. A Random Forest type classifier was trained to measure the performance of predicting histological factors using semantic features (patient data and MRI features) alone and semantic features associated with edema radiomics features. Results: edema was absent in 37 lesions and present in 127 (62 peritumoral, 26 pre-pectoral, 16 subcutaneous, 23 diffuse). The AUC-classifier obtained by associating edema radiomics with semantic features was always higher compared to the AUC-classifier obtained from semantic features alone, for all five histological classes prediction (0.645 vs. 0.520 for histological type, 0.789 vs. 0.590 for grading, 0.487 vs. 0.466 for ER, 0.659 vs. 0.546 for PgR, and 0.62 vs. 0.573 for Ki67). Conclusions: radiomic features extracted from tumor edema contribute significantly to predicting tumor histology, increasing the accuracy obtained from the combination of patient clinical characteristics and breast imaging data.

Radiomics, aptamers and nanobodies: New insights in cancer diagnostics and imaging

At present, cancer is a major health issue and the second leading cause of mortality worldwide. Researchers have been working hard on investigating not only improved therapeutics but also on early detection methods, both critical to increasing treatment efficacy and developing methods for disease prevention. Diagnosis of cancers at an early stage can promote timely medical intervention and effective treatment and will result in inhibiting tumor growth and development. Several advances have been made in the diagnostics and imagining technologies for early tumor detection and deciding an effective therapy these include radiomics, nanobodies, and aptamers. Here in this review, we summarize the main applications of radiomics, aptamers, and the use of nanobody-based probes for molecular imaging applications in diagnosis, treatment planning, and evaluations in the field of oncology to develop quantitative and personalized medicine. The preclinical data reported to date are quite promising, and it is predicted that nanobody-based molecular imaging agents will play an important role in the diagnosis and management of different cancer types in near future.

AI-enhanced breast imaging: Where are we and where are we heading?

Significant advances in imaging analysis and the development of high-throughput methods that can extract and correlate multiple imaging parameters with different clinical outcomes have led to a new direction in medical research. Radiomics and artificial intelligence (AI) studies are rapidly evolving and have many potential applications in breast imaging, such as breast cancer risk prediction, lesion detection and classification, radiogenomics, and prediction of treatment response and clinical outcomes. AI has been applied to different breast imaging modalities, including mammography, ultrasound, and magnetic resonance imaging, in different clinical scenarios. The application of AI tools in breast imaging has an unprecedented opportunity to better derive clinical value from imaging data and reshape the way we care for our patients. The aim of this study is to review the current knowledge and future applications of AI-enhanced breast imaging in clinical practice.

A Novel Approach for the Discovery of Biomarkers of Radiotherapy Response in Breast Cancer

Radiotherapy (RT) is an important treatment modality for the local control of breast cancer (BC). Unfortunately, not all patients that receive RT will obtain a therapeutic benefit, as cancer cells that either possess intrinsic radioresistance or develop resistance during treatment can reduce its efficacy. For RT treatment regimens to become personalised, there is a need to identify biomarkers that can predict and/or monitor a tumour's response to radiation. Here we describe a novel method to identify such biomarkers. Liquid chromatography-mass spectrometry (LC-MS) was used on conditioned media (CM) samples from a radiosensitive oestrogen receptor positive (ER⁺) BC cell line (MCF-7) to identify cancer-secreted biomarkers which reflected a response to radiation. A total of 33 radiation-induced secreted proteins that had higher (up to 12-fold) secretion levels at 24 h post-2 Gy radiation were identified. Secretomic results were combined with whole-transcriptome gene expression experiments, using both radiosensitive and radioresistant cells, to identify a signature related to intrinsic radiosensitivity. Gene expression analysis assessing the levels of the 33 proteins showed that 5 (YBX3, EIF4EBP2, DKK1, GNPNAT1 and TK1) had higher expression levels in the radiosensitive cells compared to their radioresistant derivatives; 3 of these proteins (DKK1, GNPNAT1 and TK1) underwent in-lab and initial clinical validation. Western blot analysis using CM samples from cell lines confirmed a significant increase in the release of each candidate biomarker from radiosensitive cells 24 h after treatment with a 2 Gy dose of radiation; no significant increase in secretion was observed in the radioresistant cells after radiation. Immunohistochemistry showed that higher intracellular protein levels of the biomarkers were associated with greater radiosensitivity. Intracellular levels were further assessed in pre-treatment biopsy tissues from patients diagnosed with ER⁺ BC that were subsequently treated with breast-conserving surgery and RT. High DKK1 and GNPNAT1 intracellular levels were associated with significantly increased recurrence-free survival times, indicating that these two candidate biomarkers have the potential to predict sensitivity to RT. We suggest that the methods highlighted in this study could be utilised for the identification of biomarkers that may have a potential clinical role in personalising and optimising RT dosing regimens, whilst limiting the administration of RT to patients who are unlikely to benefit.

Combination of computed tomography imaging-based radiomics and clinicopathological characteristics for predicting the clinical benefits of immune checkpoint inhibitors in lung cancer

BACKGROUND

In this study, we tested whether a combination of radiomic features extracted from baseline pre-immunotherapy computed tomography (CT) images and clinicopathological characteristics could be used as novel noninvasive biomarkers for predicting the clinical benefits of non-small cell lung cancer (NSCLC) patients treated with immune checkpoint inhibitors (ICIs).

METHODS

The data from 92 consecutive patients with lung cancer who had been treated with ICIs were retrospectively analyzed. In total, 88 radiomic features were selected from the pretreatment CT images for the construction of a random forest model. Radiomics model 1 was constructed based on the Rad-score. Using multivariate logistic regression analysis, the Rad-score and significant predictors were integrated into a single predictive model (radiomics nomogram model 1) to predict the durable clinical benefit (DCB) of ICIs. Radiomics model 2 was developed based on the same Rad-score as radiomics model 1.Using multivariate Cox proportional hazards regression analysis, the Rad-score, and independent risk factors, radiomics nomogram model 2 was constructed to predict the progression-free survival (PFS).

RESULTS

The models successfully predicted the patients who would benefit from ICIs. For radiomics model 1, the area under the receiver operating characteristic curve values for the training and validation cohorts were 0.848 and 0.795, respectively, whereas for radiomics nomogram model 1, the values were 0.902 and 0.877, respectively. For the PFS prediction, the Harrell's concordance indexes for the training and validation cohorts were 0.717 and 0.760, respectively, using radiomics model 2, whereas they were 0.749 and 0.791, respectively, using radiomics nomogram model 2.

CONCLUSIONS

CT-based radiomic features and clinicopathological factors can be used prior to the initiation of immunotherapy for identifying NSCLC patients who are the most likely to benefit from the therapy. This could guide the individualized treatment strategy for advanced NSCLC.

Pattern classification for breast lesion on FFDM by integration of radiomics and deep features

The radiomics model can be used in breast cancer detection via calculating quantitative image features. However, these features are explicitly designed, or handcrafted in advance, and this would limit their ability to characterize the lesion properly. This paper aims to build an integrated-features-based classification framework which cooperate the radiomics features and the deep features to classify benign and malignant breast lesions on full-filed digital mammography (FFDM). We propose a classification framework consists of three steps: (1) handcrafted features (HCFs) extraction and selection, (2) deep features (DFs) extraction and (3) the integrated features-based classification. Specifically, HCFs comprise the gray-level gap-length matrix (GLGLM) texture features and shape features, and DFs contain the pooled features and high-level fully-connected features. Then, a multi-classifier method is applied to construct our classification framework using integrated features for breast lesion classification. A total of 106 retrospective FFDM data (51 are malignant and 55 are benign) in both craniocaudal (CC) view and mediolateral oblique (MLO) view were included in this study. The areas under a receiver operating characteristic curve (AUC) value, accuracy, sensitivity, specificity and Youden's index, are used to examine the performance of our proposed method in differentiating benign and malignant breast lesion. Proposed framework trained on the concatenation of fully-connected features and HCFs can significantly improve classification performance (AUC of 94.6 %, accuracy of 96.4 %, sensitivity of 93.6 %, specificity of 98.9 % and Yonden's index of 92.5 %) compared with other features sets. Experimental results demonstrate that performance of proposed framework is improved, indicating the potential of concatenation of the fully-connected features and HCFs set in breast cancer patients.

"Real-world" radiomics from multi-vendor MRI: an original retrospective study on the prediction of nodal status and disease survival in breast cancer, as an exemplar to promote discussion of the wider issues

BACKGROUND

Most MRI radiomics studies to date, even multi-centre ones, have used "pure" datasets deliberately accrued from single-vendor, single-field-strength scanners. This does not reflect aspirations for the ultimate generalisability of AI models. We therefore investigated the development of a radiomics signature from heterogeneous data originating on six different imaging platforms, for a breast cancer exemplar, in order to provide input into future discussions of the viability of radiomics in "real-world" scenarios where image data are not controlled by specific trial protocols but reflective of routine clinical practice.

METHODS

One hundred fifty-six patients with pathologically proven breast cancer underwent multi-contrast MRI prior to neoadjuvant chemotherapy and/or surgery. From these, 92 patients were identified for whom T2-weighted, diffusion-weighted and contrast-enhanced T1-weighted sequences were available, as well as key clinicopathological variables. Regions-of-interest were drawn on the above image types and, from these, semantic and calculated radiomics features were derived. Classification models using a variety of methods, both with and without recursive feature elimination, were developed to predict pathological nodal status. Separately, we applied the same methods to analyse the information carried by the radiomic features regarding the originating scanner type and field strength. Repeated, ten-fold cross-validation was employed to verify the results. In parallel work, survival modelling was performed using random survival forests.

RESULTS

Prediction of nodal status yielded mean cross-validated AUC values of 0.735 ± 0.15 (SD) for clinical variables alone, 0.673 ± 0.16 (SD) for radiomic features only, and 0.764 ± 0.16 (SD) for radiomics and clinical features together. Prediction of scanner platform from the radiomics features yielded extremely high values of AUC between 0.91 and 1 for the different classes examined indicating the presence of confounding features for the nodal status classification task. Survival analysis, gave out-of-bag prediction errors of 19.3% (clinical features only), 36.9-51.8% (radiomic features from different combinations of image contrasts), and 26.7-35.6% (clinical plus radiomics features).

CONCLUSIONS

Radiomic classification models whose predictive ability was consistent with previous single-vendor, single-field strength studies have been obtained from multi-vendor, multi-field-strength data, despite clear confounding information being present. However, our sample size was too small to obtain useful survival modelling results.

Magnetic resonance radiomics signatures for predicting poorly differentiated hepatocellular carcinoma: A SQUIRE-compliant study

Radiomics contributes to the extraction of undetectable features with the naked eye from high-throughput quantitative images. In this study, 2 predictive models were constructed, which allowed recognition of poorly differentiated hepatocellular carcinoma (HCC). In addition, the effectiveness of the as-constructed signature was investigated in HCC patients.A retrospective study involving 188 patients (age, 29-85 years) enrolled from November 2010 to April 2018 was carried out. All patients were divided randomly into 2 cohorts, namely, the training cohort (n = 141) and the validation cohort (n = 47). The MRI images (DICOM) were collected from PACS before ablation; in addition, the radiomics features were extracted from the 3D tumor area on T1-weighted imaging (T1WI) scans, T2-weighted imaging (T2WI) scans, arterial images, portal images and delayed phase images. In total, 200 radiomics features were extracted. t test and Mann-Whitney U test were performed to exclude some radiomics signatures. Afterwards, a radiomics signature model was built through LASSO regression by RStudio Software. We constructed 2 support vector machine (SVM)-based models: 1 with a radiomics signature only (model 1) and 1 that integrated clinical and radiomics signatures (model 2). Then, the diagnostic performance of the radiomics signature was evaluated through receiver operating characteristic (ROC) analysis.The classification accuracy in the training and validation cohorts was 80.9% and 72.3%, respectively, for model 1. In the training cohort, the area under the ROC curve (AUC) was 0.623, while it was 0.576 in the validation cohort. The classification accuracy in the training and validation cohorts were 79.4% and 74.5%, respectively, for model 2. In the training cohort, the AUC was 0.721, while it was 0.681 in the validation cohort.The MRI-based radiomics signature and clinical model can distinguish HCC patients that belong in a low differentiation group from other patients, which helps in the performance of personal medical protocols.

Feasibility of MRI Radiomics for Predicting KRAS Mutation in Rectal Cancer

The mutation status of KRAS is a significant biomarker in the prognosis of rectal cancer. This study investigated the feasibility of MRI-based radiomics in predicting the mutation status of KRAS with a composite index which could be an important criterion for KRAS mutation in clinical practice. In this retrospective study, a total of 127 patients with rectal cancer were enrolled. The 3D Slicer was used to extract the radiomics features from the MRI images, and sparse support vector machine (SVM) with linear kernel was applied for feature reduction. The radiomics classifier for predicting the KRAS status was then constructed by Linear Discriminant Analysis (LDA) and its performance was evaluated. The composite index was determined with LDA model. Out of 127 rectal cancer subjects, there were 44 KRAS mutation cases and 83 wild cases. A total of 104 radiomics features were extracted, 54 features were filtered by linear SVM with L1-norm regularization and 6 features that had no significant correlations within them were finally selected. The radiomics classifier constructed using the 6 features featured an AUC value of 0.669 (specificity: 0.506; sensitivity: 0.773) with LDA. Furthermore, the composite index (Radscore) had statistically significant difference between the KRAS mutation and wild groups. It is suggested that the MRI-based radiomics has the potential in predicting the KRAS status in patients with rectal cancer, which may enhance the diagnostic value of MRI in rectal cancer.

Prediction of the clinicopathological subtypes of breast cancer using a fisher discriminant analysis model based on radiomic features of diffusion-weighted MRI

BACKGROUND

The clinicopathological classification of breast cancer is proposed according to therapeutic purposes. It is simplified and can be conducted easily in clinical practice, and this subtyping undoubtedly contributes to the treatment selection of breast cancer. This study aims to investigate the feasibility of using a Fisher discriminant analysis model based on radiomic features of diffusion-weighted MRI for predicting the clinicopathological subtypes of breast cancer.

METHODS

Patients who underwent breast magnetic resonance imaging were confirmed by retrieving data from our institutional picture archiving and communication system (PACS) between March 2013 and September 2017. Five clinicopathological subtypes were determined based on the status of ER, PR, HER2 and Ki-67 from the immunohistochemical test. The radiomic features of diffusion-weighted imaging were derived from the volume of interest (VOI) of each tumour. Fisher discriminant analysis was performed for clinicopathological subtyping by using a backward selection method. To evaluate the diagnostic performance of the radiomic features, ROC analyses were performed to differentiate between immunohistochemical biomarker-positive and -negative groups.

RESULTS

A total of 84 radiomic features of four statistical methods were included after preprocessing. The overall accuracy for predicting the clinicopathological subtypes was 96.4% by Fisher discriminant analysis, and the weighted accuracy was 96.6%. For predicting diverse clinicopathological subtypes, the prediction accuracies ranged from 92 to 100%. According to the cross-validation, the overall accuracy of the model was 82.1%, and the accuracies of the model for predicting the luminal A, luminal B_HER2-, luminal B_HER2+, HER2 positive and triple negative subtypes were 79, 77, 88, 92 and 73%, respectively. According to the ROC analysis, the radiomic features had excellent performance in differentiating between different statuses of ER, PR, HER2 and Ki-67.

CONCLUSIONS

The Fisher discriminant analysis model based on radiomic features of diffusion-weighted MRI is a reliable method for the prediction of clinicopathological breast cancer subtypes.

Machine learning prediction of axillary lymph node metastasis in breast cancer: 2D versus 3D radiomic features

PURPOSE

The purpose of this study was to distinguish axillary lymph node (ALN) status using preoperative breast DCE-MRI radiomics and compare the effects of two-dimensional (2D) and three-dimensional (3D) analysis.

METHODS

A retrospective study including 154 breast cancer patients all confirmed by pathology; 80 with ALN metastasis and 74 without. All MRI scans were achieved at a 3.0 Tesla scanner with 7 post-contrast MR phases sequentially acquired with a temporal resolution of 60 s. MRI radiomic features were extracted separately from a 2D single slice (i.e., the representative slice) and the 3D tumor volume. Several machine learning classifiers were built and compared using 2D or 3D analysis to distinguish positive vs negative ALN status. We performed independent test and 10-fold cross validation with multiple repetitions, and used bootstrap test, least absolute shrinkage selection operator, and receiver operating characteristic (ROC) curve analysis as statistical tests.

RESULTS

The highest area under the ROC curve (AUC) was 0.81 (95% confidence intervals [CI]: 0.80-0.83) and 0.82 (95% CI: 0.81-0.82) for 2D and 3D analysis, respectively; the corresponding accuracy was 79% and 80%. The linear discriminant analysis (LDA) classifier achieved the highest classification performance. None of the AUC differences between 2D and 3D analysis was statistically significant for the several tested machine learning classifiers (all P> 0.05).

CONCLUSIONS

Radiomic features from segmented tumor region in breast MRI were associated with ALN status. The separate radiomic analysis on 3D tumor volume showed a similar effect to the 2D analysis on the single representative slice in the tested machine learning classifiers.

Discovering and interpreting transcriptomic drivers of imaging traits using neural networks

MOTIVATION

Cancer heterogeneity is observed at multiple biological levels. To improve our understanding of these differences and their relevance in medicine, approaches to link organ- and tissue-level information from diagnostic images and cellular-level information from genomics are needed. However, these 'radiogenomic' studies often use linear or shallow models, depend on feature selection, or consider one gene at a time to map images to genes. Moreover, no study has systematically attempted to understand the molecular basis of imaging traits based on the interpretation of what the neural network has learned. These studies are thus limited in their ability to understand the transcriptomic drivers of imaging traits, which could provide additional context for determining clinical outcomes.

RESULTS

We present a neural network-based approach that takes high-dimensional gene expression data as input and performs non-linear mapping to an imaging trait. To interpret the models, we propose gene masking and gene saliency to extract learned relationships from radiogenomic neural networks. In glioblastoma patients, our models outperformed comparable classifiers (>0.10 AUC) and our interpretation methods were validated using a similar model to identify known relationships between genes and molecular subtypes. We found that tumor imaging traits had specific transcription patterns, e.g. edema and genes related to cellular invasion, and 10 radiogenomic traits were significantly predictive of survival. We demonstrate that neural networks can model transcriptomic heterogeneity to reflect differences in imaging and can be used to derive radiogenomic traits with clinical value.

AVAILABILITY AND IMPLEMENTATION

https://github.com/novasmedley/deepRadiogenomics.

CONTACT

whsu@mednet.ucla.edu.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Novel, non-invasive imaging approach to identify patients with advanced non-small cell lung cancer at risk of hyperprogressive disease with immune checkpoint blockade

Purpose

Hyperprogression is an atypical response pattern to immune checkpoint inhibition that has been described within non-small cell lung cancer (NSCLC). The paradoxical acceleration of tumor growth after immunotherapy has been associated with significantly shortened survival, and currently, there are no clinically validated biomarkers to identify patients at risk of hyperprogression.

Experimental design

A total of 109 patients with advanced NSCLC who underwent monotherapy with Programmed cell death protein-1 (PD1)/Programmed death-ligand-1 (PD-L1) inhibitors were included in the study. Using RECIST measurements, we divided the patients into responders (n=50) (complete/partial response or stable disease) and non-responders (n=59) (progressive disease). Tumor growth kinetics were used to further identify hyperprogressors (HPs, n=19) among non-responders. Patients were randomized into a training set (D₁=30) and a test set (D₂=79) with the essential caveat that HPs were evenly distributed among the two sets. A total of 198 radiomic textural patterns from within and around the target nodules and features relating to tortuosity of the nodule associated vasculature were extracted from the pretreatment CT scans.

Results

The random forest classifier using the top features associated with hyperprogression was able to distinguish between HP and other radiographical response patterns with an area under receiver operating curve of 0.85±0.06 in the training set (D₁=30) and 0.96 in the validation set (D₂=79). These features included one peritumoral texture feature from 5 to 10 mm outside the tumor and two nodule vessel-related tortuosity features. Kaplan-Meier survival curves showed a clear stratification between classifier predicted HPs versus non-HPs for overall survival (D₂: HR=2.66, 95% CI 1.27 to 5.55; p=0.009).

Conclusions

Our study suggests that image-based radiomics markers extracted from baseline CTs of advanced NSCLC treated with PD-1/PD-L1 inhibitors may help identify patients at risk of hyperprogressions.

Predicting Breast Cancer Molecular Subtype with MRI Dataset Utilizing Convolutional Neural Network Algorithm

To develop a convolutional neural network (CNN) algorithm that can predict the molecular subtype of a breast cancer based on MRI features. An IRB-approved study was performed in 216 patients with available pre-treatment MRIs and immunohistochemical staining pathology data. First post-contrast MRI images were used for 3D segmentation using 3D slicer. A CNN architecture was designed with 14 layers. Residual connections were used in the earlier layers to allow stabilization of gradients during backpropagation. Inception style layers were utilized deeper in the network to allow learned segregation of more complex feature mappings. Extensive regularization was utilized including dropout, L2, feature map dropout, and transition layers. The class imbalance was addressed by doubling the input of underrepresented classes and utilizing a class sensitive cost function. Parameters were tuned based on a 20% validation group. A class balanced holdout set of 40 patients was utilized as the testing set. Software code was written in Python using the TensorFlow module on a Linux workstation with one NVidia Titan X GPU. Seventy-four luminal A, 106 luminal B, 13 HER2+, and 23 basal breast tumors were evaluated. Testing set accuracy was measured at 70%. The class normalized macro area under receiver operating curve (ROC) was measured at 0.853. Non-normalized micro-aggregated AUC was measured at 0.871, representing improved discriminatory power for the highly represented Luminal A and Luminal B subtypes. Aggregate sensitivity and specificity was measured at 0.603 and 0.958. MRI analysis of breast cancers utilizing a novel CNN can predict the molecular subtype of breast cancers. Larger data sets will likely improve our model.

AI in Medical Imaging Informatics: Current Challenges and Future Directions

This paper reviews state-of-the-art research solutions across the spectrum of medical imaging informatics, discusses clinical translation, and provides future directions for advancing clinical practice. More specifically, it summarizes advances in medical imaging acquisition technologies for different modalities, highlighting the necessity for efficient medical data management strategies in the context of AI in big healthcare data analytics. It then provides a synopsis of contemporary and emerging algorithmic methods for disease classification and organ/ tissue segmentation, focusing on AI and deep learning architectures that have already become the de facto approach. The clinical benefits of in-silico modelling advances linked with evolving 3D reconstruction and visualization applications are further documented. Concluding, integrative analytics approaches driven by associate research branches highlighted in this study promise to revolutionize imaging informatics as known today across the healthcare continuum for both radiology and digital pathology applications. The latter, is projected to enable informed, more accurate diagnosis, timely prognosis, and effective treatment planning, underpinning precision medicine.

A rapid volume of interest-based approach of radiomics analysis of breast MRI for tumor decoding and phenotyping of breast cancer

BACKGROUND

Recently, radiomics has emerged as a non-invasive, imaging-based tissue characterization method in multiple cancer types. One limitation for robust and reproducible analysis lies in the inter-reader variability of the tumor annotations, which can potentially cause differences in the extracted feature sets and results. In this study, the diagnostic potential of a rapid and clinically feasible VOI (Volume of Interest)-based approach to radiomics is investigated to assess MR-derived parameters for predicting molecular subtype, hormonal receptor status, Ki67- and HER2-Expression, metastasis of lymph nodes and lymph vessel involvement as well as grading in patients with breast cancer.

METHODS

A total of 98 treatment-naïve patients (mean 59.7 years, range 28.0-89.4) with BI-RADS 5 and 6 lesions who underwent a dedicated breast MRI prior to therapy were retrospectively included in this study. The imaging protocol comprised dynamic contrast-enhanced T1-weighted imaging and T2-weighted imaging. Tumor annotations were obtained by drawing VOIs around the primary tumor lesions followed by thresholding. From each segmentation, 13.118 quantitative imaging features were extracted and analyzed with machine learning methods. Validation was performed by 5-fold cross-validation with 25 repeats.

RESULTS

Predictions for molecular subtypes obtained AUCs of 0.75 (HER2-enriched), 0.73 (triple-negative), 0.65 (luminal A) and 0.69 (luminal B). Differentiating subtypes from one another was highest for HER2-enriched vs triple-negative (AUC 0.97), followed by luminal B vs triple-negative (0.86). Receptor status predictions for Estrogen Receptor (ER), Progesterone Receptor (PR) and Hormone receptor positivity yielded AUCs of 0.67, 0.69 and 0.69, while Ki67 and HER2 Expressions achieved 0.81 and 0.62. Involvement of the lymph vessels could be predicted with an AUC of 0.8, while lymph node metastasis yielded an AUC of 0.71. Models for grading performed similar with an AUC of 0.71 for Elston-Ellis grading and 0.74 for the histological grading.

CONCLUSION

Our preliminary results of a rapid approach to VOI-based tumor-annotations for radiomics provides comparable results to current publications with the perks of clinical suitability, enabling a comprehensive non-invasive platform for breast tumor decoding and phenotyping.

Integrating tumor and nodal radiomics to predict lymph node metastasis in gastric cancer

BACKGROUND

To develop and validate a radiomics method via integrating tumor and lymph node radiomics for the preoperative prediction of lymph node (LN) status in gastric cancer (GC).

MATERIALS AND METHODS

We retrospectively collected 170 contrast-enhanced abdominal CT images from GC patients. Five times repeated random hold-out experiment was employed. Tumor and nodal radiomics features were extracted from each individual tumor and LN respectively, and then multi-step feature selection was performed. The optimal tumor and nodal features were selected using Pearson correlation analysis and sequential forward floating selection (SFFS) algorithm. After feature fusion, the SFFS algorithm was used to develop radiomics signatures. The performance of the radiomics signatures developed based on logistic regression classifier was further analyzed and compared using the area under the receiver operating characteristic curve (AUC).

RESULTS

The AUC values, reported as mean ± standard deviation, were 0.9319 ± 0.0129 and 0.8546 ± 0.0261 for the training and validation cohorts respectively. The radiomic signatures could predict LN status, especially in T2-stage, diffuse-type and moderately/well differentiated GC. After integrating clinicopathologic information, the radiomic-clinicopathologic model (training cohort, 0.9432 ± 0.0129; validation cohort, 0.8764 ± 0.0322) showed a better discrimination capability than other radiomics models and clinicopathologic model. The radiomic-clinicopathologic model also showed superior performance to the gastroenterologist' decision in all experiments, and outperformed the radiologist in some experiments.

CONCLUSION

Our proposed method presented good predictive performance and great potential for predicting LNM in GC. As a noninvasive preoperative prediction tool, it can be helpful for guiding the prognosis and treatment decision-making in GC patients.

Imaging-Genomic Study of Head and Neck Squamous Cell Carcinoma: Associations Between Radiomic Phenotypes and Genomic Mechanisms via Integration of The Cancer Genome Atlas and The Cancer Imaging Archive

PURPOSE

Recent data suggest that imaging radiomic features of a tumor could be indicative of important genomic biomarkers. Understanding the relationship between radiomic and genomic features is important for basic cancer research and future patient care. We performed a comprehensive study to discover the imaginggenomic associations in head and neck squamous cell carcinoma (HNSCC) and explore the potential of predicting tumor genomic alternations using radiomic features.

METHODS

Our retrospective study integrated whole-genome multiomics data from The Cancer Genome Atlas with matched computed tomography imaging data from The Cancer Imaging Archive for the same set of 126 patients with HNSCC. Linear regression and gene set enrichment analysis were used to identify statistically significant associations between radiomic imaging and genomic features. Random forest classifier was used to predict the status of two key HNSCC molecular biomarkers, human papillomavirus and disruptive TP53 mutation, on the basis of radiomic features.

RESULTS

Widespread and statistically significant associations were discovered between genomic features (including microRNA expression, somatic mutations, and transcriptional activity, copy number variations, and promoter region DNA methylation changes of pathways) and radiomic features characterizing the size, shape, and texture of tumor. Prediction of human papillomavirus and TP53 mutation status using radiomic features achieved areas under the receiver operating characteristic curve of 0.71 and 0.641, respectively.

CONCLUSION

Our exploratory study suggests that radiomic features are associated with genomic characteristics at multiple molecular layers in HNSCC and provides justification for continued development of radiomics as biomarkers for relevant genomic alterations in HNSCC.

eBreCaP: extreme learning-based model for breast cancer survival prediction

Breast cancer is the second leading cause of death in the world. Breast cancer research is focused towards its early prediction, diagnosis, and prognosis. Breast cancer can be predicted on omics profiles, clinical tests, and pathological images. The omics profiles comprise of genomic, proteomic, and transcriptomic profiles that are available as high-dimensional datasets. Survival prediction is carried out on omics data to predict early the onset of disease, relapse, reoccurrence of diseases, and biomarker identification. The early prediction of breast cancer is desired for the effective treatment of patients as delay can aggravate the staging of cancer. In this study, extreme learning machine (ELM) based model for breast cancer survival prediction named eBreCaP is proposed. It integrates the genomic (gene expression, copy number alteration, DNA methylation, protein expression) and pathological image datasets; and trains them using an ensemble of ELM with the six best-chosen models suitable to be applied on integrated data. eBreCaP has been evaluated on nine performance parameters, namely sensitivity, specificity, precision, accuracy, Matthews correlation coefficient, area under curve, area under precision-recall, hazard ratio, and concordance Index. eBreCaP has achieved an accuracy of 85% for early breast cancer survival prediction using the ensemble of ELM with gradient boosting.

Differentiation of Pelvic Osteosarcoma and Ewing Sarcoma Using Radiomic Analysis Based on T2-Weighted Images and Contrast-Enhanced T1-Weighted Images

Objective

To determine if osteosarcoma (OS) and Ewing sarcoma (EWS) of the pelvis based on MRI can be differentiated using radiomic analysis.

Materials and Methods

In this study, 3.0 T magnetic resonance (MR) data of 66 patients (40 males and 26 females, mean age 27.6 ± 13.9 years) with pathologically confirmed OS or EWS of the pelvis (35 with OS and 31 with EWS) taken from April 2013 to December 2017 were retrospectively reviewed. T2-weighted fat-saturated (T2-FS) and contrast-enhanced T1-weighted (CET1) images were manually segmented, and imaging features were extracted. Independent-sample t-test, Spearman's test, and the least absolute shrinkage and selection operator (LASSO) method were used to select the most useful features from the original data set. The performance of radiomic analysis was investigated by the area under the receiver operating characteristic (ROC) curve (AUC) analysis.

Results

385 initial features were extracted from T2-FS and CET1 MR data. Nine features from T2-FS and 7 features from CET1 were selected by using the LASSO method. The radiomic analysis to differentiate OS and EWS of the pelvis based on T2-FS and CET1 images using the aforementioned selected features achieved AUC values of 0.881 (95% confidence interval (CI): 0.799–0.963) and 0.765 (95% CI: 0.652–0.878), respectively.

Conclusion

Radiomic analysis showed potential in differentiating OS from EWS of the pelvis, in which T2-FS demonstrated better diagnostic value. To differentiate OS from EWS of the pelvis using our multiparametric MRI-based radiomic analysis could preoperatively improve diagnostic accuracy and greatly contribute to therapy planning.

Precision Medicine and the Role of Biomarkers of Radiotherapy Response in Breast Cancer

Radiotherapy remains an important treatment modality in nearly two thirds of all cancers, including the primary curative or palliative treatment of breast cancer. Unfortunately, largely due to tumor heterogeneity, tumor radiotherapy response rates can vary significantly, even between patients diagnosed with the same tumor type. Although in recent years significant technological advances have been made in the way radiation can be precisely delivered to tumors, it is proving more difficult to personalize radiotherapy regimens based on cancer biology. Biomarkers that provide prognostic or predictive information regarding a tumor's intrinsic radiosensitivity or its response to treatment could prove valuable in helping to personalize radiation dosing, enabling clinicians to make decisions between different treatment options whilst avoiding radiation-induced toxicity in patients unlikely to gain therapeutic benefit. Studies have investigated numerous ways in which both patient and tumor radiosensitivities can be assessed. Tumor molecular profiling has been used to develop radiosensitivity gene signatures, while the assessment of specific intracellular or secreted proteins, including circulating tumor cells, exosomes and DNA, has been performed to identify prognostic or predictive biomarkers of radiation response. Finally, the investigation of biomarkers related to radiation-induced toxicity could provide another means by which radiotherapy could become personalized. In this review, we discuss studies that have used these methods to identify or develop prognostic/predictive signatures of radiosensitivity, and how such assays could be used in the future as a means of providing personalized radiotherapy.

Combining Dynamic Contrast-Enhanced Magnetic Resonance Imaging and Apparent Diffusion Coefficient Maps for a Radiomics Nomogram to Predict Pathological Complete Response to Neoadjuvant Chemotherapy in Breast Cancer Patients

OBJECTIVE

The objective of this study was to develop a nomogrom for prediction of pathological complete response (PCR) to neoadjuvant chemotherapy in breast cancer patients.

METHODS

Ninety-one patients were analyzed. A total of 396 radiomics features were extracted from dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) and apparent diffusion coefficient (ADC) maps. The least absolute shrinkage and selection operator was selected for data dimension reduction to build a radiomics signature. Finally, the nomogram was built to predict PCR.

RESULTS

The radiomics signature of the model that combined DCE-MRI and ADC maps showed a higher performance (area under the receiver operating characteristic curve [AUC], 0.848) than the models with DCE-MRI (AUC, 0.750) or ADC maps (AUC, 0.785) alone in the training set. The proposed model, which included combined radiomics signature, estrogen receptor, and progesterone receptor, yielded a maximum AUC of 0.837 in the testing set.

CONCLUSIONS

The combined radiomics features from DCE-MRI and ADC data may serve as potential predictor markers for predicting PCR. The nomogram could be used as a quantitative tool to predict PCR.

Radiomics: from qualitative to quantitative imaging

Historically, medical imaging has been a qualitative or semi-quantitative modality. It is difficult to quantify what can be seen in an image, and to turn it into valuable predictive outcomes. As a result of advances in both computational hardware and machine learning algorithms, computers are making great strides in obtaining quantitative information from imaging and correlating it with outcomes. Radiomics, in its two forms "handcrafted and deep," is an emerging field that translates medical images into quantitative data to yield biological information and enable radiologic phenotypic profiling for diagnosis, theragnosis, decision support, and monitoring. Handcrafted radiomics is a multistage process in which features based on shape, pixel intensities, and texture are extracted from radiographs. Within this review, we describe the steps: starting with quantitative imaging data, how it can be extracted, how to correlate it with clinical and biological outcomes, resulting in models that can be used to make predictions, such as survival, or for detection and classification used in diagnostics. The application of deep learning, the second arm of radiomics, and its place in the radiomics workflow is discussed, along with its advantages and disadvantages. To better illustrate the technologies being used, we provide real-world clinical applications of radiomics in oncology, showcasing research on the applications of radiomics, as well as covering its limitations and its future direction.