Precision diagnostics based on machine learning-derived imaging signatures

Applications of Machine Learning (ML) and Mathematical Modeling (MM) in Healthcare with Special Focus on Cancer Prognosis and Anticancer Therapy: Current Status and Challenges

The use of data-driven high-throughput analytical techniques, which has given rise to computational oncology, is undisputed. The widespread use of machine learning (ML) and mathematical modeling (MM)-based techniques is widely acknowledged. These two approaches have fueled the advancement in cancer research and eventually led to the uptake of telemedicine in cancer care. For diagnostic, prognostic, and treatment purposes concerning different types of cancer research, vast databases of varied information with manifold dimensions are required, and indeed, all this information can only be managed by an automated system developed utilizing ML and MM. In addition, MM is being used to probe the relationship between the pharmacokinetics and pharmacodynamics (PK/PD interactions) of anti-cancer substances to improve cancer treatment, and also to refine the quality of existing treatment models by being incorporated at all steps of research and development related to cancer and in routine patient care. This review will serve as a consolidation of the advancement and benefits of ML and MM techniques with a special focus on the area of cancer prognosis and anticancer therapy, leading to the identification of challenges (data quantity, ethical consideration, and data privacy) which are yet to be fully addressed in current studies.

Gaussian Process-based prediction of memory performance and biomarker status in ageing and Alzheimer's disease-A systematic model evaluation

Neuroimaging markers based on Magnetic Resonance Imaging (MRI) combined with various other measures (such as genetic covariates, biomarkers, vascular risk factors, neuropsychological tests etc.) might provide useful predictions of clinical outcomes during the progression towards Alzheimer's disease (AD). The use of multiple features in predictive frameworks for clinical outcomes has become increasingly prevalent in AD research. However, many studies do not focus on systematically and accurately evaluating combinations of multiple input features. Hence, the aim of the present work is to explore and assess optimal combinations of various features for MR-based prediction of (1) cognitive status and (2) biomarker positivity with a multi-kernel learning Gaussian process framework. The explored features and parameters included (A) combinations of brain tissues, modulation, smoothing, and image resolution; (B) incorporating demographics & clinical covariates; (C) the impact of the size of the training data set; (D) the influence of dimensionality reduction and the choice of kernel types. The approach was tested in a large German cohort including 959 subjects from the multicentric longitudinal study of cognitive impairment and dementia (DELCODE). Our evaluation suggests the best prediction of memory performance was obtained for a combination of neuroimaging markers, demographics, genetic information (ApoE4) and CSF biomarkers explaining 57% of outcome variance in out-of-sample predictions. The highest performance for Aβ42/40 status classification was achieved for a combination of demographics, ApoE4, and a memory score while usage of structural MRI further improved the classification of individual patient's pTau status.

Neuroimaging-AI Endophenotypes of Brain Diseases in the General Population: Towards a Dimensional System of Vulnerability

Disease heterogeneity poses a significant challenge for precision diagnostics in both clinical and sub-clinical stages. Recent work leveraging artificial intelligence (AI) has offered promise to dissect this heterogeneity by identifying complex intermediate phenotypes - herein called dimensional neuroimaging endophenotypes (DNEs) - which subtype various neurologic and neuropsychiatric diseases. We investigate the presence of nine such DNEs derived from independent yet harmonized studies on Alzheimer's disease (AD1-2)1, autism spectrum disorder (ASD1-3)2, late-life depression (LLD1-2)3, and schizophrenia (SCZ1-2)4, in the general population of 39,178 participants in the UK Biobank study. Phenome-wide associations revealed prominent associations between the nine DNEs and phenotypes related to the brain and other human organ systems. This phenotypic landscape aligns with the SNP-phenotype genome-wide associations, revealing 31 genomic loci associated with the nine DNEs (Bonferroni corrected P-value < 5×10-8/9). The DNEs exhibited significant genetic correlations, colocalization, and causal relationships with multiple human organ systems and chronic diseases. A causal effect (odds ratio=1.25 [1.11, 1.40], P-value=8.72×1-4) was established from AD2, characterized by focal medial temporal lobe atrophy, to AD. The nine DNEs and their polygenic risk scores significantly improved the prediction accuracy for 14 systemic disease categories and mortality. These findings underscore the potential of the nine DNEs to identify individuals at a high risk of developing the four brain diseases during preclinical stages for precision diagnostics. All results are publicly available at: http://labs.loni.usc.edu/medicine/.

Patterns of Regional Brain Atrophy and Brain Aging in Middle- and Older-Aged Adults With Type 1 Diabetes

Importance

Little is known about structural brain changes in type 1 diabetes (T1D) and whether there are early manifestations of a neurodegenerative condition like Alzheimer disease (AD) or evidence of premature brain aging.

Objective

To evaluate neuroimaging markers of brain age and AD-like atrophy in participants with T1D in the Diabetes Control and Complications Trial (DCCT)/Epidemiology of Diabetes Interventions and Complications (EDIC) study, identify which brain regions are associated with the greatest changes in patients with T1D, and assess the association between cognition and brain aging indices.

Design, Setting, and Participants

This cohort study leveraged data collected during the combined DCCT (randomized clinical trial, 1983-1993) and EDIC (observational study, 1994 to present) studies at 27 clinical centers in the US and Canada. A total of 416 eligible EDIC participants and 99 demographically similar adults without diabetes were enrolled in the magnetic resonance imaging (MRI) ancillary study, which reports cross-sectional data collected in 2018 to 2019 and relates it to factors measured longitudinally in DCCT/EDIC. Data analyses were performed between July 2020 and April 2022.

Exposure

T1D diagnosis.

Main Outcomes and Measures

Psychomotor and mental efficiency were evaluated using verbal fluency, digit symbol substitution test, trail making part B, and the grooved pegboard. Immediate memory scores were derived from the logical memory subtest of the Wechsler memory scale and the Wechsler digit symbol substitution test. MRI and machine learning indices were calculated to predict brain age and quantify AD-like atrophy.

Results

This study included 416 EDIC participants with a median (range) age of 60 (44-74) years (87 of 416 [21%] were older than 65 years) and a median (range) diabetes duration of 37 (30-51) years. EDIC participants had consistently higher brain age values compared with controls without diabetes, indicative of approximately 6 additional years of brain aging (EDIC participants: β, 6.16; SE, 0.71; control participants: β, 1.04; SE, 0.04; P < .001). In contrast, AD regional atrophy was comparable between the 2 groups. Regions with atrophy in EDIC participants vs controls were observed mainly in the bilateral thalamus and putamen. Greater brain age was associated with lower psychomotor and mental efficiency among EDIC participants (β, -0.04; SE, 0.01; P < .001), but not among controls.

Conclusions and Relevance

The findings of this study suggest an increase in brain aging among individuals with T1D without any early signs of AD-related neurodegeneration. These increases were associated with reduced cognitive performance, but overall, the abnormal patterns seen in this sample were modest, even after a mean of 38 years with T1D.

Primary bone tumor detection and classification in full-field bone radiographs via YOLO deep learning model

OBJECTIVES

Automatic bone lesions detection and classifications present a critical challenge and are essential to support radiologists in making an accurate diagnosis of bone lesions. In this paper, we aimed to develop a novel deep learning model called You Only Look Once (YOLO) to handle detecting and classifying bone lesions on full-field radiographs with limited manual intervention.

METHODS

In this retrospective study, we used 1085 bone tumor radiographs and 345 normal bone radiographs from two centers between January 2009 and December 2020 to train and test our YOLO deep learning (DL) model. The trained model detected bone lesions and then classified these radiographs into normal, benign, intermediate, or malignant types. The intersection over union (IoU) was used to assess the model's performance in the detection task. Confusion matrices and Cohen's kappa scores were used for evaluating classification performance. Two radiologists compared diagnostic performance with the trained model using the external validation set.

RESULTS

In the detection task, the model achieved accuracies of 86.36% and 85.37% in the internal and external validation sets, respectively. In the DL model, radiologist 1 and radiologist 2 achieved Cohen's kappa scores of 0.8187, 0.7927, and 0.9077 for four-way classification in the external validation set, respectively. The YOLO DL model illustrated a significantly higher accuracy for intermediate bone tumor classification than radiologist 1 (95.73% vs 88.08%, p = 0.004).

CONCLUSIONS

The developed YOLO DL model could be used to assist radiologists at all stages of bone lesion detection and classification in full-field bone radiographs.

KEY POINTS

• YOLO DL model can automatically detect bone neoplasms from full-field radiographs in one shot and then simultaneously classify radiographs into normal, benign, intermediate, or malignant. • The dataset used in this retrospective study includes normal bone radiographs. • YOLO can detect even some challenging cases with small volumes.

Site effects how-to and when: An overview of retrospective techniques to accommodate site effects in multi-site neuroimaging analyses

Site differences, or systematic differences in feature distributions across multiple data-acquisition sites, are a known source of heterogeneity that may adversely affect large-scale meta- and mega-analyses of independently collected neuroimaging data. They influence nearly all multi-site imaging modalities and biomarkers, and methods to compensate for them can improve reliability and generalizability in the analysis of genetics, omics, and clinical data. The origins of statistical site effects are complex and involve both technical differences (scanner vendor, head coil, acquisition parameters, imaging processing) and differences in sample characteristics (inclusion/exclusion criteria, sample size, ancestry) between sites. In an age of expanding international consortium research, there is a growing need to disentangle technical site effects from sample characteristics of interest. Numerous statistical and machine learning methods have been developed to control for, model, or attenuate site effects - yet to date, no comprehensive review has discussed the benefits and drawbacks of each for different use cases. Here, we provide an overview of the different existing statistical and machine learning methods developed to remove unwanted site effects from independently collected neuroimaging samples. We focus on linear mixed effect models, the ComBat technique and its variants, adjustments based on image quality metrics, normative modeling, and deep learning approaches such as generative adversarial networks. For each method, we outline the statistical foundation and summarize strengths and weaknesses, including their assumptions and conditions of use. We provide information on software availability and comment on the ease of use and the applicability of these methods to different types of data. We discuss validation and comparative reports, mention caveats and provide guidance on when to use each method, depending on context and specific research questions.

Artificial Intelligence-Driven Diagnosis of Pancreatic Cancer

Pancreatic cancer is among the most challenging forms of cancer to treat, owing to its late diagnosis and aggressive nature that reduces the survival rate drastically. Pancreatic cancer diagnosis has been primarily based on imaging, but the current state-of-the-art imaging provides a poor prognosis, thus limiting clinicians' treatment options. The advancement of a cancer diagnosis has been enhanced through the integration of artificial intelligence and imaging modalities to make better clinical decisions. In this review, we examine how AI models can improve the diagnosis of pancreatic cancer using different imaging modalities along with a discussion on the emerging trends in an AI-driven diagnosis, based on cytopathology and serological markers. Ethical concerns regarding the use of these tools have also been discussed.

The University of Pennsylvania glioblastoma (UPenn-GBM) cohort: advanced MRI, clinical, genomics, & radiomics

Glioblastoma is the most common aggressive adult brain tumor. Numerous studies have reported results from either private institutional data or publicly available datasets. However, current public datasets are limited in terms of: a) number of subjects, b) lack of consistent acquisition protocol, c) data quality, or d) accompanying clinical, demographic, and molecular information. Toward alleviating these limitations, we contribute the "University of Pennsylvania Glioblastoma Imaging, Genomics, and Radiomics" (UPenn-GBM) dataset, which describes the currently largest publicly available comprehensive collection of 630 patients diagnosed with de novo glioblastoma. The UPenn-GBM dataset includes (a) advanced multi-parametric magnetic resonance imaging scans acquired during routine clinical practice, at the University of Pennsylvania Health System, (b) accompanying clinical, demographic, and molecular information, (d) perfusion and diffusion derivative volumes, (e) computationally-derived and manually-revised expert annotations of tumor sub-regions, as well as (f) quantitative imaging (also known as radiomic) features corresponding to each of these regions. This collection describes our contribution towards repeatable, reproducible, and comparative quantitative studies leading to new predictive, prognostic, and diagnostic assessments.

Radiomics and radiogenomics in pediatric neuro-oncology: A review

The current era of advanced computing has allowed for the development and implementation of the field of radiomics. In pediatric neuro-oncology, radiomics has been applied in determination of tumor histology, identification of disseminated disease, prognostication, and molecular classification of tumors (ie, radiogenomics). The field also comes with many challenges, such as limitations in study sample sizes, class imbalance, generalizability of the methods, and data harmonization across imaging centers. The aim of this review paper is twofold: first, to summarize existing literature in radiomics of pediatric neuro-oncology; second, to distill the themes and challenges of the field and discuss future directions in both a clinical and technical context.

Applications of Radiomics and Radiogenomics in High-Grade Gliomas in the Era of Precision Medicine

Simple Summary

Radiomics and radiogenomics offer new insight into high-grade glioma biology, as well as into glioma behavior in response to standard therapies. In this article, we provide neuro-oncology, neuropathology, and computational perspectives on the role of radiomics in providing more accurate diagnoses, prognostication, and surveillance of patients with high-grade glioma, and on the potential application of radiomics in clinical practice, with the overarching goal of advancing precision medicine for optimal patient care.

Abstract

Machine learning (ML) integrated with medical imaging has introduced new perspectives in precision diagnostics of high-grade gliomas, through radiomics and radiogenomics. This has raised hopes for characterizing noninvasive and in vivo biomarkers for prediction of patient survival, tumor recurrence, and genomics and therefore encouraging treatments tailored to individualized needs. Characterization of tumor infiltration based on pre-operative multi-parametric magnetic resonance imaging (MP-MRI) scans may allow prediction of the loci of future tumor recurrence and thereby aid in planning the course of treatment for the patients, such as optimizing the extent of resection and the dose and target area of radiation. Imaging signatures of tumor genomics can help in identifying the patients who benefit from certain targeted therapies. Specifying molecular properties of gliomas and prediction of their changes over time and with treatment would allow optimization of treatment. In this article, we provide neuro-oncology, neuropathology, and computational perspectives on the promise of radiomics and radiogenomics for allowing personalized treatments of patients with gliomas and discuss the challenges and limitations of these methods in multi-institutional clinical trials and suggestions to mitigate the issues and the future directions.

MRI-based machine learning for determining quantitative and qualitative characteristics affecting the survival of glioblastoma multiforme

PURPOSE

Our current study aims to consider the image biomarkers extracted from the MRI images for exploring their effects on glioblastoma multiforme (GBM) patients' survival. Determining its biomarker helps better manage the disease and evaluate treatments. It has been proven that imaging features could be used as a biomarker. The purpose of this study is to investigate the features in MRI and clinical features as the biomarker association of survival of GBM.

METHODS

55 patients were considered with five clinical features, 10 qualities pre-operative MRI image features, and six quantitative features obtained using BraTumIA software. It was run ANN, C5, Bayesian, and Cox models in two phases for determining important variables. In the first phase, we selected the quality features that occur at least in three models and quantitative in two models. In the second phase, models were run with the extracted features, and then the probability value of variables in each model was calculated.

RESULTS

The mean of accuracy, sensitivity, specificity, and area under curve (AUC) after running four machine learning techniques were 80.47, 82.54, 79.78, and 0.85, respectively. In the second step, the mean of accuracy, sensitivity, specificity, and AUC were 79.55, 78.71, 79.83, and 0.87, respectively.

CONCLUSION

We found the largest size of the width, the largest size of length, radiotherapy, volume of enhancement, volume of nCET, satellites, enhancing margin, and age feature are important features.

Machine Learning Using Multiparametric Magnetic Resonance Imaging Radiomic Feature Analysis to Predict Ki-67 in World Health Organization Grade I Meningiomas

BACKGROUND

Although World Health Organization (WHO) grade I meningiomas are considered "benign" tumors, an elevated Ki-67 is one crucial factor that has been shown to influence tumor behavior and clinical outcomes. The ability to preoperatively discern Ki-67 would confer the ability to guide surgical strategy.

OBJECTIVE

In this study, we develop a machine learning (ML) algorithm using radiomic feature analysis to predict Ki-67 in WHO grade I meningiomas.

METHODS

A retrospective analysis was performed for a cohort of 306 patients who underwent surgical resection of WHO grade I meningiomas. Preoperative magnetic resonance imaging was used to perform radiomic feature extraction followed by ML modeling using least absolute shrinkage and selection operator wrapped with support vector machine through nested cross-validation on a discovery cohort (n = 230), to stratify tumors based on Ki-67 <5% and ≥5%. The final model was independently tested on a replication cohort (n = 76).

RESULTS

An area under the receiver operating curve (AUC) of 0.84 (95% CI: 0.78-0.90) with a sensitivity of 84.1% and specificity of 73.3% was achieved in the discovery cohort. When this model was applied to the replication cohort, a similar high performance was achieved, with an AUC of 0.83 (95% CI: 0.73-0.94), sensitivity and specificity of 82.6% and 85.5%, respectively. The model demonstrated similar efficacy when applied to skull base and nonskull base tumors.

CONCLUSION

Our proposed radiomic feature analysis can be used to stratify WHO grade I meningiomas based on Ki-67 with excellent accuracy and can be applied to skull base and nonskull base tumors with similar performance achieved.

Data-driven approaches to neuroimaging biomarkers for neurological and psychiatric disorders: emerging approaches and examples

PURPOSE OF REVIEW

The 'holy grail' of clinical applications of neuroimaging to neurological and psychiatric disorders via personalized biomarkers has remained mostly elusive, despite considerable effort. However, there are many reasons to continue to be hopeful, as the field has made remarkable advances over the past few years, fueled by a variety of converging technical and data developments.

RECENT FINDINGS

We discuss a number of advances that are accelerating the push for neuroimaging biomarkers including the advent of the 'neuroscience big data' era, biomarker data competitions, the development of more sophisticated algorithms including 'guided' data-driven approaches that facilitate automation of network-based analyses, dynamic connectivity, and deep learning. Another key advance includes multimodal data fusion approaches which can provide convergent and complementary evidence pointing to possible mechanisms as well as increase predictive accuracy.

SUMMARY

The search for clinically relevant neuroimaging biomarkers for neurological and psychiatric disorders is rapidly accelerating. Here, we highlight some of these aspects, provide recent examples from studies in our group, and link to other ongoing work in the field. It is critical that access and use of these advanced approaches becomes mainstream, this will help propel the community forward and facilitate the production of robust and replicable neuroimaging biomarkers.

Multiple Subtypes of Alzheimer's Disease Base on Brain Atrophy Pattern

Alzheimer’s disease (AD) is a disease of a heterogeneous nature, which can be disentangled by exploring the characteristics of each AD subtype in the brain structure, neuropathology, and cognition. In this study, a total of 192 AD and 228 cognitively normal (CN) subjects were obtained from the Alzheimer’s disease Neuroimaging Initiative database. Based on the cortical thickness patterns, the mixture of experts method (MOE) was applied to the implicit model spectrum of transforms lined with each AD subtype, then their neuropsychological and neuropathological characteristics were analyzed. Furthermore, the piecewise linear classifiers composed of each AD subtype and CN were resolved, and each subtype was comprehensively explained. The following four distinct AD subtypes were discovered: bilateral parietal, frontal, and temporal atrophy AD subtype (occipital sparing AD subtype (OSAD), 29.2%), left temporal dominant atrophy AD subtype (LTAD, 22.4%), minimal atrophy AD subtype (MAD, 16.1%), and diffuse atrophy AD subtype (DAD, 32.3%). These four subtypes display their own characteristics in atrophy pattern, cognition, and neuropathology. Compared with the previous studies, our study found that some AD subjects showed obvious asymmetrical atrophy in left lateral temporal-parietal cortex, OSAD presented the worst cerebrospinal fluid levels, and MAD had the highest proportions of APOE ε4 and APOE ε2. The subtype characteristics were further revealed from the aspect of the model, making it easier for clinicians to understand. The results offer an effective support for individual diagnosis and prognosis.

Integrated Biophysical Modeling and Image Analysis: Application to Neuro-Oncology

Central nervous system (CNS) tumors come with vastly heterogeneous histologic, molecular, and radiographic landscapes, rendering their precise characterization challenging. The rapidly growing fields of biophysical modeling and radiomics have shown promise in better characterizing the molecular, spatial, and temporal heterogeneity of tumors. Integrative analysis of CNS tumors, including clinically acquired multi-parametric magnetic resonance imaging (mpMRI) and the inverse problem of calibrating biophysical models to mpMRI data, assists in identifying macroscopic quantifiable tumor patterns of invasion and proliferation, potentially leading to improved (a) detection/segmentation of tumor subregions and (b) computer-aided diagnostic/prognostic/predictive modeling. This article presents a summary of (a) biophysical growth modeling and simulation,(b) inverse problems for model calibration, (c) these models' integration with imaging workflows, and (d) their application to clinically relevant studies. We anticipate that such quantitative integrative analysis may even be beneficial in a future revision of the World Health Organization (WHO) classification for CNS tumors, ultimately improving patient survival prospects.

Characterizing white matter connectivity in Alzheimer's disease and mild cognitive impairment: An automated fiber quantification analysis with two independent datasets

Alzheimer's disease (AD) is a chronic neurodegenerative disease characterized by progressive dementia. Diffusion tensor imaging (DTI) has been widely used to show structural integrity and delineate white matter (WM) degeneration in AD. The automated fiber quantification (AFQ) method is a fully automated approach that can rapidly and reliably identify major WM fiber tracts and evaluate WM properties. The main aim of this study was to assess WM integrity and abnormities in a cohort of patients with amnestic mild cognitive impairment (aMCI) and AD as well as normal controls (NCs). For this purpose, we first used AFQ to identify 20 major WM tracts and assessed WM integrity and abnormalities in a cohort of 120 subjects (39 NCs, 34 aMCI patients and 47 AD patients) in a discovery dataset and 122 subjects (43 NCs, 37 aMCI patients and 42 AD patients) in a replicated dataset. Pointwise differences along WM tracts were identified in the discovery dataset and simultaneously confirmed in the replicated dataset. Next, we investigated the utility of DTI measures along WM tracts as features to distinguish patients with AD from NCs via multilevel cross validation using a support vector machine. Correlation analysis revealed the identified microstructural WM alterations and classification output to be highly associated with cognitive ability in the patient groups, suggesting that they may be a robust biomarker of AD. This systematic study provides a pipeline to examine WM integrity and its potential clinical application in AD and may be useful for studying other neurological and psychiatric disorders.

Robust Collaborative Clustering of Subjects and Radiomic Features for Cancer Prognosis

Feature dimensionality reduction plays an important role in radiomic studies with a large number of features. However, conventional radiomic approaches may suffer from noise, and feature dimensionality reduction techniques are not equipped to utilize latent supervision information of patient data under study, such as differences in patients, to learn discriminative low dimensional representations. To achieve robustness to noise and feature dimensionality reduction with improved discriminative power, we develop a robust collaborative clustering method to simultaneously cluster patients and radiomic features into distinct groups respectively under adaptive sparse regularization. Our method is built upon matrix tri-factorization enhanced by adaptive sparsity regularization for simultaneous feature dimensionality reduction and denoising. Particularly, latent grouping information of patients with distinct radiomic features is learned and utilized as supervision information to guide the feature dimensionality reduction, and noise in radiomic features is adaptively isolated in a Bayesian framework under a general assumption of Laplacian distributions of transform-domain coefficients. Experiments on synthetic data have demonstrated the effectiveness of the proposed approach in data clustering, and evaluation results on an FDG-PET/CT dataset of rectal cancer patients have demonstrated that the proposed method outperforms alternative methods in terms of both patient stratification and prediction of patient clinical outcomes.

Multi-Disease Segmentation of Gliomas and White Matter Hyperintensities in the BraTS Data Using a 3D Convolutional Neural Network

An important challenge in segmenting real-world biomedical imaging data is the presence of multiple disease processes within individual subjects. Most adults above age 60 exhibit a variable degree of small vessel ischemic disease, as well as chronic infarcts, which will manifest as white matter hyperintensities (WMH) on brain MRIs. Subjects diagnosed with gliomas will also typically exhibit some degree of abnormal T2 signal due to WMH, rather than just due to tumor. We sought to develop a fully automated algorithm to distinguish and quantify these distinct disease processes within individual subjects’ brain MRIs. To address this multi-disease problem, we trained a 3D U-Net to distinguish between abnormal signal arising from tumors vs. WMH in the 3D multi-parametric MRI (mpMRI, i.e., native T1-weighted, T1-post-contrast, T2, T2-FLAIR) scans of the International Brain Tumor Segmentation (BraTS) 2018 dataset (n_training = 285, n_validation = 66). Our trained neuroradiologist manually annotated WMH on the BraTS training subjects, finding that 69% of subjects had WMH. Our 3D U-Net model had a 4-channel 3D input patch (80 × 80 × 80) from mpMRI, four encoding and decoding layers, and an output of either four [background, active tumor (AT), necrotic core (NCR), peritumoral edematous/infiltrated tissue (ED)] or five classes (adding WMH as the fifth class). For both the four- and five-class output models, the median Dice for whole tumor (WT) extent (i.e., union of AT, ED, NCR) was 0.92 in both training and validation sets. Notably, the five-class model achieved significantly (p = 0.002) lower/better Hausdorff distances for WT extent in the training subjects. There was strong positive correlation between manually segmented and predicted volumes for WT (r = 0.96) and WMH (r = 0.89). Larger lesion volumes were positively correlated with higher/better Dice scores for WT (r = 0.33), WMH (r = 0.34), and across all lesions (r = 0.89) on a log(10) transformed scale. While the median Dice for WMH was 0.42 across training subjects with WMH, the median Dice was 0.62 for those with at least 5 cm³ of WMH. We anticipate the development of computational algorithms that are able to model multiple diseases within a single subject will be a critical step toward translating and integrating artificial intelligence systems into the heterogeneous real-world clinical workflow.

Artificial intelligence in medical imaging

The medical specialty radiology has experienced a number of extremely important and influential technical developments in the past that have affected how medical imaging is deployed. Artificial intelligence (AI) is potentially another such development that will introduce fundamental changes into the practice of radiology. In this commentary the historical evolution of some major changes in radiology are traced as background to how AI may also be embraced into practice. Potential new capabilities provided by AI offer exciting prospects for more efficient and effective use of medical images.

Discriminating schizophrenia using recurrent neural network applied on time courses of multi-site FMRI data

BACKGROUND

Current fMRI-based classification approaches mostly use functional connectivity or spatial maps as input, instead of exploring the dynamic time courses directly, which does not leverage the full temporal information.

METHODS

Motivated by the ability of recurrent neural networks (RNN) in capturing dynamic information of time sequences, we propose a multi-scale RNN model, which enables classification between 558 schizophrenia and 542 healthy controls by using time courses of fMRI independent components (ICs) directly. To increase interpretability, we also propose a leave-one-IC-out looping strategy for estimating the top contributing ICs.

FINDINGS

Accuracies of 83·2% and 80·2% were obtained respectively for the multi-site pooling and leave-one-site-out transfer classification. Subsequently, dorsal striatum and cerebellum components contribute the top two group-discriminative time courses, which is true even when adopting different brain atlases to extract time series.

INTERPRETATION

This is the first attempt to apply a multi-scale RNN model directly on fMRI time courses for classification of mental disorders, and shows the potential for multi-scale RNN-based neuroimaging classifications. FUND: Natural Science Foundation of China, the Strategic Priority Research Program of the Chinese Academy of Sciences, National Institutes of Health Grants, National Science Foundation.