Transfer learning improves resting-state functional connectivity pattern analysis using convolutional neural networks

Practical measurements distinguishing physiological and pathological stereoelectroencephalography channels based on high-frequency oscillations in the human brain

OBJECTIVE

The present study aimed to identify various distinguishing features for use in the accurate classification of stereoelectroencephalography (SEEG) channels based on high-frequency oscillations (HFOs) inside and outside the epileptogenic zone (EZ).

METHODS

HFOs were detected in patients with focal epilepsy who underwent SEEG. Subsequently, HFOs within the seizure-onset and early spread zones were defined as pathological HFOs, whereas others were defined as physiological. Three features of HFOs were identified at the channel level, namely, morphological repetition, rhythmicity, and phase-amplitude coupling (PAC). A machine-learning (ML) classifier was then built to distinguish two HFO types at the channel level by application of the above-mentioned features, and the contributions were quantified. Further verification of the characteristics and classifier performance was performed in relation to various conscious states, imaging results, EZ location, and surgical outcomes.

RESULTS

Thirty-five patients were included in this study, from whom 166  104 pathological HFOs in 255 channels and 53 374 physiological HFOs in 282 channels were entered into the analysis pipeline. The results revealed that the morphological repetitions of pathological HFOs were markedly higher than those of the physiological HFOs; this was also observed for rhythmicity and PAC. The classifier exhibited high accuracy in differentiating between the two forms of HFOs, as indicated by an area under the curve (AUC) of 0.89. Both PAC and rhythmicity contributed significantly to this distinction. The subgroup analyses supported these findings.

SIGNIFICANCE

The suggested HFO features can accurately distinguish between pathological and physiological channels substantially improving its usefulness in clinical localization.

PLAIN LANGUAGE SUMMARY

In this study, we computed three quantitative features associated with HFOs in each SEEG channel and then constructed a machine learning-based classifier for the classification of pathological and physiological channels. The classifier performed well in distinguishing the two channel types under different levels of consciousness as well as in terms of imaging results, EZ location, and patient surgical outcomes.

Feature attention graph neural network for estimating brain age and identifying important neural connections in mouse models of genetic risk for Alzheimer's disease

Alzheimer's disease (AD) remains one of the most extensively researched neurodegenerative disorders due to its widespread prevalence and complex risk factors. Age is a crucial risk factor for AD, which can be estimated by the disparity between physiological age and estimated brain age. To model AD risk more effectively, integrating biological, genetic, and cognitive markers is essential. Here, we utilized mouse models expressing the major APOE human alleles and human nitric oxide synthase 2 to replicate genetic risk for AD and a humanized innate immune response. We estimated brain age employing a multivariate dataset that includes brain connectomes, APOE genotype, subject traits such as age and sex, and behavioral data. Our methodology used Feature Attention Graph Neural Networks (FAGNN) for integrating different data types. Behavioral data were processed with a 2D Convolutional Neural Network (CNN), subject traits with a 1D CNN, brain connectomes through a Graph Neural Network using quadrant attention module. The model yielded a mean absolute error for age prediction of 31.85 days, with a root mean squared error of 41.84 days, outperforming other, reduced models. In addition, FAGNN identified key brain connections involved in the aging process. The highest weights were assigned to the connections between cingulum and corpus callosum, striatum, hippocampus, thalamus, hypothalamus, cerebellum, and piriform cortex. Our study demonstrates the feasibility of predicting brain age in models of aging and genetic risk for AD. To verify the validity of our findings, we compared Fractional Anisotropy (FA) along the tracts of regions with the highest connectivity, the Return-to-Origin Probability (RTOP), Return-to-Plane Probability (RTPP), and Return-to-Axis Probability (RTAP), which showed significant differences between young, middle-aged, and old age groups. Younger mice exhibited higher FA, RTOP, RTAP, and RTPP compared to older groups in the selected connections, suggesting that degradation of white matter tracts plays a critical role in aging and for FAGNN's selections. Our analysis suggests a potential neuroprotective role of APOE2, relative to APOE3 and APOE4, where APOE2 appears to mitigate age-related changes. Our findings highlighted a complex interplay of genetics and brain aging in the context of AD risk modeling.

Machine learning-based classification of physiological and pathological high-frequency oscillations recorded by stereoelectroencephalography

OBJECTIVE

High-frequency oscillations (HFOs) are an efficient indicator to locate the epileptogenic zone (EZ). However, physiological HFOs produced in the normal brain region may interfere with EZ localization. The present study aimed to build a machine learning-based classifier to distinguish the properties of each HFO event based on features in different domains.

METHODS

HFOs were detected in focal epilepsy patients from two different hospitals who underwent stereoelectroencephalography and subsequent resection surgery. Subsequently, 37 features in four different domains (time, frequency and time-frequency, entropy-based and nonlinear) were extracted for each HFO. After extraction, a fast correlation-based filter (FCBF) algorithm was applied for feature selection. The machine learning classifier was trained on the feature matrix with and without FCBF and then tested on the data set from patients in another hospital.

RESULTS

A dataset was compiled, consisting of 89,844 pathological HFOs and 23,613 physiological HFOs from 17 patients assigned to the training dataset. Additionally, 12,695 pathological HFOs and 5,599 physiological HFOs from 9 patients were assigned to the testing dataset. Four features (ripple band power, arithmetic mean, Petrosian fractal dimension and zero crossings) were obtained for classifier training after FCBF. The classifier showed an area under the curve (AUC) of 0.95/0.98 for FCBF/no FCBF features in the training dataset and AUC of 0.82/0.90 for FCBF/no FCBF features in the testing dataset. Our findings indicated that the classifier utilizing all features demonstrated superior performance compared to the one relying on FCBF-processed features.

CONCLUSION

Our classifier could reliably differentiate pathological HFOs from physiological ones, which could promote the development of HFOs in EZ localization.

Superior temporal gyrus functional connectivity predicts transcranial direct current stimulation response in Schizophrenia: A machine learning study

Transcranial direct current stimulation (tDCS) is a promising adjuvant treatment for persistent auditory verbal hallucinations (AVH) in Schizophrenia (SZ). Nonetheless, there is considerable inter-patient variability in the treatment response of AVH to tDCS in SZ. Machine-learned models have the potential to predict clinical response to tDCS in SZ. This study aims to examine the feasibility of identifying SZ patients with persistent AVH (SZ-AVH) who will respond to tDCS based on resting-state functional connectivity (rs-FC). Thirty-four SZ-AVH patients underwent resting-state functional MRI at baseline followed by add-on, twice-daily, 20-min sessions with tDCS (conventional/high-definition) for 5 days. A machine learning model was developed to identify tDCS treatment responders based on the rs-FC pattern, using the left superior temporal gyrus (LSTG) as the seed region. Functional connectivity between LSTG and brain regions involved in auditory and sensorimotor processing emerged as the important predictors of the tDCS treatment response. L1-regularized logistic regression model had an overall accuracy of 72.5% in classifying responders vs. non-responders. This model outperformed the state-of-the-art convolutional neural networks (CNN) model-both without (59.41%) and with pre-training (68.82%). It also outperformed the L1-logistic regression model trained with baseline demographic features and clinical scores of SZ patients. This study reports the first evidence that rs-fMRI-derived brain connectivity pattern can predict the clinical response of persistent AVH to add-on tDCS in SZ patients with 72.5% accuracy.

Prediction of Obsessive-Compulsive Disorder: Importance of Neurobiology-Aided Feature Design and Cross-Diagnosis Transfer Learning

BACKGROUND

Machine learning applications using neuroimaging provide a multidimensional, data-driven approach that captures the level of complexity necessary for objectively aiding diagnosis and prognosis in psychiatry. However, models learned from small training samples often have limited generalizability, which continues to be a problem with automated diagnosis of mental illnesses such as obsessive-compulsive disorder (OCD). Earlier studies have shown that features incorporating prior neurobiological knowledge of brain function and combining brain parcellations from various sources can potentially improve the overall prediction. However, it is unknown whether such knowledge-driven methods can provide a performance that is comparable to state-of-the-art approaches based on neural networks.

METHODS

In this study, we apply a transparent and explainable multiparcellation ensemble learning framework EMPaSchiz (Ensemble algorithm with Multiple Parcellations for Schizophrenia prediction) to the task of predicting OCD, based on a resting-state functional magnetic resonance imaging dataset of 350 subjects. Furthermore, we apply transfer learning using the features found effective for schizophrenia to OCD to leverage the commonality in brain alterations across these psychiatric diagnoses.

RESULTS

We show that our knowledge-based approach leads to a prediction performance of 80.3% accuracy for OCD diagnosis that is better than domain-agnostic and automated feature design using neural networks. Furthermore, we show that a selection of reduced feature sets can be transferred from schizophrenia to the OCD prediction model without significant loss in prediction performance.

CONCLUSIONS

This study presents a machine learning framework for OCD prediction with neurobiology-aided feature design using resting-state functional magnetic resonance imaging that is generalizable and reasonably interpretable.

Transfer Learning Approaches for Neuroimaging Analysis: A Scoping Review

Deep learning algorithms have been moderately successful in diagnoses of diseases by analyzing medical images especially through neuroimaging that is rich in annotated data. Transfer learning methods have demonstrated strong performance in tackling annotated data. It utilizes and transfers knowledge learned from a source domain to target domain even when the dataset is small. There are multiple approaches to transfer learning that result in a range of performance estimates in diagnosis, detection, and classification of clinical problems. Therefore, in this paper, we reviewed transfer learning approaches, their design attributes, and their applications to neuroimaging problems. We reviewed two main literature databases and included the most relevant studies using predefined inclusion criteria. Among 50 reviewed studies, more than half of them are on transfer learning for Alzheimer's disease. Brain mapping and brain tumor detection were second and third most discussed research problems, respectively. The most common source dataset for transfer learning was ImageNet, which is not a neuroimaging dataset. This suggests that the majority of studies preferred pre-trained models instead of training their own model on a neuroimaging dataset. Although, about one third of studies designed their own architecture, most studies used existing Convolutional Neural Network architectures. Magnetic Resonance Imaging was the most common imaging modality. In almost all studies, transfer learning contributed to better performance in diagnosis, classification, segmentation of different neuroimaging diseases and problems, than methods without transfer learning. Among different transfer learning approaches, fine-tuning all convolutional and fully-connected layers approach and freezing convolutional layers and fine-tuning fully-connected layers approach demonstrated superior performance in terms of accuracy. These recent transfer learning approaches not only show great performance but also require less computational resources and time.

Predicting Fluid Intelligence via Naturalistic Functional Connectivity Using Weighted Ensemble Model and Network Analysis

Objectives: Functional connectivity triggered by naturalistic stimuli (e.g., movie clips), coupled with machine learning techniques provide great insight in exploring brain functions such as fluid intelligence. However, functional connectivity is multi-layered while traditional machine learning is based on individual model, which is not only limited in performance, but also fails to extract multi-dimensional and multi-layered information from the brain network. Methods: In this study, inspired by multi-layer brain network structure, we propose a new method, namely weighted ensemble model and network analysis, which combines machine learning and graph theory for improved fluid intelligence prediction. Firstly, functional connectivity analysis and graphical theory were jointly employed. The functional connectivity and graphical indices computed using the preprocessed fMRI data were then all fed into an auto-encoder parallelly for automatic feature extraction to predict the fluid intelligence. In order to improve the performance, tree regression and ridge regression models were stacked and fused automatically with weighted values. Finally, layers of auto-encoder were visualized to better illustrate the connectome patterns, followed by the evaluation of the performance to justify the mechanism of brain functions. Results: Our proposed method achieved the best performance with a 3.85 mean absolute deviation, 0.66 correlation coefficient and 0.42 R-squared coefficient; this model outperformed other state-of-the-art methods. It is also worth noting that the optimization of the biological pattern extraction was automated though the auto-encoder algorithm. Conclusion: The proposed method outperforms the state-of-the-art reports, also is able to effectively capture the biological patterns of functional connectivity during a naturalistic movie state for potential clinical explorations.

Identifying cardiomegaly in chest X-rays: a cross-sectional study of evaluation and comparison between different transfer learning methods

BACKGROUND

Cardiomegaly is a relatively common incidental finding on chest X-rays; if left untreated, it can result in significant complications. Using Artificial Intelligence for diagnosing cardiomegaly could be beneficial, as this pathology may be underreported, or overlooked, especially in busy or under-staffed settings.

PURPOSE

To explore the feasibility of applying four different transfer learning methods to identify the presence of cardiomegaly in chest X-rays and to compare their diagnostic performance using the radiologists' report as the gold standard.

MATERIAL AND METHODS

Two thousand chest X-rays were utilized in the current study: 1000 were normal and 1000 had confirmed cardiomegaly. Of these exams, 80% were used for training and 20% as a holdout test dataset. A total of 2048 deep features were extracted using Google's Inception V3, VGG16, VGG19, and SqueezeNet networks. A logistic regression algorithm optimized in regularization terms was used to classify chest X-rays into those with presence or absence of cardiomegaly.

RESULTS

Diagnostic accuracy is reported by means of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), with the VGG19 network providing the best values of sensitivity (84%), specificity (83%), PPV (83%), NPV (84%), and overall accuracy (84,5%). The other networks presented sensitivity at 64.1%-82%, specificity at 77.1%-81.1%, PPV at 74%-81.4%, NPV at 68%-82%, and overall accuracy at 71%-81.3%.

CONCLUSION

Deep learning using transfer learning methods based on VGG19 network can be used for the automatic detection of cardiomegaly on chest X-ray images. However, further validation and training of each method is required before application to clinical cases.

The amplitude of low-frequency fluctuation predicts levodopa treatment response in patients with Parkinson's disease

INTRODUCTION

Levodopa has become the main therapy for motor symptoms of Parkinson's disease (PD). This study aimed to test whether the amplitude of low-frequency fluctuation (ALFF) computed by fMRI could predict individual patient's response to levodopa treatment.

METHODS

We included 40 patients. Treatment efficacy was defined based on motor symptoms improvement from the state of medication off to medication on, as assessed by the Unified Parkinson's Disease Rating Scale score III. Two machine learning models were constructed to test the prediction ability of ALFF. First, the ensemble method was implemented to predict individual treatment responses. Second, the categorical boosting (CatBoost) classification was used to predict individual levodopa responses in patients classified as moderate and superior responders, according to the 50% threshold of improvement. The age, disease duration and treatment dose were controlled as covariates.

RESULTS

No significant difference in clinical data were observed between moderate and superior responders. Using the ensemble method, the regression model showed a significant correlation between the predicted and the observed motor symptoms improvement (r = 0.61, p < 0.01, mean absolute error = 0.11 ± 0.02), measured as a continuous variable. The use of the Catboost algorithm revealed that ALFF was able to differentiate between moderate and superior responders (area under the curve = 0.90). The mainly contributed regions for both models included the bilateral primary motor cortex, the occipital cortex, the cerebellum, and the basal ganglia.

CONCLUSION

Both continuous and binary ALFF values have the potential to serve as promising predictive markers of dopaminergic therapy response in patients with PD.

Optical diagnosis of colorectal polyps using convolutional neural networks

Colonoscopy remains the gold standard investigation for colorectal cancer screening as it offers the opportunity to both detect and resect pre-malignant and neoplastic polyps. Although technologies for image-enhanced endoscopy are widely available, optical diagnosis has not been incorporated into routine clinical practice, mainly due to significant inter-operator variability. In recent years, there has been a growing number of studies demonstrating the potential of convolutional neural networks (CNN) to enhance optical diagnosis of polyps. Data suggest that the use of CNNs might mitigate the inter-operator variability amongst endoscopists, potentially enabling a “resect and discard“ or ”leave in“ strategy to be adopted in real-time. This would have significant financial benefits for healthcare systems, avoid unnecessary polypectomies of non-neoplastic polyps and improve the efficiency of colonoscopy. Here, we review advances in CNN for the optical diagnosis of colorectal polyps, current limitations and future directions.

Transfer Learning in Magnetic Resonance Brain Imaging: A Systematic Review

(1) Background: Transfer learning refers to machine learning techniques that focus on acquiring knowledge from related tasks to improve generalization in the tasks of interest. In magnetic resonance imaging (MRI), transfer learning is important for developing strategies that address the variation in MR images from different imaging protocols or scanners. Additionally, transfer learning is beneficial for reutilizing machine learning models that were trained to solve different (but related) tasks to the task of interest. The aim of this review is to identify research directions, gaps in knowledge, applications, and widely used strategies among the transfer learning approaches applied in MR brain imaging; (2) Methods: We performed a systematic literature search for articles that applied transfer learning to MR brain imaging tasks. We screened 433 studies for their relevance, and we categorized and extracted relevant information, including task type, application, availability of labels, and machine learning methods. Furthermore, we closely examined brain MRI-specific transfer learning approaches and other methods that tackled issues relevant to medical imaging, including privacy, unseen target domains, and unlabeled data; (3) Results: We found 129 articles that applied transfer learning to MR brain imaging tasks. The most frequent applications were dementia-related classification tasks and brain tumor segmentation. The majority of articles utilized transfer learning techniques based on convolutional neural networks (CNNs). Only a few approaches utilized clearly brain MRI-specific methodology, and considered privacy issues, unseen target domains, or unlabeled data. We proposed a new categorization to group specific, widely-used approaches such as pretraining and fine-tuning CNNs; (4) Discussion: There is increasing interest in transfer learning for brain MRI. Well-known public datasets have clearly contributed to the popularity of Alzheimer's diagnostics/prognostics and tumor segmentation as applications. Likewise, the availability of pretrained CNNs has promoted their utilization. Finally, the majority of the surveyed studies did not examine in detail the interpretation of their strategies after applying transfer learning, and did not compare their approach with other transfer learning approaches.

Predicting Body Mass Index From Structural MRI Brain Images Using a Deep Convolutional Neural Network

In recent years, deep learning (DL) has become more widespread in the fields of cognitive and clinical neuroimaging. Using deep neural network models to process neuroimaging data is an efficient method to classify brain disorders and identify individuals who are at increased risk of age-related cognitive decline and neurodegenerative disease. Here we investigated, for the first time, whether structural brain imaging and DL can be used for predicting a physical trait that is of significant clinical relevance—the body mass index (BMI) of the individual. We show that individual BMI can be accurately predicted using a deep convolutional neural network (CNN) and a single structural magnetic resonance imaging (MRI) brain scan along with information about age and sex. Localization maps computed for the CNN highlighted several brain structures that strongly contributed to BMI prediction, including the caudate nucleus and the amygdala. Comparison to the results obtained via a standard automatic brain segmentation method revealed that the CNN-based visualization approach yielded complementary evidence regarding the relationship between brain structure and BMI. Taken together, our results imply that predicting BMI from structural brain scans using DL represents a promising approach to investigate the relationship between brain morphological variability and individual differences in body weight and provide a new scope for future investigations regarding the potential clinical utility of brain-predicted BMI.

Ensemble Learning of Convolutional Neural Network, Support Vector Machine, and Best Linear Unbiased Predictor for Brain Age Prediction: ARAMIS Contribution to the Predictive Analytics Competition 2019 Challenge

We ranked third in the Predictive Analytics Competition (PAC) 2019 challenge by achieving a mean absolute error (MAE) of 3.33 years in predicting age from T1-weighted MRI brain images. Our approach combined seven algorithms that allow generating predictions when the number of features exceeds the number of observations, in particular, two versions of best linear unbiased predictor (BLUP), support vector machine (SVM), two shallow convolutional neural networks (CNNs), and the famous ResNet and Inception V1. Ensemble learning was derived from estimating weights via linear regression in a hold-out subset of the training sample. We further evaluated and identified factors that could influence prediction accuracy: choice of algorithm, ensemble learning, and features used as input/MRI image processing. Our prediction error was correlated with age, and absolute error was greater for older participants, suggesting to increase the training sample for this subgroup. Our results may be used to guide researchers to build age predictors on healthy individuals, which can be used in research and in the clinics as non-specific predictors of disease status.

Optimization of a Deep-Learning Method Based on the Classification of Images Generated by Parameterized Deep Snap a Novel Molecular-Image-Input Technique for Quantitative Structure-Activity Relationship (QSAR) Analysis

Numerous chemical compounds are distributed around the world and may affect the homeostasis of the endocrine system by disrupting the normal functions of hormone receptors. Although the risks associated with these compounds have been evaluated by acute toxicity testing in mammalian models, the chronic toxicity of many chemicals remains due to high cost of the compounds and the testing, etc. However, computational approaches may be promising alternatives and reduce these evaluations. Recently, deep learning (DL) has been shown to be promising prediction models with high accuracy for recognition of images, speech, signals, and videos since it greatly benefits from large datasets. Recently, a novel DL-based technique called DeepSnap was developed to conduct QSAR analysis using three-dimensional images of chemical structures. It can be used to predict the potential toxicity of many different chemicals to various receptors without extraction of descriptors. DeepSnap has been shown to have a very high capacity in tests using Tox21 quantitative qHTP datasets. Numerous parameters must be adjusted to use the DeepSnap method but they have not been optimized. In this study, the effects of these parameters on the performance of the DL prediction model were evaluated in terms of the loss in validation as an indicator for evaluating the performance of the DL using the toxicity information in the Tox21 qHTP database. The relations of the parameters of DeepSnap such as (1) number of molecules per SDF split into (2) zoom factor percentage, (3) atom size for van der waals percentage, (4) bond radius, (5) minimum bond distance, and (6) bond tolerance, with the validation loss following quadratic function curves, which suggests that optimal thresholds exist to attain the best performance with these prediction models. Using the parameter values set with the best performance, the prediction model of chemical compounds for CAR agonist was built using 64 images, at 105° angle, with AUC of 0.791. Thus, based on these parameters, the proposed DeepSnap-DL approach will be highly reliable and beneficial to establish models to assess the risk associated with various chemicals.

Transfer learning improves resting-state functional connectivity pattern analysis using convolutional neural networks

Background

Deep learning is gaining importance in the prediction of cognitive states and brain pathology based on neuroimaging data. Including multiple hidden layers in artificial neural networks enables unprecedented predictive power; however, the proper training of deep neural networks requires thousands of exemplars. Collecting this amount of data is not feasible in typical neuroimaging experiments. A handy solution to this problem, which has largely fallen outside the scope of deep learning applications in neuroimaging, is to repurpose deep networks that have already been trained on large datasets by fine-tuning them to target datasets/tasks with fewer exemplars. Here, we investigated how this method, called transfer learning, can aid age category classification and regression based on brain functional connectivity patterns derived from resting-state functional magnetic resonance imaging. We trained a connectome-convolutional neural network on a larger public dataset and then examined how the knowledge learned can be used effectively to perform these tasks on smaller target datasets collected with a different type of scanner and/or imaging protocol and pre-processing pipeline.

Results

Age classification on the target datasets benefitted from transfer learning. Significant improvement (∼9%–13% increase in accuracy) was observed when the convolutional layers’ weights were initialized based on the values learned on the public dataset and then fine-tuned to the target datasets. Transfer learning also appeared promising in improving the otherwise poor prediction of chronological age.

Conclusions

Transfer learning is a plausible solution to adapt convolutional neural networks to neuroimaging data with few exemplars and different data acquisition and pre-processing protocols.