1
|
Schmidt T, Nagy Z. SAD: semi-supervised automatic detection of BOLD activations in high temporal resolution fMRI data. MAGMA (NEW YORK, N.Y.) 2024:10.1007/s10334-024-01197-0. [PMID: 39207582 DOI: 10.1007/s10334-024-01197-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/19/2024] [Revised: 07/26/2024] [Accepted: 07/29/2024] [Indexed: 09/04/2024]
Abstract
OBJECTIVE Despite the prevalent use of the general linear model (GLM) in fMRI data analysis, assuming a pre-defined hemodynamic response function (HRF) for all voxels can lead to reduced reliability and may distort the inferences derived from it. To overcome the necessity of presuming a specific model for the hemodynamic response, we introduce a semi-supervised automatic detection (SAD) method. MATERIALS AND METHODS The proposed SAD method employs a Bi-LSTM neural network to classify high temporal resolution fMRI data. Network training utilized an fMRI dataset with 75-ms temporal resolution in an iterative scheme. Classification performance was evaluated on a second fMRI dataset from the same participant, collected on a different day. Comparative analysis with the standard GLM approach was conducted to evaluate the cooperative effectiveness of the SAD method. RESULTS The SAD method performed well based on the classification scores: true-positive rate = 0.961, area under the receiver operating curve = 0.998, true-negative rate = 0.99, F1-score = 0.979, False-negative rate = 0.038, false-discovery rate = 0.002, false-positive rate = 0.002 at 75-ms temporal resolution. CONCLUSION SAD can detect hemodynamic responses at 75-ms temporal resolution without relying on a specific shape of an HRF. Future work could expand the use cases to include more participants and different fMRI paradigms.
Collapse
Affiliation(s)
- Tim Schmidt
- Laboratory for Social and Neural Systems Research, SNS-Lab, University of Zurich, Rämistrasse 100, CH-8091, Zurich, Switzerland.
- Institute for Biomedical Engineering, ETH Zurich and University of Zurich, Zurich, Switzerland.
| | - Zoltán Nagy
- Laboratory for Social and Neural Systems Research, SNS-Lab, University of Zurich, Rämistrasse 100, CH-8091, Zurich, Switzerland
| |
Collapse
|
2
|
Mobarak-Abadi M, Mahmoudi-Aznaveh A, Dehghani H, Zarei M, Vahdat S, Doyon J, Khatibi A. DeepRetroMoCo: deep neural network-based retrospective motion correction algorithm for spinal cord functional MRI. Front Psychiatry 2024; 15:1323109. [PMID: 39006826 PMCID: PMC11239515 DOI: 10.3389/fpsyt.2024.1323109] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Accepted: 05/21/2024] [Indexed: 07/16/2024] Open
Abstract
Background and purpose There are distinct challenges in the preprocessing of spinal cord fMRI data, particularly concerning the mitigation of voluntary or involuntary movement artifacts during image acquisition. Despite the notable progress in data processing techniques for movement detection and correction, applying motion correction algorithms developed for the brain cortex to the brainstem and spinal cord remains a challenging endeavor. Methods In this study, we employed a deep learning-based convolutional neural network (CNN) named DeepRetroMoCo, trained using an unsupervised learning algorithm. Our goal was to detect and rectify motion artifacts in axial T2*-weighted spinal cord data. The training dataset consisted of spinal cord fMRI data from 27 participants, comprising 135 runs for training and 81 runs for testing. Results To evaluate the efficacy of DeepRetroMoCo, we compared its performance against the sct_fmri_moco method implemented in the spinal cord toolbox. We assessed the motion-corrected images using two metrics: the average temporal signal-to-noise ratio (tSNR) and Delta Variation Signal (DVARS) for both raw and motion-corrected data. Notably, the average tSNR in the cervical cord was significantly higher when DeepRetroMoCo was utilized for motion correction, compared to the sct_fmri_moco method. Additionally, the average DVARS values were lower in images corrected by DeepRetroMoCo, indicating a superior reduction in motion artifacts. Moreover, DeepRetroMoCo exhibited a significantly shorter processing time compared to sct_fmri_moco. Conclusion Our findings strongly support the notion that DeepRetroMoCo represents a substantial improvement in motion correction procedures for fMRI data acquired from the cervical spinal cord. This novel deep learning-based approach showcases enhanced performance, offering a promising solution to address the challenges posed by motion artifacts in spinal cord fMRI data.
Collapse
Affiliation(s)
- Mahdi Mobarak-Abadi
- Institute of Medical Science and Technology, Shahid Beheshti University, Tehran, Iran
| | | | - Hamed Dehghani
- Neuro Imaging and Analysis Group (NIAG), Research Center for Molecular and Cellular Imaging (RCMCI), Tehran University of Medical Sciences, Tehran, Iran
| | - Mojtaba Zarei
- Institute of Medical Science and Technology, Shahid Beheshti University, Tehran, Iran
| | - Shahabeddin Vahdat
- Department of Applied Physiology and Kinesiology (DAPK), University of Florida, Gainesville, FL, United States
| | - Julien Doyon
- Montreal Neurological Institute, McGill University, Montreal, QC, Canada
| | - Ali Khatibi
- Centre of Precision Rehabilitation for Spinal Pain, School of Sports Exercise and Rehabilitation Sciences, University of Birmingham, Birmingham, United Kingdom
- Centre for Human Brain Health, University of Birmingham, Birmingham, United Kingdom
| |
Collapse
|
3
|
Brookshire G, Kasper J, Blauch NM, Wu YC, Glatt R, Merrill DA, Gerrol S, Yoder KJ, Quirk C, Lucero C. Data leakage in deep learning studies of translational EEG. Front Neurosci 2024; 18:1373515. [PMID: 38765672 PMCID: PMC11099244 DOI: 10.3389/fnins.2024.1373515] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Accepted: 04/04/2024] [Indexed: 05/22/2024] Open
Abstract
A growing number of studies apply deep neural networks (DNNs) to recordings of human electroencephalography (EEG) to identify a range of disorders. In many studies, EEG recordings are split into segments, and each segment is randomly assigned to the training or test set. As a consequence, data from individual subjects appears in both the training and the test set. Could high test-set accuracy reflect data leakage from subject-specific patterns in the data, rather than patterns that identify a disease? We address this question by testing the performance of DNN classifiers using segment-based holdout (in which segments from one subject can appear in both the training and test set), and comparing this to their performance using subject-based holdout (where all segments from one subject appear exclusively in either the training set or the test set). In two datasets (one classifying Alzheimer's disease, and the other classifying epileptic seizures), we find that performance on previously-unseen subjects is strongly overestimated when models are trained using segment-based holdout. Finally, we survey the literature and find that the majority of translational DNN-EEG studies use segment-based holdout. Most published DNN-EEG studies may dramatically overestimate their classification performance on new subjects.
Collapse
Affiliation(s)
| | - Jake Kasper
- SPARK Neuro Inc., New York, NY, United States
| | - Nicholas M. Blauch
- SPARK Neuro Inc., New York, NY, United States
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, United States
| | | | - Ryan Glatt
- Pacific Brain Health Center, Pacific Neuroscience Institute and Foundation, Santa Monica, CA, United States
| | - David A. Merrill
- Pacific Brain Health Center, Pacific Neuroscience Institute and Foundation, Santa Monica, CA, United States
- Saint John's Cancer Institute at Providence Saint John's Health Center, Santa Monica, CA, United States
- Psychiatry and Biobehavioral Sciences, Semel Institute for Neuroscience and Human Behavior, David Geffen School of Medicine at University of California, Los Angeles, Los Angeles, CA, United States
| | | | | | - Colin Quirk
- SPARK Neuro Inc., New York, NY, United States
| | - Ché Lucero
- SPARK Neuro Inc., New York, NY, United States
| |
Collapse
|
4
|
Sievers B, Thornton MA. Deep social neuroscience: the promise and peril of using artificial neural networks to study the social brain. Soc Cogn Affect Neurosci 2024; 19:nsae014. [PMID: 38334747 PMCID: PMC10880882 DOI: 10.1093/scan/nsae014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 12/20/2023] [Accepted: 02/04/2024] [Indexed: 02/10/2024] Open
Abstract
This review offers an accessible primer to social neuroscientists interested in neural networks. It begins by providing an overview of key concepts in deep learning. It then discusses three ways neural networks can be useful to social neuroscientists: (i) building statistical models to predict behavior from brain activity; (ii) quantifying naturalistic stimuli and social interactions; and (iii) generating cognitive models of social brain function. These applications have the potential to enhance the clinical value of neuroimaging and improve the generalizability of social neuroscience research. We also discuss the significant practical challenges, theoretical limitations and ethical issues faced by deep learning. If the field can successfully navigate these hazards, we believe that artificial neural networks may prove indispensable for the next stage of the field's development: deep social neuroscience.
Collapse
Affiliation(s)
- Beau Sievers
- Department of Psychology, Stanford University, 420 Jane Stanford Way, Stanford, CA 94305, USA
- Department of Psychology, Harvard University, 33 Kirkland St., Cambridge, MA 02138, USA
| | - Mark A Thornton
- Department of Psychological and Brain Sciences, Dartmouth College, 6207 Moore Hall, Hanover, NH 03755, USA
| |
Collapse
|
5
|
Song L, Ren Y, Xu S, Hou Y, He X. A hybrid spatiotemporal deep belief network and sparse representation-based framework reveals multilevel core functional components in decoding multitask fMRI signals. Netw Neurosci 2023; 7:1513-1532. [PMID: 38144693 PMCID: PMC10745082 DOI: 10.1162/netn_a_00334] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Accepted: 08/17/2023] [Indexed: 12/26/2023] Open
Abstract
Decoding human brain activity on various task-based functional brain imaging data is of great significance for uncovering the functioning mechanism of the human mind. Currently, most feature extraction model-based methods for brain state decoding are shallow machine learning models, which may struggle to capture complex and precise spatiotemporal patterns of brain activity from the highly noisy fMRI raw data. Moreover, although decoding models based on deep learning methods benefit from their multilayer structure that could extract spatiotemporal features at multiscale, the relatively large populations of fMRI datasets are indispensable, and the explainability of their results is elusive. To address the above problems, we proposed a computational framework based on hybrid spatiotemporal deep belief network and sparse representations to differentiate multitask fMRI (tfMRI) signals. Using a relatively small cohort of tfMRI data as a test bed, our framework can achieve an average classification accuracy of 97.86% and define the multilevel temporal and spatial patterns of multiple cognitive tasks. Intriguingly, our model can characterize the key components for differentiating the multitask fMRI signals. Overall, the proposed framework can identify the interpretable and discriminative fMRI composition patterns at multiple scales, offering an effective methodology for basic neuroscience and clinical research with relatively small cohorts.
Collapse
Affiliation(s)
- Limei Song
- School of Information Science and Technology, Northwest University, Xi’an, China
| | - Yudan Ren
- School of Information Science and Technology, Northwest University, Xi’an, China
| | - Shuhan Xu
- School of Information Science and Technology, Northwest University, Xi’an, China
| | - Yuqing Hou
- School of Information Science and Technology, Northwest University, Xi’an, China
| | - Xiaowei He
- School of Information Science and Technology, Northwest University, Xi’an, China
| |
Collapse
|
6
|
Jimbo T, Matsuo H, Imoto Y, Sodemura T, Nishimori M, Fukui Y, Hayashi T, Furuyashiki T, Yokoyama R. Accelerated preprocessing of large numbers of brain images by parallel computing on supercomputers. Sci Rep 2023; 13:19901. [PMID: 37963952 PMCID: PMC10646110 DOI: 10.1038/s41598-023-46073-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Accepted: 10/27/2023] [Indexed: 11/16/2023] Open
Abstract
"Preprocessing" is the first step required in brain image analysis that improves the overall quality and reliability of the results. However, it is computationally demanding and time-consuming, particularly to handle and parcellate complicatedly folded cortical ribbons of the human brain. In this study, we aimed to shorten the analysis time for data preprocessing of 1410 brain images simultaneously on one of the world's highest-performing supercomputers, "Fugaku." The FreeSurfer was used as a benchmark preprocessing software for cortical surface reconstruction. All the brain images were processed simultaneously and successfully analyzed in a calculation time of 17.33 h. This result indicates that using a supercomputer for brain image preprocessing allows big data analysis to be completed shortly and flexibly, thus suggesting the possibility of supercomputers being used for expanding large data analysis and parameter optimization of preprocessing in the future.
Collapse
Affiliation(s)
- Takehiro Jimbo
- Japan Research Activity Support Inc., Kobe, Japan
- Department of Urology, Kobe University Graduate School of Medicine, Kobe, Japan
- Laboratory for Brain Connectomics Imaging, RIKEN Center for Biosystems Dynamics Research, Kobe, Japan
| | - Hidetoshi Matsuo
- Department of Radiology, Kobe University Graduate School of Medicine, Kobe, Japan
- Mediest Co., Kobe, Japan
| | - Yuya Imoto
- Japan Research Activity Support Inc., Kobe, Japan
| | | | - Makoto Nishimori
- Mediest Co., Kobe, Japan
- Division of Molecular Epidemiology, Kobe University Graduate School of Medicine, Kobe, Japan
| | - Yoshinari Fukui
- Department of Mathematics, Faculty of Science, Tokyo University of Science, Tokyo, Japan
| | - Takuya Hayashi
- Laboratory for Brain Connectomics Imaging, RIKEN Center for Biosystems Dynamics Research, Kobe, Japan
- Department of Brain Connectomics, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Tomoyuki Furuyashiki
- Division of Pharmacology, Graduate School of Medicine, Kobe University, Kobe, Japan
| | - Ryoichi Yokoyama
- Department of Extended Intelligence for Medicine, The Ishii-Ishibashi Laboratory, Keio University, 35 Shinanomachi, Shinjuku-ku, Tokyo, 160-8582, Japan.
- Yokoyama Lab, Tokyo, Japan.
| |
Collapse
|
7
|
Zhao Y, Luo H, Chen J, Loureiro R, Yang S, Zhao H. Learning based motion artifacts processing in fNIRS: a mini review. Front Neurosci 2023; 17:1280590. [PMID: 38033535 PMCID: PMC10683641 DOI: 10.3389/fnins.2023.1280590] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2023] [Accepted: 10/11/2023] [Indexed: 12/02/2023] Open
Abstract
This paper provides a concise review of learning-based motion artifacts (MA) processing methods in functional near-infrared spectroscopy (fNIRS), highlighting the challenges of maintaining optimal contact during subject movement, which can lead to MA and compromise data integrity. Traditional strategies often result in reduced reliability of the hemodynamic response and statistical power. Recognizing the limited number of studies focusing on learning-based MA removal, we examine 315 studies, identifying seven pertinent to our focus area. We discuss the current landscape of learning-based MA correction methods and highlight research gaps. Noting the absence of standard evaluation metrics for quality assessment of MA correction, we suggest a novel framework, integrating signal and model quality considerations and employing metrics like ΔSignal-to-Noise Ratio (ΔSNR), confusion matrix, and Mean Squared Error. This work aims to facilitate the application of learning-based methodologies to fNIRS and improve the accuracy and reliability of neurovascular studies.
Collapse
Affiliation(s)
- Yunyi Zhao
- HUB of Intelligent Neuro-Engineering, CREATe, IOMS, Division of Surgery and Interventional Science (DSIS), University College London, Stanmore, United Kingdom
| | - Haiming Luo
- HUB of Intelligent Neuro-Engineering, CREATe, IOMS, Division of Surgery and Interventional Science (DSIS), University College London, Stanmore, United Kingdom
| | - Jianan Chen
- HUB of Intelligent Neuro-Engineering, CREATe, IOMS, Division of Surgery and Interventional Science (DSIS), University College London, Stanmore, United Kingdom
| | - Rui Loureiro
- HUB of Intelligent Neuro-Engineering, CREATe, IOMS, Division of Surgery and Interventional Science (DSIS), University College London, Stanmore, United Kingdom
| | - Shufan Yang
- School of Computing, Engineering and Built Environment, Edinburgh Napier University, Edinburgh, United Kingdom
| | - Hubin Zhao
- HUB of Intelligent Neuro-Engineering, CREATe, IOMS, Division of Surgery and Interventional Science (DSIS), University College London, Stanmore, United Kingdom
| |
Collapse
|
8
|
Luckett PH, Park KY, Lee JJ, Lenze EJ, Wetherell JL, Eyler L, Snyder AZ, Ances BM, Shimony JS, Leuthardt EC. Data-efficient resting-state functional magnetic resonance imaging brain mapping with deep learning. J Neurosurg 2023; 139:1258-1269. [PMID: 37060318 PMCID: PMC10576012 DOI: 10.3171/2023.3.jns2314] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Accepted: 03/01/2023] [Indexed: 04/16/2023]
Abstract
OBJECTIVE Resting-state functional MRI (RS-fMRI) enables the mapping of function within the brain and is emerging as an efficient tool for the presurgical evaluation of eloquent cortex. Models capable of reliable and precise mapping of resting-state networks (RSNs) with a reduced scanning time would lead to improved patient comfort while reducing the cost per scan. The aims of the present study were to develop a deep 3D convolutional neural network (3DCNN) capable of voxel-wise mapping of language (LAN) and motor (MOT) RSNs with minimal quantities of RS-fMRI data. METHODS Imaging data were gathered from multiple ongoing studies at Washington University School of Medicine and other thoroughly characterized, publicly available data sets. All study participants (n = 2252 healthy adults) were cognitively screened and completed structural neuroimaging and RS-fMRI. Random permutations of RS-fMRI regions of interest were used to train a 3DCNN. After training, model inferences were compared using varying amounts of RS-fMRI data from the control data set as well as 5 patients with glioblastoma multiforme. RESULTS The trained model achieved 96% out-of-sample validation accuracy on data encompassing a large age range collected on multiple scanner types and varying sequence parameters. Testing on out-of-sample control data showed 97.9% similarity between results generated using either 50 or 200 RS-fMRI time points, corresponding to approximately 2.5 and 10 minutes, respectively (96.9% LAN, 96.3% MOT true-positive rate). In evaluating data from patients with brain tumors, the 3DCNN was able to accurately map LAN and MOT networks despite structural and functional alterations. CONCLUSIONS Functional maps produced by the 3DCNN can inform surgical planning in patients with brain tumors in a time-efficient manner. The authors present a highly efficient method for presurgical functional mapping and thus improved functional preservation in patients with brain tumors.
Collapse
Affiliation(s)
- Patrick H. Luckett
- Division of Neurotechnology, Department of Neurological Surgery, Washington University School of Medicine, St. Louis, Missouri
| | - Ki Yun Park
- Division of Neurotechnology, Department of Neurological Surgery, Washington University School of Medicine, St. Louis, Missouri
- Department of Neuroscience, Washington University School of Medicine, St. Louis, Missouri
| | - John J. Lee
- Mallinckrodt Institute of Radiology, Washington University School of Medicine, St. Louis, Missouri
| | - Eric J Lenze
- Department of Psychiatry, Washington University School of Medicine, St. Louis, Missouri
| | - Julie L Wetherell
- Mental Health Impact Unit 3, VA San Diego Healthcare System, San Diego, California
- Department of Psychiatry, University of California, San Diego, California
| | - Lisa Eyler
- Department of Psychiatry, University of California, San Diego, California
| | - Abraham Z. Snyder
- Mallinckrodt Institute of Radiology, Washington University School of Medicine, St. Louis, Missouri
- Department of Neurology, Washington University School of Medicine, St. Louis, Missouri
| | - Beau M. Ances
- Department of Neurology, Washington University School of Medicine, St. Louis, Missouri
| | - Joshua S. Shimony
- Mallinckrodt Institute of Radiology, Washington University School of Medicine, St. Louis, Missouri
| | - Eric C. Leuthardt
- Division of Neurotechnology, Department of Neurological Surgery, Washington University School of Medicine, St. Louis, Missouri
- Department of Neuroscience, Washington University School of Medicine, St. Louis, Missouri
- Department of Biomedical Engineering, Washington University in St. Louis, St. Louis, MO
- Department of Mechanical Engineering and Materials Science, Washington University in St. Louis, St. Louis, MO
- Center for Innovation in Neuroscience and Technology, Division of Neurotechnology, Washington University School of Medicine, St. Louis, MO
- Brain Laser Center, Washington University School of Medicine, St. Louis, Missouri
- National Center for Adaptive Neurotechnologies
| |
Collapse
|
9
|
Dai P, Lu D, Shi Y, Zhou Y, Xiong T, Zhou X, Chen Z, Zou B, Tang H, Huang Z, Liao S. Classification of recurrent major depressive disorder using a new time series feature extraction method through multisite rs-fMRI data. J Affect Disord 2023; 339:511-519. [PMID: 37467800 DOI: 10.1016/j.jad.2023.07.077] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Revised: 06/23/2023] [Accepted: 07/14/2023] [Indexed: 07/21/2023]
Abstract
BACKGROUND Major depressive disorder (MDD) has a high rate of recurrence. Identifying patients with recurrent MDD is advantageous in adopting prevention strategies to reduce the disabling effects of depression. METHOD We propose a novel feature extraction method that includes dynamic temporal information, and inputs the extracted features into a graph convolutional network (GCN) to achieve classification of recurrent MDD. We extract the average time series using an atlas from resting-state functional magnetic resonance imaging (fMRI) data. Pearson correlation was calculated between brain region sequences at each time point, representing the functional connectivity at each time point. The connectivity is used as the adjacency matrix and the brain region sequences as node features for a GCN model to classify recurrent MDD. Gradient-weighted Class Activation Mapping (Grad-CAM) was used to analyze the contribution of different brain regions to the model. Brain regions making greater contribution to classification were considered to be the regions with altered brain function in recurrent MDD. RESULT We achieved a classification accuracy of 75.8 % for recurrent MDD on the multi-site dataset, the Rest-meta-MDD. The brain regions closely related to recurrent MDD have been identified. LIMITATION The pre-processing stage may affect the final classification performance and harmonizing site differences may improve the classification performance. CONCLUSION The experimental results demonstrate that the proposed method can effectively classify recurrent MDD and extract dynamic changes of brain activity patterns in recurrent depression.
Collapse
Affiliation(s)
- Peishan Dai
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China.
| | - Da Lu
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
| | - Yun Shi
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
| | - Ying Zhou
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
| | - Tong Xiong
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
| | - Xiaoyan Zhou
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
| | - Zailiang Chen
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
| | - Beiji Zou
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
| | - Hui Tang
- Department of Psychiatry, and National Clinical Research Center for Mental Disorders, The Second Xiangya Hospital of Central South University, Changsha, Hunan, China
| | - Zhongchao Huang
- Department of Biomedical Engineering, School of Basic Medical Science, Central South University, Changsha, Hunan, China
| | - Shenghui Liao
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China.
| |
Collapse
|
10
|
Misthos LM, Krassanakis V, Merlemis N, Kesidis AL. Modeling the Visual Landscape: A Review on Approaches, Methods and Techniques. SENSORS (BASEL, SWITZERLAND) 2023; 23:8135. [PMID: 37836966 PMCID: PMC10574952 DOI: 10.3390/s23198135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Revised: 08/14/2023] [Accepted: 09/21/2023] [Indexed: 10/15/2023]
Abstract
Modeling the perception and evaluation of landscapes from the human perspective is a desirable goal for several scientific domains and applications. Human vision is the dominant sense, and human eyes are the sensors for apperceiving the environmental stimuli of our surroundings. Therefore, exploring the experimental recording and measurement of the visual landscape can reveal crucial aspects about human visual perception responses while viewing the natural or man-made landscapes. Landscape evaluation (or assessment) is another dimension that refers mainly to preferences of the visual landscape, involving human cognition as well, in ways that are often unpredictable. Yet, landscape can be approached by both egocentric (i.e., human view) and exocentric (i.e., bird's eye view) perspectives. The overarching approach of this review article lies in systematically presenting the different ways for modeling and quantifying the two 'modalities' of human perception and evaluation, under the two geometric perspectives, suggesting integrative approaches on these two 'diverging' dualities. To this end, several pertinent traditions/approaches, sensor-based experimental methods and techniques (e.g., eye tracking, fMRI, and EEG), and metrics are adduced and described. Essentially, this review article acts as a 'guide-map' for the delineation of the different activities related to landscape experience and/or management and to the valid or potentially suitable types of stimuli, sensors techniques, and metrics for each activity. Throughout our work, two main research directions are identified: (1) one that attempts to transfer the visual landscape experience/management from the one perspective to the other (and vice versa); (2) another one that aims to anticipate the visual perception of different landscapes and establish connections between perceptual processes and landscape preferences. As it appears, the research in the field is rapidly growing. In our opinion, it can be greatly advanced and enriched using integrative, interdisciplinary approaches in order to better understand the concepts and the mechanisms by which the visual landscape, as a complex set of stimuli, influences visual perception, potentially leading to more elaborate outcomes such as the anticipation of landscape preferences. As an effect, such approaches can support a rigorous, evidence-based, and socially just framework towards landscape management, protection, and decision making, based on a wide spectrum of well-suited and advanced sensor-based technologies.
Collapse
Affiliation(s)
- Loukas-Moysis Misthos
- Department of Surveying and Geoinformatics Engineering, University of West Attica, GR-12243 Athens, Greece; (L.-M.M.); (V.K.); (N.M.)
- Department of Public and One Health, University of Thessaly, GR-43100 Karditsa, Greece
| | - Vassilios Krassanakis
- Department of Surveying and Geoinformatics Engineering, University of West Attica, GR-12243 Athens, Greece; (L.-M.M.); (V.K.); (N.M.)
| | - Nikolaos Merlemis
- Department of Surveying and Geoinformatics Engineering, University of West Attica, GR-12243 Athens, Greece; (L.-M.M.); (V.K.); (N.M.)
| | - Anastasios L. Kesidis
- Department of Surveying and Geoinformatics Engineering, University of West Attica, GR-12243 Athens, Greece; (L.-M.M.); (V.K.); (N.M.)
| |
Collapse
|
11
|
Mirzabagherian H, Menhaj MB, Suratgar AA, Talebi N, Abbasi Sardari MR, Sajedin A. Temporal-spatial convolutional residual network for decoding attempted movement related EEG signals of subjects with spinal cord injury. Comput Biol Med 2023; 164:107159. [PMID: 37531857 DOI: 10.1016/j.compbiomed.2023.107159] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 05/19/2023] [Accepted: 06/07/2023] [Indexed: 08/04/2023]
Abstract
Brain Computer Interface (BCI) offers a promising approach to restoring hand functionality for people with cervical spinal cord injury (SCI). A reliable classification of brain activities based on appropriate flexibility in feature extraction could enhance BCI systems performance. In the present study, based on convolutional layers with temporal-spatial, Separable and Depthwise structures, we develop Temporal-Spatial Convolutional Residual Network)TSCR-Net(and Temporal-Spatial Convolutional Iterative Residual Network)TSCIR-Net(structures to classify electroencephalogram (EEG) signals. Using EEG signals in five different hand movement classes of SCI people, we compare the effectiveness of TSCIR-Net and TSCR-Net models with some competitive methods. We use the bayesian hyperparameter optimization algorithm to tune the hyperparameters of compact convolutional neural networks. In order to show the high generalizability of the proposed models, we compare the results of the models in different frequency ranges. Our proposed models decoded distinctive characteristics of different movement efforts and obtained higher classification accuracy than previous deep neural networks. Our findings indicate that TSCIR-Net and TSCR-Net models fulfills a better classification accuracy of 71.11%, and 64.55% for EEG_All and 57.74%, and 67.87% for EEG_Low frequency data sets than the compared methods in the literature.
Collapse
Affiliation(s)
- Hamed Mirzabagherian
- Department of Electrical Engineering, Amirkabir University of Technology, Hafez Ave. 15875-4413, Tehran, Iran.
| | - Mohammad Bagher Menhaj
- Department of Electrical Engineering, Amirkabir University of Technology, Hafez Ave. 15875-4413, Tehran, Iran.
| | - Amir Abolfazl Suratgar
- Department of Electrical Engineering, Amirkabir University of Technology, Hafez Ave. 15875-4413, Tehran, Iran.
| | - Nasibeh Talebi
- Department of Biomedical Engineering, Faculty of Engineering, Shahed University, Tehran, Iran.
| | | | - Atena Sajedin
- Department of Electrical Engineering, Amirkabir University of Technology, Hafez Ave. 15875-4413, Tehran, Iran.
| |
Collapse
|
12
|
Danks D, Davis I. Causal inference in cognitive neuroscience. WILEY INTERDISCIPLINARY REVIEWS. COGNITIVE SCIENCE 2023; 14:e1650. [PMID: 37032464 DOI: 10.1002/wcs.1650] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/04/2020] [Revised: 03/06/2023] [Accepted: 03/21/2023] [Indexed: 04/11/2023]
Abstract
Causal inference is a key step in many research endeavors in cognitive science and neuroscience, and particularly cognitive neuroscience. Statistical knowledge is sufficient for prediction and diagnosis, but causal knowledge is required for action and intervention. Most statistics courses and textbooks emphasize the difficulty of causal inference, focusing on the maxim that "correlation does not mean causation": there can be multiple causal possibilities, often many of them, consistent with given observed statistics. This paper focuses instead on the conceptual issues and assumptions that confront causal and other kinds of inference, primarily focusing on cognitive neuroscience. We connect inference methods with goals and challenges, and provide concrete guidance about how to select appropriate tools for the scientific task. This article is categorized under: Psychology > Theory and Methods Philosophy > Foundations of Cognitive Science.
Collapse
Affiliation(s)
- David Danks
- Halicioglu Data Science Institute, Department of Philosophy, University of California San Diego, La Jolla, California, USA
| | - Isaac Davis
- Department of Psychology, Yale University, New Haven, Connecticut, USA
| |
Collapse
|
13
|
Roth BJ. Can MRI Be Used as a Sensor to Record Neural Activity? SENSORS (BASEL, SWITZERLAND) 2023; 23:1337. [PMID: 36772381 PMCID: PMC9918955 DOI: 10.3390/s23031337] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 01/17/2023] [Accepted: 01/24/2023] [Indexed: 06/18/2023]
Abstract
Magnetic resonance provides exquisite anatomical images and functional MRI monitors physiological activity by recording blood oxygenation. This review attempts to answer the following question: Can MRI be used as a sensor to directly record neural behavior? It considers MRI sensing of electrical activity in the heart and in peripheral nerves before turning to the central topic: recording of brain activity. The primary hypothesis is that bioelectric current produced by a nerve or muscle creates a magnetic field that influences the magnetic resonance signal, although other mechanisms for detection are also considered. Recent studies have provided evidence that using MRI to sense neural activity is possible under ideal conditions. Whether it can be used routinely to provide functional information about brain processes in people remains an open question. The review concludes with a survey of artificial intelligence techniques that have been applied to functional MRI and may be appropriate for MRI sensing of neural activity.
Collapse
Affiliation(s)
- Bradley J Roth
- Department of Physics, Oakland University, Rochester, MI 48309, USA
| |
Collapse
|
14
|
Asadi N, Olson IR, Obradovic Z. A transformer model for learning spatiotemporal contextual representation in fMRI data. Netw Neurosci 2023; 7:22-47. [PMID: 37334006 PMCID: PMC10270708 DOI: 10.1162/netn_a_00281] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Accepted: 09/26/2022] [Indexed: 09/24/2023] Open
Abstract
Representation learning is a core component in data-driven modeling of various complex phenomena. Learning a contextually informative representation can especially benefit the analysis of fMRI data because of the complexities and dynamic dependencies present in such datasets. In this work, we propose a framework based on transformer models to learn an embedding of the fMRI data by taking the spatiotemporal contextual information in the data into account. This approach takes the multivariate BOLD time series of the regions of the brain as well as their functional connectivity network simultaneously as the input to create a set of meaningful features that can in turn be used in various downstream tasks such as classification, feature extraction, and statistical analysis. The proposed spatiotemporal framework uses the attention mechanism as well as the graph convolution neural network to jointly inject the contextual information regarding the dynamics in time series data and their connectivity into the representation. We demonstrate the benefits of this framework by applying it to two resting-state fMRI datasets, and provide further discussion on various aspects and advantages of it over a number of other commonly adopted architectures.
Collapse
Affiliation(s)
- Nima Asadi
- Department of Computer and Information Sciences, College of Science and Technology, Temple University, Philadelphia, PA, USA
| | - Ingrid R. Olson
- Department of Psychology and Neuroscience, College of Liberal Arts, Temple University, Philadelphia, PA, USA
- Decision Neuroscience, College of Liberal Arts, Temple University, Philadelphia, PA, USA
| | - Zoran Obradovic
- Department of Computer and Information Sciences, College of Science and Technology, Temple University, Philadelphia, PA, USA
| |
Collapse
|
15
|
Goel R, Tse T, Smith LJ, Floren A, Naylor B, Williams MW, Salas R, Rizzo AS, Ress D. Framework for Accurate Classification of Self-Reported Stress From Multisession Functional MRI Data of Veterans With Posttraumatic Stress. CHRONIC STRESS (THOUSAND OAKS, CALIF.) 2023; 7:24705470231203655. [PMID: 37780807 PMCID: PMC10540591 DOI: 10.1177/24705470231203655] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/02/2023] [Accepted: 09/11/2023] [Indexed: 10/03/2023]
Abstract
Background: Posttraumatic stress disorder (PTSD) is a significant burden among combat Veterans returning from the wars in Iraq and Afghanistan. While empirically supported treatments have demonstrated reductions in PTSD symptomatology, there remains a need to improve treatment effectiveness. Functional magnetic resonance imaging (fMRI) neurofeedback has emerged as a possible treatment to ameliorate PTSD symptom severity. Virtual reality (VR) approaches have also shown promise in increasing treatment compliance and outcomes. To facilitate fMRI neurofeedback-associated therapies, it would be advantageous to accurately classify internal brain stress levels while Veterans are exposed to trauma-associated VR imagery. Methods: Across 2 sessions, we used fMRI to collect neural responses to trauma-associated VR-like stimuli among male combat Veterans with PTSD symptoms (N = 8). Veterans reported their self-perceived stress level on a scale from 1 to 8 every 15 s throughout the fMRI sessions. In our proposed framework, we precisely sample the fMRI data on cortical gray matter, blurring the data along the gray-matter manifold to reduce noise and dimensionality while preserving maximum neural information. Then, we independently applied 3 machine learning (ML) algorithms to this fMRI data collected across 2 sessions, separately for each Veteran, to build individualized ML models that predicted their internal brain states (self-reported stress responses). Results: We accurately classified the 8-class self-reported stress responses with a mean (± standard error) root mean square error of 0.6 (± 0.1) across all Veterans using the best ML approach. Conclusions: The findings demonstrate the predictive ability of ML algorithms applied to whole-brain cortical fMRI data collected during individual Veteran sessions. The framework we have developed to preprocess whole-brain cortical fMRI data and train ML models across sessions would provide a valuable tool to enable individualized real-time fMRI neurofeedback during VR-like exposure therapy for PTSD.
Collapse
Affiliation(s)
- Rahul Goel
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA
| | - Teresa Tse
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA
| | - Lia J. Smith
- Department of Psychology, University of Houston, Houston, TX, USA
- Center for Translational Research on Inflammatory Diseases, Michael E. DeBakey VA Medical Center, Houston, TX, USA
| | - Andrew Floren
- Department of Electrical and Computer Engineering, University of Texas at Austin, Austin, TX, USA
| | - Bruce Naylor
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA
- Department of Electrical and Computer Engineering, University of Texas at Austin, Austin, TX, USA
| | - M. Wright Williams
- Center for Translational Research on Inflammatory Diseases, Michael E. DeBakey VA Medical Center, Houston, TX, USA
- Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
| | - Ramiro Salas
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA
- Center for Translational Research on Inflammatory Diseases, Michael E. DeBakey VA Medical Center, Houston, TX, USA
- Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
- The Menninger Clinic, Houston, TX, USA
| | - Albert S. Rizzo
- Institute for Creative Technologies, University of Southern California, Los Angeles, CA, USA
| | - David Ress
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA
| |
Collapse
|
16
|
Yu S, Shi E, Wang R, Zhao S, Liu T, Jiang X, Zhang S. A hybrid learning framework for fine-grained interpretation of brain spatiotemporal patterns during naturalistic functional magnetic resonance imaging. Front Hum Neurosci 2022; 16:944543. [PMID: 36248685 PMCID: PMC9563232 DOI: 10.3389/fnhum.2022.944543] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2022] [Accepted: 09/13/2022] [Indexed: 11/28/2022] Open
Abstract
Naturalistic stimuli, including movie, music, and speech, have been increasingly applied in the research of neuroimaging. Relative to a resting-state or single-task state, naturalistic stimuli can evoke more intense brain activities and have been proved to possess higher test–retest reliability, suggesting greater potential to study adaptive human brain function. In the current research, naturalistic functional magnetic resonance imaging (N-fMRI) has been a powerful tool to record brain states under naturalistic stimuli, and many efforts have been devoted to study the high-level semantic features from spatial or temporal representations via N-fMRI. However, integrating both spatial and temporal characteristics of brain activities for better interpreting the patterns under naturalistic stimuli is still underexplored. In this work, a novel hybrid learning framework that comprehensively investigates both the spatial (via Predictive Model) and the temporal [via convolutional neural network (CNN) model] characteristics of the brain is proposed. Specifically, to focus on certain relevant regions from the whole brain, regions of significance (ROS), which contain common spatial activation characteristics across individuals, are selected via the Predictive Model. Further, voxels of significance (VOS), whose signals contain significant temporal characteristics under naturalistic stimuli, are interpreted via one-dimensional CNN (1D-CNN) model. In this article, our proposed framework is applied onto the N-fMRI data during naturalistic classical/pop/speech audios stimuli. The promising performance is achieved via the Predictive Model to differentiate the different audio categories. Especially for distinguishing the classic and speech audios, the accuracy of classification is up to 92%. Moreover, spatial ROS and VOS are effectively obtained. Besides, temporal characteristics of the high-level semantic features are investigated on the frequency domain via convolution kernels of 1D-CNN model, and we effectively bridge the “semantic gap” between high-level semantic features of N-fMRI and low-level acoustic features of naturalistic audios in the frequency domain. Our results provide novel insights on characterizing spatiotemporal patterns of brain activities via N-fMRI and effectively explore the high-level semantic features under naturalistic stimuli, which will further benefit the understanding of the brain working mechanism and the advance of naturalistic stimuli clinical application.
Collapse
Affiliation(s)
- Sigang Yu
- Center for Brain and Brain-Inspired Computing Research, Department of Computer Science, Northwestern Polytechnical University, Xi’an, China
| | - Enze Shi
- Center for Brain and Brain-Inspired Computing Research, Department of Computer Science, Northwestern Polytechnical University, Xi’an, China
| | - Ruoyang Wang
- Center for Brain and Brain-Inspired Computing Research, Department of Computer Science, Northwestern Polytechnical University, Xi’an, China
| | - Shijie Zhao
- School of Automation, Northwestern Polytechnical University, Xi’an, China
| | - Tianming Liu
- Cortical Architecture Imaging and Discovery Lab, Department of Computer Science and Bioimaging Research Center, The University of Georgia, Athens, GA, United States
| | - Xi Jiang
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Shu Zhang
- Center for Brain and Brain-Inspired Computing Research, Department of Computer Science, Northwestern Polytechnical University, Xi’an, China
- *Correspondence: Shu Zhang,
| |
Collapse
|
17
|
A deep graph neural network architecture for modelling spatio-temporal dynamics in resting-state functional MRI data. Med Image Anal 2022; 79:102471. [PMID: 35580429 DOI: 10.1016/j.media.2022.102471] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2020] [Revised: 04/11/2022] [Accepted: 05/02/2022] [Indexed: 11/23/2022]
Abstract
Resting-state functional magnetic resonance imaging (rs-fMRI) has been successfully employed to understand the organisation of the human brain. Typically, the brain is parcellated into regions of interest (ROIs) and modelled as a graph where each ROI represents a node and association measures between ROI-specific blood-oxygen-level-dependent (BOLD) time series are edges. Recently, graph neural networks (GNNs) have seen a surge in popularity due to their success in modelling unstructured relational data. The latest developments with GNNs, however, have not yet been fully exploited for the analysis of rs-fMRI data, particularly with regards to its spatio-temporal dynamics. In this paper, we present a novel deep neural network architecture which combines both GNNs and temporal convolutional networks (TCNs) in order to learn from both the spatial and temporal components of rs-fMRI data in an end-to-end fashion. In particular, this corresponds to intra-feature learning (i.e., learning temporal dynamics with TCNs) as well as inter-feature learning (i.e., leveraging interactions between ROI-wise dynamics with GNNs). We evaluate our model with an ablation study using 35,159 samples from the UK Biobank rs-fMRI database, as well as in the smaller Human Connectome Project (HCP) dataset, both in a unimodal and in a multimodal fashion. We also demonstrate that out architecture contains explainability-related features which easily map to realistic neurobiological insights. We suggest that this model could lay the groundwork for future deep learning architectures focused on leveraging the inherently and inextricably spatio-temporal nature of rs-fMRI data.
Collapse
|
18
|
De Stefano N, Battaglini M, Pareto D, Cortese R, Zhang J, Oesingmann N, Prados F, Rocca MA, Valsasina P, Vrenken H, Gandini Wheeler-Kingshott CAM, Filippi M, Barkhof F, Rovira À. MAGNIMS recommendations for harmonization of MRI data in MS multicenter studies. Neuroimage Clin 2022; 34:102972. [PMID: 35245791 PMCID: PMC8892169 DOI: 10.1016/j.nicl.2022.102972] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Revised: 02/22/2022] [Accepted: 02/23/2022] [Indexed: 11/24/2022]
Abstract
Sharing data from cooperative studies is essential to develop new biomarkers in MS. Differences in MRI acquisition, analysis, storage represent a substantial constraint. We review the state of the art and developments in the harmonization of MRI. We provide recommendations to harmonize large MRI datasets in the MS field.
There is an increasing need of sharing harmonized data from large, cooperative studies as this is essential to develop new diagnostic and prognostic biomarkers. In the field of multiple sclerosis (MS), the issue has become of paramount importance due to the need to translate into the clinical setting some of the most recent MRI achievements. However, differences in MRI acquisition parameters, image analysis and data storage across sites, with their potential bias, represent a substantial constraint. This review focuses on the state of the art, recent technical advances, and desirable future developments of the harmonization of acquisition, analysis and storage of large-scale multicentre MRI data of MS cohorts. Huge efforts are currently being made to achieve all the requirements needed to provide harmonized MRI datasets in the MS field, as proper management of large imaging datasets is one of our greatest opportunities and challenges in the coming years. Recommendations based on these achievements will be provided here. Despite the advances that have been made, the complexity of these tasks requires further research by specialized academical centres, with dedicated technical and human resources. Such collective efforts involving different professional figures are of crucial importance to offer to MS patients a personalised management while minimizing consumption of resources.
Collapse
Affiliation(s)
- Nicola De Stefano
- Department of Medicine, Surgery and Neuroscience, University of Siena, Siena, Italy.
| | - Marco Battaglini
- Department of Medicine, Surgery and Neuroscience, University of Siena, Siena, Italy
| | - Deborah Pareto
- Section of Neuroradiology, Department of Radiology, Hospital Universitari Vall d'Hebron, Universitat Autònoma de Barcelona, Barcelona, Spain
| | - Rosa Cortese
- Department of Medicine, Surgery and Neuroscience, University of Siena, Siena, Italy
| | - Jian Zhang
- Department of Medicine, Surgery and Neuroscience, University of Siena, Siena, Italy
| | | | - Ferran Prados
- Queen Square Multiple Sclerosis Centre, Department of Neuroinflammation, UCL Queen Square Institute of Neurology, University College London, London, United Kingdom; Center for Medical Imaging Computing, Medical Physics and Biomedical Engineering, UCL, London, WC1V 6LJ, United Kingdom; e-Health Center, Universitat Oberta de Catalunya, Barcelona, Spain
| | - Maria A Rocca
- Neuroimaging Research Unit, Division of Neuroscience, and Neurology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy; Neurology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy; Vita-Salute San Raffaele University, Milan, Italy
| | - Paola Valsasina
- Neuroimaging Research Unit, Division of Neuroscience, and Neurology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Hugo Vrenken
- Amsterdam Neuroscience, MS Center Amsterdam, Department of Radiology and Nuclear Medicine, Amsterdam UMC, Amsterdam, Netherlands
| | - Claudia A M Gandini Wheeler-Kingshott
- Queen Square Multiple Sclerosis Centre, Department of Neuroinflammation, UCL Queen Square Institute of Neurology, University College London, London, United Kingdom; Brain MRI 3T Research Center, C. Mondino National Neurological Institute, Pavia, Italy; Department of Brain and Behavioural Sciences, University of Pavia, Pavia, Italy
| | - Massimo Filippi
- Neuroimaging Research Unit, Division of Neuroscience, and Neurology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy; Neurology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy; Vita-Salute San Raffaele University, Milan, Italy; Neurorehabilitation Unit, and Neurophysiology Service, IRCCS San Raffaele Scientific Institute, Milan, Italy; Neurophysiology Service, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Frederik Barkhof
- Queen Square Multiple Sclerosis Centre, Department of Neuroinflammation, UCL Queen Square Institute of Neurology, University College London, London, United Kingdom; Center for Medical Imaging Computing, Medical Physics and Biomedical Engineering, UCL, London, WC1V 6LJ, United Kingdom; Amsterdam Neuroscience, MS Center Amsterdam, Department of Radiology and Nuclear Medicine, Amsterdam UMC, Amsterdam, Netherlands
| | - Àlex Rovira
- Section of Neuroradiology, Department of Radiology, Hospital Universitari Vall d'Hebron, Universitat Autònoma de Barcelona, Barcelona, Spain
| |
Collapse
|
19
|
Qureshi MB, Azad L, Qureshi MS, Aslam S, Aljarbouh A, Fayaz M. Brain Decoding Using fMRI Images for Multiple Subjects through Deep Learning. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2022; 2022:1124927. [PMID: 35273647 PMCID: PMC8904097 DOI: 10.1155/2022/1124927] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/08/2022] [Revised: 02/06/2022] [Accepted: 02/11/2022] [Indexed: 12/02/2022]
Abstract
Substantial information related to human cerebral conditions can be decoded through various noninvasive evaluating techniques like fMRI. Exploration of the neuronal activity of the human brain can divulge the thoughts of a person like what the subject is perceiving, thinking, or visualizing. Furthermore, deep learning techniques can be used to decode the multifaceted patterns of the brain in response to external stimuli. Existing techniques are capable of exploring and classifying the thoughts of the human subject acquired by the fMRI imaging data. fMRI images are the volumetric imaging scans which are highly dimensional as well as require a lot of time for training when fed as an input in the deep learning network. However, the hassle for more efficient learning of highly dimensional high-level features in less training time and accurate interpretation of the brain voxels with less misclassification error is needed. In this research, we propose an improved CNN technique where features will be functionally aligned. The optimal features will be selected after dimensionality reduction. The highly dimensional feature vector will be transformed into low dimensional space for dimensionality reduction through autoadjusted weights and combination of best activation functions. Furthermore, we solve the problem of increased training time by using Swish activation function, making it denser and increasing efficiency of the model in less training time. Finally, the experimental results are evaluated and compared with other classifiers which demonstrated the supremacy of the proposed model in terms of accuracy.
Collapse
Affiliation(s)
- Muhammad Bilal Qureshi
- Department of Computer Science & IT, University of Lakki Marwat, Lakki Marwat 28420, KPK, Pakistan
| | - Laraib Azad
- Department of Computer Science, Shaheed Zulfikar Ali Bhutto Institute of Science and Technology, Islamabad 44000, Pakistan
| | - Muhammad Shuaib Qureshi
- Department of Computer Science, School of Arts and Sciences, University of Central Asia, Kyrgyzstan
| | - Sheraz Aslam
- Department of Electrical Engineering, Computer Engineering, and Informatics, Cyprus University of Technology, Cyprus
| | - Ayman Aljarbouh
- Department of Computer Science, School of Arts and Sciences, University of Central Asia, Kyrgyzstan
| | - Muhammad Fayaz
- Department of Computer Science, School of Arts and Sciences, University of Central Asia, Kyrgyzstan
| |
Collapse
|
20
|
Luo CY, Pearson P, Xu G, Rich SM. A Computer Vision-Based Approach for Tick Identification Using Deep Learning Models. INSECTS 2022; 13:116. [PMID: 35206690 PMCID: PMC8879515 DOI: 10.3390/insects13020116] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/06/2021] [Revised: 01/18/2022] [Accepted: 01/19/2022] [Indexed: 12/21/2022]
Abstract
A wide range of pathogens, such as bacteria, viruses, and parasites can be transmitted by ticks and can cause diseases, such as Lyme disease, anaplasmosis, or Rocky Mountain spotted fever. Landscape and climate changes are driving the geographic range expansion of important tick species. The morphological identification of ticks is critical for the assessment of disease risk; however, this process is time-consuming, costly, and requires qualified taxonomic specialists. To address this issue, we constructed a tick identification tool that can differentiate the most encountered human-biting ticks, Amblyomma americanum, Dermacentor variabilis, and Ixodes scapularis, by implementing artificial intelligence methods with deep learning algorithms. Many convolutional neural network (CNN) models (such as VGG, ResNet, or Inception) have been used for image recognition purposes but it is still a very limited application in the use of tick identification. Here, we describe the modified CNN-based models which were trained using a large-scale molecularly verified dataset to identify tick species. The best CNN model achieved a 99.5% accuracy on the test set. These results demonstrate that a computer vision system is a potential alternative tool to help in prescreening ticks for identification, an earlier diagnosis of disease risk, and, as such, could be a valuable resource for health professionals.
Collapse
Affiliation(s)
| | | | | | - Stephen M. Rich
- Department of Microbiology, University of Massachusetts, Amherst, MA 01003, USA; (C.-Y.L.); (P.P.); (G.X.)
| |
Collapse
|
21
|
|
22
|
Kazazian K, Norton L, Laforge G, Abdalmalak A, Gofton TE, Debicki D, Slessarev M, Hollywood S, Lawrence KS, Owen AM. Improving Diagnosis and Prognosis in Acute Severe Brain Injury: A Multimodal Imaging Protocol. Front Neurol 2021; 12:757219. [PMID: 34938260 PMCID: PMC8685572 DOI: 10.3389/fneur.2021.757219] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2021] [Accepted: 11/10/2021] [Indexed: 12/12/2022] Open
Abstract
Multi-modal neuroimaging techniques have the potential to dramatically improve the diagnosis of the level consciousness and prognostication of neurological outcome for patients with severe brain injury in the intensive care unit (ICU). This protocol describes a study that will utilize functional Magnetic Resonance Imaging (fMRI), electroencephalography (EEG), and functional Near Infrared Spectroscopy (fNIRS) to measure and map the brain activity of acute critically ill patients. Our goal is to investigate whether these modalities can provide objective and quantifiable indicators of good neurological outcome and reliably detect conscious awareness. To this end, we will conduct a prospective longitudinal cohort study to validate the prognostic and diagnostic utility of neuroimaging techniques in the ICU. We will recruit 350 individuals from two ICUs over the course of 7 years. Participants will undergo fMRI, EEG, and fNIRS testing several times over the first 10 days of care to assess for residual cognitive function and evidence of covert awareness. Patients who regain behavioral awareness will be asked to complete web-based neurocognitive tests for 1 year, as well as return for follow up neuroimaging to determine which acute imaging features are most predictive of cognitive and functional recovery. Ultimately, multi-modal neuroimaging techniques may improve the clinical assessments of patients' level of consciousness, aid in the prediction of outcome, and facilitate efforts to find interventional methods that improve recovery and quality of life.
Collapse
Affiliation(s)
- Karnig Kazazian
- Graduate Program in Neuroscience, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada.,Brain and Mind Institute, Western University, London, ON, Canada
| | - Loretta Norton
- Department of Psychology, King's University College at Western University, London, ON, Canada
| | - Geoffrey Laforge
- Brain and Mind Institute, Western University, London, ON, Canada.,Department of Psychology, Western University, London, ON, Canada
| | - Androu Abdalmalak
- Brain and Mind Institute, Western University, London, ON, Canada.,Department of Physiology and Pharmacology, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Teneille E Gofton
- Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Derek Debicki
- Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Marat Slessarev
- Department of Medicine, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Sarah Hollywood
- Department of Medicine, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Keith St Lawrence
- Department of Medical Biophysics, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Adrian M Owen
- Brain and Mind Institute, Western University, London, ON, Canada.,Department of Psychology, Western University, London, ON, Canada.,Department of Physiology and Pharmacology, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| |
Collapse
|
23
|
Fujibayashi D, Sakaguchi H, Ardakani I, Okuno A. Nonlinear registration as an effective preprocessing technique for Deep learning based classification of disease. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3245-3250. [PMID: 34891933 DOI: 10.1109/embc46164.2021.9631044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
A number of machine learning (ML), and particularly in recent years, deep learning (DL) approaches have been proposed for automatic classification of Alzheimer's disease (AD) using brain structural magnetic resonance imaging (MRI) data. However, the data available are limited in the case of this specific disease. Training a DL model with a large number of feature parameters on a small dataset of MRI scans will likely lead to overfitting. Overfitting reduces the generality and efficiency of the model. In this study, we show that a traditional nonlinear transformation from native space to template space, as a preprocessing stage, is effective in reducing overfitting through the reduction of spatial variations in the input data. To evaluate this effectiveness, we compare two different pre-processing approaches for DL-based AD classification task: (1) affine registration and (2) nonlinear diffeomorphic anatomical registration using exponentiated Lie algebra (DARTEL). The results show that the accuracy of the nonlinear registration based approach is much higher than the affine registration based approach. Furthermore, from the classification results obtained with noisy images, DARTEL is less susceptible to noise than affine registration. In summary, our experimental results suggest that nonlinear transformation is a preferable preprocessing step for training DL-based AD classification models on limited size datasets.
Collapse
|
24
|
A CNN Deep Local and Global ASD Classification Approach with Continuous Wavelet Transform Using Task-Based FMRI. SENSORS 2021; 21:s21175822. [PMID: 34502710 PMCID: PMC8433893 DOI: 10.3390/s21175822] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/22/2021] [Revised: 08/24/2021] [Accepted: 08/25/2021] [Indexed: 11/17/2022]
Abstract
Autism spectrum disorder (ASD) is a neurodegenerative disorder characterized by lingual and social disabilities. The autism diagnostic observation schedule is the current gold standard for ASD diagnosis. Developing objective computer aided technologies for ASD diagnosis with the utilization of brain imaging modalities and machine learning is one of main tracks in current studies to understand autism. Task-based fMRI demonstrates the functional activation in the brain by measuring blood oxygen level-dependent (BOLD) variations in response to certain tasks. It is believed to hold discriminant features for autism. A novel computer aided diagnosis (CAD) framework is proposed to classify 50 ASD and 50 typically developed toddlers with the adoption of CNN deep networks. The CAD system includes both local and global diagnosis in a response to speech task. Spatial dimensionality reduction with region of interest selection and clustering has been utilized. In addition, the proposed framework performs discriminant feature extraction with continuous wavelet transform. Local diagnosis on cingulate gyri, superior temporal gyrus, primary auditory cortex and angular gyrus achieves accuracies ranging between 71% and 80% with a four-fold cross validation technique. The fused global diagnosis achieves an accuracy of 86% with 82% sensitivity, 92% specificity. A brain map indicating ASD severity level for each brain area is created, which contributes to personalized diagnosis and treatment plans.
Collapse
|
25
|
Haweel R, Shalaby A, Mahmoud A, Seada N, Ghoniemy S, Ghazal M, Casanova MF, Barnes GN, El-Baz A. A robust DWT-CNN-based CAD system for early diagnosis of autism using task-based fMRI. Med Phys 2021; 48:2315-2326. [PMID: 33378589 DOI: 10.1002/mp.14692] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2020] [Revised: 11/27/2020] [Accepted: 12/17/2020] [Indexed: 02/06/2023] Open
Abstract
PURPOSE Task-based fMRI (TfMRI) is a diagnostic imaging modality for observing the effects of a disease or other condition on the functional activity of the brain. Autism spectrum disorder (ASD) is a pervasive developmental disorder associated with impairments in social and linguistic abilities. Machine learning algorithms have been widely utilized for brain imaging aiming for objective ASD diagnostics. Recently, deep learning methods have been gaining more attention for fMRI classification. The goal of this paper is to develop a convolutional neural network (CNN)-based framework to help in global diagnosis of ASD using TfMRI data that are collected from a response to speech experiment. METHODS To achieve this goal, the proposed framework adopts a novel imaging marker integrating both spatial and temporal information that are related to the functional activity of the brain. The developed pipeline consists of three main components. In the first step, the collected TfMRI data are preprocessed and parcellated using the Harvard-Oxford probabilistic atlas included with the fMRIB Software Library (FSL). Second, a group analysis using FSL is performed between ASD and typically developing (TD) children to identify significantly activated brain areas in response to the speech task. In order to reduce brain spatial dimensionality, a K-means clustering technique is performed on such significant brain areas. Informative blood oxygen level-dependent (BOLD) signals are extracted from each cluster. A compression step for each extracted BOLD signal using discrete wavelet transform (DWT) has been proposed. The adopted wavelets are similar to the expected hemodynamic response which enables DWT to compress the BOLD signal while highlighting its activation information. Finally, a deep learning 2D CNN network is used to classify the patients as ASD or TD based on extracted features from the previous step. RESULTS Preliminary results on 100 TfMRI dataset (50 ASD, 50 TD) obtain 80% correct global classification using tenfold cross validation (with sensitivity = 84%, specificity = 76%). CONCLUSION The experimental results show the high accuracy of the proposed framework and hold promise for the presented framework as a helpful adjunct to currently used ASD diagnostic tools.
Collapse
Affiliation(s)
- Reem Haweel
- BioImaging Laboratory, Department of Bioengineering, University of Louisville, KY, 40208, USA
- Computer Systems Department, Faculty of Computer and Information Sciences, University of Ain Shams, Cairo, 11566, Egypt
| | - Ahmed Shalaby
- BioImaging Laboratory, Department of Bioengineering, University of Louisville, KY, 40208, USA
| | - Ali Mahmoud
- BioImaging Laboratory, Department of Bioengineering, University of Louisville, KY, 40208, USA
| | - Noha Seada
- Computer Systems Department, Faculty of Computer and Information Sciences, University of Ain Shams, Cairo, 11566, Egypt
| | - Said Ghoniemy
- Computer Systems Department, Faculty of Computer and Information Sciences, University of Ain Shams, Cairo, 11566, Egypt
| | - Mohammed Ghazal
- Department of Electrical and Computer Engineering, Abu Dhabi University, Abu Dhabi, 59911, UAE
| | - Manuel F Casanova
- Biomedical Sciences, University of South Carolina, Greenville, SC, 29607, USA
| | - Gregory N Barnes
- Department of Neurology, University of Louisville, Louisville, KY, 40208, USA
| | - Ayman El-Baz
- Department of Bioengineering, University of Louisville, KY, 40208, USA
| |
Collapse
|
26
|
Jiang H, Cao P, Xu M, Yang J, Zaiane O. Hi-GCN: A hierarchical graph convolution network for graph embedding learning of brain network and brain disorders prediction. Comput Biol Med 2020; 127:104096. [DOI: 10.1016/j.compbiomed.2020.104096] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2020] [Revised: 10/24/2020] [Accepted: 10/24/2020] [Indexed: 02/06/2023]
|
27
|
Zhang X, Yao L, Wang X, Monaghan JJM, Mcalpine D, Zhang Y. A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers. J Neural Eng 2020; 18. [PMID: 33171452 DOI: 10.1088/1741-2552/abc902] [Citation(s) in RCA: 58] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2020] [Accepted: 11/10/2020] [Indexed: 12/25/2022]
Abstract
Brain signals refer to the biometric information collected from the human brain. The research on brain signals aims to discover the underlying neurological or physical status of the individuals by signal decoding. The emerging deep learning techniques have improved the study of brain signals significantly in recent years. In this work, we first present a taxonomy of non-invasive brain signals and the basics of deep learning algorithms. Then, we provide a comprehensive survey of the frontiers of applying deep learning for non-invasive brain signals analysis, by summarizing a large number of recent publications. Moreover, upon the deep learning-powered brain signal studies, we report the potential real-world applications which benefit not only disabled people but also normal individuals. Finally, we discuss the opening challenges and future directions.
Collapse
Affiliation(s)
- Xiang Zhang
- Harvard University, Cambridge, Massachusetts, UNITED STATES
| | - Lina Yao
- University of New South Wales, Sydney, New South Wales, AUSTRALIA
| | - Xianzhi Wang
- Faculty of Engineering and IT, University of Technology Sydney, 81 Broadway, Ultimo, Sydney, New South Wales, 2007, AUSTRALIA
| | | | - David Mcalpine
- Macquarie University, Sydney, New South Wales, AUSTRALIA
| | - Yu Zhang
- Stanford University, Stanford, California, 94305-6104, UNITED STATES
| |
Collapse
|
28
|
Honnorat N, Pfefferbaum A, Sullivan EV, Pohl KM. Deep Parametric Mixtures for Modeling the Functional Connectome. PREDICTIVE INTELLIGENCE IN MEDICINE. PRIME (WORKSHOP) 2020; 12329:133-143. [PMID: 33163995 PMCID: PMC7643933 DOI: 10.1007/978-3-030-59354-4_13] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Functional connectivity between brain regions is often estimated by correlating brain activity measured by resting-state fMRI in those regions. The impact of factors (e.g, disorder or substance use) are then modeled by their effects on these correlation matrices in individuals. A crucial step in better understanding their effects on brain function could lie in estimating connectomes, which encode the correlation matrices across subjects. Connectomes are mostly estimated by creating a single average for a specific cohort, which works well for binary factors (such as sex) but is unsuited for continuous ones, such as alcohol consumption. Alternative approaches based on regression methods usually model each pair of regions separately, which generally produces incoherent connectomes as correlations across multiple regions contradict each other. In this work, we address these issues by introducing a deep learning model that predicts connectomes based on factor values. The predictions are defined on a simplex spanned across correlation matrices, whose convex combination guarantees that the deep learning model generates well-formed connectomes. We present an efficient method for creating these simplexes and improve the accuracy of the entire analysis by defining loss functions based on robust norms. We show that our deep learning approach is able to produce accurate models on challenging synthetic data. Furthermore, we apply the approach to the resting-state fMRI scans of 281 subjects to study the effect of sex, alcohol, and HIV on brain function.
Collapse
Affiliation(s)
| | - Adolf Pfefferbaum
- Center for Health Sciences, SRI International, Menlo Park, CA
- Department of Psychiatry & Behavioral Sciences, Stanford University School of Medicine, Stanford, CA
| | - Edith V Sullivan
- Department of Psychiatry & Behavioral Sciences, Stanford University School of Medicine, Stanford, CA
| | - Kilian M Pohl
- Center for Health Sciences, SRI International, Menlo Park, CA
- Department of Psychiatry & Behavioral Sciences, Stanford University School of Medicine, Stanford, CA
| |
Collapse
|
29
|
Sobczak F, He Y, Sejnowski TJ, Yu X. Predicting the fMRI Signal Fluctuation with Recurrent Neural Networks Trained on Vascular Network Dynamics. Cereb Cortex 2020; 31:826-844. [PMID: 32940658 PMCID: PMC7906791 DOI: 10.1093/cercor/bhaa260] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Revised: 07/19/2020] [Accepted: 08/12/2020] [Indexed: 02/06/2023] Open
Abstract
Resting-state functional MRI (rs-fMRI) studies have revealed specific low-frequency hemodynamic signal fluctuations (<0.1 Hz) in the brain, which could be related to neuronal oscillations through the neurovascular coupling mechanism. Given the vascular origin of the fMRI signal, it remains challenging to separate the neural correlates of global rs-fMRI signal fluctuations from other confounding sources. However, the slow-oscillation detected from individual vessels by single-vessel fMRI presents strong correlation to neural oscillations. Here, we use recurrent neural networks (RNNs) to predict the future temporal evolution of the rs-fMRI slow oscillation from both rodent and human brains. The RNNs trained with vessel-specific rs-fMRI signals encode the unique brain oscillatory dynamic feature, presenting more effective prediction than the conventional autoregressive model. This RNN-based predictive modeling of rs-fMRI datasets from the Human Connectome Project (HCP) reveals brain state-specific characteristics, demonstrating an inverse relationship between the global rs-fMRI signal fluctuation with the internal default-mode network (DMN) correlation. The RNN prediction method presents a unique data-driven encoding scheme to specify potential brain state differences based on the global fMRI signal fluctuation, but not solely dependent on the global variance.
Collapse
Affiliation(s)
- Filip Sobczak
- Translational Neuroimaging and Neural Control Group, High Field Magnetic Resonance Department, Max Planck Institute for Biological Cybernetics, 72076 Tuebingen, Germany.,Graduate Training Centre of Neuroscience, International Max Planck Research School, University of Tuebingen, 72074 Tuebingen, Germany
| | - Yi He
- Translational Neuroimaging and Neural Control Group, High Field Magnetic Resonance Department, Max Planck Institute for Biological Cybernetics, 72076 Tuebingen, Germany.,Danish Research Centre for Magnetic Resonance, 2650, Hvidovre, Denmark
| | - Terrence J Sejnowski
- Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute for Biological Studies, La Jolla, CA 92037, USA.,Division of Biological Sciences, University of California, San Diego, La Jolla, CA 92093, USA
| | - Xin Yu
- Translational Neuroimaging and Neural Control Group, High Field Magnetic Resonance Department, Max Planck Institute for Biological Cybernetics, 72076 Tuebingen, Germany.,Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, Charlestown, MA 02129, USA
| |
Collapse
|
30
|
Vu H, Kim HC, Jung M, Lee JH. fMRI volume classification using a 3D convolutional neural network robust to shifted and scaled neuronal activations. Neuroimage 2020; 223:117328. [PMID: 32896633 DOI: 10.1016/j.neuroimage.2020.117328] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2019] [Revised: 07/16/2020] [Accepted: 08/20/2020] [Indexed: 12/22/2022] Open
Abstract
Deep-learning methods based on deep neural networks (DNNs) have recently been successfully utilized in the analysis of neuroimaging data. A convolutional neural network (CNN) is a type of DNN that employs a convolution kernel that covers a local area of the input sample and moves across the sample to provide a feature map for the subsequent layers. In our study, we hypothesized that a 3D-CNN model with down-sampling operations such as pooling and/or stride would have the ability to extract robust feature maps from the shifted and scaled neuronal activations in a single functional MRI (fMRI) volume for the classification of task information associated with that volume. Thus, the 3D-CNN model would be able to ameliorate the potential misalignment of neuronal activations and over-/under-activation in local brain regions caused by imperfections in spatial alignment algorithms, confounded by variability in blood-oxygenation-level-dependent (BOLD) responses across sessions and/or subjects. To this end, the fMRI volumes acquired from four sensorimotor tasks (left-hand clenching, right-hand clenching, auditory attention, and visual stimulation) were used as input for our 3D-CNN model to classify task information using a single fMRI volume. The classification performance of the 3D-CNN was systematically evaluated using fMRI volumes obtained from various minimal preprocessing scenarios applied to raw fMRI volumes that excluded spatial normalization to a template and those obtained from full preprocessing that included spatial normalization. Alternative classifier models such as the 1D fully connected DNN (1D-fcDNN) and support vector machine (SVM) were also used for comparison. The classification performance was also assessed for several k-fold cross-validation (CV) schemes, including leave-one-subject-out CV (LOOCV). Overall, the classification results of the 3D-CNN model were superior to that of the 1D-fcDNN and SVM models. When using the fully-processed fMRI volumes with LOOCV, the mean error rates (± the standard error of the mean) for the 3D-CNN, 1D-fcDNN, and SVM models were 2.1% (± 0.9), 3.1% (± 1.2), and 4.1% (± 1.5), respectively (p = 0.041 from a one-way ANOVA). The error rates for 3-fold CV were higher (2.4% ± 1.0, 4.2% ± 1.3, and 10.1% ± 2.0; p < 0.0003 from a one-way ANOVA). The mean error rates also increased considerably using the raw fMRI 3D volume data without preprocessing (26.2% for the 3D-CNN, 75.0% for the 1D-fcDNN, and 75.0% for the SVM). Furthermore, the ability of the pre-trained 3D-CNN model to handle shifted and scaled neuronal activations was demonstrated in an online scenario for five-class classification (i.e., four sensorimotor tasks and the resting state) using the real-time fMRI of three participants. The resulting classification accuracy was 78.5% (± 1.4), 26.7% (± 5.9), and 21.5% (± 3.1) for the 3D-CNN, 1D-fcDNN, and SVM models, respectively. The superior performance of the 3D-CNN compared to the 1D-fcDNN was verified by analyzing the resulting feature maps and convolution filters that handled the shifted and scaled neuronal activations and by utilizing an independent public dataset from the Human Connectome Project.
Collapse
Affiliation(s)
- Hanh Vu
- Department of Brain and Cognitive Engineering, Korea University, Anam-ro 145, Seongbuk-gu, Seoul 02841, Republic of Korea
| | - Hyun-Chul Kim
- Department of Brain and Cognitive Engineering, Korea University, Anam-ro 145, Seongbuk-gu, Seoul 02841, Republic of Korea
| | - Minyoung Jung
- Department of Brain and Cognitive Engineering, Korea University, Anam-ro 145, Seongbuk-gu, Seoul 02841, Republic of Korea
| | - Jong-Hwan Lee
- Department of Brain and Cognitive Engineering, Korea University, Anam-ro 145, Seongbuk-gu, Seoul 02841, Republic of Korea.
| |
Collapse
|
31
|
Kulkarni PH, Merchant SN, Awate SP. R-fMRI reconstruction from k-t undersampled data using a subject-invariant dictionary model and VB-EM with nested minorization. Med Image Anal 2020; 65:101752. [PMID: 32623273 DOI: 10.1016/j.media.2020.101752] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2019] [Revised: 06/01/2020] [Accepted: 06/04/2020] [Indexed: 11/30/2022]
Abstract
Higher spatial resolution in resting-state functional magnetic resonance imaging (R-fMRI) can give reliable information about the functional networks in the cerebral cortex. Typical methods can achieve higher spatial or temporal resolution by speeding up scans using either (i) complex pulse-sequence designs or (ii) k-space undersampling coupled with priors on the signal. We propose to undersample the R-fMRI acquisition in k-space and time to speedup scans in order to improve spatial resolution. We propose a novel model-based R-fMRI reconstruction framework using a robust, subject-invariant, spatially regularized dictionary prior on the signal. Furthermore, we propose a novel inference framework based on variational Bayesian expectation maximization with nested minorization (VB-EM-NM). Our inference framework allows us to provide an estimate of uncertainty of the reconstruction, unlike typical reconstruction methods. Empirical evaluation of (i) simulated R-fMRI reconstruction and (ii) functional-network estimates from brain R-fMRI reconstructions demonstrate that our framework improves over the state of the art, and, additionally, enables significantly higher spatial resolution.
Collapse
Affiliation(s)
- Prachi H Kulkarni
- Electrical Engineering (EE) Department, Indian Institute of Technology (IIT) Bombay, Mumbai, India.
| | - S N Merchant
- Electrical Engineering (EE) Department, Indian Institute of Technology (IIT) Bombay, Mumbai, India
| | - Suyash P Awate
- Computer Science and Engineering (CSE) Department, Indian Institute of Technology (IIT) Bombay, Mumbai, India
| |
Collapse
|
32
|
Hu Z, Liu G, Dong Q, Niu H. Applications of Resting-State fNIRS in the Developing Brain: A Review From the Connectome Perspective. Front Neurosci 2020; 14:476. [PMID: 32581671 PMCID: PMC7284109 DOI: 10.3389/fnins.2020.00476] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2019] [Accepted: 04/16/2020] [Indexed: 12/21/2022] Open
Abstract
Early brain development from infancy through childhood is closely related to the development of cognition and behavior in later life. Human brain connectome is a novel framework for describing topological organization of the developing brain. Resting-state functional near-infrared spectroscopy (fNIRS), with a natural scanning environment, low cost, and high portability, is considered as an emerging imaging technique and has shown valuable potential in exploring brain network architecture and its changes during the development. Here, we review the recent advances involving typical and atypical development of the brain connectome from neonates to children using resting-state fNIRS imaging. This review highlights that the combination of brain connectome and resting-state fNIRS imaging offers a promising framework for understanding human brain development.
Collapse
Affiliation(s)
- Zhishan Hu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
| | - Guangfang Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
| | - Qi Dong
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
| | - Haijing Niu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
| |
Collapse
|
33
|
Wen J, Thibeau-Sutre E, Diaz-Melo M, Samper-González J, Routier A, Bottani S, Dormont D, Durrleman S, Burgos N, Colliot O. Convolutional neural networks for classification of Alzheimer's disease: Overview and reproducible evaluation. Med Image Anal 2020; 63:101694. [PMID: 32417716 DOI: 10.1016/j.media.2020.101694] [Citation(s) in RCA: 204] [Impact Index Per Article: 51.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2019] [Revised: 03/23/2020] [Accepted: 03/27/2020] [Indexed: 10/24/2022]
Abstract
Numerous machine learning (ML) approaches have been proposed for automatic classification of Alzheimer's disease (AD) from brain imaging data. In particular, over 30 papers have proposed to use convolutional neural networks (CNN) for AD classification from anatomical MRI. However, the classification performance is difficult to compare across studies due to variations in components such as participant selection, image preprocessing or validation procedure. Moreover, these studies are hardly reproducible because their frameworks are not publicly accessible and because implementation details are lacking. Lastly, some of these papers may report a biased performance due to inadequate or unclear validation or model selection procedures. In the present work, we aim to address these limitations through three main contributions. First, we performed a systematic literature review. We identified four main types of approaches: i) 2D slice-level, ii) 3D patch-level, iii) ROI-based and iv) 3D subject-level CNN. Moreover, we found that more than half of the surveyed papers may have suffered from data leakage and thus reported biased performance. Our second contribution is the extension of our open-source framework for classification of AD using CNN and T1-weighted MRI. The framework comprises previously developed tools to automatically convert ADNI, AIBL and OASIS data into the BIDS standard, and a modular set of image preprocessing procedures, classification architectures and evaluation procedures dedicated to deep learning. Finally, we used this framework to rigorously compare different CNN architectures. The data was split into training/validation/test sets at the very beginning and only the training/validation sets were used for model selection. To avoid any overfitting, the test sets were left untouched until the end of the peer-review process. Overall, the different 3D approaches (3D-subject, 3D-ROI, 3D-patch) achieved similar performances while that of the 2D slice approach was lower. Of note, the different CNN approaches did not perform better than a SVM with voxel-based features. The different approaches generalized well to similar populations but not to datasets with different inclusion criteria or demographical characteristics. All the code of the framework and the experiments is publicly available: general-purpose tools have been integrated into the Clinica software (www.clinica.run) and the paper-specific code is available at: https://github.com/aramis-lab/AD-DL.
Collapse
Affiliation(s)
- Junhao Wen
- Institut du Cerveau et de la Moelleépinière, ICM, Paris F-75013, France; SorbonneUniversité, ParisF-75013,France; Inserm, U 1127, Paris F-75013, France; CNRS, UMR 7225, Paris F-75013, France; Inria, Aramis project-team, Paris F-75013, France
| | - Elina Thibeau-Sutre
- Institut du Cerveau et de la Moelleépinière, ICM, Paris F-75013, France; SorbonneUniversité, ParisF-75013,France; Inserm, U 1127, Paris F-75013, France; CNRS, UMR 7225, Paris F-75013, France; Inria, Aramis project-team, Paris F-75013, France
| | - Mauricio Diaz-Melo
- Inria, Aramis project-team, Paris F-75013, France; Institut du Cerveau et de la Moelleépinière, ICM, Paris F-75013, France; SorbonneUniversité, ParisF-75013,France; Inserm, U 1127, Paris F-75013, France; CNRS, UMR 7225, Paris F-75013, France
| | - Jorge Samper-González
- Inria, Aramis project-team, Paris F-75013, France; Institut du Cerveau et de la Moelleépinière, ICM, Paris F-75013, France; SorbonneUniversité, ParisF-75013,France; Inserm, U 1127, Paris F-75013, France; CNRS, UMR 7225, Paris F-75013, France
| | - Alexandre Routier
- Inria, Aramis project-team, Paris F-75013, France; Institut du Cerveau et de la Moelleépinière, ICM, Paris F-75013, France; SorbonneUniversité, ParisF-75013,France; Inserm, U 1127, Paris F-75013, France; CNRS, UMR 7225, Paris F-75013, France
| | - Simona Bottani
- Inria, Aramis project-team, Paris F-75013, France; Institut du Cerveau et de la Moelleépinière, ICM, Paris F-75013, France; SorbonneUniversité, ParisF-75013,France; Inserm, U 1127, Paris F-75013, France; CNRS, UMR 7225, Paris F-75013, France
| | - Didier Dormont
- Inria, Aramis project-team, Paris F-75013, France; Institut du Cerveau et de la Moelleépinière, ICM, Paris F-75013, France; SorbonneUniversité, ParisF-75013,France; Inserm, U 1127, Paris F-75013, France; CNRS, UMR 7225, Paris F-75013, France; Department of Neuroradiology, AP-HP, Hôpital de la PitiéSalpêtrière, Paris F-75013, France
| | - Stanley Durrleman
- Inria, Aramis project-team, Paris F-75013, France; Institut du Cerveau et de la Moelleépinière, ICM, Paris F-75013, France; SorbonneUniversité, ParisF-75013,France; Inserm, U 1127, Paris F-75013, France; CNRS, UMR 7225, Paris F-75013, France
| | - Ninon Burgos
- Institut du Cerveau et de la Moelleépinière, ICM, Paris F-75013, France; SorbonneUniversité, ParisF-75013,France; Inserm, U 1127, Paris F-75013, France; CNRS, UMR 7225, Paris F-75013, France; Inria, Aramis project-team, Paris F-75013, France
| | - Olivier Colliot
- Institut du Cerveau et de la Moelleépinière, ICM, Paris F-75013, France; SorbonneUniversité, ParisF-75013,France; Inserm, U 1127, Paris F-75013, France; CNRS, UMR 7225, Paris F-75013, France; Inria, Aramis project-team, Paris F-75013, France; Department of Neuroradiology, AP-HP, Hôpital de la PitiéSalpêtrière, Paris F-75013, France; Department of Neurology, AP-HP, Hôpital de la PitiéSalpêtrière, Paris F-75013, France.
| | | | | |
Collapse
|
34
|
Takeuchi Y, Berényi A. Oscillotherapeutics - Time-targeted interventions in epilepsy and beyond. Neurosci Res 2020; 152:87-107. [PMID: 31954733 DOI: 10.1016/j.neures.2020.01.002] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2019] [Revised: 12/18/2019] [Accepted: 12/19/2019] [Indexed: 02/09/2023]
Abstract
Oscillatory brain activities support many physiological functions from motor control to cognition. Disruptions of the normal oscillatory brain activities are commonly observed in neurological and psychiatric disorders including epilepsy, Parkinson's disease, Alzheimer's disease, schizophrenia, anxiety/trauma-related disorders, major depressive disorders, and drug addiction. Therefore, these disorders can be considered as common oscillation defects despite having distinct behavioral manifestations and genetic causes. Recent technical advances of neuronal activity recording and analysis have allowed us to study the pathological oscillations of each disorder as a possible biomarker of symptoms. Furthermore, recent advances in brain stimulation technologies enable time- and space-targeted interventions of the pathological oscillations of both neurological disorders and psychiatric disorders as possible targets for regulating their symptoms.
Collapse
Affiliation(s)
- Yuichi Takeuchi
- MTA-SZTE 'Momentum' Oscillatory Neuronal Networks Research Group, Department of Physiology, University of Szeged, Szeged, 6720, Hungary; Department of Neuropharmacology, Graduate School of Pharmaceutical Sciences, Nagoya City University, Nagoya, 467-8603, Japan.
| | - Antal Berényi
- MTA-SZTE 'Momentum' Oscillatory Neuronal Networks Research Group, Department of Physiology, University of Szeged, Szeged, 6720, Hungary; HCEMM-SZTE Magnetotherapeutics Research Group, University of Szeged, Szeged, 6720, Hungary; Neuroscience Institute, New York University, New York, NY 10016, USA.
| |
Collapse
|
35
|
Specht K. Current Challenges in Translational and Clinical fMRI and Future Directions. Front Psychiatry 2020; 10:924. [PMID: 31969840 PMCID: PMC6960120 DOI: 10.3389/fpsyt.2019.00924] [Citation(s) in RCA: 33] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/17/2019] [Accepted: 11/20/2019] [Indexed: 12/15/2022] Open
Abstract
Translational neuroscience is an important field that brings together clinical praxis with neuroscience methods. In this review article, the focus will be on functional neuroimaging (fMRI) and its applicability in clinical fMRI studies. In the light of the "replication crisis," three aspects will be critically discussed: First, the fMRI signal itself, second, current fMRI praxis, and, third, the next generation of analysis strategies. Current attempts such as resting-state fMRI, meta-analyses, and machine learning will be discussed with their advantages and potential pitfalls and disadvantages. One major concern is that the fMRI signal shows substantial within- and between-subject variability, which affects the reliability of both task-related, but in particularly resting-state fMRI studies. Furthermore, the lack of standardized acquisition and analysis methods hinders the further development of clinical relevant approaches. However, meta-analyses and machine-learning approaches may help to overcome current shortcomings in the methods by identifying new, and yet hidden relationships, and may help to build new models on disorder mechanisms. Furthermore, better control of parameters that may have an influence on the fMRI signal and that can easily be controlled for, like blood pressure, heart rate, diet, time of day, might improve reliability substantially.
Collapse
Affiliation(s)
- Karsten Specht
- Department of Biological and Medical Psychology, University of Bergen, Bergen, Norway
- Mohn Medical Imaging and Visualization Centre, Haukeland University Hospital, Bergen, Norway
- Department of Education, UiT/The Arctic University of Norway, Tromsø, Norway
| |
Collapse
|
36
|
|
37
|
The clinical feasibility of deep learning-based classification of amyloid PET images in visually equivocal cases. Eur J Nucl Med Mol Imaging 2019; 47:332-341. [PMID: 31811343 DOI: 10.1007/s00259-019-04595-y] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2019] [Accepted: 11/04/2019] [Indexed: 10/25/2022]
Abstract
PURPOSE Although most deep learning (DL) studies have reported excellent classification accuracy, these studies usually target typical Alzheimer's disease (AD) and normal cognition (NC) for which conventional visual assessment performs well. A clinically relevant issue is the selection of high-risk subjects who need active surveillance among equivocal cases. We validated the clinical feasibility of DL compared with visual rating or quantitative measurement for assessing the diagnosis and prognosis of subjects with equivocal amyloid scans. METHODS 18F-florbetaben scans of 430 cases (85 NC, 233 mild cognitive impairment, and 112 AD) were assessed through visual rating-based, quantification-based, and DL-based methods. DL was trained using 280 two-dimensional PET images (80%) and tested by randomly assigning the remaining (70 cases, 20%) cases and a clinical validation set of 54 equivocal cases. In the equivocal cases, we assessed the agreement among the visual rating, quantification, and DL and compared the clinical outcome according to each modality-based amyloid status. RESULTS The visual reading was positive in 175 cases, equivocal in 54 cases, and negative in 201 cases. The composite SUVR cutoff value was 1.32 (AUC 0.99). The subject-level performance of DL using the test set was 100%. Among the 54 equivocal cases, 37 cases were classified as positive (Eq(deep+)) by DL, 40 cases were classified by a second-round visual assessment, and 40 cases were classified by quantification. The DL- and quantification-based classifications showed good agreement (83%, κ = 0.59). The composite SUVRs differed between Eq(deep+) (1.47 [0.13]) and Eq(deep-) (1.29 [0.10]; P < 0.001). DL, but not the visual rating, showed a significant difference in the Mini-Mental Status Examination score change during the follow-up between Eq(deep+) (- 4.21 [0.57]) and Eq(deep-) (- 1.74 [0.76]; P = 0.023) (mean duration, 1.76 years). CONCLUSIONS In visually equivocal scans, DL was more related to quantification than to visual assessment, and the negative cases selected by DL showed no decline in cognitive outcome. DL is useful for clinical diagnosis and prognosis assessment in subjects with visually equivocal amyloid scans.
Collapse
|
38
|
Sarraf S, Desouza DD, Anderson J, Saverino C. MCADNNet: Recognizing Stages of Cognitive Impairment through Efficient Convolutional fMRI and MRI Neural Network Topology Models. IEEE ACCESS : PRACTICAL INNOVATIONS, OPEN SOLUTIONS 2019; 7:155584-155600. [PMID: 32021737 PMCID: PMC6999050 DOI: 10.1109/access.2019.2949577] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/07/2023]
Abstract
Mild cognitive impairment (MCI) represents the intermediate stage between normal cerebral aging and dementia associated with Alzheimer's disease (AD). Early diagnosis of MCI and AD through artificial intelligence has captured considerable scholarly interest; researchers hope to develop therapies capable of slowing or halting these processes. We developed a state-of-the-art deep learning algorithm based on an optimized convolutional neural network (CNN) topology called MCADNNet that simultaneously recognizes MCI, AD, and normally aging brains in adults over the age of 75 years, using structural and functional magnetic resonance imaging (fMRI) data. Following highly detailed preprocessing, four-dimensional (4D) fMRI and 3D MRI were decomposed to create 2D images using a lossless transformation, which enables maximum preservation of data details. The samples were shuffled and subject-level training and testing datasets were completely independent. The optimized MCADNNet was trained and extracted invariant and hierarchical features through convolutional layers followed by multi-classification in the last layer using a softmax layer. A decision-making algorithm was also designed to stabilize the outcome of the trained models. To measure the performance of classification, the accuracy rates for various pipelines were calculated before and after applying the decision-making algorithm. Accuracy rates of 99.77% 0.36% and 97.5% 1.16% were achieved for MRI and fMRI pipelines, respectively, after applying the decision-making algorithm. In conclusion, a cutting-edge and optimized topology called MCADNNet was designed and preceded a preprocessing pipeline; this was followed by a decision-making step that yielded the highest performance achieved for simultaneous classification of the three cohorts examined.
Collapse
Affiliation(s)
- Saman Sarraf
- Department of Electrical and Computer Engineering, McMaster University, Hamilton, Ontario, Canada
| | | | - John Anderson
- Kimel Family Translational Imaging Genetics Research Laboratory, The Centre for Addiction and Mental Health, University of Toronto, Toronto, Ontario, Canada
- Campbell Family Mental Health Research Institute, The Centre for Addiction and Mental 5 Health, University of Toronto, Toronto, Ontario, Canada
| | - Cristina Saverino
- Toronto Western Hospital, University Health Network, Toronto, Ontario, Canada
| |
Collapse
|
39
|
Sheng J, Wang B, Zhang Q, Liu Q, Ma Y, Liu W, Shao M, Chen B. A novel joint HCPMMP method for automatically classifying Alzheimer’s and different stage MCI patients. Behav Brain Res 2019; 365:210-221. [DOI: 10.1016/j.bbr.2019.03.004] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2019] [Revised: 02/28/2019] [Accepted: 03/01/2019] [Indexed: 10/27/2022]
|
40
|
Xin J, Zhang Y, Tang Y, Yang Y. Brain Differences Between Men and Women: Evidence From Deep Learning. Front Neurosci 2019; 13:185. [PMID: 30906246 PMCID: PMC6418873 DOI: 10.3389/fnins.2019.00185] [Citation(s) in RCA: 53] [Impact Index Per Article: 10.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2018] [Accepted: 02/15/2019] [Indexed: 01/23/2023] Open
Abstract
Do men and women have different brains? Previous neuroimage studies sought to answer this question based on morphological difference between specific brain regions, reporting unfortunately conflicting results. In the present study, we aim to use a deep learning technique to address this challenge based on a large open-access, diffusion MRI database recorded from 1,065 young healthy subjects, including 490 men and 575 women healthy subjects. Different from commonly used 2D Convolutional Neural Network (CNN), we proposed a 3D CNN method with a newly designed structure including three hidden layers in cascade with a linear layer and a terminal Softmax layer. The proposed 3D CNN was applied to the maps of factional anisotropy (FA) in the whole-brain as well as specific brain regions. The entropy measure was applied to the lowest-level image features extracted from the first hidden layer to examine the difference of brain structure complexity between men and women. The obtained results compared with the results from using the Support Vector Machine (SVM) and Tract-Based Spatial Statistics (TBSS). The proposed 3D CNN yielded a better classification result (93.3%) than the SVM (78.2%) on the whole-brain FA images, indicating gender-related differences likely exist in the whole-brain range. Moreover, high classification accuracies are also shown in several specific brain regions including the left precuneus, the left postcentral gyrus, the left cingulate gyrus, the right orbital gyrus of frontal lobe, and the left occipital thalamus in the gray matter, and middle cerebellum peduncle, genu of corpus callosum, the right anterior corona radiata, the right superior corona radiata and the left anterior limb of internal capsule in the while matter. This study provides a new insight into the structure difference between men and women, which highlights the importance of considering sex as a biological variable in brain research.
Collapse
Affiliation(s)
- Jiang Xin
- School of Computer Science and Engineering, Central South University, Changsha, China
| | - Yaoxue Zhang
- School of Computer Science and Engineering, Central South University, Changsha, China
| | - Yan Tang
- School of Computer Science and Engineering, Central South University, Changsha, China.,Department of Neurology, Xiangya Hospital, Central South University, Changsha, China
| | - Yuan Yang
- Department of Physical Therapy and Human Movement Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
| |
Collapse
|