1
|
Cao C, Fu H, Li G, Wang M, Gao X. ADHD diagnosis guided by functional brain networks combined with domain knowledge. Comput Biol Med 2024; 177:108611. [PMID: 38788375 DOI: 10.1016/j.compbiomed.2024.108611] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Revised: 04/13/2024] [Accepted: 05/11/2024] [Indexed: 05/26/2024]
Abstract
Utilizing functional magnetic resonance imaging (fMRI) to model functional brain networks (FBNs) is increasingly prominent in attention-deficit/hyperactivity disorder (ADHD) research, revealing neural impact and mechanisms through the exploration of activated brain regions. However, current FBNs-based methods face two major challenges. The primary challenge stems from the limitations of existing modeling methods in accurately capturing both regional correlations and long-distance dependencies (LDDs) within the dynamic brain, thereby affecting the diagnostic accuracy of FBNs as biomarkers. Additionally, limited sample size and class imbalance also pose a challenge to the learning performance of the model. To address the issues, we propose an automated diagnostic framework, which integrates modeling, multimodal fusion, and classification into a unified process. It aims to extract representative FBNs and efficiently incorporate domain knowledge to guide ADHD classification. Our work mainly includes three-fold: 1) A multi-head attention-based region-enhancement module (MAREM) is designed to simultaneously capture regional correlations and LDDs across the entire sequence of brain activity, which facilitates the construction of representative FBNs. 2) The multimodal supplementary learning module (MSLM) is proposed to integrate domain knowledge from phenotype data with FBNs from neuroimaging data, achieving information complementarity and alleviating the problems of insufficient medical data and unbalanced sample categories. 3) An ADHD automatic diagnosis framework guided by FBNs and domain knowledge (ADF-FAD) is proposed to help doctors make more accurate decisions, which is applied to the ADHD-200 dataset to confirm its effectiveness. The results indicate that the FBNs extracted by MAREM perform well in modeling and classification. After with MSLM, the model achieves accuracy of 92.4%, 74.4%, and 80% at NYU, PU, and KKI, respectively, demonstrating its ability to effectively capture crucial information related to ADHD diagnosis. Codes are available at https://github.com/zhuimengxuebao/ADF-FAD.
Collapse
Affiliation(s)
- Chunhong Cao
- MOE Key Laboratory of Intelligent Computing and Information Processing, Xiangtan University, Xiangtan, 411100, China
| | - Huawei Fu
- MOE Key Laboratory of Intelligent Computing and Information Processing, Xiangtan University, Xiangtan, 411100, China
| | - Gai Li
- MOE Key Laboratory of Intelligent Computing and Information Processing, Xiangtan University, Xiangtan, 411100, China
| | - Mengyang Wang
- MOE Key Laboratory of Intelligent Computing and Information Processing, Xiangtan University, Xiangtan, 411100, China
| | - Xieping Gao
- Hunan Provincial Key Laboratory of Intelligent Computing and Language Information Processing, Hunan Normal University, Changsha, 410081, China.
| |
Collapse
|
2
|
Zhao S, Fang L, Yang Y, Tang G, Luo G, Han J, Liu T, Hu X. Task sub-type states decoding via group deep bidirectional recurrent neural network. Med Image Anal 2024; 94:103136. [PMID: 38489895 DOI: 10.1016/j.media.2024.103136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 01/31/2024] [Accepted: 03/05/2024] [Indexed: 03/17/2024]
Abstract
Decoding brain states under different cognitive tasks from functional magnetic resonance imaging (fMRI) data has attracted great attention in the neuroimaging filed. However, the well-known temporal dependency in fMRI sequences has not been fully exploited in existing studies, due to the limited temporal-modeling capacity of the backbone machine learning algorithms and rigid training sample organization strategies upon which the brain decoding methods are built. To address these limitations, we propose a novel method for fine-grain brain state decoding, namely, group deep bidirectional recurrent neural network (Group-DBRNN) model. We first propose a training sample organization strategy that consists of a group-task sample generation module and a multiple-scale random fragment strategy (MRFS) module to collect training samples that contain rich task-relevant brain activity contrast (i.e., the comparison of neural activity patterns between different tasks) and maintain the temporal dependency. We then develop a novel decoding model by replacing the unidirectional RNNs that are widely used in existing brain state decoding studies with bidirectional stacked RNNs to better capture the temporal dependency, and by introducing a multi-task interaction layer (MTIL) module to effectively model the task-relevant brain activity contrast. Our experimental results on the Human Connectome Project task fMRI dataset (7 tasks consisting of 23 task sub-type states) show that the proposed model achieves an average decoding accuracy of 94.7% over the 23 fine-grain sub-type states. Meanwhile, our extensive interpretations of the intermediate features learned in the proposed model via visualizations and quantitative assessments of their discriminability and inter-subject alignment evidence that the proposed model can effectively capture the temporal dependency and task-relevant contrast.
Collapse
Affiliation(s)
- Shijie Zhao
- School of Automation, Northwestern Polytechnical University, Xi'an 710072, China; Research & Development Institute of Northwestern Polytechnical University in Shenzhen, China
| | - Long Fang
- School of Automation, Northwestern Polytechnical University, Xi'an 710072, China
| | - Yang Yang
- School of Automation, Northwestern Polytechnical University, Xi'an 710072, China
| | - Guochang Tang
- School of Automation, Northwestern Polytechnical University, Xi'an 710072, China
| | - Guoxin Luo
- Department of Ophthalmology, Nanyang First People's Hospital Affiliated to Henan University, Nanyang 473000, China
| | - Junwei Han
- School of Automation, Northwestern Polytechnical University, Xi'an 710072, China
| | - Tianming Liu
- School of Computing, The University of Georgia, GA, USA
| | - Xintao Hu
- School of Automation, Northwestern Polytechnical University, Xi'an 710072, China.
| |
Collapse
|
3
|
Duan X, Cao Z, Gao K, Yan W, Sun S, Zhou G, Wu Z, Ren F, Sun B. Memristor-Based Neuromorphic Chips. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024; 36:e2310704. [PMID: 38168750 DOI: 10.1002/adma.202310704] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/14/2023] [Revised: 12/15/2023] [Indexed: 01/05/2024]
Abstract
In the era of information, characterized by an exponential growth in data volume and an escalating level of data abstraction, there has been a substantial focus on brain-like chips, which are known for their robust processing power and energy-efficient operation. Memristors are widely acknowledged as the optimal electronic devices for the realization of neuromorphic computing, due to their innate ability to emulate the interconnection and information transfer processes witnessed among neurons. This review paper focuses on memristor-based neuromorphic chips, which provide an extensive description of the working principle and characteristic features of memristors, along with their applications in the realm of neuromorphic chips. Subsequently, a thorough discussion of the memristor array, which serves as the pivotal component of the neuromorphic chip, as well as an examination of the present mainstream neural networks, is delved. Furthermore, the design of the neuromorphic chip is categorized into three crucial sections, including synapse-neuron cores, networks on chip (NoC), and neural network design. Finally, the key performance metrics of the chip is highlighted, as well as the key metrics related to the memristor devices are employed to realize both the synaptic and neuronal components.
Collapse
Affiliation(s)
- Xuegang Duan
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Zelin Cao
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Kaikai Gao
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Wentao Yan
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Siyu Sun
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Guangdong Zhou
- College of Artificial Intelligence, Brain-inspired Computing & Intelligent Control of Chongqing Key Lab, Southwest University, Chongqing, 400715, China
| | - Zhenhua Wu
- School of Mechanical Engineering, Shanghai Jiao Tong University, 800 DongChuan Rd, Shanghai, 200240, China
| | - Fenggang Ren
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Bai Sun
- National Local Joint Engineering Research Center for Precision Surgery & Regenerative Medicine, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Department of hepatobiliary surgery, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Frontier Institute of Science and Technology (FIST), Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
- Micro-and Nano-technology Research Center, State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| |
Collapse
|
4
|
Yuan H, Li X, Wei B. Modeling default mode network patterns via a universal spatio-temporal brain attention skip network. Neuroimage 2024; 287:120522. [PMID: 38253216 DOI: 10.1016/j.neuroimage.2024.120522] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Revised: 01/16/2024] [Accepted: 01/18/2024] [Indexed: 01/24/2024] Open
Abstract
Designing a comprehensive four-dimensional resting-state functional magnetic resonance imaging (4D Rs-fMRI) based default mode network (DMN) modeling methodology to reveal the spatio-temporal patterns of individual DMN, is crucial for understanding the cognitive mechanisms of the brain and the pathogenesis of psychiatric disorders. However, there are still two limitations of existing approaches for DMN modeling. The approaches either (1) simply split the spatio-temporal components and ignore the overall character of the spatio-temporal patterns or (2) are biased in the process of feature extraction for DMN modeling, and their spatio-temporal accuracy is thus not warranted. To this end, we propose a novel Spatio-Temporal Brain Attention Skip Network (STBAS-Net) to model the personalized spatio-temporal patterns of the DMN. STBAS-Net consists of spatial and temporal components, where the multi-head attention skip connection block in the spatial component achieves detailed feature extraction and enhancement in the shallow stage. Under the guidance of spatial information, we technically fuse multiple spatio-temporal information in the temporal component, which dexterously exploits the overall spatio-temporal features and achieves mutual constraints of spatio-temporal patterns to characterize the spatio-temporal patterns of the DMN. We verify the proposed STBAS-Net on a publicly released 4D Rs-fMRI dataset and an EMCI dataset. The experimental results show that compared with existing advanced methods, the proposed network can more accurately model the personalized spatio-temporal patterns of the human brain DMN and successfully identify abnormal spatio-temporal patterns in EMCI patients. This study provides a potential tool for revealing the spatio-temporal patterns of the human brain DMN and is expected to provide an effective methodological framework for future exploration of abnormal brain spatio-temporal patterns and modeling of other functional brain networks.
Collapse
Affiliation(s)
- Hang Yuan
- Center for Medical Artificial Intelligence, Shandong University of Traditional Chinese Medicine, Qingdao, Shandong 266112, PR China; Qingdao Academy of Chinese Medical Sciences, Shandong University of Traditional Chinese Medicine, Qingdao, Shandong 266112, PR China
| | - Xiang Li
- Center for Medical Artificial Intelligence, Shandong University of Traditional Chinese Medicine, Qingdao, Shandong 266112, PR China; Qingdao Academy of Chinese Medical Sciences, Shandong University of Traditional Chinese Medicine, Qingdao, Shandong 266112, PR China
| | - Benzheng Wei
- Center for Medical Artificial Intelligence, Shandong University of Traditional Chinese Medicine, Qingdao, Shandong 266112, PR China; Qingdao Academy of Chinese Medical Sciences, Shandong University of Traditional Chinese Medicine, Qingdao, Shandong 266112, PR China.
| |
Collapse
|
5
|
Pan H, Mao Y, Liu P, Li Y, Wei G, Qiao X, Ren Y, Zhao F. Extracting transition features among brain states based on coarse-grained similarity measurement for autism spectrum disorder analysis. Med Phys 2023; 50:6269-6282. [PMID: 36995984 DOI: 10.1002/mp.16406] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Revised: 03/13/2023] [Accepted: 03/23/2023] [Indexed: 03/31/2023] Open
Abstract
BACKGROUND The abnormal brain functional connectivity (FC) of patients with mental diseases is closely linked to the transition features among brain states. However, the current research on state transition will produce certain division deviations in the measurement method of state division, and also ignore the transition features among multiple states that contain more abundant information for analyzing brain diseases. PURPOSE To investigate the potential of the proposed method based on coarse-grained similarity measurement to solve the problem of state division, and consider the transition features among multiple states to analyze the FC abnormalities of autism spectrum disorder (ASD) patients. METHODS We used resting-state functional magnetic resonance imaging to examine 45 ASD and 47 healthy controls (HC). The FC between brain regions was calculated by the sliding window and correlation algorithm, and a novel coarse-grained similarity measure method was used to cluster the FC networks into five states, and then extract the features both of the state itself and the transition features among multiple states for analysis and diagnosis. RESULTS (1) The state as divided by the coarse-grained measurement method improves the diagnostic performance of individuals with ASD compared with previous methods. (2) The transition features among multiple states can provide complementary information to the features of the state itself in the ASD analysis and diagnosis. (3) ASD individuals have different brain state transitions than HC. Specifically, the abnormalities in intra- and inter-network connectivity of ASD patients mainly occur in the default mode network, the visual network, and the cerebellum. CONCLUSIONS Such results demonstrate that our approach with new measurements and new features is effective and promising in brain state analysis and ASD diagnosis.
Collapse
Affiliation(s)
- Hongxin Pan
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai, China
| | - Yanyan Mao
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai, China
| | - Peiqiang Liu
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai, China
| | - Yuan Li
- School of Management Science and Engineering, Shandong Technology and Business University, Yantai, China
| | - Guanglan Wei
- Information Network Center, Shandong Second Provincial General Hospital, Jinan, China
| | - Xiaoyan Qiao
- School of Mathematics and Information Science, Shandong Technology and Business University, Yantai, China
| | - Yande Ren
- Department of Radiology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China
| | - Feng Zhao
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai, China
| |
Collapse
|
6
|
Li H, Srinivasan D, Zhuo C, Cui Z, Gur RE, Gur RC, Oathes DJ, Davatzikos C, Satterthwaite TD, Fan Y. Computing personalized brain functional networks from fMRI using self-supervised deep learning. Med Image Anal 2023; 85:102756. [PMID: 36706636 PMCID: PMC10103143 DOI: 10.1016/j.media.2023.102756] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Revised: 07/20/2022] [Accepted: 01/18/2023] [Indexed: 01/22/2023]
Abstract
A novel self-supervised deep learning (DL) method is developed to compute personalized brain functional networks (FNs) for characterizing brain functional neuroanatomy based on functional MRI (fMRI). Specifically, a DL model of convolutional neural networks with an encoder-decoder architecture is developed to compute personalized FNs directly from fMRI data. The DL model is trained to optimize functional homogeneity of personalized FNs without utilizing any external supervision in an end-to-end fashion. We demonstrate that a DL model trained on fMRI scans from the Human Connectome Project can identify personalized FNs and generalizes well across four different datasets. We further demonstrate that the identified personalized FNs are informative for predicting individual differences in behavior, brain development, and schizophrenia status. Taken together, the self-supervised DL allows for rapid, generalizable computation of personalized FNs.
Collapse
Affiliation(s)
- Hongming Li
- Center for Biomedical Image Computation and Analytics, University of Pennsylvania, Philadelphia, PA 19104, USA
- Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Dhivya Srinivasan
- Center for Biomedical Image Computation and Analytics, University of Pennsylvania, Philadelphia, PA 19104, USA
- Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Chuanjun Zhuo
- Key Laboratory of Brain Circuit Real Time Tracing (BCRTT-Lab), Beijing, 102206, China
| | - Zaixu Cui
- Tianjin University Affiliated Tianjin Fourth Center Hospital, Department of Psychiatry, Tianjin Medical University, Tianjin, China Chinese Institute for Brain Research, Beijing, 102206, China
| | - Raquel E. Gur
- Center for Biomedical Image Computation and Analytics, University of Pennsylvania, Philadelphia, PA 19104, USA
- Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, USA
- Department of Psychiatry, University of Pennsylvania, Philadelphia, PA 19104, USA
- Penn/CHOP Lifespan Brain Institute, University of Pennsylvania, Philadelphia, PA 19104, USA
- Department of Neurology, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Ruben C. Gur
- Center for Biomedical Image Computation and Analytics, University of Pennsylvania, Philadelphia, PA 19104, USA
- Department of Psychiatry, University of Pennsylvania, Philadelphia, PA 19104, USA
- Penn/CHOP Lifespan Brain Institute, University of Pennsylvania, Philadelphia, PA 19104, USA
- Department of Neurology, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Desmond J. Oathes
- Center for Neuromodulation in Depression and Stress, University of Pennsylvania, Philadelphia, PA 19104, USA
- Department of Psychiatry, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Christos Davatzikos
- Center for Biomedical Image Computation and Analytics, University of Pennsylvania, Philadelphia, PA 19104, USA
- Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Theodore D. Satterthwaite
- Department of Psychiatry, University of Pennsylvania, Philadelphia, PA 19104, USA
- Penn Lifespan Informatics and Neuroimaging Center, University of Pennsylvania, Philadelphia, PA 19104, USA
- Penn/CHOP Lifespan Brain Institute, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Yong Fan
- Center for Biomedical Image Computation and Analytics, University of Pennsylvania, Philadelphia, PA 19104, USA
- Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, USA
| |
Collapse
|
7
|
Fang Y, Wang M, Potter GG, Liu M. Unsupervised cross-domain functional MRI adaptation for automated major depressive disorder identification. Med Image Anal 2023; 84:102707. [PMID: 36512941 PMCID: PMC9850278 DOI: 10.1016/j.media.2022.102707] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2022] [Revised: 11/21/2022] [Accepted: 11/25/2022] [Indexed: 11/29/2022]
Abstract
Resting-state functional magnetic resonance imaging (rs-fMRI) data have been widely used for automated diagnosis of brain disorders such as major depressive disorder (MDD) to assist in timely intervention. Multi-site fMRI data have been increasingly employed to augment sample size and improve statistical power for investigating MDD. However, previous studies usually suffer from significant inter-site heterogeneity caused for instance by differences in scanners and/or scanning protocols. To address this issue, we develop a novel discrepancy-based unsupervised cross-domain fMRI adaptation framework (called UFA-Net) for automated MDD identification. The proposed UFA-Net is designed to model spatio-temporal fMRI patterns of labeled source and unlabeled target samples via an attention-guided graph convolution module, and also leverage a maximum mean discrepancy constrained module for unsupervised cross-site feature alignment between two domains. To the best of our knowledge, this is one of the first attempts to explore unsupervised rs-fMRI adaptation for cross-site MDD identification. Extensive evaluation on 681 subjects from two imaging sites shows that the proposed method outperforms several state-of-the-art methods. Our method helps localize disease-associated functional connectivity abnormalities and is therefore well interpretable and can facilitate fMRI-based analysis of MDD in clinical practice.
Collapse
Affiliation(s)
- Yuqi Fang
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, United States
| | - Mingliang Wang
- School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing 210044, China
| | - Guy G Potter
- Department of Psychiatry and Behavioral Sciences, Duke University Medical Center, Durham, NC 27710, United States.
| | - Mingxia Liu
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, United States.
| |
Collapse
|
8
|
Xu S, Ren Y, Tao Z, Song L, He X. Hierarchical Individual Naturalistic Functional Brain Networks with Group Consistency uncovered by a Two-Stage NAS-Volumetric Sparse DBN Framework. eNeuro 2022; 9:ENEURO.0200-22.2022. [PMID: 35995557 PMCID: PMC9463984 DOI: 10.1523/eneuro.0200-22.2022] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2022] [Revised: 07/30/2022] [Accepted: 08/15/2022] [Indexed: 11/21/2022] Open
Abstract
The functional magnetic resonance imaging under naturalistic paradigm (NfMRI) showed great advantages in identifying complex and interactive functional brain networks due to its dynamics and multimodal information. In recent years, various deep learning models, such as deep convolutional autoencoder (DCAE), deep belief network (DBN) and volumetric sparse deep belief network (vsDBN), can obtain hierarchical functional brain networks (FBN) and temporal features from fMRI data. Among them, the vsDBN model revealed a good capability in identifying hierarchical FBNs by modelling fMRI volume images. However, due to the high dimensionality of fMRI volumes and the diverse training parameters of deep learning methods, especially the network architecture that is the most critical parameter for uncovering the hierarchical organization of human brain function, researchers still face challenges in designing an appropriate deep learning framework with automatic network architecture optimization to model volumetric NfMRI. In addition, most of the existing deep learning models ignore the group-wise consistency and inter-subject variation properties embedded in NfMRI volumes. To solve these problems, we proposed a two-stage neural architecture search and vs DBN model (two-stage NAS-vsDBN model) to identify the hierarchical human brain spatio-temporal features possessing both group-consistency and individual-uniqueness under naturalistic condition. Moreover, our model defined reliable network structure for modelling volumetric NfMRI data via NAS framework, and the group-level and individual-level FBNs and associated temporal features exhibited great consistency. In general, our method well identified the hierarchical temporal and spatial features of the brain function and revealed the crucial properties of neural processes under natural viewing condition.Significance StatementIn this paper, we proposed and applied a novel analytical strategy - a two-stage NAS-vsDBN model to identify both group-level and individual-level spatio-temporal features at multi-scales from volumetric NfMRI data. The proposed PSO-based NAS framework can find optimal neural structure for both group-wise and individual-level vs-DBN models. Furthermore, with well-established correspondence between two stages of vsDBN models, our model can effectively detect group-level FBNs that reveal the consistency in neural processes across subjects and individual-level FBNs that maintain the subject specific variability, verifying the inherent property of brain function under naturalistic condition.
Collapse
Affiliation(s)
- Shuhan Xu
- School of Information Science & Technology, Northwest University, China
| | - Yudan Ren
- School of Information Science & Technology, Northwest University, China
| | - Zeyang Tao
- School of Information Science & Technology, Northwest University, China
| | - Limei Song
- School of Information Science & Technology, Northwest University, China
| | - Xiaowei He
- School of Information Science & Technology, Northwest University, China
| |
Collapse
|
9
|
Cao X, Chen H, Li Y, Peng Y, Zhou Y, Cheng L, Liu T, Shen D. Auto-DenseUNet: Searchable neural network architecture for mass segmentation in 3D automated breast ultrasound. Med Image Anal 2022; 82:102589. [DOI: 10.1016/j.media.2022.102589] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2021] [Revised: 07/18/2022] [Accepted: 08/17/2022] [Indexed: 11/15/2022]
|
10
|
Liu T, Siegel E, Shen D. Deep Learning and Medical Image Analysis for COVID-19 Diagnosis and Prediction. Annu Rev Biomed Eng 2022; 24:179-201. [PMID: 35316609 DOI: 10.1146/annurev-bioeng-110220-012203] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
The coronavirus disease 2019 (COVID-19) pandemic has imposed dramatic challenges to health-care organizations worldwide. To combat the global crisis, the use of thoracic imaging has played a major role in diagnosis, prediction, and management for COVID-19 patients with moderate to severe symptoms or with evidence of worsening respiratory status. In response, the medical image analysis community acted quickly to develop and disseminate deep learning models and tools to meet the urgent need of managing and interpreting large amounts of COVID-19 imaging data. This review aims to not only summarize existing deep learning and medical image analysis methods but also offer in-depth discussions and recommendations for future investigations. We believe that the wide availability of high-quality, curated, and benchmarked COVID-19 imaging data sets offers the great promise of a transformative test bed to develop, validate, and disseminate novel deep learning methods in the frontiers of data science and artificial intelligence. Expected final online publication date for the Annual Review of Biomedical Engineering, Volume 24 is June 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
Collapse
Affiliation(s)
- Tianming Liu
- Department of Computer Science, University of Georgia, Athens, Georgia, USA;
| | - Eliot Siegel
- Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland, Baltimore, Maryland, USA;
| | - Dinggang Shen
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China.,Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China;
| |
Collapse
|
11
|
Ren Y, Xu S, Tao Z, Song L, He X. Hierarchical Spatio-Temporal Modeling of Naturalistic Functional Magnetic Resonance Imaging Signals via Two-Stage Deep Belief Network With Neural Architecture Search. Front Neurosci 2021; 15:794955. [PMID: 34955738 PMCID: PMC8692564 DOI: 10.3389/fnins.2021.794955] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Accepted: 11/19/2021] [Indexed: 11/28/2022] Open
Abstract
Naturalistic functional magnetic resonance imaging (NfMRI) has become an effective tool to study brain functional activities in real-life context, which reduces the anxiety or boredom due to difficult or repetitive tasks and avoids the problem of unreliable collection of brain activity caused by the subjects’ microsleeps during resting state. Recent studies have made efforts on characterizing the brain’s hierarchical organizations from fMRI data by various deep learning models. However, most of those models have ignored the properties of group-wise consistency and inter-subject difference in brain function under naturalistic paradigm. Another critical issue is how to determine the optimal neural architecture of deep learning models, as manual design of neural architecture is time-consuming and less reliable. To tackle these problems, we proposed a two-stage deep belief network (DBN) with neural architecture search (NAS) combined framework (two-stage NAS-DBN) to model both the group-consistent and individual-specific naturalistic functional brain networks (FBNs), which reflected the hierarchical organization of brain function and the nature of brain functional activities under naturalistic paradigm. Moreover, the test-retest reliability and spatial overlap rate of the FBNs identified by our model reveal better performance than that of widely used traditional methods. In general, our model provides a promising method for characterizing hierarchical spatiotemporal features under the natural paradigm.
Collapse
Affiliation(s)
- Yudan Ren
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Shuhan Xu
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Zeyang Tao
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Limei Song
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Xiaowei He
- School of Information Science and Technology, Northwest University, Xi'an, China
| |
Collapse
|
12
|
Zhang Y, Liu M, Yu F, Zeng T, Wang Y. An O-shape Neural Network With Attention Modules to Detect Junctions in Biomedical Images Without Segmentation. IEEE J Biomed Health Inform 2021; 26:774-785. [PMID: 34197332 DOI: 10.1109/jbhi.2021.3094187] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Junction plays an important role in biomedical research such as retinal biometric identification, retinal image registration, eye-related disease diagnosis and neuron reconstruction. However, junction detection in original biomedical images is extremely challenging. For example, retinal images contain many tiny blood vessels with complicated structures and low contrast, which makes it challenging to detect junctions. In this paper, we propose an O-shape Network architecture with Attention modules (Attention O-Net), which includes Junction Detection Branch (JDB) and Local Enhancement Branch (LEB) to detect junctions in biomedical images without segmentation. In JDB, the heatmap indicating the probabilities of junctions is estimated and followed by choosing the positions with the local highest value as the junctions, whereas it is challenging to detect junctions when the images contain weak filament signals. Therefore, LEB is constructed to enhance the thin branch foreground and make the network pay more attention to the regions with low contrast, which is helpful to alleviate the imbalance of the foreground between thin and thick branches and to detect the junctions of the thin branch. Furthermore, attention modules are utilized to introduce the feature maps from LEB to JDB, which can establish a complementary relationship and further integrate local features and contextual information between the two branches. The proposed method achieves the highest average F1-scores of 0.82, 0.73 and 0.94 in two retinal datasets and one neuron dataset, respectively. The experimental results confirm that Attention O-Net outperforms other state-of-the-art detection methods, and is helpful for retinal biometric identification.
Collapse
|
13
|
Evolving fuzzy k-nearest neighbors using an enhanced sine cosine algorithm: Case study of lupus nephritis. Comput Biol Med 2021; 135:104582. [PMID: 34214940 DOI: 10.1016/j.compbiomed.2021.104582] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2021] [Revised: 06/13/2021] [Accepted: 06/13/2021] [Indexed: 02/05/2023]
Abstract
Because of its simplicity and effectiveness, fuzzy K-nearest neighbors (FKNN) is widely used in literature. The parameters have an essential impact on the performance of FKNN. Hence, the parameters need to be attuned to suit different problems. Also, choosing more representative features can enhance the performance of FKNN. This research proposes an improved optimization technique based on the sine cosine algorithm (LSCA), which introduces a linear population size reduction mechanism for enhancing the original algorithm's performance. Moreover, we developed an FKNN model based on the LSCA, it simultaneously performs feature selection and parameter optimization. Firstly, the search performance of LSCA is verified on the IEEE CEC2017 benchmark test function compared to the classical and improved algorithms. Secondly, the validity of the LSCA-FKNN model is verified on three medical datasets. Finally, we used the proposed LSCA-FKNN to predict lupus nephritis classes, and the model showed competitive results. The paper will be supported by an online web service for any question at https://aliasgharheidari.com.
Collapse
|