1
|
You S, De Leon Barba A, Cruz Tamayo V, Yun HJ, Yang E, Grant PE, Im K. Automatic cortical surface parcellation in the fetal brain using attention-gated spherical U-net. Front Neurosci 2024; 18:1410936. [PMID: 38872945 PMCID: PMC11169851 DOI: 10.3389/fnins.2024.1410936] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2024] [Accepted: 05/20/2024] [Indexed: 06/15/2024] Open
Abstract
Cortical surface parcellation for fetal brains is essential for the understanding of neurodevelopmental trajectories during gestations with regional analyses of brain structures and functions. This study proposes the attention-gated spherical U-net, a novel deep-learning model designed for automatic cortical surface parcellation of the fetal brain. We trained and validated the model using MRIs from 55 typically developing fetuses [gestational weeks: 32.9 ± 3.3 (mean ± SD), 27.4-38.7]. The proposed model was compared with the surface registration-based method, SPHARM-net, and the original spherical U-net. Our model demonstrated significantly higher accuracy in parcellation performance compared to previous methods, achieving an overall Dice coefficient of 0.899 ± 0.020. It also showed the lowest error in terms of the median boundary distance, 2.47 ± 1.322 (mm), and mean absolute percent error in surface area measurement, 10.40 ± 2.64 (%). In this study, we showed the efficacy of the attention gates in capturing the subtle but important information in fetal cortical surface parcellation. Our precise automatic parcellation model could increase sensitivity in detecting regional cortical anomalies and lead to the potential for early detection of neurodevelopmental disorders in fetuses.
Collapse
Affiliation(s)
- Sungmin You
- Fetal Neonatal Neuroimaging and Developmental Science Center, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
- Division of Newborn Medicine, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
| | - Anette De Leon Barba
- Fetal Neonatal Neuroimaging and Developmental Science Center, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
| | - Valeria Cruz Tamayo
- Fetal Neonatal Neuroimaging and Developmental Science Center, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
| | - Hyuk Jin Yun
- Fetal Neonatal Neuroimaging and Developmental Science Center, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
- Division of Newborn Medicine, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
- Department of Pediatrics, Harvard Medical School, Boston, MA, United States
| | - Edward Yang
- Department of Radiology, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
| | - P. Ellen Grant
- Fetal Neonatal Neuroimaging and Developmental Science Center, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
- Division of Newborn Medicine, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
- Department of Radiology, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
| | - Kiho Im
- Fetal Neonatal Neuroimaging and Developmental Science Center, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
- Division of Newborn Medicine, Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
- Department of Pediatrics, Harvard Medical School, Boston, MA, United States
| |
Collapse
|
2
|
Liu S, Ye H, Yang B, Li M, Cao F. A joint parcellation and boundary network with multi-rate-shared dilated graph attention for cortical surface parcellation. Med Biol Eng Comput 2024; 62:537-549. [PMID: 37945794 DOI: 10.1007/s11517-023-02942-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Accepted: 09/27/2023] [Indexed: 11/12/2023]
Abstract
Cortical surface parcellation aims to segment the surface into anatomically and functionally significant regions, which are crucial for diagnosing and treating numerous neurological diseases. However, existing methods generally ignore the difficulty in learning labeling patterns of boundaries, hindering the performance of parcellation. To this end, this paper proposes a joint parcellation and boundary network (JPBNet) to promote the effectiveness of cortical surface parcellation. Its core is developing a multi-rate-shared dilated graph attention (MDGA) module and incorporating boundary learning into the parcellation process. The former, in particular, constructs a dilated graph attention strategy, extending the dilated convolution from regular data to irregular graph data. We fuse it with different dilated rates to extract context information in various scales by devising a shared graph attention layer. After that, a boundary enhancement module and a parcellation enhancement module based on graph attention mechanisms are built in each layer, forcing MDGA to capture informative and valuable features for boundary detection and parcellation tasks. Integrating MDGA, the boundary enhancement module, and the parcellation enhancement module at each layer to supervise boundary and parcellation information, an effective JPBNet is formed by stacking multiple layers. Experiments on the public dataset reveal that the proposed method outperforms comparison methods and performs well on boundaries for cortical surface parcellation.
Collapse
Affiliation(s)
- Siqi Liu
- College of Sciences, China Jiliang University, Hangzhou, 310018, China
| | - Hailiang Ye
- College of Sciences, China Jiliang University, Hangzhou, 310018, China.
| | - Bing Yang
- College of Sciences, China Jiliang University, Hangzhou, 310018, China
| | - Ming Li
- Key Laboratory of Intelligent Education Technology and Application, Zhejiang Normal University, Jinhua, 321004, China
| | - Feilong Cao
- College of Sciences, China Jiliang University, Hangzhou, 310018, China.
| |
Collapse
|
3
|
Stoebner ZA, Hett K, Lyu I, Johnson H, Paulsen JS, Long JD, Oguz I. Comprehensive shape analysis of the cortex in Huntington's disease. Hum Brain Mapp 2023; 44:1417-1431. [PMID: 36409662 PMCID: PMC9921229 DOI: 10.1002/hbm.26125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Revised: 09/15/2022] [Accepted: 09/28/2022] [Indexed: 11/22/2022] Open
Abstract
The striatum has traditionally been the focus of Huntington's disease research due to the primary insult to this region and its central role in motor symptoms. Beyond the striatum, evidence of cortical alterations caused by Huntington's disease has surfaced. However, findings are not coherent between studies which have used cortical thickness for Huntington's disease since it is the well-established cortical metric of interest in other diseases. In this study, we propose a more comprehensive approach to cortical morphology in Huntington's disease using cortical thickness, sulcal depth, and local gyrification index. Our results show consistency with prior findings in cortical thickness, including its limitations. Our comparison between cortical thickness and local gyrification index underscores the complementary nature of these two measures-cortical thickness detects changes in the sensorimotor and posterior areas while local gyrification index identifies insular differences. Since local gyrification index and cortical thickness measures detect changes in different regions, the two used in tandem could provide a clinically relevant measure of disease progression. Our findings suggest that differences in insular regions may correspond to earlier neurodegeneration and may provide a complementary cortical measure for detection of subtle early cortical changes due to Huntington's disease.
Collapse
Affiliation(s)
- Zachary A Stoebner
- Department of Computer Science, Vanderbilt University, Nashville, Tennessee, USA.,University of Texas at Austin, Austin, Texas, USA
| | - Kilian Hett
- Department of Computer Science, Vanderbilt University, Nashville, Tennessee, USA
| | - Ilwoo Lyu
- Department of Computer Science, Vanderbilt University, Nashville, Tennessee, USA.,Department of Computer Science and Engineering, UNIST, Ulsan, South Korea
| | - Hans Johnson
- Department of Electrical and Computer Engineering, University of Iowa, Iowa City, Iowa, USA
| | - Jane S Paulsen
- Department of Neurology, University of Wisconsin, Madison, Wisconsin, USA
| | - Jeffrey D Long
- Department of Psychiatry, University of Iowa, Iowa City, Iowa, USA.,Department of Biostatistics, University of Iowa, Iowa City, Iowa, USA
| | - Ipek Oguz
- Department of Computer Science, Vanderbilt University, Nashville, Tennessee, USA
| |
Collapse
|
4
|
Bao S, Boyd BD, Kanakaraj P, Ramadass K, Meyer FAC, Liu Y, Duett WE, Huo Y, Lyu I, Zald DH, Smith SA, Rogers BP, Landman BA. Integrating the BIDS Neuroimaging Data Format and Workflow Optimization for Large-Scale Medical Image Analysis. J Digit Imaging 2022; 35:1576-1589. [PMID: 35922700 PMCID: PMC9712842 DOI: 10.1007/s10278-022-00679-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2021] [Revised: 06/21/2022] [Accepted: 07/08/2022] [Indexed: 01/01/2023] Open
Abstract
A robust medical image computing infrastructure must host massive multimodal archives, perform extensive analysis pipelines, and execute scalable job management. An emerging data format standard, the Brain Imaging Data Structure (BIDS), introduces complexities for interfacing with XNAT archives. Moreover, workflow integration is combinatorically problematic when matching large amount of processing to large datasets. Historically, workflow engines have been focused on refining workflows themselves instead of actual job generation. However, such an approach is incompatible with data centric architecture that hosts heterogeneous medical image computing. Distributed automation for XNAT toolkit (DAX) provides large-scale image storage and analysis pipelines with an optimized job management tool. Herein, we describe developments for DAX that allows for integration of XNAT and BIDS standards. We also improve DAX's efficiencies of diverse containerized workflows in a high-performance computing (HPC) environment. Briefly, we integrate YAML configuration processor scripts to abstract workflow data inputs, data outputs, commands, and job attributes. Finally, we propose an online database-driven mechanism for DAX to efficiently identify the most recent updated sessions, thereby improving job building efficiency on large projects. We refer the proposed overall DAX development in this work as DAX-1 (DAX version 1). To validate the effectiveness of the new features, we verified (1) the efficiency of converting XNAT data to BIDS format and the correctness of the conversion using a collection of BIDS standard containerized neuroimaging workflows, (2) how YAML-based processor simplified configuration setup via a sequence of application pipelines, and (3) the productivity of DAX-1 on generating actual HPC processing jobs compared with earlier DAX baseline method. The empirical results show that (1) DAX-1 converting XNAT data to BIDS has similar speed as accessing XNAT data only; (2) YAML can integrate to the DAX-1 with shallow learning curve for users, and (3) DAX-1 reduced the job/assessor generation latency by finding recent modified sessions. Herein, we present approaches for efficiently integrating XNAT and modern image formats with a scalable workflow engine for the large-scale dataset access and processing.
Collapse
Affiliation(s)
- Shunxing Bao
- Computer Science, Vanderbilt University, Nashville, TN USA
- Electrical and Computer Engineering, Vanderbilt University, Nashville, TN USA
| | - Brian D. Boyd
- Department of Psychiatry and Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN USA
| | | | | | | | - Yuqian Liu
- Institute of Imaging Science, Vanderbilt University, Nashville, TN USA
| | - William E. Duett
- Institute of Imaging Science, Vanderbilt University, Nashville, TN USA
| | - Yuankai Huo
- Computer Science, Vanderbilt University, Nashville, TN USA
- Electrical and Computer Engineering, Vanderbilt University, Nashville, TN USA
- Data Science Institute, Vanderbilt University, Nashville, TN USA
| | - Ilwoo Lyu
- Computer Science and Engineering, Ulsan National Institute of Science and Technology, Ulsan, South Korea
| | - David H. Zald
- Department of Psychology, Vanderbilt University, Nashville, TN USA
- Department of Psychiatry, Brain Health Institute, Rutgers University, Piscataway, NJ USA
| | - Seth A. Smith
- Department of Psychiatry and Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN USA
- Institute of Imaging Science, Vanderbilt University, Nashville, TN USA
- Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN USA
| | - Baxter P. Rogers
- Department of Psychiatry and Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN USA
- Institute of Imaging Science, Vanderbilt University, Nashville, TN USA
- Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN USA
| | - Bennett A. Landman
- Computer Science, Vanderbilt University, Nashville, TN USA
- Institute of Imaging Science, Vanderbilt University, Nashville, TN USA
- Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN USA
- Electrical and Computer Engineering, Vanderbilt University, Nashville, TN USA
- Biomedical Engineering, Vanderbilt University, Nashville, TN USA
| |
Collapse
|
5
|
Ha S, Lyu I. SPHARM-Net: Spherical Harmonics-Based Convolution for Cortical Parcellation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:2739-2751. [PMID: 35436188 DOI: 10.1109/tmi.2022.3168670] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
We present a spherical harmonics-based convolutional neural network (CNN) for cortical parcellation, which we call SPHARM-Net. Recent advances in CNNs offer cortical parcellation on a fine-grained triangle mesh of the cortex. Yet, most CNNs designed for cortical parcellation employ spatial convolution that depends on extensive data augmentation and allows only predefined neighborhoods of specific spherical tessellation. On the other hand, a rotation-equivariant convolutional filter avoids data augmentation, and rotational equivariance can be achieved in spectral convolution independent of a neighborhood definition. Nevertheless, the limited resources of a modern machine enable only a finite set of spectral components that might lose geometric details. In this paper, we propose (1) a constrained spherical convolutional filter that supports an infinite set of spectral components and (2) an end-to-end framework without data augmentation. The proposed filter encodes all the spectral components without the full expansion of spherical harmonics. We show that rotational equivariance drastically reduces the training time while achieving accurate cortical parcellation. Furthermore, the proposed convolution is fully composed of matrix transformations, which offers efficient and fast spectral processing. In the experiments, we validate SPHARM-Net on two public datasets with manual labels: Mindboggle-101 (N=101) and NAMIC (N=39). The experimental results show that the proposed method outperforms the state-of-the-art methods on both datasets even with fewer learnable parameters without rigid alignment and data augmentation. Our code is publicly available at https://github.com/Shape-Lab/SPHARM-Net.
Collapse
|
6
|
Cai LY, Rheault F, Kerley CI, Aboud KS, Beason-Held LL, Shafer AT, Resnick SM, Jordan LC, Anderson AW, Schilling KG, Landman BA. Joint independent component analysis for hypothesizing spatiotemporal relationships between longitudinal gray and white matter changes in preclinical Alzheimer's disease. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12032:120321H. [PMID: 36303573 PMCID: PMC9603731 DOI: 10.1117/12.2611562] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Characterizing relationships between gray matter (GM) and white matter (WM) in early Alzheimer's disease (AD) would improve understanding of how and when AD impacts the brain. However, modeling these relationships across brain regions and longitudinally remains a challenge. Thus, we propose extending joint independent component analysis (jICA) into spatiotemporal modeling of regional cortical thickness and WM bundle volumes leveraging multimodal MRI. We jointly characterize these GM and WM features in a normal aging (n=316) and an age- and sex-matched preclinical AD cohort (n=81) at each of two imaging sessions spaced three years apart, training on the normal aging population in cross-validation and interrogating the preclinical AD cohort. We find this joint model identifies reproducible, longitudinal changes in GM and WM between the two imaging sessions and that these changes are associated with preclinical AD and are plausible considering the literature. We compare this joint model to two focused models: (1) GM features at the first session and WM at the second and (2) vice versa. The joint model identifies components that correlate poorly with those from the focused models, suggesting the different models resolve different patterns. We find the strength of association with preclinical AD is improved in the GM to WM model, which supports the hypothesis that medial temporal and frontal thinning precedes volume loss in the uncinate fasciculus and inferior anterior-posterior association fibers. These results suggest that jICA effectively generates spatiotemporal hypotheses about GM and WM in preclinical AD, especially when specific intermodality relationships are considered a priori.
Collapse
Affiliation(s)
- Leon Y Cai
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, USA
| | - Francois Rheault
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA
| | - Cailey I Kerley
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA
| | - Katherine S Aboud
- Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA
| | - Lori L Beason-Held
- Laboratory of Behavioral Neuroscience, National Institute on Aging, National Institutes of Health,Baltimore, MD, USA
| | - Andrea T Shafer
- Laboratory of Behavioral Neuroscience, National Institute on Aging, National Institutes of Health,Baltimore, MD, USA
| | - Susan M Resnick
- Laboratory of Behavioral Neuroscience, National Institute on Aging, National Institutes of Health,Baltimore, MD, USA
| | - Lori C Jordan
- Departments of Pediatrics and Neurology, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Adam W Anderson
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, USA
- Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
- Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, USA
| | - Kurt G Schilling
- Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
- Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, USA
| | - Bennett A Landman
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, USA
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA
- Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
- Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, USA
| |
Collapse
|
7
|
Cai LY, Tanase C, Anderson AW, Ramadass K, Rheault F, Lee CA, Patel NJ, Jones S, LeStourgeon LM, Mahon A, Pruthi S, Gwal K, Ozturk A, Kang H, Glaser N, Ghetti S, Jaser SS, Jordan LC, Landman BA. Multimodal neuroimaging in pediatric type 1 diabetes: a pilot multisite feasibility study of acquisition quality, motion, and variability. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12032:120323U. [PMID: 36303580 PMCID: PMC9604061 DOI: 10.1117/12.2611553] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Type 1 diabetes (T1D) affects over 200,000 children and is associated with an increased risk of cognitive dysfunction. Prior imaging studies suggest the neurological changes underlying this risk are multifactorial, including macrostructural, microstructural, and inflammatory changes. However, these studies have yet to be integrated, limiting investigation into how these phenomena interact. To better understand these complex mechanisms of brain injury, a well-powered, prospective, multisite, and multimodal neuroimaging study is needed. We take the first step in accomplishing this with a preliminary characterization of multisite, multimodal MRI quality, motion, and variability in pediatric T1D. We acquire structural T1 weighted (T1w) MRI, diffusion tensor MRI (DTI), functional MRI (fMRI), and magnetic resonance spectroscopy (MRS) of 5-7 participants from each of two sites. First, we assess the contrast-to-noise ratio of the T1w MRI and find no differences between sites. Second, we characterize intervolume motion in DTI and fMRI and find it to be on the subvoxel level. Third, we investigate variability in regional gray matter volumes and local gyrification indices, bundle-wise DTI microstructural measures, and N-acetylaspartate to creatine ratios. We find the T1-based measures to be comparable between sites before harmonization and the DTI and MRS-based measures to be comparable after. We find a 5-15% coefficient of variation for most measures, suggesting ~150-200 participants per group on average are needed to detect a 5% difference across these modalities at 0.9 power. We conclude that multisite, multimodal neuroimaging of pediatric T1D is feasible with low motion artifact after harmonization of DTI and MRS.
Collapse
Affiliation(s)
- Leon Y. Cai
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, USA
| | - Costin Tanase
- Department of Psychiatry, University of California, Davis, Davis, CA, USA
| | - Adam W. Anderson
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, USA,Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, USA
| | - Karthik Ramadass
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA
| | - Francois Rheault
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA
| | - Chelsea A. Lee
- Department of Pediatrics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Niral J. Patel
- Department of Pediatrics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Sky Jones
- Department of Pediatrics, Vanderbilt University Medical Center, Nashville, TN, USA,Department of Neurology, Vanderbilt University Medical Center, Nashville, TN, USA
| | | | - Alix Mahon
- Department of Psychology, University of California, Davis, Davis, CA, USA
| | - Sumit Pruthi
- Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Kriti Gwal
- Department of Radiology, UC Davis Health, UC Davis School of Medicine, Sacramento, CA, USA
| | - Arzu Ozturk
- Department of Radiology, UC Davis Health, UC Davis School of Medicine, Sacramento, CA, USA
| | - Hakmook Kang
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Nicole Glaser
- Department of Pediatrics, UC Davis Health, UC Davis School of Medicine, Sacramento, CA, USA
| | - Simona Ghetti
- Department of Psychology, University of California, Davis, Davis, CA, USA
| | - Sarah S. Jaser
- Department of Pediatrics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Lori C. Jordan
- Department of Pediatrics, Vanderbilt University Medical Center, Nashville, TN, USA,Department of Neurology, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Bennett A. Landman
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, USA,Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, USA,Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA,Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
| |
Collapse
|
8
|
Zhao F, Wu Z, Wang F, Lin W, Xia S, Shen D, Wang L, Li G. S3Reg: Superfast Spherical Surface Registration Based on Deep Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:1964-1976. [PMID: 33784617 PMCID: PMC8424532 DOI: 10.1109/tmi.2021.3069645] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/20/2023]
Abstract
Cortical surface registration is an essential step and prerequisite for surface-based neuroimaging analysis. It aligns cortical surfaces across individuals and time points to establish cross-sectional and longitudinal cortical correspondences to facilitate neuroimaging studies. Though achieving good performance, available methods are either time consuming or not flexible to extend to multiple or high dimensional features. Considering the explosive availability of large-scale and multimodal brain MRI data, fast surface registration methods that can flexibly handle multimodal features are desired. In this study, we develop a Superfast Spherical Surface Registration (S3Reg) framework for the cerebral cortex. Leveraging an end-to-end unsupervised learning strategy, S3Reg offers great flexibility in the choice of input feature sets and output similarity measures for registration, and meanwhile reduces the registration time significantly. Specifically, we exploit the powerful learning capability of spherical Convolutional Neural Network (CNN) to directly learn the deformation fields in spherical space and implement diffeomorphic design with "scaling and squaring" layers to guarantee topology-preserving deformations. To handle the polar-distortion issue, we construct a novel spherical CNN model using three orthogonal Spherical U-Nets. Experiments are performed on two different datasets to align both adult and infant multimodal cortical features. Results demonstrate that our S3Reg shows superior or comparable performance with state-of-the-art methods, while improving the registration time from 1 min to 10 sec.
Collapse
|
9
|
Ahmedt-Aristizabal D, Armin MA, Denman S, Fookes C, Petersson L. Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future. SENSORS (BASEL, SWITZERLAND) 2021; 21:4758. [PMID: 34300498 PMCID: PMC8309939 DOI: 10.3390/s21144758] [Citation(s) in RCA: 41] [Impact Index Per Article: 13.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/07/2021] [Revised: 07/05/2021] [Accepted: 07/07/2021] [Indexed: 01/17/2023]
Abstract
With the advances of data-driven machine learning research, a wide variety of prediction problems have been tackled. It has become critical to explore how machine learning and specifically deep learning methods can be exploited to analyse healthcare data. A major limitation of existing methods has been the focus on grid-like data; however, the structure of physiological recordings are often irregular and unordered, which makes it difficult to conceptualise them as a matrix. As such, graph neural networks have attracted significant attention by exploiting implicit information that resides in a biological system, with interacting nodes connected by edges whose weights can be determined by either temporal associations or anatomical junctions. In this survey, we thoroughly review the different types of graph architectures and their applications in healthcare. We provide an overview of these methods in a systematic manner, organized by their domain of application including functional connectivity, anatomical structure, and electrical-based analysis. We also outline the limitations of existing techniques and discuss potential directions for future research.
Collapse
Affiliation(s)
- David Ahmedt-Aristizabal
- Imaging and Computer Vision Group, CSIRO Data61, Canberra 2601, Australia; (M.A.A.); (L.P.)
- Signal Processing, Artificial Intelligence and Vision Technologies (SAIVT) Research Program, Queensland University of Technology, Brisbane 4000, Australia; (S.D.); (C.F.)
| | - Mohammad Ali Armin
- Imaging and Computer Vision Group, CSIRO Data61, Canberra 2601, Australia; (M.A.A.); (L.P.)
| | - Simon Denman
- Signal Processing, Artificial Intelligence and Vision Technologies (SAIVT) Research Program, Queensland University of Technology, Brisbane 4000, Australia; (S.D.); (C.F.)
| | - Clinton Fookes
- Signal Processing, Artificial Intelligence and Vision Technologies (SAIVT) Research Program, Queensland University of Technology, Brisbane 4000, Australia; (S.D.); (C.F.)
| | - Lars Petersson
- Imaging and Computer Vision Group, CSIRO Data61, Canberra 2601, Australia; (M.A.A.); (L.P.)
| |
Collapse
|
10
|
Lyu I, Bao S, Hao L, Yao J, Miller JA, Voorhies W, Taylor WD, Bunge SA, Weiner KS, Landman BA. Labeling lateral prefrontal sulci using spherical data augmentation and context-aware training. Neuroimage 2021; 229:117758. [PMID: 33497773 PMCID: PMC8366030 DOI: 10.1016/j.neuroimage.2021.117758] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2020] [Revised: 12/18/2020] [Accepted: 01/07/2021] [Indexed: 02/06/2023] Open
Abstract
The inference of cortical sulcal labels often focuses on deep (primary and secondary) sulcal regions, whereas shallow (tertiary) sulcal regions are largely overlooked in the literature due to the scarcity of manual/well-defined annotations and their large neuroanatomical variability. In this paper, we present an automated framework for regional labeling of both primary/secondary and tertiary sulci of the dorsal portion of lateral prefrontal cortex (LPFC) using spherical convolutional neural networks. We propose two core components that enhance the inference of sulcal labels to overcome such large neuroanatomical variability: (1) surface data augmentation and (2) context-aware training. (1) To take into account neuroanatomical variability, we synthesize training data from the proposed feature space that embeds intermediate deformation trajectories of spherical data in a rigid to non-rigid fashion, which bridges an augmentation gap in conventional rotation data augmentation. (2) Moreover, we design a two-stage training process to improve labeling accuracy of tertiary sulci by informing the biological associations in neuroanatomy: inference of primary/secondary sulci and then their spatial likelihood to guide the definition of tertiary sulci. In the experiments, we evaluate our method on 13 deep and shallow sulci of human LPFC in two independent data sets with different age ranges: pediatric (N=60) and adult (N=36) cohorts. We compare the proposed method with a conventional multi-atlas approach and spherical convolutional neural networks without/with rotation data augmentation. In both cohorts, the proposed data augmentation improves labeling accuracy of deep and shallow sulci over the baselines, and the proposed context-aware training offers further improvement in the labeling of shallow sulci over the proposed data augmentation. We share our tools with the field and discuss applications of our results for understanding neuroanatomical-functional organization of LPFC and the rest of cortex (https://github.com/ilwoolyu/SphericalLabeling).
Collapse
Affiliation(s)
- Ilwoo Lyu
- Electrical Engineering and Computer Science, Vanderbilt University, Nashville TN, 37235 USA.
| | - Shuxing Bao
- Electrical Engineering and Computer Science, Vanderbilt University, Nashville TN, 37235 USA
| | - Lingyan Hao
- Institute for Computational & Mathematical Engineering, Stanford University, Stanford, CA 94305, USA
| | - Jewelia Yao
- Department of Psychology, The University of California, Berkeley, CA 94720, USA
| | - Jacob A Miller
- Helen Wills Neuroscience Institute, The University of California, Berkeley, CA 94720, USA
| | - Willa Voorhies
- Department of Psychology, The University of California, Berkeley, CA 94720, USA; Helen Wills Neuroscience Institute, The University of California, Berkeley, CA 94720, USA
| | - Warren D Taylor
- Psychiatry & Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN 37203 USA
| | - Silvia A Bunge
- Department of Psychology, The University of California, Berkeley, CA 94720, USA; Helen Wills Neuroscience Institute, The University of California, Berkeley, CA 94720, USA
| | - Kevin S Weiner
- Department of Psychology, The University of California, Berkeley, CA 94720, USA; Helen Wills Neuroscience Institute, The University of California, Berkeley, CA 94720, USA
| | - Bennett A Landman
- Electrical Engineering and Computer Science, Vanderbilt University, Nashville TN, 37235 USA
| |
Collapse
|
11
|
Zhao F, Wu Z, Wang L, Lin W, Gilmore JH, Xia S, Shen D, Li G. Spherical Deformable U-Net: Application to Cortical Surface Parcellation and Development Prediction. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:1217-1228. [PMID: 33417540 PMCID: PMC8016713 DOI: 10.1109/tmi.2021.3050072] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/13/2023]
Abstract
Convolutional Neural Networks (CNNs) have achieved overwhelming success in learning-related problems for 2D/3D images in the Euclidean space. However, unlike in the Euclidean space, the shapes of many structures in medical imaging have an inherent spherical topology in a manifold space, e.g., the convoluted brain cortical surfaces represented by triangular meshes. There is no consistent neighborhood definition and thus no straightforward convolution/pooling operations for such cortical surface data. In this paper, leveraging the regular and hierarchical geometric structure of the resampled spherical cortical surfaces, we create the 1-ring filter on spherical cortical triangular meshes and accordingly develop convolution/pooling operations for constructing Spherical U-Net for cortical surface data. However, the regular nature of the 1-ring filter makes it inherently limited to model fixed geometric transformations. To further enhance the transformation modeling capability of Spherical U-Net, we introduce the deformable convolution and deformable pooling to cortical surface data and accordingly propose the Spherical Deformable U-Net (SDU-Net). Specifically, spherical offsets are learned to freely deform the 1-ring filter on the sphere to adaptively localize cortical structures with different sizes and shapes. We then apply the SDU-Net to two challenging and scientifically important tasks in neuroimaging: cortical surface parcellation and cortical attribute map prediction. Both applications validate the competitive performance of our approach in accuracy and computational efficiency in comparison with state-of-the-art methods.
Collapse
|
12
|
Liu Y, Englot DJ, Morgan VL, Taylor WD, Wei Y, Oguz I, Landman BA, Lyu I. Establishing Surface Correspondence for Post-surgical Cortical Thickness Changes in Temporal Lobe Epilepsy. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11596. [PMID: 34531630 DOI: 10.1117/12.2580808] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
In pre- and post-surgical surface shape analysis, establishing shape correspondence is necessary to investigate the postoperative surface changes. However, structural absence after the operation accompanies focal non-rigid changes, which leads to challenges in existing surface registration methods. In this paper, we present a fully automatic particle-based method to establish surface correspondence that can handle partial structural abnormality in the temporal lobe resection. Our method optimizes the coordinates of points which are modeled as particles on surfaces in a hierarchical way to reduce a chance of being trapped in a local minimum during the optimization. In the experiments, we evaluate the effectiveness of our method in comparison with conventional spherical registration (FreeSurfer) on two scenarios: cortical thickness changes in healthy controls within a short scan-rescan time window and patients with temporal lobe resection. The post-surgical scan is acquired at least 1 year after the presurgical scan. In region of interest-wise (ROI-wise) analysis, no changes on cortical thickness are found in both methods on the healthy control group. In patients, since there is no ground truth available, we instead investigated the disagreement between our method and FreeSurfer. We see poorly matched ROIs and large cortical thickness changes using FreeSurfer. On the contrary, our method shows well-matched ROIs and subtle cortical thickness changes. This suggests that the proposed method can establish a stable shape correspondence, which is not fully captured in a conventional spherical registration.
Collapse
Affiliation(s)
- Yue Liu
- College of Information Science and Engineering, Northeastern University, Shenyang, China.,Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Dario J Englot
- Neurological Surgery, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Victoria L Morgan
- Radiology & Radiological Science, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Warren D Taylor
- Psychiatry & Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Ying Wei
- College of Information Science and Engineering, Northeastern University, Shenyang, China
| | - Ipek Oguz
- Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Bennett A Landman
- Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Ilwoo Lyu
- Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA
| |
Collapse
|
13
|
Kerley CI, Cai LY, Yu C, Crawford LM, Elenberger JM, Singh ES, Schilling KG, Aboud KS, Landman BA, Rex TS. Joint analysis of structural connectivity and cortical surface features: correlates with mild traumatic brain injury. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11596:115960R. [PMID: 34354324 PMCID: PMC8336656 DOI: 10.1117/12.2580902] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Mild traumatic brain injury (mTBI) is a complex syndrome that affects up to 600 per 100,000 individuals, with a particular concentration among military personnel. About half of all mTBI patients experience a diverse array of chronic symptoms which persist long after the acute injury. Hence, there is an urgent need for better understanding of the white matter and gray matter pathologies associated with mTBI to map which specific brain systems are impacted and identify courses of intervention. Previous works have linked mTBI to disruptions in white matter pathways and cortical surface abnormalities. Herein, we examine these hypothesized links in an exploratory study of joint structural connectivity and cortical surface changes associated with mTBI and its chronic symptoms. Briefly, we consider a cohort of 12 mTBI and 26 control subjects. A set of 588 cortical surface metrics and 4,753 structural connectivity metrics were extracted from cortical surface regions and diffusion weighted magnetic resonance imaging in each subject. Principal component analysis (PCA) was used to reduce the dimensionality of each metric set. We then applied independent component analysis (ICA) both to each PCA space individually and together in a joint ICA approach. We identified a stable independent component across the connectivity-only and joint ICAs which presented significant group differences in subject loadings (p<0.05, corrected). Additionally, we found that two mTBI symptoms, slowed thinking and forgetfulness, were significantly correlated (p<0.05, corrected) with mTBI subject loadings in a surface-only ICA. These surface-only loadings captured an increase in bilateral cortical thickness.
Collapse
Affiliation(s)
| | - Leon Y Cai
- Department of Biomedical Engineering, Vanderbilt University
| | - Chang Yu
- Department of Computer Science, Vanderbilt University
| | - Logan M Crawford
- Department of Ophthalmology and Visual Sciences, Vanderbilt University Medical Center
| | - Jason M Elenberger
- Department of Ophthalmology and Visual Sciences, Vanderbilt University Medical Center
| | - Eden S Singh
- Department of Ophthalmology and Visual Sciences, Vanderbilt University Medical Center
| | - Kurt G Schilling
- Vanderbilt University Institute of Imaging Science, Vanderbilt University
| | | | - Bennett A Landman
- Department of Electrical Engineering, Vanderbilt University
- Department of Biomedical Engineering, Vanderbilt University
- Department of Computer Science, Vanderbilt University
- Vanderbilt University Institute of Imaging Science, Vanderbilt University
- Vanderbilt Brain Institute, Vanderbilt University
| | - Tonia S Rex
- Department of Ophthalmology and Visual Sciences, Vanderbilt University Medical Center
| |
Collapse
|
14
|
Cai LY, Kerley CI, Yu C, Aboud KS, Beason-Held LL, Shafer AT, Resnick SM, Jordan LC, Anderson AW, Schilling KG, Lyu I, Landman BA. Joint cortical surface and structural connectivity analysis of Alzheimer's Disease. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11596:1159630. [PMID: 34354323 PMCID: PMC8336655 DOI: 10.1117/12.2580956] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
Prior neuroimaging studies have demonstrated isolated structural and connectivity changes in the brain due to Alzheimer's Disease (AD). However, how these changes relate to each other is not well understood. We present a preliminary study to begin to fill this gap by leveraging joint independent component analysis (jICA). We explore how jICA performs in an analysis of T1 and diffusion weighted MRI by characterizing the joint changes of complex cortical surface and structural connectivity metrics in AD in subjects from the Baltimore Longitudinal Study of Aging. We calculate 588 region-based cortical metrics and 4,753 fractional anisotropy-based connectivity metrics and project them into a low-dimensional manifold with principal component analysis. We perform jICA on the manifold and subsequently backproject the independent components to the original data space. We demonstrate component stability with 3-fold cross validation and find differential component loadings between 776 cognitively unimpaired control subjects and 23 with AD that generalizes across folds. In addition, we perform the same analysis on the surface and connectivity metrics separately and find that the joint approach identifies both novel and similar components to the separate approaches. To illustrate the joint approach's primary utility, we provide an example hypothesis for how surface and connectivity components may vary together with AD. These preliminary results suggest jointly varying independent cortical surface and structural connectivity components can be consistently extracted from MRI data and provide a data-driven way for generating novel hypotheses about AD that may not be captured by separate analyses.
Collapse
Affiliation(s)
- Leon Y Cai
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, USA
| | - Cailey I Kerley
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Chang Yu
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Katherine S Aboud
- Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA
| | - Lori L Beason-Held
- Laboratory of Behavioral Neuroscience, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA
| | - Andrea T Shafer
- Laboratory of Behavioral Neuroscience, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA
| | - Susan M Resnick
- Laboratory of Behavioral Neuroscience, National Institute on Aging, National Institutes of Health, Baltimore, MD, USA
| | - Lori C Jordan
- Department of Pediatrics, Division of Pediatric Neurology, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Adam W Anderson
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, USA
- Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
- Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, USA
| | - Kurt G Schilling
- Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
- Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, USA
| | - Ilwoo Lyu
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Bennett A Landman
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, USA
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA
- Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
- Vanderbilt University Institute of Imaging Science, Vanderbilt University, Nashville, TN, USA
| |
Collapse
|
15
|
Hao L, Bao S, Tang Y, Gao R, Parvathaneni P, Miller JA, Voorhies W, Yao J, Bunge SA, Weiner KS, Landman BA, Lyu I. AUTOMATIC LABELING OF CORTICAL SULCI USING SPHERICAL CONVOLUTIONAL NEURAL NETWORKS IN A DEVELOPMENTAL COHORT. PROCEEDINGS. IEEE INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING 2020; 2020:412-415. [PMID: 32547677 PMCID: PMC7296783 DOI: 10.1109/isbi45749.2020.9098414] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
In this paper, we present the automatic labeling framework for sulci in the human lateral prefrontal cortex (PFC). We adapt an existing spherical U-Net architecture with our recent surface data augmentation technique to improve the sulcal labeling accuracy in a developmental cohort. Specifically, our framework consists of the following key components: (1) augmented geometrical features being generated during cortical surface registration, (2) spherical U-Net architecture to efficiently fit the augmented features, and (3) postrefinement of sulcal labeling by optimizing spatial coherence via a graph cut technique. We validate our method on 30 healthy subjects with manual labeling of sulcal regions within PFC. In the experiments, we demonstrate significantly improved labeling performance (0.7749) in mean Dice overlap compared to that of multi-atlas (0.6410) and standard spherical U-Net (0.7011) approaches, respectively (p < 0.05). Additionally, the proposed method achieves a full set of sulcal labels in 20 seconds in this developmental cohort.
Collapse
Affiliation(s)
- Lingyan Hao
- Department of Mathematics, Vanderbilt University, TN, USA
- Electrical Engineering and Computer Science, Vanderbilt University, TN, USA
| | - Shunxing Bao
- Electrical Engineering and Computer Science, Vanderbilt University, TN, USA
| | - Yucheng Tang
- Electrical Engineering and Computer Science, Vanderbilt University, TN, USA
| | - Riqiang Gao
- Electrical Engineering and Computer Science, Vanderbilt University, TN, USA
| | - Prasanna Parvathaneni
- National Institute of Neurological Disorders and Stroke, National Institutes of Health, MD, USA
| | - Jacob A Miller
- Helen Wills Neuroscience Institute, University of California at Berkeley, CA, USA
| | - Willa Voorhies
- Department of Psychology, University of California at Berkeley, CA, USA
- Helen Wills Neuroscience Institute, University of California at Berkeley, CA, USA
| | - Jewelia Yao
- Department of Psychology, University of California at Berkeley, CA, USA
| | - Silvia A Bunge
- Department of Psychology, University of California at Berkeley, CA, USA
- Helen Wills Neuroscience Institute, University of California at Berkeley, CA, USA
| | - Kevin S Weiner
- Department of Psychology, University of California at Berkeley, CA, USA
- Helen Wills Neuroscience Institute, University of California at Berkeley, CA, USA
| | - Bennett A Landman
- Electrical Engineering and Computer Science, Vanderbilt University, TN, USA
| | - Ilwoo Lyu
- Electrical Engineering and Computer Science, Vanderbilt University, TN, USA
| |
Collapse
|