1
|
Boorboor S, Mathew S, Ananth M, Talmage D, Role LW, Kaufman AE. NeuRegenerate: A Framework for Visualizing Neurodegeneration. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2023; 29:1625-1637. [PMID: 34757909 PMCID: PMC10070008 DOI: 10.1109/tvcg.2021.3127132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Recent advances in high-resolution microscopy have allowed scientists to better understand the underlying brain connectivity. However, due to the limitation that biological specimens can only be imaged at a single timepoint, studying changes to neural projections over time is limited to observations gathered using population analysis. In this article, we introduce NeuRegenerate, a novel end-to-end framework for the prediction and visualization of changes in neural fiber morphology within a subject across specified age-timepoints. To predict projections, we present neuReGANerator, a deep-learning network based on cycle-consistent generative adversarial network (GAN) that translates features of neuronal structures across age-timepoints for large brain microscopy volumes. We improve the reconstruction quality of the predicted neuronal structures by implementing a density multiplier and a new loss function, called the hallucination loss. Moreover, to alleviate artifacts that occur due to tiling of large input volumes, we introduce a spatial-consistency module in the training pipeline of neuReGANerator. Finally, to visualize the change in projections, predicted using neuReGANerator, NeuRegenerate offers two modes: (i) neuroCompare to simultaneously visualize the difference in the structures of the neuronal projections, from two age domains (using structural view and bounded view), and (ii) neuroMorph, a vesselness-based morphing technique to interactively visualize the transformation of the structures from one age-timepoint to the other. Our framework is designed specifically for volumes acquired using wide-field microscopy. We demonstrate our framework by visualizing the structural changes within the cholinergic system of the mouse brain between a young and old specimen.
Collapse
|
2
|
Afzal S, Ghani S, Hittawe MM, Rashid SF, Knio OM, Hadwiger M, Hoteit I. Visualization and Visual Analytics Approaches for Image and Video Datasets: A Survey. ACM T INTERACT INTEL 2023. [DOI: 10.1145/3576935] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Abstract
Image and video data analysis has become an increasingly important research area with applications in different domains such as security surveillance, healthcare, augmented and virtual reality, video and image editing, activity analysis and recognition, synthetic content generation, distance education, telepresence, remote sensing, sports analytics, art, non-photorealistic rendering, search engines, and social media. Recent advances in Artificial Intelligence (AI) and particularly deep learning have sparked new research challenges and led to significant advancements, especially in image and video analysis. These advancements have also resulted in significant research and development in other areas such as visualization and visual analytics, and have created new opportunities for future lines of research. In this survey paper, we present the current state of the art at the intersection of visualization and visual analytics, and image and video data analysis. We categorize the visualization papers included in our survey based on different taxonomies used in visualization and visual analytics research. We review these papers in terms of task requirements, tools, datasets, and application areas. We also discuss insights based on our survey results, trends and patterns, the current focus of visualization research, and opportunities for future research.
Collapse
Affiliation(s)
- Shehzad Afzal
- King Abdullah University of Science & Technology, Saudi Arabia
| | - Sohaib Ghani
- King Abdullah University of Science & Technology, Saudi Arabia
| | | | | | - Omar M Knio
- King Abdullah University of Science & Technology, Saudi Arabia
| | - Markus Hadwiger
- King Abdullah University of Science & Technology, Saudi Arabia
| | - Ibrahim Hoteit
- King Abdullah University of Science & Technology, Saudi Arabia
| |
Collapse
|
3
|
Ghahremani P, Boorboor S, Mirhosseini P, Gudisagar C, Ananth M, Talmage D, Role LW, Kaufman AE. NeuroConstruct: 3D Reconstruction and Visualization of Neurites in Optical Microscopy Brain Images. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2022; 28:4951-4965. [PMID: 34478372 DOI: 10.1109/tvcg.2021.3109460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
We introduce NeuroConstruct, a novel end-to-end application for the segmentation, registration, and visualization of brain volumes imaged using wide-field microscopy. NeuroConstruct offers a Segmentation Toolbox with various annotation helper functions that aid experts to effectively and precisely annotate micrometer resolution neurites. It also offers an automatic neurites segmentation using convolutional neuronal networks (CNN) trained by the Toolbox annotations and somas segmentation using thresholding. To visualize neurites in a given volume, NeuroConstruct offers a hybrid rendering by combining iso-surface rendering of high-confidence classified neurites, along with real-time rendering of raw volume using a 2D transfer function for voxel classification score versus voxel intensity value. For a complete reconstruction of the 3D neurites, we introduce a Registration Toolbox that provides automatic coarse-to-fine alignment of serially sectioned samples. The quantitative and qualitative analysis show that NeuroConstruct outperforms the state-of-the-art in all design aspects. NeuroConstruct was developed as a collaboration between computer scientists and neuroscientists, with an application to the study of cholinergic neurons, which are severely affected in Alzheimer's disease.
Collapse
|
4
|
Sugashi T, Yuki H, Niizawa T, Takuwa H, Kanno I, Masamoto K. Three-dimensional microvascular network reconstruction from in vivo images with adaptation of the regional inhomogeneity in the signal-to-noise ratio. Microcirculation 2021; 28:e12697. [PMID: 33786951 DOI: 10.1111/micc.12697] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2021] [Revised: 02/19/2021] [Accepted: 03/22/2021] [Indexed: 11/27/2022]
Abstract
OBJECTIVE Quantification of angiographic images with two-photon laser scanning fluorescence microscopy (2PLSM) relies on proper segmentation of the vascular images. However, the images contain inhomogeneities in the signal-to-noise ratio (SNR) arising from regional effects of light scattering and absorption. The present study developed a semiautomated quantification method for volume images of 2PLSM angiography by adjusting the binarization threshold according to local SNR along the vessel centerlines. METHODS A phantom model made with fluorescent microbeads was used to incorporate a region-dependent binarization threshold. RESULTS The recommended SNR for imaging was found to be 4.2-10.6 that provide the true size of imaged objects if the binarization threshold was fixed at 50% of SNR. However, angiographic images in the mouse cortex showed variable SNR up to 45 over the depths. To minimize the errors caused by variable SNR and a spatial extent of the imaged objects in an axial direction, the microvascular networks were three-dimensionally reconstructed based on the cross-sectional diameters measured along the vessel centerline from the XY-plane images with adapted binarization threshold. The arterial volume was relatively constant over depths of 0-500 µm, and the capillary volume (1.7% relative to the scanned volume) showed the larger volumes than the artery (0.8%) and vein (0.6%). CONCLUSIONS The present methods allow consistent segmentation of microvasculature by adapting the local inhomogeneity in the SNR, which will be useful for quantitative comparison of the microvascular networks, such as under disease conditions where SNR in the 2PLSM images varies over space and time.
Collapse
Affiliation(s)
- Takuma Sugashi
- Department of Mechanical Engineering and Intelligent Systems, Graduate School of Informatics and Engineering, University of Electro-Communications, Chofu, Japan
| | - Hiroya Yuki
- Department of Mechanical Engineering and Intelligent Systems, Graduate School of Informatics and Engineering, University of Electro-Communications, Chofu, Japan
| | - Tomoya Niizawa
- Department of Mechanical Engineering and Intelligent Systems, Graduate School of Informatics and Engineering, University of Electro-Communications, Chofu, Japan
| | - Hiroyuki Takuwa
- Functional Brain Imaging Research, National Institute of Radiological Sciences, Chiba, Japan
| | - Iwao Kanno
- Functional Brain Imaging Research, National Institute of Radiological Sciences, Chiba, Japan
| | - Kazuto Masamoto
- Department of Mechanical Engineering and Intelligent Systems, Graduate School of Informatics and Engineering, University of Electro-Communications, Chofu, Japan.,Functional Brain Imaging Research, National Institute of Radiological Sciences, Chiba, Japan.,Center for Neuroscience and Biomedical Engineering, University of Electro-Communications, Chofu, Japan
| |
Collapse
|
5
|
McDonald T, Usher W, Morrical N, Gyulassy A, Petruzza S, Federer F, Angelucci A, Pascucci V. Improving the Usability of Virtual Reality Neuron Tracing with Topological Elements. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2021; 27:744-754. [PMID: 33055032 PMCID: PMC7891492 DOI: 10.1109/tvcg.2020.3030363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Researchers in the field of connectomics are working to reconstruct a map of neural connections in the brain in order to understand at a fundamental level how the brain processes information. Constructing this wiring diagram is done by tracing neurons through high-resolution image stacks acquired with fluorescence microscopy imaging techniques. While a large number of automatic tracing algorithms have been proposed, these frequently rely on local features in the data and fail on noisy data or ambiguous cases, requiring time-consuming manual correction. As a result, manual and semi-automatic tracing methods remain the state-of-the-art for creating accurate neuron reconstructions. We propose a new semi-automatic method that uses topological features to guide users in tracing neurons and integrate this method within a virtual reality (VR) framework previously used for manual tracing. Our approach augments both visualization and interaction with topological elements, allowing rapid understanding and tracing of complex morphologies. In our pilot study, neuroscientists demonstrated a strong preference for using our tool over prior approaches, reported less fatigue during tracing, and commended the ability to better understand possible paths and alternatives. Quantitative evaluation of the traces reveals that users' tracing speed increased, while retaining similar accuracy compared to a fully manual approach.
Collapse
|
6
|
Bai C, Qian J, Dang S, Peng T, Min J, Lei M, Dan D, Yao B. Full-color optically-sectioned imaging by wide-field microscopy via deep-learning. BIOMEDICAL OPTICS EXPRESS 2020; 11:2619-2632. [PMID: 32499948 PMCID: PMC7249807 DOI: 10.1364/boe.389852] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/05/2020] [Revised: 03/10/2020] [Accepted: 04/11/2020] [Indexed: 06/11/2023]
Abstract
Wide-field microscopy (WFM) is broadly used in experimental studies of biological specimens. However, combining the out-of-focus signals with the in-focus plane reduces the signal-to-noise ratio (SNR) and axial resolution of the image. Therefore, structured illumination microscopy (SIM) with white light illumination has been used to obtain full-color 3D images, which can capture high SNR optically-sectioned images with improved axial resolution and natural specimen colors. Nevertheless, this full-color SIM (FC-SIM) has a data acquisition burden for 3D-image reconstruction with a shortened depth-of-field, especially for thick samples such as insects and large-scale 3D imaging using stitching techniques. In this paper, we propose a deep-learning-based method for full-color WFM, i.e., FC-WFM-Deep, which can reconstruct high-quality full-color 3D images with an extended optical sectioning capability directly from the FC-WFM z-stack data. Case studies of different specimens with a specific imaging system are used to illustrate this method. Consequently, the image quality achievable with this FC-WFM-Deep method is comparable to the FC-SIM method in terms of 3D information and spatial resolution, while the reconstruction data size is 21-fold smaller and the in-focus depth is doubled. This technique significantly reduces the 3D data acquisition requirements without losing detail and improves the 3D imaging speed by extracting the optical sectioning in the depth-of-field. This cost-effective and convenient method offers a promising tool to observe high-precision color 3D spatial distributions of biological samples.
Collapse
Affiliation(s)
- Chen Bai
- State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China
- These authors contributed equally to this work
| | - Jia Qian
- State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China
- University of Chinese Academy of Sciences, Beijing 100049, China
- These authors contributed equally to this work
| | - Shipei Dang
- State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China
- University of Chinese Academy of Sciences, Beijing 100049, China
| | - Tong Peng
- State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China
| | - Junwei Min
- State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China
| | - Ming Lei
- State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China
| | - Dan Dan
- State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China
| | - Baoli Yao
- State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China
| |
Collapse
|
7
|
Li A, Guan Y, Gong H, Luo Q. Challenges of Processing and Analyzing Big Data in Mesoscopic Whole-brain Imaging. GENOMICS, PROTEOMICS & BIOINFORMATICS 2019; 17:337-343. [PMID: 31805368 PMCID: PMC6943785 DOI: 10.1016/j.gpb.2019.10.001] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/17/2019] [Revised: 09/15/2019] [Accepted: 10/12/2019] [Indexed: 01/09/2023]
Affiliation(s)
- Anan Li
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China; MOE Key Laboratory for Biomedical Photonics, School of Engineering Sciences, Huazhong University of Science and Technology, Wuhan 430074, China; HUST-Suzhou Institute for Brainsmatics, JITRI Institute for Brainsmatics, Suzhou 215125, China
| | - Yue Guan
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China; MOE Key Laboratory for Biomedical Photonics, School of Engineering Sciences, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Hui Gong
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China; MOE Key Laboratory for Biomedical Photonics, School of Engineering Sciences, Huazhong University of Science and Technology, Wuhan 430074, China; HUST-Suzhou Institute for Brainsmatics, JITRI Institute for Brainsmatics, Suzhou 215125, China
| | - Qingming Luo
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China; MOE Key Laboratory for Biomedical Photonics, School of Engineering Sciences, Huazhong University of Science and Technology, Wuhan 430074, China.
| |
Collapse
|
8
|
Záborszky L, Gombkoto P, Varsanyi P, Gielow MR, Poe G, Role LW, Ananth M, Rajebhosale P, Talmage DA, Hasselmo ME, Dannenberg H, Minces VH, Chiba AA. Specific Basal Forebrain-Cortical Cholinergic Circuits Coordinate Cognitive Operations. J Neurosci 2018; 38:9446-9458. [PMID: 30381436 PMCID: PMC6209837 DOI: 10.1523/jneurosci.1676-18.2018] [Citation(s) in RCA: 118] [Impact Index Per Article: 19.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2018] [Revised: 09/11/2018] [Accepted: 09/12/2018] [Indexed: 11/21/2022] Open
Abstract
Based on recent molecular genetics, as well as functional and quantitative anatomical studies, the basal forebrain (BF) cholinergic projections, once viewed as a diffuse system, are emerging as being remarkably specific in connectivity. Acetylcholine (ACh) can rapidly and selectively modulate activity of specific circuits and ACh release can be coordinated in multiple areas that are related to particular aspects of cognitive processing. This review discusses how a combination of multiple new approaches with more established techniques are being used to finally reveal how cholinergic neurons, together with other BF neurons, provide temporal structure for behavior, contribute to local cortical state regulation, and coordinate activity between different functionally related cortical circuits. ACh selectively modulates dynamics for encoding and attention within individual cortical circuits, allows for important transitions during sleep, and shapes the fidelity of sensory processing by changing the correlation structure of neural firing. The importance of this system for integrated and fluid behavioral function is underscored by its disease-modifying role; the demise of BF cholinergic neurons has long been established in Alzheimer's disease and recent studies have revealed the involvement of the cholinergic system in modulation of anxiety-related circuits. Therefore, the BF cholinergic system plays a pivotal role in modulating the dynamics of the brain during sleep and behavior, as foretold by the intricacies of its anatomical map.
Collapse
Affiliation(s)
- Laszlo Záborszky
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark 07102,
| | - Peter Gombkoto
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark 07102
| | - Peter Varsanyi
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark 07102
| | - Matthew R Gielow
- Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark 07102
| | - Gina Poe
- Department of Integrative Biology and Physiology, University of California, Los Angeles 90095
| | - Lorna W Role
- Department of Neurobiology and Center for Nervous System Disorders, Stony Brook University, Stony Brook, New York 11794
| | - Mala Ananth
- Program in Neuroscience and Center for Nervous System Disorders, Stony Brook University, Stony Brook, New York 11794
| | - Prithviraj Rajebhosale
- Program in Neuroscience and Center for Nervous System Disorders, Stony Brook University, Stony Brook, New York 11794
| | - David A Talmage
- Department of Pharmacological Sciences and Center for Nervous System Disorders, Stony Brook University, Stony Brook, New York 11794
| | - Michael E Hasselmo
- Center for Systems Neuroscience and Department of Psychological and Brain Sciences, Boston University, Boston, Massachusetts 02215, and
| | - Holger Dannenberg
- Center for Systems Neuroscience and Department of Psychological and Brain Sciences, Boston University, Boston, Massachusetts 02215, and
| | - Victor H Minces
- Department of Cognitive Science, University of California, San Diego 92093
| | - Andrea A Chiba
- Department of Cognitive Science, University of California, San Diego 92093
| |
Collapse
|