Visualization of Neuronal Structures in Wide-Field Microscopy Brain Images

NeuRegenerate: A Framework for Visualizing Neurodegeneration

Recent advances in high-resolution microscopy have allowed scientists to better understand the underlying brain connectivity. However, due to the limitation that biological specimens can only be imaged at a single timepoint, studying changes to neural projections over time is limited to observations gathered using population analysis. In this article, we introduce NeuRegenerate, a novel end-to-end framework for the prediction and visualization of changes in neural fiber morphology within a subject across specified age-timepoints. To predict projections, we present neuReGANerator, a deep-learning network based on cycle-consistent generative adversarial network (GAN) that translates features of neuronal structures across age-timepoints for large brain microscopy volumes. We improve the reconstruction quality of the predicted neuronal structures by implementing a density multiplier and a new loss function, called the hallucination loss. Moreover, to alleviate artifacts that occur due to tiling of large input volumes, we introduce a spatial-consistency module in the training pipeline of neuReGANerator. Finally, to visualize the change in projections, predicted using neuReGANerator, NeuRegenerate offers two modes: (i) neuroCompare to simultaneously visualize the difference in the structures of the neuronal projections, from two age domains (using structural view and bounded view), and (ii) neuroMorph, a vesselness-based morphing technique to interactively visualize the transformation of the structures from one age-timepoint to the other. Our framework is designed specifically for volumes acquired using wide-field microscopy. We demonstrate our framework by visualizing the structural changes within the cholinergic system of the mouse brain between a young and old specimen.

Visualization and Visual Analytics Approaches for Image and Video Datasets: A Survey

Image and video data analysis has become an increasingly important research area with applications in different domains such as security surveillance, healthcare, augmented and virtual reality, video and image editing, activity analysis and recognition, synthetic content generation, distance education, telepresence, remote sensing, sports analytics, art, non-photorealistic rendering, search engines, and social media. Recent advances in Artificial Intelligence (AI) and particularly deep learning have sparked new research challenges and led to significant advancements, especially in image and video analysis. These advancements have also resulted in significant research and development in other areas such as visualization and visual analytics, and have created new opportunities for future lines of research. In this survey paper, we present the current state of the art at the intersection of visualization and visual analytics, and image and video data analysis. We categorize the visualization papers included in our survey based on different taxonomies used in visualization and visual analytics research. We review these papers in terms of task requirements, tools, datasets, and application areas. We also discuss insights based on our survey results, trends and patterns, the current focus of visualization research, and opportunities for future research.

NeuroConstruct: 3D Reconstruction and Visualization of Neurites in Optical Microscopy Brain Images

We introduce NeuroConstruct, a novel end-to-end application for the segmentation, registration, and visualization of brain volumes imaged using wide-field microscopy. NeuroConstruct offers a Segmentation Toolbox with various annotation helper functions that aid experts to effectively and precisely annotate micrometer resolution neurites. It also offers an automatic neurites segmentation using convolutional neuronal networks (CNN) trained by the Toolbox annotations and somas segmentation using thresholding. To visualize neurites in a given volume, NeuroConstruct offers a hybrid rendering by combining iso-surface rendering of high-confidence classified neurites, along with real-time rendering of raw volume using a 2D transfer function for voxel classification score versus voxel intensity value. For a complete reconstruction of the 3D neurites, we introduce a Registration Toolbox that provides automatic coarse-to-fine alignment of serially sectioned samples. The quantitative and qualitative analysis show that NeuroConstruct outperforms the state-of-the-art in all design aspects. NeuroConstruct was developed as a collaboration between computer scientists and neuroscientists, with an application to the study of cholinergic neurons, which are severely affected in Alzheimer's disease.

Three-dimensional microvascular network reconstruction from in vivo images with adaptation of the regional inhomogeneity in the signal-to-noise ratio

OBJECTIVE

Quantification of angiographic images with two-photon laser scanning fluorescence microscopy (2PLSM) relies on proper segmentation of the vascular images. However, the images contain inhomogeneities in the signal-to-noise ratio (SNR) arising from regional effects of light scattering and absorption. The present study developed a semiautomated quantification method for volume images of 2PLSM angiography by adjusting the binarization threshold according to local SNR along the vessel centerlines.

METHODS

A phantom model made with fluorescent microbeads was used to incorporate a region-dependent binarization threshold.

RESULTS

The recommended SNR for imaging was found to be 4.2-10.6 that provide the true size of imaged objects if the binarization threshold was fixed at 50% of SNR. However, angiographic images in the mouse cortex showed variable SNR up to 45 over the depths. To minimize the errors caused by variable SNR and a spatial extent of the imaged objects in an axial direction, the microvascular networks were three-dimensionally reconstructed based on the cross-sectional diameters measured along the vessel centerline from the XY-plane images with adapted binarization threshold. The arterial volume was relatively constant over depths of 0-500 µm, and the capillary volume (1.7% relative to the scanned volume) showed the larger volumes than the artery (0.8%) and vein (0.6%).

CONCLUSIONS

The present methods allow consistent segmentation of microvasculature by adapting the local inhomogeneity in the SNR, which will be useful for quantitative comparison of the microvascular networks, such as under disease conditions where SNR in the 2PLSM images varies over space and time.

Improving the Usability of Virtual Reality Neuron Tracing with Topological Elements

Researchers in the field of connectomics are working to reconstruct a map of neural connections in the brain in order to understand at a fundamental level how the brain processes information. Constructing this wiring diagram is done by tracing neurons through high-resolution image stacks acquired with fluorescence microscopy imaging techniques. While a large number of automatic tracing algorithms have been proposed, these frequently rely on local features in the data and fail on noisy data or ambiguous cases, requiring time-consuming manual correction. As a result, manual and semi-automatic tracing methods remain the state-of-the-art for creating accurate neuron reconstructions. We propose a new semi-automatic method that uses topological features to guide users in tracing neurons and integrate this method within a virtual reality (VR) framework previously used for manual tracing. Our approach augments both visualization and interaction with topological elements, allowing rapid understanding and tracing of complex morphologies. In our pilot study, neuroscientists demonstrated a strong preference for using our tool over prior approaches, reported less fatigue during tracing, and commended the ability to better understand possible paths and alternatives. Quantitative evaluation of the traces reveals that users' tracing speed increased, while retaining similar accuracy compared to a fully manual approach.

Full-color optically-sectioned imaging by wide-field microscopy via deep-learning

Wide-field microscopy (WFM) is broadly used in experimental studies of biological specimens. However, combining the out-of-focus signals with the in-focus plane reduces the signal-to-noise ratio (SNR) and axial resolution of the image. Therefore, structured illumination microscopy (SIM) with white light illumination has been used to obtain full-color 3D images, which can capture high SNR optically-sectioned images with improved axial resolution and natural specimen colors. Nevertheless, this full-color SIM (FC-SIM) has a data acquisition burden for 3D-image reconstruction with a shortened depth-of-field, especially for thick samples such as insects and large-scale 3D imaging using stitching techniques. In this paper, we propose a deep-learning-based method for full-color WFM, i.e., FC-WFM-Deep, which can reconstruct high-quality full-color 3D images with an extended optical sectioning capability directly from the FC-WFM z-stack data. Case studies of different specimens with a specific imaging system are used to illustrate this method. Consequently, the image quality achievable with this FC-WFM-Deep method is comparable to the FC-SIM method in terms of 3D information and spatial resolution, while the reconstruction data size is 21-fold smaller and the in-focus depth is doubled. This technique significantly reduces the 3D data acquisition requirements without losing detail and improves the 3D imaging speed by extracting the optical sectioning in the depth-of-field. This cost-effective and convenient method offers a promising tool to observe high-precision color 3D spatial distributions of biological samples.

Specific Basal Forebrain-Cortical Cholinergic Circuits Coordinate Cognitive Operations

Based on recent molecular genetics, as well as functional and quantitative anatomical studies, the basal forebrain (BF) cholinergic projections, once viewed as a diffuse system, are emerging as being remarkably specific in connectivity. Acetylcholine (ACh) can rapidly and selectively modulate activity of specific circuits and ACh release can be coordinated in multiple areas that are related to particular aspects of cognitive processing. This review discusses how a combination of multiple new approaches with more established techniques are being used to finally reveal how cholinergic neurons, together with other BF neurons, provide temporal structure for behavior, contribute to local cortical state regulation, and coordinate activity between different functionally related cortical circuits. ACh selectively modulates dynamics for encoding and attention within individual cortical circuits, allows for important transitions during sleep, and shapes the fidelity of sensory processing by changing the correlation structure of neural firing. The importance of this system for integrated and fluid behavioral function is underscored by its disease-modifying role; the demise of BF cholinergic neurons has long been established in Alzheimer's disease and recent studies have revealed the involvement of the cholinergic system in modulation of anxiety-related circuits. Therefore, the BF cholinergic system plays a pivotal role in modulating the dynamics of the brain during sleep and behavior, as foretold by the intricacies of its anatomical map.