Mesoscale microscopy and image analysis tools for understanding the brain

The Brain Image Library: A Community-Contributed Microscopy Resource for Neuroscientists

Advancements in microscopy techniques and computing technologies have enabled researchers to digitally reconstruct brains at micron scale. As a result, community efforts like the BRAIN Initiative Cell Census Network (BICCN) have generated thousands of whole-brain imaging datasets to trace neuronal circuitry and comprehensively map cell types. This data holds valuable information that extends beyond initial analyses, opening avenues for variation studies and robust classification of cell types in specific brain regions. However, the size and heterogeneity of these imaging data have historically made storage, sharing, and analysis difficult for individual investigators and impractical on a broad community scale. Here, we introduce the Brain Image Library (BIL), a public resource serving the neuroscience community that provides a persistent centralized repository for brain microscopy data. BIL currently holds thousands of brain datasets and provides an integrated analysis ecosystem, allowing for exploration, visualization, and data access without the need to download, thus encouraging scientific discovery and data reuse.

Automated identification of protein expression intensity and classification of protein cellular locations in mouse brain regions from immunofluorescence images

Knowledge of protein expression in mammalian brains at regional and cellular levels can facilitate understanding of protein functions and associated diseases. As the mouse brain is a typical mammalian brain considering cell type and structure, several studies have been conducted to analyze protein expression in mouse brains. However, labeling protein expression using biotechnology is costly and time-consuming. Therefore, automated models that can accurately recognize protein expression are needed. Here, we constructed machine learning models to automatically annotate the protein expression intensity and cellular location in different mouse brain regions from immunofluorescence images. The brain regions and sub-regions were segmented through learning image features using an autoencoder and then performing K-means clustering and registration to align with the anatomical references. The protein expression intensities for those segmented structures were computed on the basis of the statistics of the image pixels, and patch-based weakly supervised methods and multi-instance learning were used to classify the cellular locations. Results demonstrated that the models achieved high accuracy in the expression intensity estimation, and the F1 score of the cellular location prediction was 74.5%. This work established an automated pipeline for analyzing mouse brain images and provided a foundation for further study of protein expression and functions.

Fine-tuning TrailMap: The utility of transfer learning to improve the performance of deep learning in axon segmentation of light-sheet microscopy images

Light-sheet microscopy has made possible the 3D imaging of both fixed and live biological tissue, with samples as large as the entire mouse brain. However, segmentation and quantification of that data remains a time-consuming manual undertaking. Machine learning methods promise the possibility of automating this process. This study seeks to advance the performance of prior models through optimizing transfer learning. We fine-tuned the existing TrailMap model using expert-labeled data from noradrenergic axonal structures in the mouse brain. By changing the cross-entropy weights and using augmentation, we demonstrate a generally improved adjusted F1-score over using the originally trained TrailMap model within our test datasets.

Between neurons and networks: investigating mesoscale brain connectivity in neurological and psychiatric disorders

The study of brain connectivity has been a cornerstone in understanding the complexities of neurological and psychiatric disorders. It has provided invaluable insights into the functional architecture of the brain and how it is perturbed in disorders. However, a persistent challenge has been achieving the proper spatial resolution, and developing computational algorithms to address biological questions at the multi-cellular level, a scale often referred to as the mesoscale. Historically, neuroimaging studies of brain connectivity have predominantly focused on the macroscale, providing insights into inter-regional brain connections but often falling short of resolving the intricacies of neural circuitry at the cellular or mesoscale level. This limitation has hindered our ability to fully comprehend the underlying mechanisms of neurological and psychiatric disorders and to develop targeted interventions. In light of this issue, our review manuscript seeks to bridge this critical gap by delving into the domain of mesoscale neuroimaging. We aim to provide a comprehensive overview of conditions affected by aberrant neural connections, image acquisition techniques, feature extraction, and data analysis methods that are specifically tailored to the mesoscale. We further delineate the potential of brain connectivity research to elucidate complex biological questions, with a particular focus on schizophrenia and epilepsy. This review encompasses topics such as dendritic spine quantification, single neuron morphology, and brain region connectivity. We aim to showcase the applicability and significance of mesoscale neuroimaging techniques in the field of neuroscience, highlighting their potential for gaining insights into the complexities of neurological and psychiatric disorders.

The Locare workflow: representing neuroscience data locations as geometric objects in 3D brain atlases

Neuroscientists employ a range of methods and generate increasing amounts of data describing brain structure and function. The anatomical locations from which observations or measurements originate represent a common context for data interpretation, and a starting point for identifying data of interest. However, the multimodality and abundance of brain data pose a challenge for efforts to organize, integrate, and analyze data based on anatomical locations. While structured metadata allow faceted data queries, different types of data are not easily represented in a standardized and machine-readable way that allow comparison, analysis, and queries related to anatomical relevance. To this end, three-dimensional (3D) digital brain atlases provide frameworks in which disparate multimodal and multilevel neuroscience data can be spatially represented. We propose to represent the locations of different neuroscience data as geometric objects in 3D brain atlases. Such geometric objects can be specified in a standardized file format and stored as location metadata for use with different computational tools. We here present the Locare workflow developed for defining the anatomical location of data elements from rodent brains as geometric objects. We demonstrate how the workflow can be used to define geometric objects representing multimodal and multilevel experimental neuroscience in rat or mouse brain atlases. We further propose a collection of JSON schemas (LocareJSON) for specifying geometric objects by atlas coordinates, suitable as a starting point for co-visualization of different data in an anatomical context and for enabling spatial data queries.

Deep learning in mesoscale brain image analysis: A review

Mesoscale microscopy images of the brain contain a wealth of information which can help us understand the working mechanisms of the brain. However, it is a challenging task to process and analyze these data because of the large size of the images, their high noise levels, the complex morphology of the brain from the cellular to the regional and anatomical levels, the inhomogeneous distribution of fluorescent labels in the cells and tissues, and imaging artifacts. Due to their impressive ability to extract relevant information from images, deep learning algorithms are widely applied to microscopy images of the brain to address these challenges and they perform superiorly in a wide range of microscopy image processing and analysis tasks. This article reviews the applications of deep learning algorithms in brain mesoscale microscopy image processing and analysis, including image synthesis, image segmentation, object detection, and neuron reconstruction and analysis. We also discuss the difficulties of each task and possible directions for further research.

Fine-tuning TrailMap: The utility of transfer learning to improve the performance of deep learning in axon segmentation of light-sheet microscopy images

Light-sheet microscopy has made possible the 3D imaging of both fixed and live biological tissue, with samples as large as the entire mouse brain. However, segmentation and quantification of that data remains a time-consuming manual undertaking. Machine learning methods promise the possibility of automating this process. This study seeks to advance the performance of prior models through optimizing transfer learning. We fine-tuned the existing TrailMap model using expert-labeled data from noradrenergic axonal structures in the mouse brain. By fine-tuning the final two layers of the neural network at a lower learning rate of the TrailMap model, we demonstrate an improved recall and an occasionally improved adjusted F1-score within our test dataset over using the originally trained TrailMap model.

A cellular resolution atlas of Broca's area

Brain cells are arranged in laminar, nuclear, or columnar structures, spanning a range of scales. Here, we construct a reliable cell census in the frontal lobe of human cerebral cortex at micrometer resolution in a magnetic resonance imaging (MRI)-referenced system using innovative imaging and analysis methodologies. MRI establishes a macroscopic reference coordinate system of laminar and cytoarchitectural boundaries. Cell counting is obtained with a digital stereological approach on the 3D reconstruction at cellular resolution from a custom-made inverted confocal light-sheet fluorescence microscope (LSFM). Mesoscale optical coherence tomography enables the registration of the distorted histological cell typing obtained with LSFM to the MRI-based atlas coordinate system. The outcome is an integrated high-resolution cellular census of Broca's area in a human postmortem specimen, within a whole-brain reference space atlas.

Scaling up cell-counting efforts in neuroscience through semi-automated methods

Quantifying how the cellular composition of brain regions vary across development, aging, sex, and disease, is crucial in experimental neuroscience, and the accuracy of different counting methods is continuously debated. Due to the tedious nature of most counting procedures, studies are often restricted to one or a few brain regions. Recently, there have been considerable methodological advances in combining semi-automated feature extraction with brain atlases for cell quantification. Such methods hold great promise for scaling up cell-counting efforts. However, little focus has been paid to how these methods should be implemented and reported to support reproducibility. Here, we provide an overview of practices for conducting and reporting cell counting in mouse and rat brains, showing that critical details for interpretation are typically lacking. We go on to discuss how novel methods may increase efficiency and reproducibility of cell counting studies. Lastly, we provide practical recommendations for researchers planning cell counting.

AtOM, an ontology model to standardize use of brain atlases in tools, workflows, and data infrastructures

Brain atlases are important reference resources for accurate anatomical description of neuroscience data. Open access, three-dimensional atlases serve as spatial frameworks for integrating experimental data and defining regions-of-interest in analytic workflows. However, naming conventions, parcellation criteria, area definitions, and underlying mapping methodologies differ considerably between atlases and across atlas versions. This lack of standardized description impedes use of atlases in analytic tools and registration of data to different atlases. To establish a machine-readable standard for representing brain atlases, we identified four fundamental atlas elements, defined their relations, and created an ontology model. Here we present our Atlas Ontology Model (AtOM) and exemplify its use by applying it to mouse, rat, and human brain atlases. We discuss how AtOM can facilitate atlas interoperability and data integration, thereby increasing compliance with the FAIR guiding principles. AtOM provides a standardized framework for communication and use of brain atlases to create, use, and refer to specific atlas elements and versions. We argue that AtOM will accelerate analysis, sharing, and reuse of neuroscience data.

A neuroscientist's guide to using murine brain atlases for efficient analysis and transparent reporting

Brain atlases are widely used in neuroscience as resources for conducting experimental studies, and for integrating, analyzing, and reporting data from animal models. A variety of atlases are available, and it may be challenging to find the optimal atlas for a given purpose and to perform efficient atlas-based data analyses. Comparing findings reported using different atlases is also not trivial, and represents a barrier to reproducible science. With this perspective article, we provide a guide to how mouse and rat brain atlases can be used for analyzing and reporting data in accordance with the FAIR principles that advocate for data to be findable, accessible, interoperable, and re-usable. We first introduce how atlases can be interpreted and used for navigating to brain locations, before discussing how they can be used for different analytic purposes, including spatial registration and data visualization. We provide guidance on how neuroscientists can compare data mapped to different atlases and ensure transparent reporting of findings. Finally, we summarize key considerations when choosing an atlas and give an outlook on the relevance of increased uptake of atlas-based tools and workflows for FAIR data sharing.

BrainLine: An Open Pipeline for Connectivity Analysis of Heterogeneous Whole-Brain Fluorescence Volumes

Whole-brain fluorescence images require several stages of computational processing to fully reveal the neuron morphology and connectivity information they contain. However, these computational tools are rarely part of an integrated pipeline. Here we present BrainLine, an open-source pipeline that interfaces with existing software to provide registration, axon segmentation, soma detection, visualization and analysis of results. By implementing a feedback based training paradigm with BrainLine, we were able to use a single learning algorithm to accurately process a diverse set of whole-brain images generated by light-sheet microscopy. BrainLine is available as part of our Python package brainlit: http://brainlit.neurodata.io/ .

Multimodal 3D Mouse Brain Atlas Framework with the Skull-Derived Coordinate System

Magnetic resonance imaging (MRI) and light-sheet fluorescence microscopy (LSFM) are technologies that enable non-disruptive 3-dimensional imaging of whole mouse brains. A combination of complementary information from both modalities is desirable for studying neuroscience in general, disease progression and drug efficacy. Although both technologies rely on atlas mapping for quantitative analyses, the translation of LSFM recorded data to MRI templates has been complicated by the morphological changes inflicted by tissue clearing and the enormous size of the raw data sets. Consequently, there is an unmet need for tools that will facilitate fast and accurate translation of LSFM recorded brains to in vivo, non-distorted templates. In this study, we have developed a bidirectional multimodal atlas framework that includes brain templates based on both imaging modalities, region delineations from the Allen's Common Coordinate Framework, and a skull-derived stereotaxic coordinate system. The framework also provides algorithms for bidirectional transformation of results obtained using either MR or LSFM (iDISCO cleared) mouse brain imaging while the coordinate system enables users to easily assign in vivo coordinates across the different brain templates.

Fluorescent transgenic mouse models for whole-brain imaging in health and disease

A paradigm shift is occurring in neuroscience and in general in life sciences converting biomedical research from a descriptive discipline into a quantitative, predictive, actionable science. Living systems are becoming amenable to quantitative description, with profound consequences for our ability to predict biological phenomena. New experimental tools such as tissue clearing, whole-brain imaging, and genetic engineering technologies have opened the opportunity to embrace this new paradigm, allowing to extract anatomical features such as cell number, their full morphology, and even their structural connectivity. These tools will also allow the exploration of new features such as their geometrical arrangement, within and across brain regions. This would be especially important to better characterize brain function and pathological alterations in neurological, neurodevelopmental, and neurodegenerative disorders. New animal models for mapping fluorescent protein-expressing neurons and axon pathways in adult mice are key to this aim. As a result of both developments, relevant cell populations with endogenous fluorescence signals can be comprehensively and quantitatively mapped to whole-brain images acquired at submicron resolution. However, they present intrinsic limitations: weak fluorescent signals, unequal signal strength across the same cell type, lack of specificity of fluorescent labels, overlapping signals in cell types with dense labeling, or undetectable signal at distal parts of the neurons, among others. In this review, we discuss the recent advances in the development of fluorescent transgenic mouse models that overcome to some extent the technical and conceptual limitations and tradeoffs between different strategies. We also discuss the potential use of these strains for understanding disease.

Brain-wide analysis of the supraspinal connectome reveals anatomical correlates to functional recovery after spinal injury

The supraspinal connectome is essential for normal behavior and homeostasis and consists of numerous sensory, motor, and autonomic projections from brain to spinal cord. Study of supraspinal control and its restoration after damage has focused mostly on a handful of major populations that carry motor commands, with only limited consideration of dozens more that provide autonomic or crucial motor modulation. Here, we assemble an experimental workflow to rapidly profile the entire supraspinal mesoconnectome in adult mice and disseminate the output in a web-based resource. Optimized viral labeling, 3D imaging, and registration to a mouse digital neuroanatomical atlas assigned tens of thousands of supraspinal neurons to 69 identified regions. We demonstrate the ability of this approach to clarify essential points of topographic mapping between spinal levels, measure population-specific sensitivity to spinal injury, and test the relationships between region-specific neuronal sparing and variability in functional recovery. This work will spur progress by broadening understanding of essential but understudied supraspinal populations.

Challenges and advances in optical 3D mesoscale imaging

Optical mesoscale imaging is a rapidly developing field that allows the visualisation of larger samples than is possible with standard light microscopy, and fills a gap between cell and organism resolution. It spans from advanced fluorescence imaging of micrometric cell clusters to centimetre-size complete organisms. However, with larger volume specimens, new problems arise. Imaging deeper into tissues at high resolution poses challenges ranging from optical distortions to shadowing from opaque structures. This manuscript discusses the latest developments in mesoscale imaging and highlights limitations, namely labelling, clearing, absorption, scattering, and also sample handling. We then focus on approaches that seek to turn mesoscale imaging into a more quantitative technique, analogous to quantitative tomography in medical imaging, highlighting a future role for digital and physical phantoms as well as artificial intelligence.

Digital Brain Maps and Virtual Neuroscience: An Emerging Role for Light-Sheet Fluorescence Microscopy in Drug Development

The mammalian brain is by far the most advanced organ to have evolved and the underlying biology is extremely complex. However, with aging populations and sedentary lifestyles, the prevalence of neurological disorders is increasing around the world. Consequently, there is a dire need for technologies that can help researchers to better understand the complexity of the brain and thereby accelerate therapies for diseases with origin in the central nervous system. One such technology is light-sheet fluorescence microscopy (LSFM) which in combination with whole organ immunolabelling has made it possible to visualize an intact mouse brain with single cell resolution. However, the price for this level of detail comes in form of enormous datasets that often challenges extraction of quantitative information. One approach for analyzing whole brain data is to align the scanned brains to a reference brain atlas. Having a fixed spatial reference provides each voxel of the sample brains with x-, y-, z-coordinates from which it is possible to obtain anatomical information on the observed fluorescence signal. An additional and important benefit of aligning light sheet data to a reference brain is that the aligned data provides a digital map of gene expression or cell counts which can be deposited in databases or shared with other scientists. This review focuses on the emerging field of virtual neuroscience using digital brain maps and discusses some of challenges incurred when registering LSFM recorded data to a standardized brain template.

The mouse brain after foot shock in four dimensions: Temporal dynamics at a single-cell resolution

Acute stress involves the majority of brain areas, which can be sequentially organized in functional brain networks as shown by our study with foot shock in mice. We used whole-brain microscopy to investigate different spatial resolutions over time. From mesoscale region–based analyses, we identified the order of activation of brain areas. With single-cell analyses, we analyzed shifts in activation over time within small nuclei—a result impossible to achieve with functional MRI’s resolution. These findings required the development of a four-dimensional (4D) analytical pipeline, which is made available as an R package. This “atlas” of foot shock can be visualized in 4D in our interactive web portal.

Acute stress leads to sequential activation of functional brain networks. A biologically relevant question is exactly which (single) cells belonging to brain networks are changed in activity over time after acute stress across the entire brain. We developed a preprocessing and analytical pipeline to chart whole-brain immediate early genes’ expression—as proxy for cellular activity—after a single stressful foot shock in four dimensions: that is, from functional networks up to three-dimensional (3D) single-cell resolution and over time. The pipeline is available as an R package. Most brain areas (96%) showed increased numbers of c-fos+ cells after foot shock, yet hypothalamic areas stood out as being most active and prompt in their activation, followed by amygdalar, prefrontal, hippocampal, and finally, thalamic areas. At the cellular level, c-fos+ density clearly shifted over time across subareas, as illustrated for the basolateral amygdala. Moreover, some brain areas showed increased numbers of c-fos+ cells, while others—like the dentate gyrus—dramatically increased c-fos intensity in just a subset of cells, reminiscent of engrams; importantly, this “strategy” changed after foot shock in half of the brain areas. One of the strengths of our approach is that single-cell data were simultaneously examined across all of the 90 brain areas and can be visualized in 3D in our interactive web portal.

Accurate determination of marker location within whole-brain microscopy images

High-resolution whole-brain microscopy provides a means for post hoc determination of the location of implanted devices and labelled cell populations that are necessary to interpret in vivo experiments designed to understand brain function. Here we have developed two plugins (brainreg and brainreg-segment) for the Python-based image viewer napari, to accurately map any object in a common coordinate space. We analysed the position of dye-labelled electrode tracks and two-photon imaged cell populations expressing fluorescent proteins. The precise location of probes and cells were physiologically interrogated and revealed accurate segmentation with near-cellular resolution.

Seeing the Forest and Its Trees Together: Implementing 3D Light Microscopy Pipelines for Cell Type Mapping in the Mouse Brain

The brain is composed of diverse neuronal and non-neuronal cell types with complex regional connectivity patterns that create the anatomical infrastructure underlying cognition. Remarkable advances in neuroscience techniques enable labeling and imaging of these individual cell types and their interactions throughout intact mammalian brains at a cellular resolution allowing neuroscientists to examine microscopic details in macroscopic brain circuits. Nevertheless, implementing these tools is fraught with many technical and analytical challenges with a need for high-level data analysis. Here we review key technical considerations for implementing a brain mapping pipeline using the mouse brain as a primary model system. Specifically, we provide practical details for choosing methods including cell type specific labeling, sample preparation (e.g., tissue clearing), microscopy modalities, image processing, and data analysis (e.g., image registration to standard atlases). We also highlight the need to develop better 3D atlases with standardized anatomical labels and nomenclature across species and developmental time points to extend the mapping to other species including humans and to facilitate data sharing, confederation, and integrative analysis. In summary, this review provides key elements and currently available resources to consider while developing and implementing high-resolution mapping methods.