Computational methods and challenges for large-scale circuit mapping

Unsupervised classification of brain-wide axons reveals the presubiculum neuronal projection blueprint

We present a quantitative strategy to identify all projection neuron types from a given region with statistically different patterns of anatomical targeting. We first validate the technique with mouse primary motor cortex layer 6 data, yielding two clusters consistent with cortico-thalamic and intra-telencephalic neurons. We next analyze the presubiculum, a less-explored region, identifying five classes of projecting neurons with unique patterns of divergence, convergence, and specificity. We report several findings: individual classes target multiple subregions along defined functions; all hypothalamic regions are exclusively targeted by the same class also invading midbrain and agranular retrosplenial cortex; Cornu Ammonis receives input from a single class of presubicular axons also projecting to granular retrosplenial cortex; path distances from the presubiculum to the same targets differ significantly between classes, as do the path distances to distinct targets within most classes; the identified classes have highly non-uniform abundances; and presubicular somata are topographically segregated among classes. This study thus demonstrates that statistically distinct projections shed light on the functional organization of their circuit.

Unsupervised classification of brain-wide axons reveals neuronal projection blueprint

Long-range axonal projections are quintessential determinants of network connectivity, linking cellular organization and circuit architecture. Here we introduce a quantitative strategy to identify, from a given source region, all "projection neuron types" with statistically different patterns of anatomical targeting. We first validate the proposed technique with well-characterized data from layer 6 of the mouse primary motor cortex. The results yield two clusters, consistent with previously discovered cortico-thalamic and intra-telencephalic neuron classes. We next analyze neurons from the presubiculum, a less-explored region. Extending sparse knowledge from earlier retrograde tracing studies, we identify five classes of presubicular projecting neurons, revealing unique patterns of divergence, convergence, and specificity. We thus report several findings: (1) individual classes target multiple subregions along defined functions, such as spatial representation vs. sensory integration and visual vs. auditory input; (2) all hypothalamic regions are exclusively targeted by the same class also invading midbrain, a sharp subset of thalamic nuclei, and agranular retrosplenial cortex; (3) Cornu Ammonis, in contrast, receives input from the same presubicular axons projecting to granular retrosplenial cortex, also the purview of a single class; (4) path distances from the presubiculum to the same targets differ significantly between classes, as do the path distances to distinct targets within most classes, suggesting fine temporal coordination in activating distant areas; (5) the identified classes have highly non-uniform abundances, with substantially more neurons projecting to midbrain and hypothalamus than to medial and lateral entorhinal cortex; (6) lastly, presubicular soma locations are segregated among classes, indicating topographic organization of projections. This study thus demonstrates that classifying neurons based on statistically distinct axonal projection patterns sheds light on the functional organizational of their circuit.

Deep residual contextual and subpixel convolution network for automated neuronal structure segmentation in micro-connectomics

BACKGROUND AND OBJECTIVE

The goal of micro-connectomics research is to reconstruct the connectome and elucidate the mechanisms and functions of the nervous system via electron microscopy (EM). Due to the enormous variety of neuronal structures, neuron segmentation is among most difficult tasks in connectome reconstruction, and neuroanatomists desperately need a reliable neuronal structure segmentation method to reduce the burden of manual labeling and validation.

METHODS

In this article, we proposed an effective deep learning method based on a deep residual contextual and subpixel convolution network to obtain the neuronal structure segmentation in anisotropic EM image stacks. Furthermore, lifted multicut is used for post-processing to optimize the prediction and obtain the reconstruction results.

RESULTS

On the ISBI EM segmentation challenge, the proposed method ranks among the top of the leader board and yields a Rand score of 0.98788. On the public data set of mouse piriform cortex, it achieves a Rand score of 0.9562 and 0.9318 in the different testing stacks. The evaluation scores of our method are significantly improved when compared with those of state-of-the-art methods.

CONCLUSIONS

The proposed automatic method contributes to the development of micro-connectomics, which improves the accuracy of neuronal structure segmentation and provides neuroanatomists with an effective approach to obtain the segmentation and reconstruction of neurons.

Finding the best clearing approach - Towards 3D wide-scale multimodal imaging of aged human brain tissue

The accessibility of new wide-scale multimodal imaging techniques led to numerous clearing techniques emerging over the last decade. However, clearing mesoscopic-sized blocks of aged human brain tissue remains an extremely challenging task. Homogenizing refractive indices and reducing light absorption and scattering are the foundation of tissue clearing. Due to its dense and highly myelinated nature, especially in white matter, the human brain poses particular challenges to clearing techniques. Here, we present a comparative study of seven tissue clearing approaches and their impact on aged human brain tissue blocks (> 5 mm). The goal was to identify the most practical and efficient method in regards to macroscopic transparency, brief clearing time, compatibility with immunohistochemical processing and wide-scale multimodal microscopic imaging. We successfully cleared 26 × 26 × 5 mm³-sized human brain samples with two hydrophilic and two hydrophobic clearing techniques. Optical properties as well as light and antibody penetration depths highly vary between these methods. In addition to finding the best clearing approach, we compared three microscopic imaging setups (the Zeiss Laser Scanning Microscope (LSM) 880 , the Miltenyi Biotec Ultramicroscope ll (UM ll) and the 3i Marianas LightSheet microscope) regarding optimal imaging of large-scale tissue samples. We demonstrate that combining the CLARITY technique (Clear Lipid-exchanged Acrylamide-hybridized Rigid Imaging compatible Tissue hYdrogel) with the Zeiss LSM 880 and combining the iDISCO technique (immunolabeling-enabled three-dimensional imaging of solvent-cleared organs) with the Miltenyi Biotec UM ll are the most practical and efficient approaches to sufficiently clear aged human brain tissue and generate 3D microscopic images. Our results point out challenges that arise from seven clearing and three imaging techniques applied to non-standardized tissue samples such as aged human brain tissue.

Pursuit of precision medicine: Systems biology approaches in Alzheimer's disease mouse models

Alzheimer's disease (AD) is a complex disease that is mediated by numerous factors and manifests in various forms. A systems biology approach to studying AD involves analyses of various body systems, biological scales, environmental elements, and clinical outcomes to understand the genotype to phenotype relationship that potentially drives AD development. Currently, there are many research investigations probing how modifiable and nonmodifiable factors impact AD symptom presentation. This review specifically focuses on how imaging modalities can be integrated into systems biology approaches using model mouse populations to link brain level functional and structural changes to disease onset and progression. Combining imaging and omics data promotes the classification of AD into subtypes and paves the way for precision medicine solutions to prevent and treat AD.

Foreground Estimation in Neuronal Images With a Sparse-Smooth Model for Robust Quantification

3D volume imaging has been regarded as a basic tool to explore the organization and function of the neuronal system. Foreground estimation from neuronal image is essential in the quantification and analysis of neuronal image such as soma counting, neurite tracing and neuron reconstruction. However, the complexity of neuronal structure itself and differences in the imaging procedure, including different optical systems and biological labeling methods, result in various and complex neuronal images, which greatly challenge foreground estimation from neuronal image. In this study, we propose a robust sparse-smooth model (RSSM) to separate the foreground and the background of neuronal image. The model combines the different smoothness levels of the foreground and the background, and the sparsity of the foreground. These prior constraints together contribute to the robustness of foreground estimation from a variety of neuronal images. We demonstrate the proposed RSSM method could promote some best available tools to trace neurites or locate somas from neuronal images with their default parameters, and the quantified results are similar or superior to the results that generated from the original images. The proposed method is proved to be robust in the foreground estimation from different neuronal images, and helps to improve the usability of current quantitative tools on various neuronal images with several applications.

Methods for analyzing neuronal structure and activity in Caenorhabditis elegans

The model research animal Caenorhabditis elegans has unique properties making it particularly advantageous for studies of the nervous system. The nervous system is composed of a stereotyped complement of neurons connected in a consistent manner. Here, we describe methods for studying nervous system structure and function. The transparency of the animal makes it possible to visualize and identify neurons in living animals with fluorescent probes. These methods have been recently enhanced for the efficient use of neuron-specific reporter genes. Because of its simple structure, for a number of years, C. elegans has been at the forefront of connectomic studies defining synaptic connectivity by electron microscopy. This field is burgeoning with new, more powerful techniques, and recommended up-to-date methods are here described that encourage the possibility of new work in C. elegans. Fluorescent probes for single synapses and synaptic connections have allowed verification of the EM reconstructions and for experimental approaches to synapse formation. Advances in microscopy and in fluorescent reporters sensitive to Ca2+ levels have opened the way to observing activity within single neurons across the entire nervous system.

webTDat: A Web-Based, Real-Time, 3D Visualization Framework for Mesoscopic Whole-Brain Images

The popularity of mesoscopic whole-brain imaging techniques has increased dramatically, but these techniques generate teravoxel-sized volumetric image data. Visualizing or interacting with these massive data is both necessary and essential in the bioimage analysis pipeline; however, due to their size, researchers have difficulty using typical computers to process them. The existing solutions do not consider applying web visualization and three-dimensional (3D) volume rendering methods simultaneously to reduce the number of data copy operations and provide a better way to visualize 3D structures in bioimage data. Here, we propose webTDat, an open-source, web-based, real-time 3D visualization framework for mesoscopic-scale whole-brain imaging datasets. webTDat uses an advanced rendering visualization method designed with an innovative data storage format and parallel rendering algorithms. webTDat loads the primary information in the image first and then decides whether it needs to load the secondary information in the image. By performing validation on TB-scale whole-brain datasets, webTDat achieves real-time performance during web visualization. The webTDat framework also provides a rich interface for annotation, making it a useful tool for visualizing mesoscopic whole-brain imaging data.

Semantic segmentation of microscopic neuroanatomical data by combining topological priors with encoder-decoder deep networks

Understanding of neuronal circuitry at cellular resolution within the brain has relied on neuron tracing methods which involve careful observation and interpretation by experienced neuroscientists. With recent developments in imaging and digitization, this approach is no longer feasible with the large scale (terabyte to petabyte range) images. Machine learning based techniques, using deep networks, provide an efficient alternative to the problem. However, these methods rely on very large volumes of annotated images for training and have error rates that are too high for scientific data analysis, and thus requires a significant volume of human-in-the-loop proofreading. Here we introduce a hybrid architecture combining prior structure in the form of topological data analysis methods, based on discrete Morse theory, with the best-in-class deep-net architectures for the neuronal connectivity analysis. We show significant performance gains using our hybrid architecture on detection of topological structure (e.g. connectivity of neuronal processes and local intensity maxima on axons corresponding to synaptic swellings) with precision/recall close to 90% compared with human observers. We have adapted our architecture to a high performance pipeline capable of semantic segmentation of light microscopic whole-brain image data into a hierarchy of neuronal compartments. We expect that the hybrid architecture incorporating discrete Morse techniques into deep nets will generalize to other data domains.

A Brief History of Simulation Neuroscience

Our knowledge of the brain has evolved over millennia in philosophical, experimental and theoretical phases. We suggest that the next phase is simulation neuroscience. The main drivers of simulation neuroscience are big data generated at multiple levels of brain organization and the need to integrate these data to trace the causal chain of interactions within and across all these levels. Simulation neuroscience is currently the only methodology for systematically approaching the multiscale brain. In this review, we attempt to reconstruct the deep historical paths leading to simulation neuroscience, from the first observations of the nerve cell to modern efforts to digitally reconstruct and simulate the brain. Neuroscience began with the identification of the neuron as the fundamental unit of brain structure and function and has evolved towards understanding the role of each cell type in the brain, how brain cells are connected to each other, and how the seemingly infinite networks they form give rise to the vast diversity of brain functions. Neuronal mapping is evolving from subjective descriptions of cell types towards objective classes, subclasses and types. Connectivity mapping is evolving from loose topographic maps between brain regions towards dense anatomical and physiological maps of connections between individual genetically distinct neurons. Functional mapping is evolving from psychological and behavioral stereotypes towards a map of behaviors emerging from structural and functional connectomes. We show how industrialization of neuroscience and the resulting large disconnected datasets are generating demand for integrative neuroscience, how the scale of neuronal and connectivity maps is driving digital atlasing and digital reconstruction to piece together the multiple levels of brain organization, and how the complexity of the interactions between molecules, neurons, microcircuits and brain regions is driving brain simulation to understand the interactions in the multiscale brain.

DeepBouton: Automated Identification of Single-Neuron Axonal Boutons at the Brain-Wide Scale

Fine morphological reconstruction of individual neurons across the entire brain is essential for mapping brain circuits. Inference of presynaptic axonal boutons, as a key part of single-neuron fine reconstruction, is critical for interpreting the patterns of neural circuit wiring schemes. However, automated bouton identification remains challenging for current neuron reconstruction tools, as they focus mainly on neurite skeleton drawing and have difficulties accurately quantifying bouton morphology. Here, we developed an automated method for recognizing single-neuron axonal boutons in whole-brain fluorescence microscopy datasets. The method is based on deep convolutional neural networks and density-peak clustering. High-dimensional feature representations of bouton morphology can be learned adaptively through convolutional networks and used for bouton recognition and subtype classification. We demonstrate that the approach is effective for detecting single-neuron boutons at the brain-wide scale for both long-range pyramidal projection neurons and local interneurons.

Comparative Principles for Next-Generation Neuroscience

Neuroscience is enjoying a renaissance of discovery due in large part to the implementation of next-generation molecular technologies. The advent of genetically encoded tools has complemented existing methods and provided researchers the opportunity to examine the nervous system with unprecedented precision and to reveal facets of neural function at multiple scales. The weight of these discoveries, however, has been technique-driven from a small number of species amenable to the most advanced gene-editing technologies. To deepen interpretation and build on these breakthroughs, an understanding of nervous system evolution and diversity are critical. Evolutionary change integrates advantageous variants of features into lineages, but is also constrained by pre-existing organization and function. Ultimately, each species’ neural architecture comprises both properties that are species-specific and those that are retained and shared. Understanding the evolutionary history of a nervous system provides interpretive power when examining relationships between brain structure and function. The exceptional diversity of nervous systems and their unique or unusual features can also be leveraged to advance research by providing opportunities to ask new questions and interpret findings that are not accessible in individual species. As new genetic and molecular technologies are added to the experimental toolkits utilized in diverse taxa, the field is at a key juncture to revisit the significance of evolutionary and comparative approaches for next-generation neuroscience as a foundational framework for understanding fundamental principles of neural function.

Tracing Tubular Structures from Teravoxel-Sized Microscope Images

Tracing vasculature and neurites from teravoxel sized light-microscopy data-sets is a challenge impeding the availability of processed data to the research community. This is because (1) Holding terabytes of data during run-time is not easy for a regular PC. (2) Processing all the data at once would be slow and inefficient. In this paper, we propose a way to mitigate this challenge by Divide Conquer and Combine (DCC) method. We first split the volume into many smaller and manageable sub-volumes before tracing. These sub-volumes can then be traced individually in parallel (or otherwise). We propose an algorithm to stitch together the traced data from these sub-volumes. This algorithm is robust and handles challenging scenarios like (1) sub-optimal tracing at edges (2) densely packed structures and (3) different depths of trace termination. We validate our results using whole mouse brain vasculature data-set obtained from the Knife-Edge Scanning Microscopy (KESM) based automated tissue scanner.

as-PSOCT: Volumetric microscopic imaging of human brain architecture and connectivity

Polarization sensitive optical coherence tomography (PSOCT) with serial sectioning has enabled the investigation of 3D structures in mouse and human brain tissue samples. By using intrinsic optical properties of back-scattering and birefringence, PSOCT reliably images cytoarchitecture, myeloarchitecture and fiber orientations. In this study, we developed a fully automatic serial sectioning polarization sensitive optical coherence tomography (as-PSOCT) system to enable volumetric reconstruction of human brain samples with unprecedented sample size and resolution. The 3.5 μm in-plane resolution and 50 μm through-plane voxel size allow inspection of cortical layers that are a single-cell in width, as well as small crossing fibers. We show the abilities of as-PSOCT in quantifying layer thicknesses of the cerebellar cortex and creating microscopic tractography of intricate fiber networks in the subcortical nuclei and internal capsule regions, all based on volumetric reconstructions. as-PSOCT provides a viable tool for studying quantitative cytoarchitecture and myeloarchitecture and mapping connectivity with microscopic resolution in the human brain.

TDat: An Efficient Platform for Processing Petabyte-Scale Whole-Brain Volumetric Images

Three-dimensional imaging of whole mammalian brains at single-neuron resolution has generated terabyte (TB)- and even petabyte (PB)-sized datasets. Due to their size, processing these massive image datasets can be hindered by the computer hardware and software typically found in biological laboratories. To fill this gap, we have developed an efficient platform named TDat, which adopts a novel data reformatting strategy by reading cuboid data and employing parallel computing. In data reformatting, TDat is more efficient than any other software. In data accessing, we adopted parallelization to fully explore the capability for data transmission in computers. We applied TDat in large-volume data rigid registration and neuron tracing in whole-brain data with single-neuron resolution, which has never been demonstrated in other studies. We also showed its compatibility with various computing platforms, image processing software and imaging systems.

Toward Whole-Body Connectomics

Recent advances in neuro-technologies have revolutionized knowledge of brain structure and functions. Governments and private organizations worldwide have initiated several large-scale brain connectome projects, to further understand how the brain works at the systems levels. Most recent projects focus on only brain neurons, with the exception of an early effort to reconstruct the 302 neurons that comprise the whole body of the small worm, Caenorhabditis elegans However, to fully elucidate the neural circuitry of complex behavior, it is crucial to understand brain interactions with the whole body, which can be achieved only by mapping the whole-body connectome. In this article, we discuss the current state of connectomics study, focusing on novel optical approaches and related imaging technologies. We also discuss the challenges encountered by scientists who endeavor to map these whole-body connectomes in large animals.

Fast assembling of neuron fragments in serial 3D sections

Reconstructing neurons from 3D image-stacks of serial sections of thick brain tissue is very time-consuming and often becomes a bottleneck in high-throughput brain mapping projects. We developed NeuronStitcher, a software suite for stitching non-overlapping neuron fragments reconstructed in serial 3D image sections. With its efficient algorithm and user-friendly interface, NeuronStitcher has been used successfully to reconstruct very large and complex human and mouse neurons.

Quantification of neuronal density across cortical depth using automated 3D analysis of confocal image stacks

A new framework for measuring densities of immunolabeled neurons across cortical layers was implemented that combines a confocal microscopy sampling strategy with automated analysis of 3D image stacks. Its utility was demonstrated by quantifying neuronal density in macaque cortical areas V1 and V2. A series of overlapping confocal image stacks were acquired, each spanning from the pial surface to the white matter. DAPI channel images were automatically thresholded, and contiguous regions that included multiple clumped nuclear profiles were split using k-means clustering of image pixels for a set of candidate k values determined based on the clump's area; the most likely candidate segmentation was selected based on criteria that capture expected nuclear profile shape and size. The centroids of putative nuclear profiles estimated from 2D images were then grouped across z planes in an image stack to identify the positions of nuclei in x-y-z. 3D centroids falling outside user-specified exclusion boundaries were deleted, nuclei were classified by the presence or absence of signal in a channel corresponding to an immunolabeled antigen (e.g., the pan-neuronal marker NeuN) at the nuclear centroid location, and the set of classified cells was combined across image stacks to estimate density across cortical depth. The method was validated by comparison with conventional stereological methods. The average neuronal density across cortical layers was 230 × 10³ neurons per mm³ in V1 and 130 × 10³ neurons per mm³ in V2. The method is accurate, flexible, and general enough to measure densities of neurons of various molecularly identified types.

Residual Deconvolutional Networks for Brain Electron Microscopy Image Segmentation

Accurate reconstruction of anatomical connections between neurons in the brain using electron microscopy (EM) images is considered to be the gold standard for circuit mapping. A key step in obtaining the reconstruction is the ability to automatically segment neurons with a precision close to human-level performance. Despite the recent technical advances in EM image segmentation, most of them rely on hand-crafted features to some extent that are specific to the data, limiting their ability to generalize. Here, we propose a simple yet powerful technique for EM image segmentation that is trained end-to-end and does not rely on prior knowledge of the data. Our proposed residual deconvolutional network consists of two information pathways that capture full-resolution features and contextual information, respectively. We showed that the proposed model is very effective in achieving the conflicting goals in dense output prediction; namely preserving full-resolution predictions and including sufficient contextual information. We applied our method to the ongoing open challenge of 3D neurite segmentation in EM images. Our method achieved one of the top results on this open challenge. We demonstrated the generality of our technique by evaluating it on the 2D neurite segmentation challenge dataset where consistently high performance was obtained. We thus expect our method to generalize well to other dense output prediction problems.

Clustered structural and functional plasticity of dendritic spines

The configuration of synaptic circuits underlies their ability to process and store information. Research on dendritic spines has revealed that their structural and functional alterations are clustered along the parent dendrite. Here we review the evidence supporting such notion of clustered synaptic plasticity, discuss its functional implications and possible contributing factors, and suggest potential strategies to deal with open challenges.

Analysis of Brain Mitochondria Using Serial Block-Face Scanning Electron Microscopy

Human brain is a high energy consuming organ that mainly relies on glucose as a fuel source. Glucose is catabolized by brain mitochondria via glycolysis, tri-carboxylic acid (TCA) cycle and oxidative phosphorylation (OXPHOS) pathways to produce cellular energy in the form of adenosine triphosphate (ATP). Impairment of mitochondrial ATP production causes mitochondrial disorders, which present clinically with prominent neurological and myopathic symptoms. Mitochondrial defects are also present in neurodevelopmental disorders (e.g. autism spectrum disorder) and neurodegenerative disorders (e.g. amyotrophic lateral sclerosis, Alzheimer's and Parkinson's diseases). Thus, there is an increased interest in the field for performing 3D analysis of mitochondrial morphology, structure and distribution under both healthy and disease states. The brain mitochondrial morphology is extremely diverse, with some mitochondria especially those in the synaptic region being in the range of <200 nm diameter, which is below the resolution limit of traditional light microscopy. Expressing a mitochondrially-targeted green fluorescent protein (GFP) in the brain significantly enhances the organellar detection by confocal microscopy. However, it does not overcome the constraints on the sensitivity of detection of relatively small sized mitochondria without oversaturating the images of large sized mitochondria. While serial transmission electron microscopy has been successfully used to characterize mitochondria at the neuronal synapse, this technique is extremely time-consuming especially when comparing multiple samples. The serial block-face scanning electron microscopy (SBFSEM) technique involves an automated process of sectioning, imaging blocks of tissue and data acquisition. Here, we provide a protocol to perform SBFSEM of a defined region from rodent brain to rapidly reconstruct and visualize mitochondrial morphology. This technique could also be used to provide accurate information on mitochondrial number, volume, size and distribution in a defined brain region. Since the obtained image resolution is high (typically under 10 nm) any gross mitochondrial morphological defects may also be detected.

Optimizing the 3D-reconstruction technique for serial block-face scanning electron microscopy

BACKGROUND

Elucidating the anatomy of neuronal circuits and localizing the synaptic connections between neurons, can give us important insights in how the neuronal circuits work. We are using serial block-face scanning electron microscopy (SBEM) to investigate the anatomy of a collision detection circuit including the Lobula Giant Movement Detector (LGMD) neuron in the locust, Locusta migratoria. For this, thousands of serial electron micrographs are produced that allow us to trace the neuronal branching pattern.

NEW METHOD

The reconstruction of neurons was previously done manually by drawing cell outlines of each cell in each image separately. This approach was very time consuming and troublesome. To make the process more efficient a new interactive software was developed. It uses the contrast between the neuron under investigation and its surrounding for semi-automatic segmentation.

RESULTS

For segmentation the user sets starting regions manually and the algorithm automatically selects a volume within the neuron until the edges corresponding to the neuronal outline are reached. Internally the algorithm optimizes a 3D active contour segmentation model formulated as a cost function taking the SEM image edges into account. This reduced the reconstruction time, while staying close to the manual reference segmentation result.

COMPARISON WITH EXISTING METHODS

Our algorithm is easy to use for a fast segmentation process, unlike previous methods it does not require image training nor an extended computing capacity.

CONCLUSION

Our semi-automatic segmentation algorithm led to a dramatic reduction in processing time for the 3D-reconstruction of identified neurons.

The BRAIN Initiative: developing technology to catalyse neuroscience discovery

The evolution of the field of neuroscience has been propelled by the advent of novel technological capabilities, and the pace at which these capabilities are being developed has accelerated dramatically in the past decade. Capitalizing on this momentum, the United States launched the Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative to develop and apply new tools and technologies for revolutionizing our understanding of the brain. In this article, we review the scientific vision for this initiative set forth by the National Institutes of Health and discuss its implications for the future of neuroscience research. Particular emphasis is given to its potential impact on the mapping and study of neural circuits, and how this knowledge will transform our understanding of the complexity of the human brain and its diverse array of behaviours, perceptions, thoughts and emotions.

NeuroGPS-Tree: automatic reconstruction of large-scale neuronal populations with dense neurites

The reconstruction of neuronal populations, a key step in understanding neural circuits, remains a challenge in the presence of densely packed neurites. Here we achieved automatic reconstruction of neuronal populations by partially mimicking human strategies to separate individual neurons. For populations not resolvable by other methods, we obtained recall and precision rates of approximately 80%. We also demonstrate the reconstruction of 960 neurons within 3 h.

Automated in situ brain imaging for mapping the Drosophila connectome

Mapping the connectome, a wiring diagram of the entire brain, requires large-scale imaging of numerous single neurons with diverse morphology. It is a formidable challenge to reassemble these neurons into a virtual brain and correlate their structural networks with neuronal activities, which are measured in different experiments to analyze the informational flow in the brain. Here, we report an in situ brain imaging technique called Fly Head Array Slice Tomography (FHAST), which permits the reconstruction of structural and functional data to generate an integrative connectome in Drosophila. Using FHAST, the head capsules of an array of flies can be opened with a single vibratome sectioning to expose the brains, replacing the painstaking and inconsistent brain dissection process. FHAST can reveal in situ brain neuroanatomy with minimal distortion to neuronal morphology and maintain intact neuronal connections to peripheral sensory organs. Most importantly, it enables the automated 3D imaging of 100 intact fly brains in each experiment. The established head model with in situ brain neuroanatomy allows functional data to be accurately registered and associated with 3D images of single neurons. These integrative data can then be shared, searched, visualized, and analyzed for understanding how brain-wide activities in different neurons within the same circuit function together to control complex behaviors.

Unsupervised learning of electrocorticography motifs with binary descriptors of wavelet features and hierarchical clustering

We describe a novel method for data mining spectro-spatiotemporal network motifs from electrocorticographic (ECoG) data. The method utilizes wavelet feature extraction from ECoG data, generation of compact binary vectors from these features, and binary vector hierarchical clustering. The potential utility of this method in the discovery of recurring neural patterns is demonstrated in an example showing clustering of ictal and post-ictal gamma activity patterns. The method allows for the efficient and scalable retrieval and clustering of neural motifs occurring in massive amounts of neural data, such as in prolonged EEG/ECoG recordings and in brain computer interfaces.

An efficient conditional random field approach for automatic and interactive neuron segmentation

We present a new graphical-model-based method for automatic and interactive segmentation of neuron structures from electron microscopy (EM) images. For automated reconstruction, our learning based model selects a collection of nodes from a hierarchical merging tree as the proposed segmentation. More specifically, this is achieved by training a conditional random field (CRF) whose underlying graph is the watershed merging tree. The maximum a posteriori (MAP) prediction of the CRF is the output segmentation. Our results are comparable to the results of state-of-the-art methods. Furthermore, both the inference and the training are very efficient as the graph is tree-structured. The problem of neuron segmentation requires extremely high segmentation quality. Therefore, proofreading, namely, interactively correcting mistakes of the automatic method, is a necessary module in the pipeline. Based on our efficient tree-structured inference algorithm, we develop an interactive segmentation framework which only selects locations where the model is uncertain for a user to proofread. The uncertainty is measured by the marginals of the graphical model. Only giving a limited number of choices makes the user interaction very efficient. Based on user corrections, our framework modifies the merging tree and thus improves the segmentation globally.

Large-scale automatic reconstruction of neuronal processes from electron microscopy images

Automated sample preparation and electron microscopy enables acquisition of very large image data sets. These technical advances are of special importance to the field of neuroanatomy, as 3D reconstructions of neuronal processes at the nm scale can provide new insight into the fine grained structure of the brain. Segmentation of large-scale electron microscopy data is the main bottleneck in the analysis of these data sets. In this paper we present a pipeline that provides state-of-the art reconstruction performance while scaling to data sets in the GB-TB range. First, we train a random forest classifier on interactive sparse user annotations. The classifier output is combined with an anisotropic smoothing prior in a Conditional Random Field framework to generate multiple segmentation hypotheses per image. These segmentations are then combined into geometrically consistent 3D objects by segmentation fusion. We provide qualitative and quantitative evaluation of the automatic segmentation and demonstrate large-scale 3D reconstructions of neuronal processes from a 27,000 μm(3) volume of brain tissue over a cube of 30 μm in each dimension corresponding to 1000 consecutive image sections. We also introduce Mojo, a proofreading tool including semi-automated correction of merge errors based on sparse user scribbles.

The circuit architecture of whole brains at the mesoscopic scale

Vertebrate brains of even moderate size are composed of astronomically large numbers of neurons and show a great degree of individual variability at the microscopic scale. This variation is presumably the result of phenotypic plasticity and individual experience. At a larger scale, however, relatively stable species-typical spatial patterns are observed in neuronal architecture, e.g., the spatial distributions of somata and axonal projection patterns, probably the result of a genetically encoded developmental program. The mesoscopic scale of analysis of brain architecture is the transitional point between a microscopic scale where individual variation is prominent and the macroscopic level where a stable, species-typical neural architecture is observed. The empirical existence of this scale, implicit in neuroanatomical atlases, combined with advances in computational resources, makes studying the circuit architecture of entire brains a practical task. A methodology has previously been proposed that employs a shotgun-like grid-based approach to systematically cover entire brain volumes with injections of neuronal tracers. This methodology is being employed to obtain mesoscale circuit maps in mouse and should be applicable to other vertebrate taxa. The resulting large data sets raise issues of data representation, analysis, and interpretation, which must be resolved. Even for data representation the challenges are nontrivial: the conventional approach using regional connectivity matrices fails to capture the collateral branching patterns of projection neurons. Future success of this promising research enterprise depends on the integration of previous neuroanatomical knowledge, partly through the development of suitable computational tools that encapsulate such expertise.

Semi-automatic organelle detection on transmission electron microscopic images

Recent advances in the acquisition of large-scale datasets of transmission electron microscope images have allowed researchers to determine the number and the distribution of subcellular ultrastructures at both the cellular level and the tissue level. For this purpose, it would be very useful to have a computer-assisted system to detect the structures of interest, such as organelles. Using our original image recognition framework CARTA (Clustering-Aided Rapid Training Agent), combined with procedures to highlight and enlarge regions of interest on the image, we have developed a successful method for the semi-automatic detection of plant organelles including mitochondria, amyloplasts, chloroplasts, etioplasts, and Golgi stacks in transmission electron microscope images. Our proposed semi-automatic detection system will be helpful for labelling organelles in the interpretation and/or quantitative analysis of large-scale electron microscope imaging data.

VolRoverN: enhancing surface and volumetric reconstruction for realistic dynamical simulation of cellular and subcellular function

Establishing meaningful relationships between cellular structure and function requires accurate morphological reconstructions. In particular, there is an unmet need for high quality surface reconstructions to model subcellular and synaptic interactions among neurons and glia at nanometer resolution. We address this need with VolRoverN, a software package that produces accurate, efficient, and automated 3D surface reconstructions from stacked 2D contour tracings. While many techniques and tools have been developed in the past for 3D visualization of cellular structure, the reconstructions from VolRoverN meet specific quality criteria that are important for dynamical simulations. These criteria include manifoldness, water-tightness, lack of self- and object-object-intersections, and geometric accuracy. These enhanced surface reconstructions are readily extensible to any cell type and are used here on spiny dendrites with complex morphology and axons from mature rat hippocampal area CA1. Both spatially realistic surface reconstructions and reduced skeletonizations are produced and formatted by VolRoverN for easy input into analysis software packages for neurophysiological simulations at multiple spatial and temporal scales ranging from ion electro-diffusion to electrical cable models.

Brain-mapping projects using the common marmoset

Globally, there is an increasing interest in brain-mapping projects, including the Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative project in the USA, the Human Brain Project (HBP) in Europe, and the Brain Mapping by Integrated Neurotechnologies for Disease Studies (Brain/MINDS) project in Japan. These projects aim to map the structure and function of neuronal circuits to ultimately understand the vast complexity of the human brain. Brain/MINDS is focused on structural and functional mapping of the common marmoset (Callithrix jacchus) brain. This non-human primate has numerous advantages for brain mapping, including a well-developed frontal cortex and a compact brain size, as well as the availability of transgenic technologies. In the present review article, we discuss strategies for structural and functional mapping of the marmoset brain and the relation of the common marmoset to other animals models.

Virtual finger boosts three-dimensional imaging and microsurgery as well as terabyte volume image visualization and analysis

Three-dimensional (3D) bioimaging, visualization and data analysis are in strong need of powerful 3D exploration techniques. We develop virtual finger (VF) to generate 3D curves, points and regions-of-interest in the 3D space of a volumetric image with a single finger operation, such as a computer mouse stroke, or click or zoom from the 2D-projection plane of an image as visualized with a computer. VF provides efficient methods for acquisition, visualization and analysis of 3D images for roundworm, fruitfly, dragonfly, mouse, rat and human. Specifically, VF enables instant 3D optical zoom-in imaging, 3D free-form optical microsurgery, and 3D visualization and annotation of terabytes of whole-brain image volumes. VF also leads to orders of magnitude better efficiency of automated 3D reconstruction of neurons and similar biostructures over our previous systems. We use VF to generate from images of 1,107 Drosophila GAL4 lines a projectome of a Drosophila brain.

Active learning of neuron morphology for accurate automated tracing of neurites

Automating the process of neurite tracing from light microscopy stacks of images is essential for large-scale or high-throughput quantitative studies of neural circuits. While the general layout of labeled neurites can be captured by many automated tracing algorithms, it is often not possible to differentiate reliably between the processes belonging to different cells. The reason is that some neurites in the stack may appear broken due to imperfect labeling, while others may appear fused due to the limited resolution of optical microscopy. Trained neuroanatomists routinely resolve such topological ambiguities during manual tracing tasks by combining information about distances between branches, branch orientations, intensities, calibers, tortuosities, colors, as well as the presence of spines or boutons. Likewise, to evaluate different topological scenarios automatically, we developed a machine learning approach that combines many of the above mentioned features. A specifically designed confidence measure was used to actively train the algorithm during user-assisted tracing procedure. Active learning significantly reduces the training time and makes it possible to obtain less than 1% generalization error rates by providing few training examples. To evaluate the overall performance of the algorithm a number of image stacks were reconstructed automatically, as well as manually by several trained users, making it possible to compare the automated traces to the baseline inter-user variability. Several geometrical and topological features of the traces were selected for the comparisons. These features include the total trace length, the total numbers of branch and terminal points, the affinity of corresponding traces, and the distances between corresponding branch and terminal points. Our results show that when the density of labeled neurites is sufficiently low, automated traces are not significantly different from manual reconstructions obtained by trained users.

Digital reconstruction of the cell body in dense neural circuits using a spherical-coordinated variational model

Mapping the neuronal circuits is essential to understand brain function. Recent technological advancements have made it possible to acquire the brain atlas at single cell resolution. Digital reconstruction of the neural circuits down to this level across the whole brain would significantly facilitate brain studies. However, automatic reconstruction of the dense neural connections from microscopic image still remains a challenge. Here we developed a spherical-coordinate based variational model to reconstruct the shape of the cell body i.e. soma, as one of the procedures for this purpose. When intuitively processing the volumetric images in the spherical coordinate system, the reconstruction of somas with variational model is no longer sensitive to the interference of the complicated neuronal morphology, and could automatically and robustly achieve accurate soma shape regardless of the dense spatial distribution, and diversity in cell size, and morphology. We believe this method would speed drawing the neural circuits and boost brain studies.

NeuroGPS: automated localization of neurons for brain circuits using L1 minimization model

Drawing the map of neuronal circuits at microscopic resolution is important to explain how brain works. Recent progresses in fluorescence labeling and imaging techniques have enabled measuring the whole brain of a rodent like a mouse at submicron-resolution. Considering the huge volume of such datasets, automatic tracing and reconstruct the neuronal connections from the image stacks is essential to form the large scale circuits. However, the first step among which, automated location the soma across different brain areas remains a challenge. Here, we addressed this problem by introducing L1 minimization model. We developed a fully automated system, NeuronGlobalPositionSystem (NeuroGPS) that is robust to the broad diversity of shape, size and density of the neurons in a mouse brain. This method allows locating the neurons across different brain areas without human intervention. We believe this method would facilitate the analysis of the neuronal circuits for brain function and disease studies.

Automated image computing reshapes computational neuroscience

We briefly identify several critical issues in current computational neuroscience, and present our opinions on potential solutions based on bioimage informatics, especially automated image computing.

Cellular-resolution connectomics: challenges of dense neural circuit reconstruction

Neuronal networks are high-dimensional graphs that are packed into three-dimensional nervous tissue at extremely high density. Comprehensively mapping these networks is therefore a major challenge. Although recent developments in volume electron microscopy imaging have made data acquisition feasible for circuits comprising a few hundreds to a few thousands of neurons, data analysis is massively lagging behind. The aim of this perspective is to summarize and quantify the challenges for data analysis in cellular-resolution connectomics and describe current solutions involving online crowd-sourcing and machine-learning approaches.

Automatic landmark annotation and dense correspondence registration for 3D human facial images

Background

Traditional anthropometric studies of human face rely on manual measurements of simple features, which are labor intensive and lack of full comprehensive inference. Dense surface registration of three-dimensional (3D) human facial images holds great potential for high throughput quantitative analyses of complex facial traits. However there is a lack of automatic high density registration method for 3D faical images. Furthermore, current approaches of landmark recognition require further improvement in accuracy to support anthropometric applications.

Result

Here we describe a novel non-rigid registration method for fully automatic 3D facial image mapping. This method comprises two steps: first, seventeen facial landmarks are automatically annotated, mainly via PCA-based feature recognition following 3D-to-2D data transformation. Second, an efficient thin-plate spline (TPS) protocol is used to establish the dense anatomical correspondence between facial images, under the guidance of the predefined landmarks. We demonstrate that this method is highly accurate in landmark recognition, with an average RMS error of ~1.7 mm. The registration process is highly robust, even for different ethnicities.

Conclusion

This method supports fully automatic registration of dense 3D facial images, with 17 landmarks annotated at greatly improved accuracy. A stand-alone software has been implemented to assist high-throughput high-content anthropometric analysis.

The mind-brain relationship as a mathematical problem

This paper aims to frame certain fundamental aspects of the human mind (content and meaning of mental states) and foundational elements of brain computation (spatial and temporal patterns of neural activity) so as to enable at least in principle their integration within one and the same quantitative representation. Through the history of science, similar approaches have been instrumental to bridge other seemingly mysterious scientific phenomena, such as thermodynamics and statistical mechanics, optics and electromagnetism, or chemistry and quantum physics, among several other examples. Identifying the relevant levels of analysis is important to define proper mathematical formalisms for describing the brain and the mind, such that they could be mapped onto each other in order to explain their equivalence. Based on these premises, we overview the potential of neural connectivity to provide highly informative constraints on brain computational process. Moreover, we outline approaches for representing cognitive and emotional states geometrically with semantic maps. Next, we summarize leading theoretical framework that might serve as an explanatory bridge between neural connectivity and mental space. Furthermore, we discuss the implications of this framework for human communication and our view of reality. We conclude by analyzing the practical requirements to manage the necessary data for solving the mind-brain problem from this perspective.