The Mind of a Mouse

Fast imaging of millimeter-scale areas with beam deflection transmission electron microscopy

Serial section transmission electron microscopy (TEM) has proven to be one of the leading methods for millimeter-scale 3D imaging of brain tissues at nanoscale resolution. It is important to further improve imaging efficiency to acquire larger and more brain volumes. We report here a threefold increase in the speed of TEM by using a beam deflecting mechanism to enable highly efficient acquisition of multiple image tiles (nine) for each motion of the mechanical stage. For millimeter-scale areas, the duty cycle of imaging doubles to more than 30%, yielding a net average imaging rate of 0.3 gigapixels per second. If fully utilized, an array of four beam deflection TEMs should be capable of imaging a dataset of cubic millimeter scale in five weeks.

Neural Control of Naturalistic Behavior Choices

In the natural world, animals make decisions on an ongoing basis, continuously selecting which action to undertake next. In the lab, however, the neural bases of decision processes have mostly been studied using artificial trial structures. New experimental tools based on the genetic toolkit of model organisms now make it experimentally feasible to monitor and manipulate neural activity in small subsets of neurons during naturalistic behaviors. We thus propose a new approach to investigating decision processes, termed reverse neuroethology. In this approach, experimenters select animal models based on experimental accessibility and then utilize cutting-edge tools such as connectomes and genetically encoded reagents to analyze the flow of information through an animal's nervous system during naturalistic choice behaviors. We describe how the reverse neuroethology strategy has been applied to understand the neural underpinnings of innate, rapid decision making, with a focus on defensive behavioral choices in the vinegar fly Drosophila melanogaster.

A strong direct link from the layer 3/4 border to layer 6 of cat primary visual cortex

The cat primary visual cortex (V1) is a cortical area for which we have one of the most detailed estimates of the connection 'weights' (expressed as number of synapses) between different neural populations in different layers (Binzegger et al in J Neurosci 24:8441-8453, 2004). Nevertheless, the majority of excitatory input sources to layer 6, the deepest layer in a local translaminar excitatory feedforward loop, was not accounted for by the known neuron types used to generate the quantitative Binzegger diagram. We aimed to fill this gap by using a retrograde tracer that would label neural cell bodies in and outside V1 that directly connect to layer 6 of V1. We found that more than 80% of labeled neurons projecting to layer 6 were within V1 itself. Our data indicate that a substantial fraction of the missing input is provided by a previously unidentified population of layer 3/4 border neurons, laterally distributed and connecting more strongly to layer 6 than the typical superficial layer pyramidal neurons considered by Binzegger et al. (Binzegger et al in J Neurosci 24:8441-8453, 2004). This layer 3/4 to layer 6 connection may be a parallel route to the layer 3 - layer 5 - layer 6 feedforward pathway, be associated with the fast-conducting, movement-related Y pathway and provide convergent input from distant (5-10 degrees) regions of the visual field.

SmartEM: machine-learning guided electron microscopy

Connectomics provides essential nanometer-resolution, synapse-level maps of neural circuits to understand brain activity and behavior. However, few researchers have access to the high-throughput electron microscopes necessary to generate enough data for whole circuit or brain reconstruction. To date, machine-learning methods have been used after the collection of images by electron microscopy (EM) to accelerate and improve neuronal segmentation, synapse reconstruction and other data analysis. With the computational improvements in processing EM images, acquiring EM images has now become the rate-limiting step. Here, in order to speed up EM imaging, we integrate machine-learning into real-time image acquisition in a singlebeam scanning electron microscope. This SmartEM approach allows an electron microscope to perform intelligent, data-aware imaging of specimens. SmartEM allocates the proper imaging time for each region of interest - scanning all pixels equally rapidly, then re-scanning small subareas more slowly where a higher quality signal is required to achieve accurate segmentability, in significantly less time. We demonstrate that this pipeline achieves a 7-fold acceleration of image acquisition time for connectomics using a commercial single-beam SEM. We apply SmartEM to reconstruct a portion of mouse cortex with the same accuracy as traditional microscopy but in less time.

Gene-environmental regulation of the postnatal post-mitotic neuronal maturation

Embryonic neurodevelopment, particularly neural progenitor differentiation into post-mitotic neurons, has been extensively studied. While the number and composition of post-mitotic neurons remain relatively constant from birth to adulthood, the brain undergoes significant postnatal maturation marked by major property changes frequently disrupted in neural diseases. This review first summarizes recent characterizations of the functional and molecular maturation of the postnatal nervous system. We then review regulatory mechanisms controlling the precise gene expression changes crucial for the intricate sequence of maturation events, highlighting experience-dependent versus cell-intrinsic genetic timer mechanisms. Despite significant advances in understanding of the gene-environmental regulation of postnatal neuronal maturation, many aspects remain unknown. The review concludes with our perspective on exciting future research directions in the next decade.

Measuring the replicability of our own research

In the study of transgenic mouse models of neurodevelopmental and neurodegenerative disorders, we use batteries of tests to measure deficits in behaviour and from the results of these tests, we make inferences about the mental states of the mice that we interpret as deficits in "learning", "memory", "anxiety", "depression", etc. This paper discusses the problems of determining whether a particular transgenic mouse is a valid mouse model of disease X, the problem of background strains, and the question of whether our behavioural tests are measuring what we say they are. The problem of the reliability of results is then discussed: are they replicable between labs and can we replicate our results in our own lab? This involves the study of intra- and inter- experimenter reliability. The variables that influence replicability and the importance of conducting a complete behavioural phenotype: sensory, motor, cognitive and social emotional behaviour are discussed. Then the thorny question of failure to replicate is examined: Is it a curse or a blessing? Finally, the role of failure in research and what it tells us about our research paradigms is examined.

Photoacoustic Tomography with Temporal Encoding Reconstruction (PATTERN) for cross-modal individual analysis of the whole brain

Cross-modal analysis of the same whole brain is an ideal strategy to uncover brain function and dysfunction. However, it remains challenging due to the slow speed and destructiveness of traditional whole-brain optical imaging techniques. Here we develop a new platform, termed Photoacoustic Tomography with Temporal Encoding Reconstruction (PATTERN), for non-destructive, high-speed, 3D imaging of ex vivo rodent, ferret, and non-human primate brains. Using an optimally designed image acquisition scheme and an accompanying machine-learning algorithm, PATTERN extracts signals of genetically-encoded probes from photobleaching-based temporal modulation and enables reliable visualization of neural projection in the whole central nervous system with 3D isotropic resolution. Without structural and biological perturbation to the sample, PATTERN can be combined with other whole-brain imaging modalities to acquire the whole-brain image with both high resolution and morphological fidelity. Furthermore, cross-modal transcriptome analysis of an individual brain is achieved by PATTERN imaging. Together, PATTERN provides a compatible and versatile strategy for brain-wide cross-modal analysis at the individual level.

RoboEM: automated 3D flight tracing for synaptic-resolution connectomics

Mapping neuronal networks from three-dimensional electron microscopy (3D-EM) data still poses substantial reconstruction challenges, in particular for thin axons. Currently available automated image segmentation methods require manual proofreading for many types of connectomic analysis. Here we introduce RoboEM, an artificial intelligence-based self-steering 3D 'flight' system trained to navigate along neurites using only 3D-EM data as input. Applied to 3D-EM data from mouse and human cortex, RoboEM substantially improves automated state-of-the-art segmentations and can replace manual proofreading for more complex connectomic analysis problems, yielding computational annotation cost for cortical connectomes about 400-fold lower than the cost of manual error correction.

Fibration symmetries and cluster synchronization in the Caenorhabditis elegans connectome

Capturing how the Caenorhabditis elegans connectome structure gives rise to its neuron functionality remains unclear. It is through fiber symmetries found in its neuronal connectivity that synchronization of a group of neurons can be determined. To understand these we investigate graph symmetries and search for such in the symmetrized versions of the forward and backward locomotive sub-networks of the Caenorhabditi elegans worm neuron network. The use of ordinarily differential equations simulations admissible to these graphs are used to validate the predictions of these fiber symmetries and are compared to the more restrictive orbit symmetries. Additionally fibration symmetries are used to decompose these graphs into their fundamental building blocks which reveal units formed by nested loops or multilayered fibers. It is found that fiber symmetries of the connectome can accurately predict neuronal synchronization even under not idealized connectivity as long as the dynamics are within stable regimes of simulations.

Exploration-Exploitation Paradigm for Networked Biological Systems

The stochastic exploration of the configuration space and the exploitation of functional states underlie many biological processes. The evolutionary dynamics stands out as a remarkable example. Here, we introduce a novel formalism that mimics evolution and encodes a general exploration-exploitation dynamics for biological networks. We apply it to the brain wiring problem, focusing on the maturation of that of the nematode Caenorhabditis elegans. We demonstrate that a parsimonious maxent description of the adult brain combined with our framework is able to track down the entire developmental trajectory.

Network Statistics of the Whole-Brain Connectome of Drosophila

Brains comprise complex networks of neurons and connections. Network analysis applied to the wiring diagrams of brains can offer insights into how brains support computations and regulate information flow. The completion of the first whole-brain connectome of an adult Drosophila, the largest connectome to date, containing 130,000 neurons and millions of connections, offers an unprecedented opportunity to analyze its network properties and topological features. To gain insights into local connectivity, we computed the prevalence of two- and three-node network motifs, examined their strengths and neurotransmitter compositions, and compared these topological metrics with wiring diagrams of other animals. We discovered that the network of the fly brain displays rich club organization, with a large population (30% percent of the connectome) of highly connected neurons. We identified subsets of rich club neurons that may serve as integrators or broadcasters of signals. Finally, we examined subnetworks based on 78 anatomically defined brain regions or neuropils. These data products are shared within the FlyWire Codex and will serve as a foundation for models and experiments exploring the relationship between neural activity and anatomical structure.

Connectivity Matrix Seriation via Relaxation

Volume electron microscopy together with computer-based image analysis are yielding neural circuit diagrams of ever larger regions of the brain. These datasets are usually represented in a cell-to-cell connectivity matrix and contain important information about prevalent circuit motifs allowing to directly test various theories on the computation in that brain structure. Of particular interest are the detection of cell assemblies and the quantification of feedback, which can profoundly change circuit properties. While the ordering of cells along the rows and columns doesn't change the connectivity, it can make special connectivity patterns recognizable. For example, ordering the cells along the flow of information, feedback and feedforward connections are segregated above and below the main matrix diagonal, respectively. Different algorithms are used to renumber matrices such as to minimize a given cost function, but either their performance becomes unsatisfying at a given size of the circuit or the CPU time needed to compute them scales in an unfavorable way with increasing number of neurons. Based on previous ideas, I describe an algorithm which is effective in matrix reordering with respect to both its performance as well as to its scaling in computing time. Rather than trying to reorder the matrix in discrete steps, the algorithm transiently relaxes the integer program by assigning a real-valued parameter to each cell describing its location on a continuous axis ('smooth-index') and finds the parameter set that minimizes the cost. I find that the smooth-index algorithm outperforms all algorithms I compared it to, including those based on topological sorting.

Cell type-specific connectome predicts distributed working memory activity in the mouse brain

Recent advances in connectomics and neurophysiology make it possible to probe whole-brain mechanisms of cognition and behavior. We developed a large-scale model of the multiregional mouse brain for a cardinal cognitive function called working memory, the brain's ability to internally hold and process information without sensory input. The model is built on mesoscopic connectome data for interareal cortical connections and endowed with a macroscopic gradient of measured parvalbumin-expressing interneuron density. We found that working memory coding is distributed yet exhibits modularity; the spatial pattern of mnemonic representation is determined by long-range cell type-specific targeting and density of cell classes. Cell type-specific graph measures predict the activity patterns and a core subnetwork for memory maintenance. The model shows numerous attractor states, which are self-sustained internal states (each engaging a distinct subset of areas). This work provides a framework to interpret large-scale recordings of brain activity during cognition, while highlighting the need for cell type-specific connectomics.

Analyzing Thalamocortical Tract-Tracing Experiments in a Common Reference Space

Current mesoscale connectivity atlases provide limited information about the organization of thalamocortical projections in the mouse brain. Labeling the projections of spatially restricted neuron populations in thalamus can provide a functionally relevant level of connectomic analysis, but these need to be integrated within the same common reference space. Here, we present a pipeline for the segmentation, registration, integration and analysis of multiple tract-tracing experiments. The key difference with other workflows is that the data is transformed to fit the reference template. As a test-case, we investigated the axonal projections and intranuclear arrangement of seven neuronal populations of the ventral posteromedial nucleus of the thalamus (VPM), which we labeled with an anterograde tracer. Their soma positions corresponded, from dorsal to ventral, to cortical representations of the whiskers, nose and mouth. They strongly targeted layer 4, with the majority exclusively targeting one cortical area and the ones in ventrolateral VPM branching to multiple somatosensory areas. We found that our experiments were more topographically precise than similar experiments from the Allen Institute and projections to the primary somatosensory area were in agreement with single-neuron morphological reconstructions from publicly available databases. This pilot study sets the basis for a shared virtual connectivity atlas that could be enriched with additional data for studying the topographical organization of different thalamic nuclei. The pipeline is accessible with only minimal programming skills via a Jupyter Notebook, and offers multiple visualization tools such as cortical flatmaps, subcortical plots and 3D renderings and can be used with custom anatomical delineations.

Combinatorial quantification of distinct neural projections from retrograde tracing

Comprehensive quantification of neuronal architectures underlying anatomical brain connectivity remains challenging. We introduce a method to identify distinct axonal projection patterns from a source to a set of target regions and the count of neurons with each pattern. A source region projecting to n targets could have 2n-1 theoretically possible projection types, although only a subset of these types typically exists. By injecting uniquely labeled retrograde tracers in k target regions (k < n), one can experimentally count the cells expressing different color combinations in the source region. The neuronal counts for different color combinations from n-choose-k experiments provide constraints for a model that is robustly solvable using evolutionary algorithms. Here, we demonstrate this method's reliability for 4 targets using simulated triple injection experiments. Furthermore, we illustrate the experimental application of this framework by quantifying the projections of male mouse primary motor cortex to the primary and secondary somatosensory and motor cortices.

Neural signal propagation atlas of Caenorhabditis elegans

Establishing how neural function emerges from network properties is a fundamental problem in neuroscience1. Here, to better understand the relationship between the structure and the function of a nervous system, we systematically measure signal propagation in 23,433 pairs of neurons across the head of the nematode Caenorhabditis elegans by direct optogenetic activation and simultaneous whole-brain calcium imaging. We measure the sign (excitatory or inhibitory), strength, temporal properties and causal direction of signal propagation between these neurons to create a functional atlas. We find that signal propagation differs from model predictions that are based on anatomy. Using mutants, we show that extrasynaptic signalling not visible from anatomy contributes to this difference. We identify many instances of dense-core-vesicle-dependent signalling, including on timescales of less than a second, that evoke acute calcium transients-often where no direct wired connection exists but where relevant neuropeptides and receptors are expressed. We propose that, in such cases, extrasynaptically released neuropeptides serve a similar function to that of classical neurotransmitters. Finally, our measured signal propagation atlas better predicts the neural dynamics of spontaneous activity than do models based on anatomy. We conclude that both synaptic and extrasynaptic signalling drive neural dynamics on short timescales, and that measurements of evoked signal propagation are crucial for interpreting neural function.

Shadow imaging for panoptical visualization of brain tissue in vivo

Progress in neuroscience research hinges on technical advances in visualizing living brain tissue with high fidelity and facility. Current neuroanatomical imaging approaches either require tissue fixation (electron microscopy), do not have cellular resolution (magnetic resonance imaging) or only give a fragmented view (fluorescence microscopy). Here, we show how regular light microscopy together with fluorescence labeling of the interstitial fluid in the extracellular space provide comprehensive optical access in real-time to the anatomical complexity and dynamics of living brain tissue at submicron scale. Using several common fluorescence microscopy modalities (confocal, light-sheet and 2-photon microscopy) in mouse organotypic and acute brain slices and the intact mouse brain in vivo, we demonstrate the value of this straightforward 'shadow imaging' approach by revealing neurons, microglia, tumor cells and blood capillaries together with their complete anatomical tissue contexts. In addition, we provide quantifications of perivascular spaces and the volume fraction of the extracellular space of brain tissue in vivo.

A Scalable Staining Strategy for Whole-Brain Connectomics

Mapping the complete synaptic connectivity of a mammalian brain would be transformative, revealing the pathways underlying perception, behavior, and memory. Serial section electron microscopy, via membrane staining using osmium tetroxide, is ideal for visualizing cells and synaptic connections but, in whole brain samples, faces significant challenges related to chemical treatment and volume changes. These issues can adversely affect both the ultrastructural quality and macroscopic tissue integrity. By leveraging time-lapse X-ray imaging and brain proxies, we have developed a 12-step protocol, ODeCO, that effectively infiltrates osmium throughout an entire mouse brain while preserving ultrastructure without any cracks or fragmentation, a necessary prerequisite for constructing the first comprehensive mouse brain connectome.

Architectural organization of ∼1,500-neuron modular minicolumnar disinhibitory circuits in healthy and Alzheimer's cortices

Acquisition of neuronal circuit architectures, central to understanding brain function and dysfunction, remains prohibitively challenging. Here I report the development of a simultaneous and sequential octuple-sexdecuple whole-cell patch-clamp recording system that enables architectural reconstruction of complex cortical circuits. The method unveils the canonical layer 1 single bouquet cell (SBC)-led disinhibitory neuronal circuits across the mouse somatosensory, motor, prefrontal, and medial entorhinal cortices. The ∼1,500-neuron modular circuits feature the translaminar, unidirectional, minicolumnar, and independent disinhibition and optimize cortical complexity, subtlety, plasticity, variation, and redundancy. Moreover, architectural reconstruction uncovers age-dependent deficits at SBC-disinhibited synapses in the senescence-accelerated mouse prone 8, an animal model of Alzheimer's disease. The deficits exhibit the characteristic Alzheimer's-like cortical spread and correlation with cognitive impairments. These findings decrypt operations of the elementary processing units in healthy and Alzheimer's mouse cortices and validate the efficacy of octuple-sexdecuple patch-clamp recordings for architectural reconstruction of complex neuronal circuits.

Neuroscience Needs Network Science

The brain is a complex system comprising a myriad of interacting neurons, posing significant challenges in understanding its structure, function, and dynamics. Network science has emerged as a powerful tool for studying such interconnected systems, offering a framework for integrating multiscale data and complexity. To date, network methods have significantly advanced functional imaging studies of the human brain and have facilitated the development of control theory-based applications for directing brain activity. Here, we discuss emerging frontiers for network neuroscience in the brain atlas era, addressing the challenges and opportunities in integrating multiple data streams for understanding the neural transitions from development to healthy function to disease. We underscore the importance of fostering interdisciplinary opportunities through workshops, conferences, and funding initiatives, such as supporting students and postdoctoral fellows with interests in both disciplines. By bringing together the network science and neuroscience communities, we can develop novel network-based methods tailored to neural circuits, paving the way toward a deeper understanding of the brain and its functions, as well as offering new challenges for network science.

How to incorporate biological insights into network models and why it matters

Due to the staggering complexity of the brain and its neural circuitry, neuroscientists rely on the analysis of mathematical models to elucidate its function. From Hodgkin and Huxley's detailed description of the action potential in 1952 to today, new theories and increasing computational power have opened up novel avenues to study how neural circuits implement the computations that underlie behaviour. Computational neuroscientists have developed many models of neural circuits that differ in complexity, biological realism or emergent network properties. With recent advances in experimental techniques for detailed anatomical reconstructions or large-scale activity recordings, rich biological data have become more available. The challenge when building network models is to reflect experimental results, either through a high level of detail or by finding an appropriate level of abstraction. Meanwhile, machine learning has facilitated the development of artificial neural networks, which are trained to perform specific tasks. While they have proven successful at achieving task-oriented behaviour, they are often abstract constructs that differ in many features from the physiology of brain circuits. Thus, it is unclear whether the mechanisms underlying computation in biological circuits can be investigated by analysing artificial networks that accomplish the same function but differ in their mechanisms. Here, we argue that building biologically realistic network models is crucial to establishing causal relationships between neurons, synapses, circuits and behaviour. More specifically, we advocate for network models that consider the connectivity structure and the recorded activity dynamics while evaluating task performance.

Simulations of cortical networks using spatially extended conductance-based neuronal models

The Hodgkin-Huxley model of action potential generation and propagation, published in the Journal of Physiology in 1952, initiated the field of biophysically detailed computational modelling in neuroscience, which has expanded to encompass a variety of species and components of the nervous system. Here we review the developments in this area with a focus on efforts in the community towards modelling the mammalian neocortex using spatially extended conductance-based neuronal models. The Hodgkin-Huxley formalism and related foundational contributions, such as Rall's cable theory, remain widely used in these efforts to the current day. We argue that at present the field is undergoing a qualitative change due to new very rich datasets describing the composition, connectivity and functional activity of cortical circuits, which are being integrated systematically into large-scale network models. This trend, combined with the accelerating development of convenient software tools supporting such complex modelling projects, is giving rise to highly detailed models of the cortex that are extensively constrained by the data, enabling computational investigation of a multitude of questions about cortical structure and function.

CAVE: Connectome Annotation Versioning Engine

Advances in Electron Microscopy, image segmentation and computational infrastructure have given rise to large-scale and richly annotated connectomic datasets which are increasingly shared across communities. To enable collaboration, users need to be able to concurrently create new annotations and correct errors in the automated segmentation by proofreading. In large datasets, every proofreading edit relabels cell identities of millions of voxels and thousands of annotations like synapses. For analysis, users require immediate and reproducible access to this constantly changing and expanding data landscape. Here, we present the Connectome Annotation Versioning Engine (CAVE), a computational infrastructure for immediate and reproducible connectome analysis in up-to petascale datasets (~1mm3) while proofreading and annotating is ongoing. For segmentation, CAVE provides a distributed proofreading infrastructure for continuous versioning of large reconstructions. Annotations in CAVE are defined by locations such that they can be quickly assigned to the underlying segment which enables fast analysis queries of CAVE's data for arbitrary time points. CAVE supports schematized, extensible annotations, so that researchers can readily design novel annotation types. CAVE is already used for many connectomics datasets, including the largest datasets available to date.

Whole-brain annotation and multi-connectome cell typing quantifies circuit stereotypy in Drosophila

The fruit fly Drosophila melanogaster combines surprisingly sophisticated behaviour with a highly tractable nervous system. A large part of the fly's success as a model organism in modern neuroscience stems from the concentration of collaboratively generated molecular genetic and digital resources. As presented in our FlyWire companion paper 1 , this now includes the first full brain connectome of an adult animal. Here we report the systematic and hierarchical annotation of this ~130,000-neuron connectome including neuronal classes, cell types and developmental units (hemilineages). This enables any researcher to navigate this huge dataset and find systems and neurons of interest, linked to the literature through the Virtual Fly Brain database 2 . Crucially, this resource includes 4,552 cell types. 3,094 are rigorous consensus validations of cell types previously proposed in the hemibrain connectome 3 . In addition, we propose 1,458 new cell types, arising mostly from the fact that the FlyWire connectome spans the whole brain, whereas the hemibrain derives from a subvolume. Comparison of FlyWire and the hemibrain showed that cell type counts and strong connections were largely stable, but connection weights were surprisingly variable within and across animals. Further analysis defined simple heuristics for connectome interpretation: connections stronger than 10 unitary synapses or providing >1% of the input to a target cell are highly conserved. Some cell types showed increased variability across connectomes: the most common cell type in the mushroom body, required for learning and memory, is almost twice as numerous in FlyWire as the hemibrain. We find evidence for functional homeostasis through adjustments of the absolute amount of excitatory input while maintaining the excitation-inhibition ratio. Finally, and surprisingly, about one third of the cell types proposed in the hemibrain connectome could not yet be reliably identified in the FlyWire connectome. We therefore suggest that cell types should be defined to be robust to inter-individual variation, namely as groups of cells that are quantitatively more similar to cells in a different brain than to any other cell in the same brain. Joint analysis of the FlyWire and hemibrain connectomes demonstrates the viability and utility of this new definition. Our work defines a consensus cell type atlas for the fly brain and provides both an intellectual framework and open source toolchain for brain-scale comparative connectomics.

mEMbrain: an interactive deep learning MATLAB tool for connectomic segmentation on commodity desktops

Connectomics is fundamental in propelling our understanding of the nervous system's organization, unearthing cells and wiring diagrams reconstructed from volume electron microscopy (EM) datasets. Such reconstructions, on the one hand, have benefited from ever more precise automatic segmentation methods, which leverage sophisticated deep learning architectures and advanced machine learning algorithms. On the other hand, the field of neuroscience at large, and of image processing in particular, has manifested a need for user-friendly and open source tools which enable the community to carry out advanced analyses. In line with this second vein, here we propose mEMbrain, an interactive MATLAB-based software which wraps algorithms and functions that enable labeling and segmentation of electron microscopy datasets in a user-friendly user interface compatible with Linux and Windows. Through its integration as an API to the volume annotation and segmentation tool VAST, mEMbrain encompasses functions for ground truth generation, image preprocessing, training of deep neural networks, and on-the-fly predictions for proofreading and evaluation. The final goals of our tool are to expedite manual labeling efforts and to harness MATLAB users with an array of semi-automatic approaches for instance segmentation. We tested our tool on a variety of datasets that span different species at various scales, regions of the nervous system and developmental stages. To further expedite research in connectomics, we provide an EM resource of ground truth annotation from four different animals and five datasets, amounting to around 180 h of expert annotations, yielding more than 1.2 GB of annotated EM images. In addition, we provide a set of four pre-trained networks for said datasets. All tools are available from https://lichtman.rc.fas.harvard.edu/mEMbrain/. With our software, our hope is to provide a solution for lab-based neural reconstructions which does not require coding by the user, thus paving the way to affordable connectomics.

High-resolution volumetric imaging constrains compartmental models to explore synaptic integration and temporal processing by cochlear nucleus globular bushy cells

Globular bushy cells (GBCs) of the cochlear nucleus play central roles in the temporal processing of sound. Despite investigation over many decades, fundamental questions remain about their dendrite structure, afferent innervation, and integration of synaptic inputs. Here, we use volume electron microscopy (EM) of the mouse cochlear nucleus to construct synaptic maps that precisely specify convergence ratios and synaptic weights for auditory nerve innervation and accurate surface areas of all postsynaptic compartments. Detailed biophysically based compartmental models can help develop hypotheses regarding how GBCs integrate inputs to yield their recorded responses to sound. We established a pipeline to export a precise reconstruction of auditory nerve axons and their endbulb terminals together with high-resolution dendrite, soma, and axon reconstructions into biophysically detailed compartmental models that could be activated by a standard cochlear transduction model. With these constraints, the models predict auditory nerve input profiles whereby all endbulbs onto a GBC are subthreshold (coincidence detection mode), or one or two inputs are suprathreshold (mixed mode). The models also predict the relative importance of dendrite geometry, soma size, and axon initial segment length in setting action potential threshold and generating heterogeneity in sound-evoked responses, and thereby propose mechanisms by which GBCs may homeostatically adjust their excitability. Volume EM also reveals new dendritic structures and dendrites that lack innervation. This framework defines a pathway from subcellular morphology to synaptic connectivity, and facilitates investigation into the roles of specific cellular features in sound encoding. We also clarify the need for new experimental measurements to provide missing cellular parameters, and predict responses to sound for further in vivo studies, thereby serving as a template for investigation of other neuron classes.

Afferent convergence to a shared population of interneuron AMPA receptors

Precise alignment of pre- and postsynaptic elements optimizes the activation of glutamate receptors at excitatory synapses. Nonetheless, glutamate that diffuses out of the synaptic cleft can have actions at distant receptors, a mode of transmission called spillover. To uncover the extrasynaptic actions of glutamate, we localized AMPA receptors (AMPARs) mediating spillover transmission between climbing fibers and molecular layer interneurons in the cerebellar cortex. We found that climbing fiber spillover generates calcium transients mediated by Ca2+-permeable AMPARs at parallel fiber synapses. Spillover occludes parallel fiber synaptic currents, indicating that separate, independently regulated afferent pathways converge onto a common pool of AMPARs. Together these findings demonstrate a circuit motif wherein glutamate 'spill-in' from an unconnected afferent pathway co-opts synaptic receptors, allowing activation of postsynaptic AMPARs even when canonical glutamate release is suppressed.

Connecting Connectomes to Physiology

With the advent of volumetric EM techniques, large connectomic datasets are being created, providing neuroscience researchers with knowledge about the full connectivity of neural circuits under study. This allows for numerical simulation of detailed, biophysical models of each neuron participating in the circuit. However, these models typically include a large number of parameters, and insight into which of these are essential for circuit function is not readily obtained. Here, we review two mathematical strategies for gaining insight into connectomics data: linear dynamical systems analysis and matrix reordering techniques. Such analytical treatment can allow us to make predictions about time constants of information processing and functional subunits in large networks.SIGNIFICANCE STATEMENT This viewpoint provides a concise overview on how to extract important insights from Connectomics data by mathematical methods. First, it explains how new dynamics and new time constants can evolve, simply through connectivity between neurons. These new time-constants can be far longer than the intrinsic membrane time-constants of the individual neurons. Second, it summarizes how structural motifs in the circuit can be discovered. Specifically, there are tools to decide whether or not a circuit is strictly feed-forward or whether feed-back connections exist. Only by reordering connectivity matrices can such motifs be made visible.

High-contrast en bloc staining of mouse whole-brain and human brain samples for EM-based connectomics

Connectomes of human cortical gray matter require high-contrast homogeneously stained samples sized at least 2 mm on a side, and a mouse whole-brain connectome requires samples sized at least 5-10 mm on a side. Here we report en bloc staining and embedding protocols for these and other applications, removing a key obstacle for connectomic analyses at the mammalian whole-brain level.

Comparative connectomics reveals noncanonical wiring for color vision in human foveal retina

The Old World macaque monkey and New World common marmoset provide fundamental models for human visual processing, yet the human ancestral lineage diverged from these monkey lineages over 25 Mya. We therefore asked whether fine-scale synaptic wiring in the nervous system is preserved across these three primate families, despite long periods of independent evolution. We applied connectomic electron microscopy to the specialized foveal retina where circuits for highest acuity and color vision reside. Synaptic motifs arising from the cone photoreceptor type sensitive to short (S) wavelengths and associated with "blue-yellow" (S-ON and S-OFF) color-coding circuitry were reconstructed. We found that distinctive circuitry arises from S cones for each of the three species. The S cones contacted neighboring L and M (long- and middle-wavelength sensitive) cones in humans, but such contacts were rare or absent in macaques and marmosets. We discovered a major S-OFF pathway in the human retina and established its absence in marmosets. Further, the S-ON and S-OFF chromatic pathways make excitatory-type synaptic contacts with L and M cone types in humans, but not in macaques or marmosets. Our results predict that early-stage chromatic signals are distinct in the human retina and imply that solving the human connectome at the nanoscale level of synaptic wiring will be critical for fully understanding the neural basis of human color vision.

A Convenient All-Cell Optical Imaging Method Compatible with Serial SEM for Brain Mapping

The mammalian brain, with its complexity and intricacy, poses significant challenges for researchers aiming to understand its inner workings. Optical multilayer interference tomography (OMLIT) is a novel, promising imaging technique that enables the mapping and reconstruction of mesoscale all-cell brain atlases and is seamlessly compatible with tape-based serial scanning electron microscopy (SEM) for microscale mapping in the same tissue. However, currently, OMLIT suffers from imperfect coatings, leading to background noise and image contamination. In this study, we introduced a new imaging configuration using carbon spraying to eliminate the tape-coating step, resulting in reduced noise and enhanced imaging quality. We demonstrated the improved imaging quality and validated its applicability through a correlative light-electron imaging workflow. Our method successfully reconstructed all cells and vasculature within a large OMLIT dataset, enabling basic morphological classification and analysis. We also show that this approach can perform effectively on thicker sections, extending its applicability to sub-micron scale slices, saving sample preparation and imaging time, and increasing imaging throughput. Consequently, this method emerges as a promising candidate for high-speed, high-throughput brain tissue reconstruction and analysis. Our findings open new avenues for exploring the structure and function of the brain using OMLIT images.

mEMbrain: an interactive deep learning MATLAB tool for connectomic segmentation on commodity desktops

Connectomics is fundamental in propelling our understanding of the nervous system’s organization, unearthing cells and wiring diagrams reconstructed from volume electron microscopy (EM) datasets. Such reconstructions, on the one hand, have benefited from ever more precise automatic segmentation methods, which leverage sophisticated deep learning architectures and advanced machine learning algorithms. On the other hand, the field of neuroscience at large, and of image processing in particular, has manifested a need for user-friendly and open source tools which enable the community to carry out advanced analyses. In line with this second vein, here we propose mEMbrain, an interactive MATLAB-based software which wraps algorithms and functions that enable labeling and segmentation of electron microscopy datasets in a user-friendly user interface compatible with Linux and Windows. Through its integration as an API to the volume annotation and segmentation tool VAST, mEMbrain encompasses functions for ground truth generation, image preprocessing, training of deep neural networks, and on-the-fly predictions for proofreading and evaluation. The final goals of our tool are to expedite manual labeling efforts and to harness MATLAB users with an array of semi-automatic approaches for instance segmentation. We tested our tool on a variety of datasets that span different species at various scales, regions of the nervous system and developmental stages. To further expedite research in connectomics, we provide an EM resource of ground truth annotation from 4 different animals and 5 datasets, amounting to around 180 hours of expert annotations, yielding more than 1.2 GB of annotated EM images. In addition, we provide a set of 4 pre-trained networks for said datasets. All tools are available from https://lichtman.rc.fas.harvard.edu/mEMbrain/ . With our software, our hope is to provide a solution for lab-based neural reconstructions which does not require coding by the user, thus paving the way to affordable connectomics.

Generative network modeling reveals quantitative definitions of bilateral symmetry exhibited by a whole insect brain connectome

Comparing connectomes can help explain how neural connectivity is related to genetics, disease, development, learning, and behavior. However, making statistical inferences about the significance and nature of differences between two networks is an open problem, and such analysis has not been extensively applied to nanoscale connectomes. Here, we investigate this problem via a case study on the bilateral symmetry of a larval Drosophila brain connectome. We translate notions of 'bilateral symmetry' to generative models of the network structure of the left and right hemispheres, allowing us to test and refine our understanding of symmetry. We find significant differences in connection probabilities both across the entire left and right networks and between specific cell types. By rescaling connection probabilities or removing certain edges based on weight, we also present adjusted definitions of bilateral symmetry exhibited by this connectome. This work shows how statistical inferences from networks can inform the study of connectomes, facilitating future comparisons of neural structures.

Integration of reconfigurable microchannels into aligned three-dimensional neural networks for spatially controllable neuromodulation

Anisotropically organized neural networks are indispensable routes for functional connectivity in the brain, which remains largely unknown. While prevailing animal models require additional preparation and stimulation-applying devices and have exhibited limited capabilities regarding localized stimulation, no in vitro platform exists that permits spatiotemporal control of chemo-stimulation in anisotropic three-dimensional (3D) neural networks. We present the integration of microchannels seamlessly into a fibril-aligned 3D scaffold by adapting a single fabrication principle. We investigated the underlying physics of elastic microchannels' ridges and interfacial sol-gel transition of collagen under compression to determine a critical window of geometry and strain. We demonstrated the spatiotemporally resolved neuromodulation in an aligned 3D neural network by local deliveries of KCl and Ca²⁺ signal inhibitors, such as tetrodotoxin, nifedipine, and mibefradil, and also visualized Ca²⁺ signal propagation with a speed of ~3.7 μm/s. We anticipate that our technology will pave the way to elucidate functional connectivity and neurological diseases associated with transsynaptic propagation.

SynapseCLR: Uncovering features of synapses in primary visual cortex through contrastive representation learning

3D electron microscopy (EM) connectomics image volumes are surpassing 1 mm³, providing information-dense, multi-scale visualizations of brain circuitry and necessitating scalable analysis techniques. We present SynapseCLR, a self-supervised contrastive learning method for 3D EM data, and use it to extract features of synapses from mouse visual cortex. SynapseCLR feature representations separate synapses by appearance and functionally important structural annotations. We demonstrate SynapseCLR's utility for valuable downstream tasks, including one-shot identification of defective synapse segmentations, dataset-wide similarity-based querying, and accurate imputation of annotations for unlabeled synapses, using manual annotation of only 0.2% of the dataset's synapses. In particular, excitatory versus inhibitory neuronal types can be assigned with >99.8% accuracy to individual synapses and highly truncated neurites, enabling neurite-enhanced connectomics analysis. Finally, we present a data-driven, unsupervised study of synaptic structural variation on the representation manifold, revealing its intrinsic axes of variation and showing that representations contain inhibitory subtype information.

The plasticitome of cortical interneurons

Hebb postulated that, to store information in the brain, assemblies of excitatory neurons coding for a percept are bound together via associative long-term synaptic plasticity. In this view, it is unclear what role, if any, is carried out by inhibitory interneurons. Indeed, some have argued that inhibitory interneurons are not plastic. Yet numerous recent studies have demonstrated that, similar to excitatory neurons, inhibitory interneurons also undergo long-term plasticity. Here, we discuss the many diverse forms of long-term plasticity that are found at inputs to and outputs from several types of cortical inhibitory interneuron, including their plasticity of intrinsic excitability and their homeostatic plasticity. We explain key plasticity terminology, highlight key interneuron plasticity mechanisms, extract overarching principles and point out implications for healthy brain functionality as well as for neuropathology. We introduce the concept of the plasticitome - the synaptic plasticity counterpart to the genome or the connectome - as well as nomenclature and definitions for dealing with this rich diversity of plasticity. We argue that the great diversity of interneuron plasticity rules is best understood at the circuit level, for example as a way of elucidating how the credit-assignment problem is solved in deep biological neural networks.

Local shape descriptors for neuron segmentation

We present an auxiliary learning task for the problem of neuron segmentation in electron microscopy volumes. The auxiliary task consists of the prediction of local shape descriptors (LSDs), which we combine with conventional voxel-wise direct neighbor affinities for neuron boundary detection. The shape descriptors capture local statistics about the neuron to be segmented, such as diameter, elongation, and direction. On a study comparing several existing methods across various specimen, imaging techniques, and resolutions, auxiliary learning of LSDs consistently increases segmentation accuracy of affinity-based methods over a range of metrics. Furthermore, the addition of LSDs promotes affinity-based segmentation methods to be on par with the current state of the art for neuron segmentation (flood-filling networks), while being two orders of magnitudes more efficient-a critical requirement for the processing of future petabyte-sized datasets.

Igneous: Distributed dense 3D segmentation meshing, neuron skeletonization, and hierarchical downsampling

Three-dimensional electron microscopy images of brain tissue and their dense segmentations are now petascale and growing. These volumes require the mass production of dense segmentation-derived neuron skeletons, multi-resolution meshes, image hierarchies (for both modalities) for visualization and analysis, and tools to manage the large amount of data. However, open tools for large-scale meshing, skeletonization, and data management have been missing. Igneous is a Python-based distributed computing framework that enables economical meshing, skeletonization, image hierarchy creation, and data management using cloud or cluster computing that has been proven to scale horizontally. We sketch Igneous's computing framework, show how to use it, and characterize its performance and data storage.

Scale matters: The nested human connectome

A comprehensive description of how neurons and entire brain regions are interconnected is fundamental for a mechanistic understanding of brain function and dysfunction. Neuroimaging has shaped the way to approaching the human brain’s connectivity on the basis of diffusion magnetic resonance imaging and tractography. At the same time, polarization, fluorescence, and electron microscopy became available, which pushed spatial resolution and sensitivity to the axonal or even to the synaptic level. New methods are mandatory to inform and constrain whole-brain tractography by regional, high-resolution connectivity data and local fiber geometry. Machine learning and simulation can provide predictions where experimental data are missing. Future interoperable atlases require new concepts, including high-resolution templates and directionality, to represent variants of tractography solutions and estimates of their accuracy.

MouseNet: A biologically constrained convolutional neural network model for the mouse visual cortex

Convolutional neural networks trained on object recognition derive inspiration from the neural architecture of the visual system in mammals, and have been used as models of the feedforward computation performed in the primate ventral stream. In contrast to the deep hierarchical organization of primates, the visual system of the mouse has a shallower arrangement. Since mice and primates are both capable of visually guided behavior, this raises questions about the role of architecture in neural computation. In this work, we introduce a novel framework for building a biologically constrained convolutional neural network model of the mouse visual cortex. The architecture and structural parameters of the network are derived from experimental measurements, specifically the 100-micrometer resolution interareal connectome, the estimates of numbers of neurons in each area and cortical layer, and the statistics of connections between cortical layers. This network is constructed to support detailed task-optimized models of mouse visual cortex, with neural populations that can be compared to specific corresponding populations in the mouse brain. Using a well-studied image classification task as our working example, we demonstrate the computational capability of this mouse-sized network. Given its relatively small size, MouseNet achieves roughly 2/3rds the performance level on ImageNet as VGG16. In combination with the large scale Allen Brain Observatory Visual Coding dataset, we use representational similarity analysis to quantify the extent to which MouseNet recapitulates the neural representation in mouse visual cortex. Importantly, we provide evidence that optimizing for task performance does not improve similarity to the corresponding biological system beyond a certain point. We demonstrate that the distributions of some physiological quantities are closer to the observed distributions in the mouse brain after task training. We encourage the use of the MouseNet architecture by making the code freely available.

Task-driven deep neural networks have shown great potential in predicting functional responses of biological neurons. Nevertheless, they are not precise biological analogues, raising questions about how they should be interpreted. Here, we build new deep neural network models of the mouse visual cortex (MouseNet) that are biologically constrained in detail, not only in terms of the basic structure of their connectivity, but also in terms of the count and hence density of neurons within each area, and the spatial extent of their projections. Equipped with the MouseNet model, we can address key questions about mesoscale brain architecture and its role in task learning and performance. We ask, and provide a first set of answers, to: What is the performance of a mouse brain-sized—and mouse brain-structured—model on benchmark image classification tasks? How does the training of a network on this task affect the functional properties of specified layers within the biologically constrained architecture—both overall, and in comparison with recorded function of mouse neurons? We anticipate much future work on allied questions, and the development of more sophisticated models in both mouse and other species, based on the freely available MouseNet model and code which we develop and provide here.

What is a cell type and how to define it?

Cell types are the basic functional units of an organism. Cell types exhibit diverse phenotypic properties at multiple levels, making them challenging to define, categorize, and understand. This review provides an overview of the basic principles of cell types rooted in evolution and development and discusses approaches to characterize and classify cell types and investigate how they contribute to the organism's function, using the mammalian brain as a primary example. I propose a roadmap toward a conceptual framework and knowledge base of cell types that will enable a deeper understanding of the dynamic changes of cellular function under healthy and diseased conditions.

neuPrint: An open access tool for EM connectomics

Due to advances in electron microscopy and deep learning, it is now practical to reconstruct a connectome, a description of neurons and the chemical synapses between them, for significant volumes of neural tissue. Smaller past reconstructions were primarily used by domain experts, could be handled by downloading data, and performance was not a serious problem. But new and much larger reconstructions upend these assumptions. These networks now contain tens of thousands of neurons and tens of millions of connections, with yet larger reconstructions pending, and are of interest to a large community of non-specialists. Allowing other scientists to make use of this data needs more than publication—it requires new tools that are publicly available, easy to use, and efficiently handle large data. We introduce neuPrint to address these data analysis challenges. Neuprint contains two major components—a web interface and programmer APIs. The web interface is designed to allow any scientist worldwide, using only a browser, to quickly ask and answer typical biological queries about a connectome. The neuPrint APIs allow more computer-savvy scientists to make more complex or higher volume queries. NeuPrint also provides features for assessing reconstruction quality. Internally, neuPrint organizes connectome data as a graph stored in a neo4j database. This gives high performance for typical queries, provides access though a public and well documented query language Cypher, and will extend well to future larger connectomics databases. Our experience is also an experiment in open science. We find a significant fraction of the readers of the article proceed to examine the data directly. In our case preprints worked exactly as intended, with data inquiries and PDF downloads starting immediately after pre-print publication, and little affected by formal publication later. From this we deduce that many readers are more interested in our data than in our analysis of our data, suggesting that data-only papers can be well appreciated and that public data release can speed up the propagation of scientific results by many months. We also find that providing, and keeping, the data available for online access imposes substantial additional costs to connectomics research.

Theory of the Multiregional Neocortex: Large-Scale Neural Dynamics and Distributed Cognition

The neocortex is a complex neurobiological system with many interacting regions. How these regions work together to subserve flexible behavior and cognition has become increasingly amenable to rigorous research. Here, I review recent experimental and theoretical work on the modus operandi of a multiregional cortex. These studies revealed several general principles for the neocortical interareal connectivity, low-dimensional macroscopic gradients of biological properties across cortical areas, and a hierarchy of timescales for information processing. Theoretical work suggests testable predictions regarding differential excitation and inhibition along feedforward and feedback pathways in the cortical hierarchy. Furthermore, modeling of distributed working memory and simple decision-making has given rise to a novel mathematical concept, dubbed bifurcation in space, that potentially explains how different cortical areas, with a canonical circuit organization but gradients of biological heterogeneities, are able to subserve their respective (e.g., sensory coding versus executive control) functions in a modularly organized brain.

Volume electron microscopy

Life exists in three dimensions, but until the turn of the century most electron microscopy methods provided only 2D image data. Recently, electron microscopy techniques capable of delving deep into the structure of cells and tissues have emerged, collectively called volume electron microscopy (vEM). Developments in vEM have been dubbed a quiet revolution as the field evolved from established transmission and scanning electron microscopy techniques, so early publications largely focused on the bioscience applications rather than the underlying technological breakthroughs. However, with an explosion in the uptake of vEM across the biosciences and fast-paced advances in volume, resolution, throughput and ease of use, it is timely to introduce the field to new audiences. In this Primer, we introduce the different vEM imaging modalities, the specialized sample processing and image analysis pipelines that accompany each modality and the types of information revealed in the data. We showcase key applications in the biosciences where vEM has helped make breakthrough discoveries and consider limitations and future directions. We aim to show new users how vEM can support discovery science in their own research fields and inspire broader uptake of the technology, finally allowing its full adoption into mainstream biological imaging.

Edge-colored directed subgraph enumeration on the connectome

Following significant advances in image acquisition, synapse detection, and neuronal segmentation in connectomics, researchers have extracted an increasingly diverse set of wiring diagrams from brain tissue. Neuroscientists frequently represent these wiring diagrams as graphs with nodes corresponding to a single neuron and edges indicating synaptic connectivity. The edges can contain "colors" or "labels", indicating excitatory versus inhibitory connections, among other things. By representing the wiring diagram as a graph, we can begin to identify motifs, the frequently occurring subgraphs that correspond to specific biological functions. Most analyses on these wiring diagrams have focused on hypothesized motifs-those we expect to find. However, one of the goals of connectomics is to identify biologically-significant motifs that we did not previously hypothesize. To identify these structures, we need large-scale subgraph enumeration to find the frequencies of all unique motifs. Exact subgraph enumeration is a computationally expensive task, particularly in the edge-dense wiring diagrams. Furthermore, most existing methods do not differentiate between types of edges which can significantly affect the function of a motif. We propose a parallel, general-purpose subgraph enumeration strategy to count motifs in the connectome. Next, we introduce a divide-and-conquer community-based subgraph enumeration strategy that allows for enumeration per brain region. Lastly, we allow for differentiation of edges by types to better reflect the underlying biological properties of the graph. We demonstrate our results on eleven connectomes and publish for future analyses extensive overviews for the 26 trillion subgraphs enumerated that required approximately 9.25 years of computation time.

A set of hub neurons and non-local connectivity features support global brain dynamics in C. elegans

The wiring architecture of neuronal networks is assumed to be a strong determinant of their dynamical computations. An ongoing effort in neuroscience is therefore to generate comprehensive synapse-resolution connectomes alongside brain-wide activity maps. However, the structure-function relationship, i.e., how the anatomical connectome and neuronal dynamics relate to each other on a global scale, remains unsolved. Systematically, comparing graph features in the C. elegans connectome with correlations in nervous system-wide neuronal dynamics, we found that few local connectivity motifs and mostly other non-local features such as triplet motifs and input similarities can predict functional relationships between neurons. Surprisingly, quantities such as connection strength and amount of common inputs do not improve these predictions, suggesting that the network's topology is sufficient. We demonstrate that hub neurons in the connectome are key to these relevant graph features. Consistently, inhibition of multiple hub neurons specifically disrupts brain-wide correlations. Thus, we propose that a set of hub neurons and non-local connectivity features provide an anatomical substrate for global brain dynamics.

Learning accurate path integration in ring attractor models of the head direction system

Ring attractor models for angular path integration have received strong experimental support. To function as integrators, head direction circuits require precisely tuned connectivity, but it is currently unknown how such tuning could be achieved. Here, we propose a network model in which a local, biologically plausible learning rule adjusts synaptic efficacies during development, guided by supervisory allothetic cues. Applied to the Drosophila head direction system, the model learns to path-integrate accurately and develops a connectivity strikingly similar to the one reported in experiments. The mature network is a quasi-continuous attractor and reproduces key experiments in which optogenetic stimulation controls the internal representation of heading, and where the network remaps to integrate with different gains in rodents. Our model predicts that path integration requires self-supervised learning during a developmental phase, and proposes a general framework to learn to path-integrate with gain-1 even in architectures that lack the physical topography of a ring.