NEURD: automated proofreading and feature extraction for connectomics

We are now in the era of millimeter-scale electron microscopy (EM) volumes collected at nanometer resolution (Shapson-Coe et al., 2021; Consortium et al., 2021). Dense reconstruction of cellular compartments in these EM volumes has been enabled by recent advances in Machine Learning (ML) (Lee et al., 2017; Wu et al., 2021; Lu et al., 2021; Macrina et al., 2021). Automated segmentation methods can now yield exceptionally accurate reconstructions of cells, but despite this accuracy, laborious post-hoc proofreading is still required to generate large connectomes free of merge and split errors. The elaborate 3-D meshes of neurons produced by these segmentations contain detailed morphological information, from the diameter, shape, and branching patterns of axons and dendrites, down to the fine-scale structure of dendritic spines. However, extracting information about these features can require substantial effort to piece together existing tools into custom workflows. Building on existing open-source software for mesh manipulation, here we present "NEURD", a software package that decomposes each meshed neuron into a compact and extensively-annotated graph representation. With these feature-rich graphs, we implement workflows to automate a variety of tasks that would otherwise require extensive manual effort, such as state of the art automated post-hoc proofreading of merge errors, cell classification, spine detection, axon-dendritic proximities, and computation of other features. These features enable many downstream analyses of neural morphology and connectivity, making these new massive and complex datasets more accessible to neuroscience researchers focused on a variety of scientific questions.

Cell-type-specific inhibitory circuitry from a connectomic census of mouse visual cortex

Mammalian cortex features a vast diversity of neuronal cell types, each with characteristic anatomical, molecular and functional properties. Synaptic connectivity powerfully shapes how each cell type participates in the cortical circuit, but mapping connectivity rules at the resolution of distinct cell types remains difficult. Here, we used millimeter-scale volumetric electron microscopy1 to investigate the connectivity of all inhibitory neurons across a densely-segmented neuronal population of 1352 cells spanning all layers of mouse visual cortex, producing a wiring diagram of inhibitory connections with more than 70,000 synapses. Taking a data-driven approach inspired by classical neuroanatomy, we classified inhibitory neurons based on the relative targeting of dendritic compartments and other inhibitory cells and developed a novel classification of excitatory neurons based on the morphological and synaptic input properties. The synaptic connectivity between inhibitory cells revealed a novel class of disinhibitory specialist targeting basket cells, in addition to familiar subclasses. Analysis of the inhibitory connectivity onto excitatory neurons found widespread specificity, with many interneurons exhibiting differential targeting of certain subpopulations spatially intermingled with other potential targets. Inhibitory targeting was organized into "motif groups," diverse sets of cells that collectively target both perisomatic and dendritic compartments of the same excitatory targets. Collectively, our analysis identified new organizing principles for cortical inhibition and will serve as a foundation for linking modern multimodal neuronal atlases with the cortical wiring diagram.

CAVE: Connectome Annotation Versioning Engine

Advances in Electron Microscopy, image segmentation and computational infrastructure have given rise to large-scale and richly annotated connectomic datasets which are increasingly shared across communities. To enable collaboration, users need to be able to concurrently create new annotations and correct errors in the automated segmentation by proofreading. In large datasets, every proofreading edit relabels cell identities of millions of voxels and thousands of annotations like synapses. For analysis, users require immediate and reproducible access to this constantly changing and expanding data landscape. Here, we present the Connectome Annotation Versioning Engine (CAVE), a computational infrastructure for immediate and reproducible connectome analysis in up-to petascale datasets (~1mm3) while proofreading and annotating is ongoing. For segmentation, CAVE provides a distributed proofreading infrastructure for continuous versioning of large reconstructions. Annotations in CAVE are defined by locations such that they can be quickly assigned to the underlying segment which enables fast analysis queries of CAVE's data for arbitrary time points. CAVE supports schematized, extensible annotations, so that researchers can readily design novel annotation types. CAVE is already used for many connectomics datasets, including the largest datasets available to date.

Expansion-assisted selective plane illumination microscopy for nanoscale imaging of centimeter-scale tissues

Recent advances in tissue processing, labeling, and fluorescence microscopy are providing unprecedented views of the structure of cells and tissues at sub-diffraction resolutions and near single molecule sensitivity, driving discoveries in diverse fields of biology, including neuroscience. Biological tissue is organized over scales of nanometers to centimeters. Harnessing molecular imaging across three-dimensional samples on this scale requires new types of microscopes with larger fields of view and working distance, as well as higher imaging throughput. We present a new expansion-assisted selective plane illumination microscope (ExA-SPIM) with diffraction-limited and aberration-free performance over a large field of view (85 mm 2 ) and working distance (35 mm). Combined with new tissue clearing and expansion methods, the microscope allows nanoscale imaging of centimeter-scale samples, including entire mouse brains, with diffraction-limited resolutions and high contrast without sectioning. We illustrate ExA-SPIM by reconstructing individual neurons across the mouse brain, imaging cortico-spinal neurons in the macaque motor cortex, and tracing axons in human white matter.

Functional connectomics reveals general wiring rule in mouse visual cortex

To understand how the brain computes, it is important to unravel the relationship between circuit connectivity and function. Previous research has shown that excitatory neurons in layer 2/3 of the primary visual cortex of mice with similar response properties are more likely to form connections. However, technical challenges of combining synaptic connectivity and functional measurements have limited these studies to few, highly local connections. Utilizing the millimeter scale and nanometer resolution of the MICrONS dataset, we studied the connectivity-function relationship in excitatory neurons of the mouse visual cortex across interlaminar and interarea projections, assessing connection selectivity at the coarse axon trajectory and fine synaptic formation levels. A digital twin model of this mouse, that accurately predicted responses to arbitrary video stimuli, enabled a comprehensive characterization of the function of neurons. We found that neurons with highly correlated responses to natural videos tended to be connected with each other, not only within the same cortical area but also across multiple layers and visual areas, including feedforward and feedback connections, whereas we did not find that orientation preference predicted connectivity. The digital twin model separated each neuron's tuning into a feature component (what the neuron responds to) and a spatial component (where the neuron's receptive field is located). We show that the feature, but not the spatial component, predicted which neurons were connected at the fine synaptic scale. Together, our results demonstrate the "like-to-like" connectivity rule generalizes to multiple connection types, and the rich MICrONS dataset is suitable to further refine a mechanistic understanding of circuit structure and function.

Integrating EM and Patch-seq data: Synaptic connectivity and target specificity of predicted Sst transcriptomic types

Neural circuit function is shaped both by the cell types that comprise the circuit and the connections between those cell types ¹ . Neural cell types have previously been defined by morphology ^{2, 3} , electrophysiology ^{4, 5} , transcriptomic expression ^6-8 , connectivity ^9-13 , or even a combination of such modalities ^14-16 . More recently, the Patch-seq technique has enabled the characterization of morphology (M), electrophysiology (E), and transcriptomic (T) properties from individual cells ^17-20 . Using this technique, these properties were integrated to define 28, inhibitory multimodal, MET-types in mouse primary visual cortex ²¹ . It is unknown how these MET-types connect within the broader cortical circuitry however. Here we show that we can predict the MET-type identity of inhibitory cells within a large-scale electron microscopy (EM) dataset and these MET-types have distinct ultrastructural features and synapse connectivity patterns. We found that EM Martinotti cells, a well defined morphological cell type ^{22, 23} known to be Somatostatin positive (Sst+) ^{24, 25} , were successfully predicted to belong to Sst+ MET-types. Each identified MET-type had distinct axon myelination patterns and synapsed onto specific excitatory targets. Our results demonstrate that morphological features can be used to link cell type identities across imaging modalities, which enables further comparison of connectivity in relation to transcriptomic or electrophysiological properties. Furthermore, our results show that MET-types have distinct connectivity patterns, supporting the use of MET-types and connectivity to meaningfully define cell types.

Binary and analog variation of synapses between cortical pyramidal neurons

Learning from experience depends at least in part on changes in neuronal connections. We present the largest map of connectivity to date between cortical neurons of a defined type (layer 2/3 [L2/3] pyramidal cells in mouse primary visual cortex), which was enabled by automated analysis of serial section electron microscopy images with improved handling of image defects (250 × 140 × 90 μm3 volume). We used the map to identify constraints on the learning algorithms employed by the cortex. Previous cortical studies modeled a continuum of synapse sizes by a log-normal distribution. A continuum is consistent with most neural network models of learning, in which synaptic strength is a continuously graded analog variable. Here, we show that synapse size, when restricted to synapses between L2/3 pyramidal cells, is well modeled by the sum of a binary variable and an analog variable drawn from a log-normal distribution. Two synapses sharing the same presynaptic and postsynaptic cells are known to be correlated in size. We show that the binary variables of the two synapses are highly correlated, while the analog variables are not. Binary variation could be the outcome of a Hebbian or other synaptic plasticity rule depending on activity signals that are relatively uniform across neuronal arbors, while analog variation may be dominated by other influences such as spontaneous dynamical fluctuations. We discuss the implications for the longstanding hypothesis that activity-dependent plasticity switches synapses between bistable states.

A scalable and modular automated pipeline for stitching of large electron microscopy datasets

Serial-section electronmicroscopy (ssEM) is themethod of choice for studyingmacroscopic biological samples at extremely high resolution in three dimensions. In the nervous system, nanometer-scale images are necessary to reconstruct dense neural wiring diagrams in the brain, so called connectomes. In order to use this data, consisting of up to 10⁸ individual EM images, it must be assembled into a volume, requiring seamless 2D stitching from each physical section followed by 3D alignment of the stitched sections. The high throughput of ssEM necessitates 2D stitching to be done at the pace of imaging, which currently produces tens of terabytes per day. To achieve this, we present a modular volume assembly software pipeline ASAP (Assembly Stitching and Alignment Pipeline) that is scalable to datasets containing petabytes of data and parallelized to work in a distributed computational environment. The pipeline is built on top of the Render (27) services used in the volume assembly of the brain of adult Drosophilamelanogaster (30). It achieves high throughput by operating on themeta-data and transformations of each image stored in a database, thus eliminating the need to render intermediate output. ASAP ismodular, allowing for easy incorporation of new algorithms without significant changes in the workflow. The entire software pipeline includes a complete set of tools for stitching, automated quality control, 3D section alignment, and final rendering of the assembled volume to disk. ASAP has been deployed for continuous stitching of several large-scale datasets of the mouse visual cortex and human brain samples including one cubic millimeter of mouse visual cortex (28; 8) at speeds that exceed imaging. The pipeline also has multi-channel processing capabilities and can be applied to fluorescence and multi-modal datasets like array tomography.

Reconstruction of neocortex: Organelles, compartments, cells, circuits, and activity

We assembled a semi-automated reconstruction of L2/3 mouse primary visual cortex from ∼250 × 140 × 90 μm3 of electron microscopic images, including pyramidal and non-pyramidal neurons, astrocytes, microglia, oligodendrocytes and precursors, pericytes, vasculature, nuclei, mitochondria, and synapses. Visual responses of a subset of pyramidal cells are included. The data are publicly available, along with tools for programmatic and three-dimensional interactive access. Brief vignettes illustrate the breadth of potential applications relating structure to function in cortical circuits and neuronal cell biology. Mitochondria and synapse organization are characterized as a function of path length from the soma. Pyramidal connectivity motif frequencies are predicted accurately using a configuration model of random graphs. Pyramidal cells receiving more connections from nearby cells exhibit stronger and more reliable visual responses. Sample code shows data access and analysis.

Structure and function of axo-axonic inhibition

Inhibitory neurons in mammalian cortex exhibit diverse physiological, morphological, molecular, and connectivity signatures. While considerable work has measured the average connectivity of several interneuron classes, there remains a fundamental lack of understanding of the connectivity distribution of distinct inhibitory cell types with synaptic resolution, how it relates to properties of target cells, and how it affects function. Here, we used large-scale electron microscopy and functional imaging to address these questions for chandelier cells in layer 2/3 of the mouse visual cortex. With dense reconstructions from electron microscopy, we mapped the complete chandelier input onto 153 pyramidal neurons. We found that synapse number is highly variable across the population and is correlated with several structural features of the target neuron. This variability in the number of axo-axonic ChC synapses is higher than the variability seen in perisomatic inhibition. Biophysical simulations show that the observed pattern of axo-axonic inhibition is particularly effective in controlling excitatory output when excitation and inhibition are co-active. Finally, we measured chandelier cell activity in awake animals using a cell-type-specific calcium imaging approach and saw highly correlated activity across chandelier cells. In the same experiments, in vivo chandelier population activity correlated with pupil dilation, a proxy for arousal. Together, these results suggest that chandelier cells provide a circuit-wide signal whose strength is adjusted relative to the properties of target neurons.

Laminar distribution and arbor density of two functional classes of thalamic inputs to primary visual cortex

Motion/direction-sensitive and location-sensitive neurons are the two major functional types in mouse visual thalamus that project to the primary visual cortex (V1). It is under debate whether motion/direction-sensitive inputs preferentially target the superficial layers in V1, as opposed to the location-sensitive inputs, which preferentially target the middle layers. Here, by using calcium imaging to measure the activity of motion/direction-sensitive and location-sensitive axons in V1, we find evidence against these cell-type-specific laminar biases at the population level. Furthermore, using an approach to reconstruct axon arbors with identified in vivo response types, we show that, at the single-axon level, the motion/direction-sensitive axons project more densely to the middle layers than the location-sensitive axons. Overall, our results demonstrate that motion/direction-sensitive thalamic neurons project extensively to the middle layers of V1 at both the population and single-cell levels, providing further insight into the organization of thalamocortical projection in the mouse visual system.

The Mind of a Mouse

Large scientific projects in genomics and astronomy are influential not because they answer any single question but because they enable investigation of continuously arising new questions from the same data-rich sources. Advances in automated mapping of the brain's synaptic connections (connectomics) suggest that the complicated circuits underlying brain function are ripe for analysis. We discuss benefits of mapping a mouse brain at the level of synapses.

Autoencoder networks extract latent variables and encode these variables in their connectomes

Advances in electron microscopy and data processing techniques are leading to increasingly large and complete microscale connectomes. At the same time, advances in artificial neural networks have produced model systems that perform comparably rich computations with perfectly specified connectivity. This raises an exciting scientific opportunity for the study of both biological and artificial neural networks: to infer the underlying circuit function from the structure of its connectivity. A potential roadblock, however, is that - even with well constrained neural dynamics - there are in principle many different connectomes that could support a given computation. Here, we define a tractable setting in which the problem of inferring circuit function from circuit connectivity can be analyzed in detail: the function of input compression and reconstruction, in an autoencoder network with a single hidden layer. Here, in general there is substantial ambiguity in the weights that can produce the same circuit function, because largely arbitrary changes to input weights can be undone by applying the inverse modifications to the output weights. However, we use mathematical arguments and simulations to show that adding simple, biologically motivated regularization of connectivity resolves this ambiguity in an interesting way: weights are constrained such that the latent variable structure underlying the inputs can be extracted from the weights by using nonlinear dimensionality reduction methods.

Relationship between simultaneously recorded spiking activity and fluorescence signal in GCaMP6 transgenic mice

Fluorescent calcium indicators are often used to investigate neural dynamics, but the relationship between fluorescence and action potentials (APs) remains unclear. Most APs can be detected when the soma almost fills the microscope’s field of view, but calcium indicators are used to image populations of neurons, necessitating a large field of view, generating fewer photons per neuron, and compromising AP detection. Here, we characterized the AP-fluorescence transfer function in vivo for 48 layer 2/3 pyramidal neurons in primary visual cortex, with simultaneous calcium imaging and cell-attached recordings from transgenic mice expressing GCaMP6s or GCaMP6f. While most APs were detected under optimal conditions, under conditions typical of population imaging studies, only a minority of 1 AP and 2 AP events were detected (often <10% and ~20–30%, respectively), emphasizing the limits of AP detection under more realistic imaging conditions.

Neurons, the cells that make up the nervous system, transmit information using electrical signals known as action potentials or spikes. Studying the spiking patterns of neurons in the brain is essential to understand perception, memory, thought, and behaviour. One way to do that is by recording electrical activity with microelectrodes. Another way to study neuronal activity is by using molecules that change how they interact with light when calcium binds to them, since changes in calcium concentration can be indicative of neuronal spiking. That change can be observed with specialized microscopes know as two-photon fluorescence microscopes. Using calcium indicators, it is possible to simultaneously record hundreds or even thousands of neurons. However, calcium fluorescence and spikes do not translate one-to-one.

In order to interpret fluorescence data, it is important to understand the relationship between the fluorescence signals and the spikes associated with individual neurons. The only way to directly measure this relationship is by using calcium imaging and electrical recording simultaneously to record activity from the same neuron. However, this is extremely challenging experimentally, so this type of data is rare.

To shed some light on this, Huang, Ledochowitsch et al. used mice that had been genetically modified to produce a calcium indicator in neurons of the visual cortex and simultaneously obtained both fluorescence measurements and electrical recordings from these neurons. These experiments revealed that, while the majority of time periods containing multi-spike neural activity could be identified using calcium imaging microscopy, on average, less than 10% of isolated single spikes were detectable. This is an important caveat that researchers need to take into consideration when interpreting calcium imaging results.

These findings are intended to serve as a guide for interpreting calcium imaging studies that look at neurons in the mammalian brain at the population level. In addition, the data provided will be useful as a reference for the development of activity sensors, and to benchmark and improve computational approaches for detecting and predicting spikes.

Survey of spiking in the mouse visual system reveals functional hierarchy

The anatomy of the mammalian visual system, from the retina to the neocortex, is organized hierarchically¹. However, direct observation of cellular-level functional interactions across this hierarchy is lacking due to the challenge of simultaneously recording activity across numerous regions. Here we describe a large, open dataset-part of the Allen Brain Observatory²-that surveys spiking from tens of thousands of units in six cortical and two thalamic regions in the brains of mice responding to a battery of visual stimuli. Using cross-correlation analysis, we reveal that the organization of inter-area functional connectivity during visual stimulation mirrors the anatomical hierarchy from the Allen Mouse Brain Connectivity Atlas³. We find that four classical hierarchical measures-response latency, receptive-field size, phase-locking to drifting gratings and response decay timescale-are all correlated with the hierarchy. Moreover, recordings obtained during a visual task reveal that the correlation between neural activity and behavioural choice also increases along the hierarchy. Our study provides a foundation for understanding coding and signal propagation across hierarchically organized cortical and thalamic visual areas.

A petascale automated imaging pipeline for mapping neuronal circuits with high-throughput transmission electron microscopy

Electron microscopy (EM) is widely used for studying cellular structure and network connectivity in the brain. We have built a parallel imaging pipeline using transmission electron microscopes that scales this technology, implements 24/7 continuous autonomous imaging, and enables the acquisition of petascale datasets. The suitability of this architecture for large-scale imaging was demonstrated by acquiring a volume of more than 1 mm3 of mouse neocortex, spanning four different visual areas at synaptic resolution, in less than 6 months. Over 26,500 ultrathin tissue sections from the same block were imaged, yielding a dataset of more than 2 petabytes. The combined burst acquisition rate of the pipeline is 3 Gpixel per sec and the net rate is 600 Mpixel per sec with six microscopes running in parallel. This work demonstrates the feasibility of acquiring EM datasets at the scale of cortical microcircuits in multiple brain regions and species.

A large-scale standardized physiological survey reveals functional organization of the mouse visual cortex

To understand how the brain processes sensory information to guide behavior, we must know how stimulus representations are transformed throughout the visual cortex. Here we report an open, large-scale physiological survey of activity in the awake mouse visual cortex: the Allen Brain Observatory Visual Coding dataset. This publicly available dataset includes the cortical activity of nearly 60,000 neurons from six visual areas, four layers, and 12 transgenic mouse lines in a total of 243 adult mice, in response to a systematic set of visual stimuli. We classify neurons on the basis of joint reliabilities to multiple stimuli and validate this functional classification with models of visual responses. While most classes are characterized by responses to specific subsets of the stimuli, the largest class is not reliably responsive to any of the stimuli and becomes progressively larger in higher visual areas. These classes reveal a functional organization wherein putative dorsal areas show specialization for visual motion signals.

Visual physiology of the layer 4 cortical circuit in silico

Despite advances in experimental techniques and accumulation of large datasets concerning the composition and properties of the cortex, quantitative modeling of cortical circuits under in-vivo-like conditions remains challenging. Here we report and publicly release a biophysically detailed circuit model of layer 4 in the mouse primary visual cortex, receiving thalamo-cortical visual inputs. The 45,000-neuron model was subjected to a battery of visual stimuli, and results were compared to published work and new in vivo experiments. Simulations reproduced a variety of observations, including effects of optogenetic perturbations. Critical to the agreement between responses in silico and in vivo were the rules of functional synaptic connectivity between neurons. Interestingly, after extreme simplification the model still performed satisfactorily on many measurements, although quantitative agreement with experiments suffered. These results emphasize the importance of functional rules of cortical wiring and enable a next generation of data-driven models of in vivo neural activity and computations.

How can we capture the incredible complexity of brain circuits in quantitative models, and what can such models teach us about mechanisms underlying brain activity? To answer these questions, we set out to build extensive, bio-realistic models of brain circuitry by employing systematic datasets on brain structure and function. Here we report the first modeling results of this project, focusing on the layer 4 of the primary visual cortex (V1) of the mouse. Our simulations reproduced a variety of experimental observations in response to a large battery of visual stimuli. The results elucidated circuit mechanisms determining patters of neuronal activity in layer 4 –in particular, the roles of feedforward thalamic inputs and specific patterns of intracortical connectivity in producing tuning of neuronal responses to the orientation of motion. Simplification of neuronal models led to specific deficiencies in reproducing experimental data, giving insights into how biological details contribute to various aspects of brain activity. To enable future development of more sophisticated models, we make the software code, the model, and simulation results publicly available.

Large-scale neuroanatomy using LASSO: Loop-based Automated Serial Sectioning Operation

Serial section transmission electron microscopy (ssTEM) is the most promising tool for investigating the three-dimensional anatomy of the brain with nanometer resolution. Yet as the field progresses to larger volumes of brain tissue, new methods for high-yield, low-cost, and high-throughput serial sectioning are required. Here, we introduce LASSO (Loop-based Automated Serial Sectioning Operation), in which serial sections are processed in “batches.” Batches are quantized groups of individual sections that, in LASSO, are cut with a diamond knife, picked up from an attached waterboat, and placed onto microfabricated TEM substrates using rapid, accurate, and repeatable robotic tools. Additionally, we introduce mathematical models for ssTEM with respect to yield, throughput, and cost to access ssTEM scalability. To validate the method experimentally, we processed 729 serial sections of human brain tissue (~40 nm x 1 mm x 1 mm). Section yield was 727/729 (99.7%). Sections were placed accurately and repeatably (x-direction: -20 ± 110 μm (1 s.d.), y-direction: 60 ± 150 μm (1 s.d.)) with a mean cycle time of 43 s ± 12 s (1 s.d.). High-magnification (2.5 nm/px) TEM imaging was conducted to measure the image quality. We report no significant distortion, information loss, or substrate-derived artifacts in the TEM images. Quantitatively, the edge spread function across vesicle edges and image contrast were comparable, suggesting that LASSO does not negatively affect image quality. In total, LASSO compares favorably with traditional serial sectioning methods with respect to throughput, yield, and cost for large-scale experiments, and represents a flexible, scalable, and accessible technology platform to enable the next generation of neuroanatomical studies.

Transport and trapping of nanosheets via hydrodynamic forces and curvature-induced capillary quadrupolar interactions

HYPOTHESIS

The manipulation of nanosheets on a fluid-fluid interface remains a significant challenge. At this interface, hydrodynamic forces can be used for long-range transport (>1× capillary length) but are difficult to utilize for accurate and repeatable positioning. While capillary multipole interactions have been used for particle trapping, how these interactions manifest on large but thin objects, i.e., nanosheets, remains an open question. Hence, we posit hydrodynamic forces in conjunction with capillary multipole interactions can be used for nanosheet transport and trapping.

EXPERIMENTS

We designed and characterized a fluidic device for transporting and trapping nanosheets on the water-air interface. Analytical models were compared against optical measurements of the nanosheet behavior to investigate capillary multipole interactions. Energy-based modeling and dimensional analysis were used to study trapping stability.

FINDINGS

Hydrodynamic forces and capillary interactions successfully transported and trapped nanosheets at a designated trapping location with a repeatability of 10% of the nanosheet's length and 12% of its width (length = 1500 µm, width = 1000 µm) and an accuracy of 20% of their length and width. Additionally, this is the first report that surface tension forces acting upon nanoscale-thick objects manifest as capillary quadrupolar interactions and can be used for precision manipulation of nanosheets.

Nontoxic, double-deletion-mutant rabies viral vectors for retrograde targeting of projection neurons

Recombinant rabies viral vectors have proven useful for applications including retrograde targeting of projection neurons and monosynaptic tracing, but their cytotoxicity has limited their use to short-term experiments. Here we introduce a new class of double-deletion-mutant rabies viral vectors that left transduced cells alive and healthy indefinitely. Deletion of the viral polymerase gene abolished cytotoxicity and reduced transgene expression to trace levels but left vectors still able to retrogradely infect projection neurons and express recombinases, allowing downstream expression of other transgene products such as fluorophores and calcium indicators. The morphology of retrogradely targeted cells appeared unperturbed at 1 year postinjection. Whole-cell patch-clamp recordings showed no physiological abnormalities at 8 weeks. Longitudinal two-photon structural and functional imaging in vivo, tracking thousands of individual neurons for up to 4 months, showed that transduced neurons did not die but retained stable visual response properties even at the longest time points imaged.

Mouse color and wavelength-specific luminance contrast sensitivity are non-uniform across visual space

Mammalian visual behaviors, as well as responses in the neural systems underlying these behaviors, are driven by luminance and color contrast. With constantly improving tools for measuring activity in cell-type-specific populations in the mouse during visual behavior, it is important to define the extent of luminance and color information that is behaviorally accessible to the mouse. A non-uniform distribution of cone opsins in the mouse retina potentially complicates both luminance and color sensitivity; opposing gradients of short (UV-shifted) and middle (blue/green) cone opsins suggest that color discrimination and wavelength-specific luminance contrast sensitivity may differ with retinotopic location. Here we ask how well mice can discriminate color and wavelength-specific luminance changes across visuotopic space. We found that mice were able to discriminate color and were able to do so more broadly across visuotopic space than expected from the cone-opsin distribution. We also found wavelength-band-specific differences in luminance sensitivity.

Spatial Organization of Chromatic Pathways in the Mouse Dorsal Lateral Geniculate Nucleus

In both dichromats and trichromats, cone opsin signals are maintained independently in cones and combined at the bipolar and retinal ganglion cell level, creating parallel color opponent pathways to the central visual system. Like other dichromats, the mouse retina expresses a short-wavelength (S) and a medium-wavelength (M) opsin, with the S-opsin shifted to peak sensitivity in the ultraviolet (UV) range. Unlike in primates, nonuniform opsin expression across the retina and coexpression in single cones creates a mostly mixed chromatic signal. Here, we describe the visuotopic and chromatic organization of spiking responses in the dorsal lateral geniculate and of the local field potentials in their recipient zone in primary visual cortex (V1). We used an immersive visual stimulus dome that allowed us to present spatiotemporally modulated UV and green luminance in any region of the visual field of an awake, head-fixed mouse. Consistent with retinal expression of opsins, we observed graded UV-to-green dominated responses from the upper to lower visual fields, with a smaller difference across azimuth. In addition, we identified a subpopulation of cells (<10%) that exhibited spectrally opponent responses along the S-M axis. Luminance signals of each wavelength and color signals project to the middle layers of V1.

SIGNIFICANCE STATEMENT

In natural environments, color information is useful for guiding behavior. How small terrestrial mammals such as mice use graded expression of cone opsins to extract visual information from their environments is not clear, even as the use of mice for studying visually guided behavior grows. In this study, we examined the color signals that the retina sends to the visual cortex via the lateral geniculate nucleus of the thalamus. We found that green dominated responses in the lower and nasal visual field and ultraviolet dominated responses in the upper visual field. We describe a subset of cells that exhibit color opponent responses.

Transgenic mice for intersectional targeting of neural sensors and effectors with high specificity and performance

An increasingly powerful approach for studying brain circuits relies on targeting genetically encoded sensors and effectors to specific cell types. However, current approaches for this are still limited in functionality and specificity. Here we utilize several intersectional strategies to generate multiple transgenic mouse lines expressing high levels of novel genetic tools with high specificity. We developed driver and double reporter mouse lines and viral vectors using the Cre/Flp and Cre/Dre double recombinase systems and established a new, retargetable genomic locus, TIGRE, which allowed the generation of a large set of Cre/tTA-dependent reporter lines expressing fluorescent proteins, genetically encoded calcium, voltage, or glutamate indicators, and optogenetic effectors, all at substantially higher levels than before. High functionality was shown in example mouse lines for GCaMP6, YCX2.60, VSFP Butterfly 1.2, and Jaws. These novel transgenic lines greatly expand the ability to monitor and manipulate neuronal activities with increased specificity.

VIDEO ABSTRACT

Removable cranial windows for long-term imaging in awake mice

Cranial window implants in head-fixed rodents are becoming a preparation of choice for stable optical access to large areas of the cortex over extended periods of time. Here we provide a highly detailed and reliable surgical protocol for a cranial window implantation procedure for chronic wide-field and cellular imaging in awake, head-fixed mice, which enables subsequent window removal and replacement in the weeks and months after the initial craniotomy. This protocol has facilitated awake, chronic imaging in adolescent and adult mice over several months from a large number of cortical brain regions; targeted virus and tracer injections from data obtained using prior awake functional mapping; and functionally targeted two-photon imaging across all cortical layers in awake mice using a microprism attachment to the cranial window. Collectively, these procedures extend the reach of chronic imaging of cortical function and dysfunction in behaving animals.

In vivo two-photon calcium imaging in the visual system

Two-photon imaging of calcium-sensitive dyes in vivo has become a common tool used by neuroscientists, largely because of the development of bolus loading techniques, which can label every neuron in a local circuit with calcium-sensitive dye. Like multielectrode recordings, two-photon imaging paired with bolus loading provides a method for monitoring many neurons at once, but, in addition, it provides a means for determining the precise location of every neuron. Thus, it is an ideal method for studying the fine-scale functional architecture of the cortex and guiding the experimenter to individual neurons that can be targeted for further anatomical study. Two-photon calcium imaging enables study of the fine structure of functional maps in the visual cortex in cats and rodents. In mice, it can allow the characterization of specific cell types when paired with transgenic or retrograde labeling.

A mouse model of higher visual cortical function

During sensory experience, the retina transmits a diverse array of signals to the brain, which must be parsed to generate meaningful percepts that can guide decisions and actions. Decades of anatomical and physiological studies in primates and carnivores have revealed a complex parallel and hierarchical organization by which distinct visual features are distributed to, and processed by, different brain regions. However, these studies have been limited in their ability to dissect the circuit mechanisms involved in the transformation of sensory inputs into complex cortical representations and action patterns. Multiple groups have therefore pushed to explore the organization and function of higher visual areas in the mouse. Here we review the anatomical and physiological findings of these recent explorations in mouse visual cortex. These studies find that sensory input is processed in a diverse set of higher areas that are each interconnected with specific limbic and motor systems. This hierarchical and parallel organization is consistent with the multiple streams that have been found in the higher visual areas of primates. We therefore propose that the mouse visual system is a useful model to explore the circuits underlying the transformation of sensory inputs into goal-directed perceptions and actions.

Chronic cellular imaging of entire cortical columns in awake mice using microprisms

Two-photon imaging of cortical neurons in vivo has provided unique insights into the structure, function, and plasticity of cortical networks, but this method does not currently allow simultaneous imaging of neurons in the superficial and deepest cortical layers. Here, we describe a simple modification that enables simultaneous, long-term imaging of all cortical layers. Using a chronically implanted glass microprism in barrel cortex, we could image the same fluorescently labeled deep-layer pyramidal neurons across their entire somatodendritic axis for several months. We could also image visually evoked and endogenous calcium activity in hundreds of cell bodies or long-range axon terminals, across all six layers in visual cortex of awake mice. Electrophysiology and calcium imaging of evoked and endogenous activity near the prism face were consistent across days and comparable with previous observations. These experiments extend the reach of in vivo two-photon imaging to chronic, simultaneous monitoring of entire cortical columns.

The Open Connectome Project Data Cluster: Scalable Analysis and Vision for High-Throughput Neuroscience

We describe a scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data, initially for 3-d electron microscopy image stacks, but for time-series and multi-channel data as well. The system was designed primarily for workloads that build connectomes- neural connectivity maps of the brain-using the parallel execution of computer vision algorithms on high-performance compute clusters. These services and open-science data sets are publicly available at openconnecto.me. The system design inherits much from NoSQL scale-out and data-intensive computing architectures. We distribute data to cluster nodes by partitioning a spatial index. We direct I/O to different systems-reads to parallel disk arrays and writes to solid-state storage-to avoid I/O interference and maximize throughput. All programming interfaces are RESTful Web services, which are simple and stateless, improving scalability and usability. We include a performance evaluation of the production system, highlighting the effec-tiveness of spatial data organization.

From functional architecture to functional connectomics

"Receptive Fields, Binocular Interaction and Functional Architecture in the Cat's Visual Cortex" by Hubel and Wiesel (1962) reported several important discoveries: orientation columns, the distinct structures of simple and complex receptive fields, and binocular integration. But perhaps the paper's greatest influence came from the concept of functional architecture (the complex relationship between in vivo physiology and the spatial arrangement of neurons) and several models of functionally specific connectivity. They thus identified two distinct concepts, topographic specificity and functional specificity, which together with cell-type specificity constitute the major determinants of nonrandom cortical connectivity. Orientation columns are iconic examples of topographic specificity, whereby axons within a column connect with cells of a single orientation preference. Hubel and Wiesel also saw the need for functional specificity at a finer scale in their model of thalamic inputs to simple cells, verified in the 1990s. The difficult but potentially more important question of functional specificity between cortical neurons is only now becoming tractable with new experimental techniques.

Functional specialization of mouse higher visual cortical areas

The mouse is emerging as an important model for understanding how sensory neocortex extracts cues to guide behavior, yet little is known about how these cues are processed beyond primary cortical areas. Here, we used two-photon calcium imaging in awake mice to compare visual responses in primary visual cortex (V1) and in two downstream target areas, AL and PM. Neighboring V1 neurons had diverse stimulus preferences spanning five octaves in spatial and temporal frequency. By contrast, AL and PM neurons responded best to distinct ranges of stimulus parameters. Most strikingly, AL neurons preferred fast-moving stimuli while PM neurons preferred slow-moving stimuli. By contrast, neurons in V1, AL, and PM demonstrated similar selectivity for stimulus orientation but not for stimulus direction. Based on these findings, we predict that area AL helps guide behaviors involving fast-moving stimuli (e.g., optic flow), while area PM helps guide behaviors involving slow-moving objects.

A multiscale parallel computing architecture for automated segmentation of the brain connectome

Several groups in neurobiology have embarked into deciphering the brain circuitry using large-scale imaging of a mouse brain and manual tracing of the connections between neurons. Creating a graph of the brain circuitry, also called a connectome, could have a huge impact on the understanding of neurodegenerative diseases such as Alzheimer's disease. Although considerably smaller than a human brain, a mouse brain already exhibits one billion connections and manually tracing the connectome of a mouse brain can only be achieved partially. This paper proposes to scale up the tracing by using automated image segmentation and a parallel computing approach designed for domain experts. We explain the design decisions behind our parallel approach and we present our results for the segmentation of the vasculature and the cell nuclei, which have been obtained without any manual intervention.

Accelerating image registration with the Johnson-Lindenstrauss lemma: application to imaging 3-D neural ultrastructure with electron microscopy

We present a novel algorithm to accelerate feature based registration, and demonstrate the utility of the algorithm for the alignment of large transmission electron microscopy (TEM) images to create 3-D images of neural ultrastructure. In contrast to the most similar algorithms, which achieve small computation times by truncated search, our algorithm uses a novel randomized projection to accelerate feature comparison and to enable global search. Further, we demonstrate robust estimation of nonrigid transformations with a novel probabilistic correspondence framework, that enables large TEM images to be rapidly brought into alignment, removing characteristic distortions of the tissue fixation and imaging process. We analyze the impact of randomized projections upon correspondence detection, and upon transformation accuracy, and demonstrate that accuracy is maintained. We provide experimental results that demonstrate significant reduction in computation time and successful alignment of TEM images.

Network anatomy and in vivo physiology of visual cortical neurons

In the cerebral cortex, local circuits consist of tens of thousands of neurons, each of which makes thousands of synaptic connections. Perhaps the biggest impediment to understanding these networks is that we have no wiring diagrams of their interconnections. Even if we had a partial or complete wiring diagram, however, understanding the network would also require information about each neuron's function. Here we show that the relationship between structure and function can be studied in the cortex with a combination of in vivo physiology and network anatomy. We used two-photon calcium imaging to characterize a functional property—the preferred stimulus orientation—of a group of neurons in the mouse primary visual cortex. We then used large-scale electron microscopy (EM) of serial thin sections to trace a portion of these neurons’ local network. Consistent with a prediction from recent physiological experiments, inhibitory interneurons received convergent anatomical input from nearby excitatory neurons with a broad range of preferred orientations, although weak biases could not be rejected.

Interactive histology of large-scale biomedical image stacks

Histology is the study of the structure of biological tissue using microscopy techniques. As digital imaging technology advances, high resolution microscopy of large tissue volumes is becoming feasible; however, new interactive tools are needed to explore and analyze the enormous datasets. In this paper we present a visualization framework that specifically targets interactive examination of arbitrarily large image stacks. Our framework is built upon two core techniques: display-aware processing and GPU-accelerated texture compression. With display-aware processing, only the currently visible image tiles are fetched and aligned on-the-fly, reducing memory bandwidth and minimizing the need for time-consuming global pre-processing. Our novel texture compression scheme for GPUs is tailored for quick browsing of image stacks. We evaluate the usability of our viewer for two histology applications: digital pathology and visualization of neural structure at nanoscale-resolution in serial electron micrographs.

Broadly tuned response properties of diverse inhibitory neuron subtypes in mouse visual cortex

Different subtypes of GABAergic neurons in sensory cortex exhibit diverse morphology, histochemical markers, and patterns of connectivity. These subtypes likely play distinct roles in cortical function, but their in vivo response properties remain unclear. We used in vivo calcium imaging, combined with immunohistochemical and genetic labels, to record visual responses in excitatory neurons and up to three distinct subtypes of GABAergic neurons (immunoreactive for parvalbumin, somatostatin, or vasoactive intestinal peptide) in layer 2/3 of mouse visual cortex. Excitatory neurons had sharp response selectivity for stimulus orientation and spatial frequency, while all GABAergic subtypes had broader selectivity. Further, bias in the responses of GABAergic neurons toward particular orientations or spatial frequencies tended to reflect net biases of the surrounding neurons. These results suggest that the sensory responses of layer 2/3 GABAergic neurons reflect the pooled activity of the surrounding population--a principle that may generalize across species and sensory modalities.

Cellular imaging of visual cortex reveals the spatial and functional organization of spontaneous activity

The cerebral cortex is never silent; even in primary sensory areas there is ongoing neural activity in the absence of sensory input. Correlations in spontaneous activity can provide clues about network structure, but it has been difficult to record from enough nearby neurons to sample these correlations well. We used in vivo two-photon calcium imaging to demonstrate sparse patterns of correlated spontaneous activity among groups of ∼150 simultaneously imaged cells. In cat visual cortex, correlations fell off sharply with distance, by 50% within ∼240 μm, but in the rat there was little dependence on spatial separation up to 400 μm. In both species, cells that responded best to visual contours of a specific orientation were spontaneously co-active, suggesting that functionally related cells are organized into distinct subnetworks. Although these subnetworks are clustered in the cat, they are intermingled in the rodent, arguing for specific connections within the local cortical network.

Ssecrett and NeuroTrace: interactive visualization and analysis tools for large-scale neuroscience data sets

Data sets imaged with modern electron microscopes can range from tens of terabytes to about one petabyte. Two new tools, Ssecrett and NeuroTrace, support interactive exploration and analysis of large-scale optical-and electron-microscopy images to help scientists reconstruct complex neural circuits of the mammalian nervous system.

Chronic cellular imaging of mouse visual cortex during operant behavior and passive viewing

Nearby neurons in mammalian neocortex demonstrate a great diversity of cell types and connectivity patterns. The importance of this diversity for computation is not understood. While extracellular recording studies in visual cortex have provided a particularly rich description of behavioral modulation of neural activity, new methods are needed to dissect the contribution of specific circuit elements in guiding visual perception. Here, we describe a method for three-dimensional cellular imaging of neural activity in the awake mouse visual cortex during active discrimination and passive viewing of visual stimuli. Head-fixed mice demonstrated robust discrimination for many hundred trials per day after initial task acquisition. To record from multiple neurons during operant behavior with single-trial resolution and minimal artifacts, we built a sensitive microscope for two-photon calcium imaging, capable of rapid tracking of neurons in three dimensions. We demonstrate stable recordings of cellular calcium activity during discrimination behavior across hours, days, and weeks, using both synthetic and genetically encoded calcium indicators. When combined with molecular and genetic technologies in mice (e.g., cell-type specific transgenic labeling), this approach allows the identification of neuronal classes in vivo. Physiological measurements from distinct classes of neighboring neurons will enrich our understanding of the coordinated roles of diverse elements of cortical microcircuits in guiding sensory perception and perceptual learning. Further, our method provides a high-throughput, chronic in vivo assay of behavioral influences on cellular activity that is applicable to a wide range of mouse models of neurologic disease.

Direct activation of sparse, distributed populations of cortical neurons by electrical microstimulation

For over a century, electrical microstimulation has been the most direct method for causally linking brain function with behavior. Despite this long history, it is still unclear how the activity of neural populations is affected by stimulation. For example, there is still no consensus on where activated cells lie or on the extent to which neural processes such as passing axons near the electrode are also activated. Past studies of this question have proven difficult because microstimulation interferes with electrophysiological recordings, which in any case provide only coarse information about the location of activated cells. We used two-photon calcium imaging, an optical method, to circumvent these hurdles. We found that microstimulation sparsely activates neurons around the electrode, sometimes as far as millimeters away, even at low currents. Our results indicate that the pattern of activated neurons likely arises from the direct activation of axons in a volume tens of microns in diameter.

Simulations of electrode placement for a thalamic visual prosthesis

In this paper, placement parameters for microstimulation electrodes in a visual prosthesis are evaluated based on retinotopic models of macaque and human lateral geniculate nucleus. Phosphene patterns were simulated for idealized microwire electrodes as well as for currently available clinical electrodes. For idealized microwire electrodes, spacing as large as 600 microm in three dimensions would allow for over 250 phosphenes per visual hemifield in macaques, and over 800 in humans.

Specificity and randomness in the visual cortex

Research on the functional anatomy of visual cortical circuits has recently zoomed in from the macroscopic level to the microscopic. High-resolution functional imaging has revealed that the functional architecture of orientation maps in higher mammals is built with single-cell precision. By contrast, orientation selectivity in rodents is dispersed on visual cortex in a salt-and-pepper fashion, despite highly tuned visual responses. Recent studies of synaptic physiology indicate that there are disjoint subnetworks of interconnected cells in the rodent visual cortex. These intermingled subnetworks, described in vitro, may relate to the intermingled ensembles of cells tuned to different orientations, described in vivo. This hypothesis may soon be tested with new anatomic techniques that promise to reveal the detailed wiring diagram of cortical circuits.

Homeostatic regulation of eye-specific responses in visual cortex during ocular dominance plasticity

Experience-dependent plasticity is crucial for the precise formation of neuronal connections during development. It is generally thought to depend on Hebbian forms of synaptic plasticity. In addition, neurons possess other, homeostatic means of compensating for changes in sensory input, but their role in cortical plasticity is unclear. We used two-photon calcium imaging to investigate whether homeostatic response regulation contributes to changes of eye-specific responsiveness after monocular deprivation (MD) in mouse visual cortex. Short MD durations decreased deprived-eye responses in neurons with binocular input. Longer MD periods strengthened open-eye responses, and surprisingly, also increased deprived-eye responses in neurons devoid of open-eye input. These bidirectional response adjustments effectively preserved the net visual drive for each neuron. Our finding that deprived-eye responses were either weaker or stronger after MD, depending on the amount of open-eye input a cell received, argues for both Hebbian and homeostatic mechanisms regulating neuronal responsiveness during experience-dependent plasticity.

Demonstration of artificial visual percepts generated through thalamic microstimulation

Electrical stimulation of the visual system might serve as the foundation for a prosthetic device for the blind. We examined whether microstimulation of the dorsal lateral geniculate nucleus of the thalamus can generate localized visual percepts in alert monkeys. To assess electrically generated percepts, an eye-movement task was used with targets presented on a computer screen (optically) or through microstimulation of the lateral geniculate nucleus (electrically). Saccades (fast, direct eye movements) made to electrical targets were comparable to saccades made to optical targets. Gaze locations for electrical targets were well predicted by measured visual response maps of cells at the electrode tips. With two electrodes, two distinct targets could be independently created. A sequential saccade task verified that electrical targets were processed not in motor coordinates, but in visual spatial coordinates. Microstimulation produced predictable visual percepts, showing that this technique may be useful for a visual prosthesis.

Highly ordered arrangement of single neurons in orientation pinwheels

In the visual cortex of higher mammals, neurons are arranged across the cortical surface in an orderly map of preferred stimulus orientations. This map contains 'orientation pinwheels', structures that are arranged like the spokes of a wheel such that orientation changes continuously around a centre. Conventional optical imaging first demonstrated these pinwheels, but the technique lacked the spatial resolution to determine the response properties and arrangement of cells near pinwheel centres. Electrophysiological recordings later demonstrated sharply selective neurons near pinwheel centres, but it remained unclear whether they were arranged randomly or in an orderly fashion. Here we use two-photon calcium imaging in vivo to determine the microstructure of pinwheel centres in cat visual cortex with single-cell resolution. We find that pinwheel centres are highly ordered: neurons selective to different orientations are clearly segregated even in the very centre. Thus, pinwheel centres truly represent singularities in the cortical map. This highly ordered arrangement at the level of single cells suggests great precision in the development of cortical circuits underlying orientation selectivity.

Attention modulates the responses of simple cells in monkey primary visual cortex

Spatial attention has long been postulated to act as a spotlight that increases the salience of visual stimuli at the attended location. We examined the effects of attention on the receptive fields of simple cells in primary visual cortex (V1) by training macaque monkeys to perform a task with two modes. In the attended mode, the stimuli relevant to the animal's task overlay the receptive field of the neuron being recorded. In the unattended mode, the animal was cued to attend to stimuli outside the receptive field of that neuron. The relevant stimulus, a colored pixel, was briefly presented within a white-noise stimulus, a flickering grid of black and white pixels. The receptive fields of the neurons were mapped by correlating spikes with the white-noise stimulus in both attended and unattended modes. We found that attention could cause significant modulation of the visually evoked response despite an absence of significant effects on the overall firing rates. On further examination of the relationship between the strength of the visual stimulation and the firing rate, we found that attention appears to cause multiplicative scaling of the visually evoked responses of simple cells, demonstrating that attention reaches back to the initial stages of visual cortical processing.

Single-neuron responses and neuronal decisions in a vernier task

Vernier acuity is a measure of the smallest horizontal offset between two vertical lines that can be behaviorally discriminated. To examine the link between the neuronal responses in a retinotopic mosaic and vernier acuity, we recorded the responses of single cells in cat lateral geniculate nucleus to a vertical bar stimulus that was stepped in small increments through the receptive fields of cells. Based on the single-trial responses evoked by stimuli at different positions, we calculated the spatial resolution that could be achieved. If the stimulus could fall anywhere in their receptive fields, single neurons had spatial resolutions two times worse than previously reported vernier thresholds. Given the known coverage factor in a cat retina, we developed a two-stage decision model to examine how the responses of neurons in a retinotopic mosaic could be processed to achieve vernier acuity. In order for psychophysical thresholds to be accounted for by the responses of a single cell, the stimulus must fall in the quarter of the receptive field that provides the most information about stimulus position. Alternatively, both the absolute psychophysical threshold for vernier acuity and its dependence on stimulus length can be realized by pooling the responses of a few neurons, all located on one side of the bar stimulus.