Network anatomy and in vivo physiology of visual cortical neurons

Deciphering prefrontal circuits underlying stress and depression: exploring the potential of volume electron microscopy

Adapting to environmental changes and formulating behavioral strategies are central to the nervous system, with the prefrontal cortex being crucial. Chronic stress impacts this region, leading to disorders including major depression. This review discusses the roles for prefrontal cortex and the effects of stress, highlighting similarities and differences between human/primates and rodent brains. Notably, the rodent medial prefrontal cortex is analogous to the human subgenual anterior cingulate cortex in terms of emotional regulation, sharing similarities in cytoarchitecture and circuitry, while also performing cognitive functions similar to the human dorsolateral prefrontal cortex. It has been shown that chronic stress induces atrophic changes in the rodent mPFC, which mirrors the atrophy observed in the subgenual anterior cingulate cortex and dorsolateral prefrontal cortex of depression patients. However, the precise alterations in neural circuitry due to chronic stress are yet to be fully unraveled. The use of advanced imaging techniques, particularly volume electron microscopy, is emphasized as critical for the detailed examination of synaptic changes, providing a deeper understanding of stress and depression at the molecular, cellular and circuit levels. This approach offers invaluable insights into the alterations in neuronal circuits within the medial prefrontal cortex caused by chronic stress, significantly enriching our understanding of stress and depression pathologies.

Calcium-permeable AMPA receptors govern PV neuron feature selectivity

The brain helps us survive by forming internal representations of the external world1,2. Excitatory cortical neurons are often precisely tuned to specific external stimuli3,4. However, inhibitory neurons, such as parvalbumin-positive (PV) interneurons, are generally less selective5. PV interneurons differ from excitatory neurons in their neurotransmitter receptor subtypes, including AMPA (α-amino-3-hydroxy-5-methyl-4-isoxazole propionic acid) receptors (AMPARs)6,7. Excitatory neurons express calcium-impermeable AMPARs that contain the GluA2 subunit (encoded by GRIA2), whereas PV interneurons express receptors that lack the GluA2 subunit and are calcium-permeable (CP-AMPARs). Here we demonstrate a causal relationship between CP-AMPAR expression and the low feature selectivity of PV interneurons. We find low expression stoichiometry of GRIA2 mRNA relative to other subunits in PV interneurons that is conserved across ferrets, rodents, marmosets and humans, and causes abundant CP-AMPAR expression. Replacing CP-AMPARs in PV interneurons with calcium-impermeable AMPARs increased their orientation selectivity in the visual cortex. Manipulations to induce sparse CP-AMPAR expression demonstrated that this increase was cell-autonomous and could occur with changes beyond development. Notably, excitatory-PV interneuron connectivity rates and unitary synaptic strength were unaltered by CP-AMPAR removal, which suggested that the selectivity of PV interneurons can be altered without markedly changing connectivity. In Gria2-knockout mice, in which all AMPARs are calcium-permeable, excitatory neurons showed significantly degraded orientation selectivity, which suggested that CP-AMPARs are sufficient to drive lower selectivity regardless of cell type. Moreover, hippocampal PV interneurons, which usually exhibit low spatial tuning, became more spatially selective after removing CP-AMPARs, which indicated that CP-AMPARs suppress the feature selectivity of PV interneurons independent of modality. These results reveal a new role of CP-AMPARs in maintaining low-selectivity sensory representation in PV interneurons and implicate a conserved molecular mechanism that distinguishes this cell type in the neocortex.

NEURD offers automated proofreading and feature extraction for connectomics

We are now in the era of millimeter-scale electron microscopy (EM) volumes collected at nanometer resolution. Dense reconstruction of cellular compartments in these EM volumes has been enabled by recent advances in Machine Learning (ML). Automated segmentation methods produce exceptionally accurate reconstructions of cells, but post-hoc proofreading is still required to generate large connectomes free of merge and split errors. The elaborate 3-D meshes of neurons in these volumes contain detailed morphological information at multiple scales, from the diameter, shape, and branching patterns of axons and dendrites, down to the fine-scale structure of dendritic spines. However, extracting these features can require substantial effort to piece together existing tools into custom workflows. Building on existing open-source software for mesh manipulation, here we present "NEURD", a software package that decomposes meshed neurons into compact and extensively-annotated graph representations. With these feature-rich graphs, we automate a variety of tasks such as state of the art automated proofreading of merge errors, cell classification, spine detection, axon-dendritic proximities, and other annotations. These features enable many downstream analyses of neural morphology and connectivity, making these massive and complex datasets more accessible to neuroscience researchers focused on a variety of scientific questions.

Collaborative augmented reconstruction of 3D neuron morphology in mouse and human brains

Digital reconstruction of the intricate 3D morphology of individual neurons from microscopic images is a crucial challenge in both individual laboratories and large-scale projects focusing on cell types and brain anatomy. This task often fails in both conventional manual reconstruction and state-of-the-art artificial intelligence (AI)-based automatic reconstruction algorithms. It is also challenging to organize multiple neuroanatomists to generate and cross-validate biologically relevant and mutually agreed upon reconstructions in large-scale data production. Based on collaborative group intelligence augmented by AI, we developed a collaborative augmented reconstruction (CAR) platform for neuron reconstruction at scale. This platform allows for immersive interaction and efficient collaborative editing of neuron anatomy using a variety of devices, such as desktop workstations, virtual reality headsets and mobile phones, enabling users to contribute anytime and anywhere and to take advantage of several AI-based automation tools. We tested CAR's applicability for challenging mouse and human neurons toward scaled and faithful data production.

2P-NucTag: on-demand phototagging for molecular analysis of functionally identified cortical neurons

Neural circuits are characterized by genetically and functionally diverse cell types. A mechanistic understanding of circuit function is predicated on linking the genetic and physiological properties of individual neurons. However, it remains highly challenging to map the transcriptional properties to functionally heterogeneous neuronal subtypes in mammalian cortical circuits in vivo. Here, we introduce a high-throughput two-photon nuclear phototagging (2P-NucTag) approach optimized for on-demand and indelible labeling of single neurons via a photoactivatable red fluorescent protein following in vivo functional characterization in behaving mice. We demonstrate the utility of this function-forward pipeline by selectively labeling and transcriptionally profiling previously inaccessible 'place' and 'silent' cells in the mouse hippocampus. Our results reveal unexpected differences in gene expression between these hippocampal pyramidal neurons with distinct spatial coding properties. Thus, 2P-NucTag opens a new way to uncover the molecular principles that govern the functional organization of neural circuits.

Cooperative thalamocortical circuit mechanism for sensory prediction errors

The brain functions as a prediction machine, utilizing an internal model of the world to anticipate sensations and the outcomes of our actions. Discrepancies between expected and actual events, referred to as prediction errors, are leveraged to update the internal model and guide our attention towards unexpected events1-10. Despite the importance of prediction-error signals for various neural computations across the brain, surprisingly little is known about the neural circuit mechanisms responsible for their implementation. Here we describe a thalamocortical disinhibitory circuit that is required for generating sensory prediction-error signals in mouse primary visual cortex (V1). We show that violating animals' predictions by an unexpected visual stimulus preferentially boosts responses of the layer 2/3 V1 neurons that are most selective for that stimulus. Prediction errors specifically amplify the unexpected visual input, rather than representing non-specific surprise or difference signals about how the visual input deviates from the animal's predictions. This selective amplification is implemented by a cooperative mechanism requiring thalamic input from the pulvinar and cortical vasoactive-intestinal-peptide-expressing (VIP) inhibitory interneurons. In response to prediction errors, VIP neurons inhibit a specific subpopulation of somatostatin-expressing inhibitory interneurons that gate excitatory pulvinar input to V1, resulting in specific pulvinar-driven response amplification of the most stimulus-selective neurons in V1. Therefore, the brain prioritizes unpredicted sensory information by selectively increasing the salience of unpredicted sensory features through the synergistic interaction of thalamic input and neocortical disinhibitory circuits.

Strategies to dissect microglia-synaptic interactions during aging and in Alzheimer's disease

Age is the largest risk factor for developing Alzheimer's disease (AD), a neurodegenerative disorder that causes a progressive and severe dementia. The underlying cause of cognitive deficits seen in AD is thought to be the disconnection of neural circuits that control memory and executive functions. Insight into the mechanisms by which AD diverges from normal aging will require identifying precisely which cellular events are driven by aging and which are impacted by AD-related pathologies. Since microglia, the brain-resident macrophages, are known to have critical roles in the formation and maintenance of neural circuits through synaptic pruning, they are well-positioned to modulate synaptic connectivity in circuits sensitive to aging or AD. In this review, we provide an overview of the current state of the field and on emerging technologies being employed to elucidate microglia-synaptic interactions in aging and AD. We also discuss the importance of leveraging genetic diversity to study how these interactions are shaped across more realistic contexts. We propose that these approaches will be essential to define specific aging- and disease-relevant trajectories for more personalized therapeutics aimed at reducing the effects of age or AD pathologies on the brain. This article is part of the Special Issue on "Microglia".

Fast imaging of millimeter-scale areas with beam deflection transmission electron microscopy

Serial section transmission electron microscopy (TEM) has proven to be one of the leading methods for millimeter-scale 3D imaging of brain tissues at nanoscale resolution. It is important to further improve imaging efficiency to acquire larger and more brain volumes. We report here a threefold increase in the speed of TEM by using a beam deflecting mechanism to enable highly efficient acquisition of multiple image tiles (nine) for each motion of the mechanical stage. For millimeter-scale areas, the duty cycle of imaging doubles to more than 30%, yielding a net average imaging rate of 0.3 gigapixels per second. If fully utilized, an array of four beam deflection TEMs should be capable of imaging a dataset of cubic millimeter scale in five weeks.

Electron holography observation of electron spin polarization around charged insulating wire

We report direct observation by electron holography of the spin polarization of electrons in a vacuum region around a charged SiO2 wire coated with Pt-Pd. Irradiating the SiO2 wire with 300 keV electrons caused the wire to become positively charged due to the emission of secondary electrons. The spin polarization of these electrons interacting with the charged wire was observed in situ using a phase reconstruction process under an external magnetic field. The magnetic field of the spin-polarized electrons was simulated taking into account the distribution of secondary electrons and the effect of the external magnetic field.

Volume electron microscopy for genetically and molecularly defined neural circuits

The brain networks responsible for adaptive behavioral changes are based on the physical connections between neurons. Light and electron microscopy have long been used to study neural projections and the physical connections between neurons. Volume electron microscopy has recently expanded its scale of analysis due to methodological advances, resulting in complete wiring maps of neurites in a large volume of brain tissues and even entire nervous systems in a growing number of species. However, structural approaches frequently suffer from inherent limitations in which elements in images are identified solely by morphological criteria. Recently, an increasing number of tools and technologies have been developed to characterize cells and cellular components in the context of molecules and gene expression. These advancements include newly developed probes for visualization in electron microscopic images as well as correlative integration methods for the same elements across multiple microscopic modalities. Such approaches advance our understanding of interactions between specific neurons and circuits and may help to elucidate novel aspects of the basal ganglia network involving dopamine neurons. These advancements are expected to reveal mechanisms for processing adaptive changes in specific neural circuits that modulate brain functions.

SmartEM: machine-learning guided electron microscopy

Connectomics provides essential nanometer-resolution, synapse-level maps of neural circuits to understand brain activity and behavior. However, few researchers have access to the high-throughput electron microscopes necessary to generate enough data for whole circuit or brain reconstruction. To date, machine-learning methods have been used after the collection of images by electron microscopy (EM) to accelerate and improve neuronal segmentation, synapse reconstruction and other data analysis. With the computational improvements in processing EM images, acquiring EM images has now become the rate-limiting step. Here, in order to speed up EM imaging, we integrate machine-learning into real-time image acquisition in a singlebeam scanning electron microscope. This SmartEM approach allows an electron microscope to perform intelligent, data-aware imaging of specimens. SmartEM allocates the proper imaging time for each region of interest - scanning all pixels equally rapidly, then re-scanning small subareas more slowly where a higher quality signal is required to achieve accurate segmentability, in significantly less time. We demonstrate that this pipeline achieves a 7-fold acceleration of image acquisition time for connectomics using a commercial single-beam SEM. We apply SmartEM to reconstruct a portion of mouse cortex with the same accuracy as traditional microscopy but in less time.

Deciphering the genetic code of neuronal type connectivity through bilinear modeling

Understanding how different neuronal types connect and communicate is critical to interpreting brain function and behavior. However, it has remained a formidable challenge to decipher the genetic underpinnings that dictate the specific connections formed between neuronal types. To address this, we propose a novel bilinear modeling approach that leverages the architecture similar to that of recommendation systems. Our model transforms the gene expressions of presynaptic and postsynaptic neuronal types, obtained from single-cell transcriptomics, into a covariance matrix. The objective is to construct this covariance matrix that closely mirrors a connectivity matrix, derived from connectomic data, reflecting the known anatomical connections between these neuronal types. When tested on a dataset of Caenorhabditis elegans, our model achieved a performance comparable to, if slightly better than, the previously proposed spatial connectome model (SCM) in reconstructing electrical synaptic connectivity based on gene expressions. Through a comparative analysis, our model not only captured all genetic interactions identified by the SCM but also inferred additional ones. Applied to a mouse retinal neuronal dataset, the bilinear model successfully recapitulated recognized connectivity motifs between bipolar cells and retinal ganglion cells, and provided interpretable insights into genetic interactions shaping the connectivity. Specifically, it identified unique genetic signatures associated with different connectivity motifs, including genes important to cell-cell adhesion and synapse formation, highlighting their role in orchestrating specific synaptic connections between these neurons. Our work establishes an innovative computational strategy for decoding the genetic programming of neuronal type connectivity. It not only sets a new benchmark for single-cell transcriptomic analysis of synaptic connections but also paves the way for mechanistic studies of neural circuit assembly and genetic manipulation of circuit wiring.

Ultrastructural differences impact cilia shape and external exposure across cell classes in the visual cortex

A primary cilium is a membrane-bound extension from the cell surface that contains receptors for perceiving and transmitting signals that modulate cell state and activity. Primary cilia in the brain are less accessible than cilia on cultured cells or epithelial tissues because in the brain they protrude into a deep, dense network of glial and neuronal processes. Here, we investigated cilia frequency, internal structure, shape, and position in large, high-resolution transmission electron microscopy volumes of mouse primary visual cortex. Cilia extended from the cell bodies of nearly all excitatory and inhibitory neurons, astrocytes, and oligodendrocyte precursor cells (OPCs) but were absent from oligodendrocytes and microglia. Ultrastructural comparisons revealed that the base of the cilium and the microtubule organization differed between neurons and glia. Investigating cilia-proximal features revealed that many cilia were directly adjacent to synapses, suggesting that cilia are poised to encounter locally released signaling molecules. Our analysis indicated that synapse proximity is likely due to random encounters in the neuropil, with no evidence that cilia modulate synapse activity as would be expected in tetrapartite synapses. The observed cell class differences in proximity to synapses were largely due to differences in external cilia length. Many key structural features that differed between neuronal and glial cilia influenced both cilium placement and shape and, thus, exposure to processes and synapses outside the cilium. Together, the ultrastructure both within and around neuronal and glial cilia suggest differences in cilia formation and function across cell types in the brain.

Germline PTEN genotype-dependent phenotypic divergence during the early neural developmental process of forebrain organoids

PTEN germline mutations account for ~0.2-1% of all autism spectrum disorder (ASD) cases, as well as ~17% of ASD patients with macrocephaly, making it one of the top ASD-associated risk genes. Individuals with germline PTEN mutations receive the molecular diagnosis of PTEN Hamartoma Tumor Syndrome (PHTS), an inherited cancer predisposition syndrome, about 20-23% of whom are diagnosed with ASD. We generated forebrain organoid cultures from gene-edited isogenic human induced pluripotent stem cells (hiPSCs) harboring a PTENG132D (ASD) or PTENM134R (cancer) mutant allele to model how these mutations interrupt neurodevelopmental processes. Here, we show that the PTENG132D allele disrupts early neuroectoderm formation during the first several days of organoid generation, and results in deficient electrophysiology. While organoids generated from PTENM134R hiPSCs remained morphologically similar to wild-type organoids during this early stage in development, we observed disrupted neuronal differentiation, radial glia positioning, and cortical layering in both PTEN-mutant organoids at the later stage of 72+ days of development. Perifosine, an AKT inhibitor, reduced over-activated AKT and partially corrected the abnormalities in cellular organization observed in PTENG132D organoids. Single cell RNAseq analyses on early-stage organoids revealed that genes related to neural cell fate were decreased in PTENG132D mutant organoids, and AKT inhibition was capable of upregulating gene signatures related to neuronal cell fate and CNS maturation pathways. These findings demonstrate that different PTEN missense mutations can have a profound impact on neurodevelopment at diverse stages which in turn may predispose PHTS individuals to ASD. Further study will shed light on ways to mitigate pathological impact of PTEN mutants on neurodevelopment by stage-specific manipulation of downstream PTEN signaling components.

A petavoxel fragment of human cerebral cortex reconstructed at nanoscale resolution

To fully understand how the human brain works, knowledge of its structure at high resolution is needed. Presented here is a computationally intensive reconstruction of the ultrastructure of a cubic millimeter of human temporal cortex that was surgically removed to gain access to an underlying epileptic focus. It contains about 57,000 cells, about 230 millimeters of blood vessels, and about 150 million synapses and comprises 1.4 petabytes. Our analysis showed that glia outnumber neurons 2:1, oligodendrocytes were the most common cell, deep layer excitatory neurons could be classified on the basis of dendritic orientation, and among thousands of weak connections to each neuron, there exist rare powerful axonal inputs of up to 50 synapses. Further studies using this resource may bring valuable insights into the mysteries of the human brain.

RoboEM: automated 3D flight tracing for synaptic-resolution connectomics

Mapping neuronal networks from three-dimensional electron microscopy (3D-EM) data still poses substantial reconstruction challenges, in particular for thin axons. Currently available automated image segmentation methods require manual proofreading for many types of connectomic analysis. Here we introduce RoboEM, an artificial intelligence-based self-steering 3D 'flight' system trained to navigate along neurites using only 3D-EM data as input. Applied to 3D-EM data from mouse and human cortex, RoboEM substantially improves automated state-of-the-art segmentations and can replace manual proofreading for more complex connectomic analysis problems, yielding computational annotation cost for cortical connectomes about 400-fold lower than the cost of manual error correction.

Morphomics via next-generation electron microscopy

The living body is composed of innumerable fine and complex structures. Although these structures have been studied in the past, a vast amount of information pertaining to them still remains unknown. When attempting to observe these ultra-structures, the use of electron microscopy (EM) has become indispensable. However, conventional EM settings are limited to a narrow tissue area, which can bias observations. Recently, new trends in EM research have emerged, enabling coverage of far broader, nano-scale fields of view for two-dimensional wide areas and three-dimensional large volumes. Moreover, cutting-edge bioimage informatics conducted via deep learning has accelerated the quantification of complex morphological bioimages. Taken together, these technological and analytical advances have led to the comprehensive acquisition and quantification of cellular morphology, which now arises as a new omics science termed 'morphomics'.

Local origin of excitatory-inhibitory tuning equivalence in a cortical network

The interplay between excitation and inhibition determines the fidelity of cortical representations. The receptive fields of excitatory neurons are often finely tuned to encoded features, but the principles governing the tuning of inhibitory neurons remain elusive. In this study, we recorded populations of neurons in the mouse postsubiculum (PoSub), where the majority of excitatory neurons are head-direction (HD) cells. We show that the tuning of fast-spiking (FS) cells, the largest class of cortical inhibitory neurons, was broad and frequently radially symmetrical. By decomposing tuning curves using the Fourier transform, we identified an equivalence in tuning between PoSub-FS and PoSub-HD cell populations. Furthermore, recordings, optogenetic manipulations of upstream thalamic populations and computational modeling provide evidence that the tuning of PoSub-FS cells has a local origin. These findings support the notion that the equivalence of neuronal tuning between excitatory and inhibitory cell populations is an intrinsic property of local cortical networks.

Array tomography: trails to discovery

Tissue slicing is at the core of many approaches to studying biological structures. Among the modern volume electron microscopy (vEM) methods, array tomography (AT) is based on serial ultramicrotomy, section collection onto solid support, imaging via light and/or scanning electron microscopy, and re-assembly of the serial images into a volume for analysis. While AT largely uses standard EM equipment, it provides several advantages, including long-term preservation of the sample and compatibility with multi-scale and multi-modal imaging. Furthermore, the collection of serial ultrathin sections improves axial resolution and provides access for molecular labeling, which is beneficial for light microscopy and immunolabeling, and facilitates correlation with EM. Despite these benefits, AT techniques are underrepresented in imaging facilities and labs, due to their perceived difficulty and lack of training opportunities. Here we point towards novel developments in serial sectioning and image analysis that facilitate the AT pipeline, and solutions to overcome constraints. Because no single vEM technique can serve all needs regarding field of view and resolution, we sketch a decision tree to aid researchers in navigating the plethora of options available. Lastly, we elaborate on the unexplored potential of AT approaches to add valuable insight in diverse biological fields.

FAST-EM array tomography: a workflow for multibeam volume electron microscopy

Elucidating the 3D nanoscale structure of tissues and cells is essential for understanding the complexity of biological processes. Electron microscopy (EM) offers the resolution needed for reliable interpretation, but the limited throughput of electron microscopes has hindered its ability to effectively image large volumes. We report a workflow for volume EM with FAST-EM, a novel multibeam scanning transmission electron microscope that speeds up acquisition by scanning the sample in parallel with 64 electron beams. FAST-EM makes use of optical detection to separate the signals of the individual beams. The acquisition and 3D reconstruction of ultrastructural data from multiple biological samples is demonstrated. The results show that the workflow is capable of producing large reconstructed volumes with high resolution and contrast to address biological research questions within feasible acquisition time frames.

The application and development of electron microscopy for three-dimensional reconstruction in life science: a review

Imaging technologies have played a pivotal role in advancing biological research by enabling visualization of biological structures and processes. While traditional electron microscopy (EM) produces two-dimensional images, emerging techniques now allow high-resolution three-dimensional (3D) characterization of specimens in situ, meeting growing needs in molecular and cellular biology. Combining transmission electron microscopy (TEM) with serial sectioning inaugurated 3D imaging, attracting biologists seeking to explore cell ultrastructure and driving advancement of 3D EM reconstruction. By comprehensively and precisely rendering internal structure and distribution, 3D TEM reconstruction provides unparalleled ultrastructural insights into cells and molecules, holding tremendous value for elucidating structure-function relationships and broadly propelling structural biology. Here, we first introduce the principle of 3D reconstruction of cells and tissues by classical approaches in TEM and then discuss modern technologies utilizing TEM and on new SEM-based as well as cryo-electron microscope (cryo-EM) techniques. 3D reconstruction techniques from serial sections, electron tomography (ET), and the recent single-particle analysis (SPA) are examined; the focused ion beam scanning electron microscopy (FIB-SEM), the serial block-face scanning electron microscopy (SBF-SEM), and automatic tape-collecting lathe ultramicrotome (ATUM-SEM) for 3D reconstruction of large volumes are discussed. Finally, we review the challenges and development prospects of these technologies in life science. It aims to provide an informative reference for biological researchers.

Photoemission electron microscopy for connectomics

Detailing the physical basis of neural circuits with large-volume serial electron microscopy (EM), 'connectomics', has emerged as an invaluable tool in the neuroscience armamentarium. However, imaging synaptic resolution connectomes is currently limited to either transmission electron microscopy (TEM) or scanning electron microscopy (SEM). Here, we describe a third way, using photoemission electron microscopy (PEEM) which illuminates ultra-thin brain slices collected on solid substrates with UV light and images the photoelectron emission pattern with a wide-field electron microscope. PEEM works with existing sample preparations for EM and routinely provides sufficient resolution and contrast to reveal myelinated axons, somata, dendrites, and sub-cellular organelles. Under optimized conditions, PEEM provides synaptic resolution; and simulation and experiments show that PEEM can be transformatively fast, at Gigahertz pixel rates. We conclude that PEEM imaging leverages attractive aspects of SEM and TEM, namely reliable sample collection on robust substrates combined with fast wide-field imaging, and could enable faster data acquisition for next-generation circuit mapping.

GAUSS-EM, guided accumulation of ultrathin serial sections with a static magnetic field for volume electron microscopy

Serial sectioning electron microscopy (EM) of millimeter-scale three-dimensional (3D) anatomical volumes requires the collection of thousands of ultrathin sections. Here, we report a high-throughput automated approach, GAUSS-EM (guided accumulation of ultrathin serial sections-EM), utilizing a static magnetic field to collect and densely pack thousands of sections onto individual silicon wafers. The method is capable of sectioning hundreds of microns of tissue per day at section thicknesses down to 35 nm. Relative to other automated volume EM approaches, GAUSS-EM democratizes the ability to collect large 3D EM volumes because it is simple and inexpensive to implement. We present two exemplar EM volumes of a zebrafish eye and mouse olfactory bulb collected with the method.

Functional specificity of recurrent inhibition in visual cortex

In the neocortex, neural activity is shaped by the interaction of excitatory and inhibitory neurons, defined by the organization of their synaptic connections. Although connections among excitatory pyramidal neurons are sparse and functionally tuned, inhibitory connectivity is thought to be dense and largely unstructured. By measuring in vivo visual responses and synaptic connectivity of parvalbumin-expressing (PV+) inhibitory cells in mouse primary visual cortex, we show that the synaptic weights of their connections to nearby pyramidal neurons are specifically tuned according to the similarity of the cells' responses. Individual PV+ cells strongly inhibit those pyramidal cells that provide them with strong excitation and share their visual selectivity. This structured organization of inhibitory synaptic weights provides a circuit mechanism for tuned inhibition onto pyramidal cells despite dense connectivity, stabilizing activity within feature-specific excitatory ensembles while supporting competition between them.

Synaptic wiring motifs in posterior parietal cortex support decision-making

The posterior parietal cortex exhibits choice-selective activity during perceptual decision-making tasks1-10. However, it is not known how this selective activity arises from the underlying synaptic connectivity. Here we combined virtual-reality behaviour, two-photon calcium imaging, high-throughput electron microscopy and circuit modelling to analyse how synaptic connectivity between neurons in the posterior parietal cortex relates to their selective activity. We found that excitatory pyramidal neurons preferentially target inhibitory interneurons with the same selectivity. In turn, inhibitory interneurons preferentially target pyramidal neurons with opposite selectivity, forming an opponent inhibition motif. This motif was present even between neurons with activity peaks in different task epochs. We developed neural-circuit models of the computations performed by these motifs, and found that opponent inhibition between neural populations with opposite selectivity amplifies selective inputs, thereby improving the encoding of trial-type information. The models also predict that opponent inhibition between neurons with activity peaks in different task epochs contributes to creating choice-specific sequential activity. These results provide evidence for how synaptic connectivity in cortical circuits supports a learned decision-making task.

Network Statistics of the Whole-Brain Connectome of Drosophila

Brains comprise complex networks of neurons and connections. Network analysis applied to the wiring diagrams of brains can offer insights into how brains support computations and regulate information flow. The completion of the first whole-brain connectome of an adult Drosophila, the largest connectome to date, containing 130,000 neurons and millions of connections, offers an unprecedented opportunity to analyze its network properties and topological features. To gain insights into local connectivity, we computed the prevalence of two- and three-node network motifs, examined their strengths and neurotransmitter compositions, and compared these topological metrics with wiring diagrams of other animals. We discovered that the network of the fly brain displays rich club organization, with a large population (30% percent of the connectome) of highly connected neurons. We identified subsets of rich club neurons that may serve as integrators or broadcasters of signals. Finally, we examined subnetworks based on 78 anatomically defined brain regions or neuropils. These data products are shared within the FlyWire Codex and will serve as a foundation for models and experiments exploring the relationship between neural activity and anatomical structure.

Vision-driven metasurfaces for perception enhancement

Metasurfaces have exhibited unprecedented degree of freedom in manipulating electromagnetic (EM) waves and thus provide fantastic front-end interfaces for smart systems. Here we show a framework for perception enhancement based on vision-driven metasurface. Human's eye movements are matched with microwave radiations to extend the humans' perception spectrum. By this means, our eyes can "sense" visual information and invisible microwave information. Several experimental demonstrations are given for specific implementations, including a physiological-signal-monitoring system, an "X-ray-glasses" system, a "glimpse-and-forget" tracking system and a speech reception system for deaf people. Both the simulation and experiment results verify evident advantages in perception enhancement effects and improving information acquisition efficiency. This framework can be readily integrated into healthcare systems to monitor physiological signals and to offer assistance for people with disabilities. This work provides an alternative framework for perception enhancement and may find wide applications in healthcare, wearable devices, search-and-rescue and others.

Petascale pipeline for precise alignment of images from serial section electron microscopy

The reconstruction of neural circuits from serial section electron microscopy (ssEM) images is being accelerated by automatic image segmentation methods. Segmentation accuracy is often limited by the preceding step of aligning 2D section images to create a 3D image stack. Precise and robust alignment in the presence of image artifacts is challenging, especially as datasets are attaining the petascale. We present a computational pipeline for aligning ssEM images with several key elements. Self-supervised convolutional nets are trained via metric learning to encode and align image pairs, and they are used to initialize iterative fine-tuning of alignment. A procedure called vector voting increases robustness to image artifacts or missing image data. For speedup the series is divided into blocks that are distributed to computational workers for alignment. The blocks are aligned to each other by composing transformations with decay, which achieves a global alignment without resorting to a time-consuming global optimization. We apply our pipeline to a whole fly brain dataset, and show improved accuracy relative to prior state of the art. We also demonstrate that our pipeline scales to a cubic millimeter of mouse visual cortex. Our pipeline is publicly available through two open source Python packages.

The logic of recurrent circuits in the primary visual cortex

Recurrent cortical activity sculpts visual perception by refining, amplifying or suppressing visual input. However, the rules that govern the influence of recurrent activity remain enigmatic. We used ensemble-specific two-photon optogenetics in the mouse visual cortex to isolate the impact of recurrent activity from external visual input. We found that the spatial arrangement and the visual feature preference of the stimulated ensemble and the neighboring neurons jointly determine the net effect of recurrent activity. Photoactivation of these ensembles drives suppression in all cells beyond 30 µm but uniformly drives activation in closer similarly tuned cells. In nonsimilarly tuned cells, compact, cotuned ensembles drive net suppression, while diffuse, cotuned ensembles drive activation. Computational modeling suggests that highly local recurrent excitatory connectivity and selective convergence onto inhibitory neurons explain these effects. Our findings reveal a straightforward logic in which space and feature preference of cortical ensembles determine their impact on local recurrent activity.

msemalign: a pipeline for serial section multibeam scanning electron microscopy volume alignment

Serial section multibeam scanning electron microscopy (ssmSEM) is currently among the fastest technologies available for acquiring 3D anatomical data spanning relatively large neural tissue volumes, on the order of 1 mm3 or larger, at a resolution sufficient to resolve the fine detail of neuronal morphologies and synapses. These petabyte-scale volumes can be analyzed to create connectomes, datasets that contain detailed anatomical information including synaptic connectivity, neuronal morphologies and distributions of cellular organelles. The mSEM acquisition process creates hundreds of millions of individual image tiles for a single cubic-millimeter-sized dataset and these tiles must be aligned to create 3D volumes. Here we introduce msemalign, an alignment pipeline that strives for scalability and design simplicity. The pipeline can align petabyte-scale datasets such that they contain smooth transitions as the dataset is navigated in all directions, but critically that does so in a fashion that minimizes the overall magnitude of section distortions relative to the originally acquired micrographs.

Multi-layered maps of neuropil with segmentation-guided contrastive learning

Maps of the nervous system that identify individual cells along with their type, subcellular components and connectivity have the potential to elucidate fundamental organizational principles of neural circuits. Nanometer-resolution imaging of brain tissue provides the necessary raw data, but inferring cellular and subcellular annotation layers is challenging. We present segmentation-guided contrastive learning of representations (SegCLR), a self-supervised machine learning technique that produces representations of cells directly from 3D imagery and segmentations. When applied to volumes of human and mouse cortex, SegCLR enables accurate classification of cellular subcompartments and achieves performance equivalent to a supervised approach while requiring 400-fold fewer labeled examples. SegCLR also enables inference of cell types from fragments as small as 10 μm, which enhances the utility of volumes in which many neurites are truncated at boundaries. Finally, SegCLR enables exploration of layer 5 pyramidal cell subtypes and automated large-scale analysis of synaptic partners in mouse visual cortex.

Nanometer-scale views of visual cortex reveal anatomical features of primary cilia poised to detect synaptic spillover

A primary cilium is a thin membrane-bound extension off a cell surface that contains receptors for perceiving and transmitting signals that modulate cell state and activity. While many cell types have a primary cilium, little is known about primary cilia in the brain, where they are less accessible than cilia on cultured cells or epithelial tissues and protrude from cell bodies into a deep, dense network of glial and neuronal processes. Here, we investigated cilia frequency, internal structure, shape, and position in large, high-resolution transmission electron microscopy volumes of mouse primary visual cortex. Cilia extended from the cell bodies of nearly all excitatory and inhibitory neurons, astrocytes, and oligodendrocyte precursor cells (OPCs), but were absent from oligodendrocytes and microglia. Structural comparisons revealed that the membrane structure at the base of the cilium and the microtubule organization differed between neurons and glia. OPC cilia were distinct in that they were the shortest and contained pervasive internal vesicles only occasionally observed in neuron and astrocyte cilia. Investigating cilia-proximal features revealed that many cilia were directly adjacent to synapses, suggesting cilia are well poised to encounter locally released signaling molecules. Cilia proximity to synapses was random, not enriched, in the synapse-rich neuropil. The internal anatomy, including microtubule changes and centriole location, defined key structural features including cilium placement and shape. Together, the anatomical insights both within and around neuron and glia cilia provide new insights into cilia formation and function across cell types in the brain.

Mutual generation in neuronal activity across the brain via deep neural approach, and its network interpretation

In the brain, many regions work in a network-like association, yet it is not known how durable these associations are in terms of activity and could survive without structural connections. To assess the association or similarity between brain regions with a generating approach, this study evaluated the similarity of activities of neurons within each region after disconnecting between regions. The "generation" approach here refers to using a multi-layer LSTM (Long Short-Term Memory) model to learn the rules of activity generation in one region and then apply that knowledge to generate activity in other regions. Surprisingly, the results revealed that activity generation from one region to disconnected regions was possible with similar accuracy to generation between the same regions in many cases. Notably, firing rates and synchronization of firing between neuron pairs, often used as neuronal representations, could be reproduced with precision. Additionally, accuracies were associated with the relative angle between brain regions and the strength of the structural connections that initially connected them. This outcome enables us to look into trends governing non-uniformity of the cortex based on the potential to generate informative data and reduces the need for animal experiments.

Optical STEM detection for scanning electron microscopy

Recent advances in electron microscopy techniques have led to a significant scale up in volumetric imaging of biological tissue. The throughput of electron microscopes, however, remains a limiting factor for the volume that can be imaged in high resolution within reasonable time. Faster detection methods will improve throughput. Here, we have characterized and benchmarked a novel detection technique for scanning electron microscopy: optical scanning transmission electron microscopy (OSTEM). A qualitative and quantitative comparison was performed between OSTEM, secondary and backscattered electron detection and annular dark field detection in scanning transmission electron microscopy. Our analysis shows that OSTEM produces images similar to backscattered electron detection in terms of contrast, resolution and signal-to-noise ratio. OSTEM can complement large scale imaging with (scanning) transmission electron microscopy and has the potential to speed up imaging in single-beam scanning electron microscope.

Rehydration of Freeze Substituted Brain Tissue for Pre-embedding Immunoelectron Microscopy

Electron microscopy (EM) volume reconstruction is a powerful tool for investigating the fundamental structure of brain circuits, but the full potential of this technique is limited by the difficulty of integrating molecular information. High quality ultrastructural preservation is necessary for EM reconstruction, and intact, highly contrasted cell membranes are essential for following small neuronal processes through serial sections. Unfortunately, the antibody labeling methods used to identify most endogenous molecules result in compromised morphology, especially of membranes. Cryofixation can produce superior morphological preservation and has the additional advantage of allowing indefinite storage of valuable samples. We have developed a method based on cryofixation that allows sensitive immunolabeling of endogenous molecules, preserves excellent ultrastructure, and is compatible with high-contrast staining for serial EM reconstruction.

Correlative light and volume electron microscopy to study brain development

Recent advances in volume electron microscopy (EM) have been driving our thorough understanding of the brain architecture. Volume EM becomes increasingly powerful when cells and their subcellular structures that are imaged in light microscopy are correlated to those in ultramicrographs obtained with EM. This correlative approach, called correlative light and volume electron microscopy (vCLEM), is used to link three-dimensional ultrastructural information with physiological data such as intracellular Ca2+ dynamics. Genetic tools to express fluorescent proteins and/or an engineered form of a soybean ascorbate peroxidase allow us to perform vCLEM using natural landmarks including blood vessels without immunohistochemical staining. This immunostaining-free vCLEM has been successfully employed in two-photon Ca2+ imaging in vivo as well as in studying complex synaptic connections in thalamic neurons that receive a variety of specialized inputs from the cerebral cortex. In this mini-review, we overview how volume EM and vCLEM have contributed to studying the developmental processes of the brain. We also discuss potential applications of genetic manipulation of target cells using clustered regularly interspaced short palindromic repeats-associated protein 9 and subsequent volume EM to the analysis of protein localization as well as to loss-of-function studies of genes regulating brain development. We give examples for the combinatorial usage of genetic tools with vCLEM that will further enhance our understanding of regulatory mechanisms underlying brain development.

A novel registration method for long-serial section images of EM with a serial split technique based on unsupervised optical flow network

MOTIVATION

The registration of serial section electron microscope images is a critical step in reconstructing biological tissue volumes, and it aims to eliminate complex nonlinear deformations from sectioning and replicate the correct neurite structure. However, due to the inherent properties of biological structures and the challenges posed by section preparation of biological tissues, achieving an accurate registration of serial sections remains a significant challenge. Conventional nonlinear registration techniques, which are effective in eliminating nonlinear deformation, can also eliminate the natural morphological variation of neurites across sections. Additionally, accumulation of registration errors alters the neurite structure.

RESULTS

This article proposes a novel method for serial section registration that utilizes an unsupervised optical flow network to measure feature similarity rather than pixel similarity to eliminate nonlinear deformation and achieve pairwise registration between sections. The optical flow network is then employed to estimate and compensate for cumulative registration error, thereby allowing for the reconstruction of the structure of biological tissues. Based on the novel serial section registration method, a serial split technique is proposed for long-serial sections. Experimental results demonstrate that the state-of-the-art method proposed here effectively improves the spatial continuity of serial sections, leading to more accurate registration and improved reconstruction of the structure of biological tissues.

AVAILABILITY AND IMPLEMENTATION

The source code and data are available at https://github.com/TongXin-CASIA/EFSR.

Serial Block Face-Scanning Electron Microscopy as a Burgeoning Technology

Serial block face scanning electron microscopy (SBF-SEM), also referred to as serial block-face electron microscopy, is an advanced ultrastructural imaging technique that enables three-dimensional visualization that provides largerx- and y-axis ranges than other volumetric EM techniques. While SEM is first introduced in the 1930s, SBF-SEM is developed as a novel method to resolve the 3D architecture of neuronal networks across large volumes with nanometer resolution by Denk and Horstmann in 2004. Here, the authors provide an accessible overview of the advantages and challenges associated with SBF-SEM. Beyond this, the applications of SBF-SEM in biochemical domains as well as potential future clinical applications are briefly reviewed. Finally, the alternative forms of artificial intelligence-based segmentation which may contribute to devising a feasible workflow involving SBF-SEM, are also considered.

How to incorporate biological insights into network models and why it matters

Due to the staggering complexity of the brain and its neural circuitry, neuroscientists rely on the analysis of mathematical models to elucidate its function. From Hodgkin and Huxley's detailed description of the action potential in 1952 to today, new theories and increasing computational power have opened up novel avenues to study how neural circuits implement the computations that underlie behaviour. Computational neuroscientists have developed many models of neural circuits that differ in complexity, biological realism or emergent network properties. With recent advances in experimental techniques for detailed anatomical reconstructions or large-scale activity recordings, rich biological data have become more available. The challenge when building network models is to reflect experimental results, either through a high level of detail or by finding an appropriate level of abstraction. Meanwhile, machine learning has facilitated the development of artificial neural networks, which are trained to perform specific tasks. While they have proven successful at achieving task-oriented behaviour, they are often abstract constructs that differ in many features from the physiology of brain circuits. Thus, it is unclear whether the mechanisms underlying computation in biological circuits can be investigated by analysing artificial networks that accomplish the same function but differ in their mechanisms. Here, we argue that building biologically realistic network models is crucial to establishing causal relationships between neurons, synapses, circuits and behaviour. More specifically, we advocate for network models that consider the connectivity structure and the recorded activity dynamics while evaluating task performance.

The evolution of Big Data in neuroscience and neurology

Neurological diseases are on the rise worldwide, leading to increased healthcare costs and diminished quality of life in patients. In recent years, Big Data has started to transform the fields of Neuroscience and Neurology. Scientists and clinicians are collaborating in global alliances, combining diverse datasets on a massive scale, and solving complex computational problems that demand the utilization of increasingly powerful computational resources. This Big Data revolution is opening new avenues for developing innovative treatments for neurological diseases. Our paper surveys Big Data's impact on neurological patient care, as exemplified through work done in a comprehensive selection of areas, including Connectomics, Alzheimer's Disease, Stroke, Depression, Parkinson's Disease, Pain, and Addiction (e.g., Opioid Use Disorder). We present an overview of research and the methodologies utilizing Big Data in each area, as well as their current limitations and technical challenges. Despite the potential benefits, the full potential of Big Data in these fields currently remains unrealized. We close with recommendations for future research aimed at optimizing the use of Big Data in Neuroscience and Neurology for improved patient outcomes.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40537-023-00751-2.

Multiplexed volumetric CLEM enabled by antibody derivatives provides new insights into the cytology of the mouse cerebellar cortex

Mapping neuronal networks that underlie behavior has become a central focus in neuroscience. While serial section electron microscopy (ssEM) can reveal the fine structure of neuronal networks (connectomics), it does not provide the molecular information that helps identify cell types or their functional properties. Volumetric correlated light and electron microscopy (vCLEM) combines ssEM and volumetric fluorescence microscopy to incorporate molecular labeling into ssEM datasets. We developed an approach that uses small fluorescent single-chain variable fragment (scFv) immuno-probes to perform multiplexed detergent-free immuno-labeling and ssEM on the same samples. We generated eight such fluorescent scFvs that targeted useful markers for brain studies (green fluorescent protein, glial fibrillary acidic protein, calbindin, parvalbumin, voltage-gated potassium channel subfamily A member 2, vesicular glutamate transporter 1, postsynaptic density protein 95, and neuropeptide Y). To test the vCLEM approach, six different fluorescent probes were imaged in a sample of the cortex of a cerebellar lobule (Crus 1), using confocal microscopy with spectral unmixing, followed by ssEM imaging of the same sample. The results show excellent ultrastructure with superimposition of the multiple fluorescence channels. Using this approach we could document a poorly described cell type in the cerebellum, two types of mossy fiber terminals, and the subcellular localization of one type of ion channel. Because scFvs can be derived from existing monoclonal antibodies, hundreds of such probes can be generated to enable molecular overlays for connectomic studies.

Exploring nonlinear dynamics and network structures in Kuramoto systems using machine learning approaches

Recent advances in machine learning (ML) have facilitated its application to a wide range of systems, from complex to quantum. Reservoir computing algorithms have proven particularly effective for studying nonlinear dynamical systems that exhibit collective behaviors, such as synchronizations and chaotic phenomena, some of which still remain unclear. Here, we apply ML approaches to the Kuramoto model to address several intriguing problems, including identifying the transition point and criticality of a hybrid synchronization transition, predicting future chaotic behaviors, and understanding network structures from chaotic patterns. Our proposed method also has further implications, such as inferring the structure of neural networks from electroencephalogram signals. This study, finally, highlights the potential of ML approaches for advancing our understanding of complex systems.

mEMbrain: an interactive deep learning MATLAB tool for connectomic segmentation on commodity desktops

Connectomics is fundamental in propelling our understanding of the nervous system's organization, unearthing cells and wiring diagrams reconstructed from volume electron microscopy (EM) datasets. Such reconstructions, on the one hand, have benefited from ever more precise automatic segmentation methods, which leverage sophisticated deep learning architectures and advanced machine learning algorithms. On the other hand, the field of neuroscience at large, and of image processing in particular, has manifested a need for user-friendly and open source tools which enable the community to carry out advanced analyses. In line with this second vein, here we propose mEMbrain, an interactive MATLAB-based software which wraps algorithms and functions that enable labeling and segmentation of electron microscopy datasets in a user-friendly user interface compatible with Linux and Windows. Through its integration as an API to the volume annotation and segmentation tool VAST, mEMbrain encompasses functions for ground truth generation, image preprocessing, training of deep neural networks, and on-the-fly predictions for proofreading and evaluation. The final goals of our tool are to expedite manual labeling efforts and to harness MATLAB users with an array of semi-automatic approaches for instance segmentation. We tested our tool on a variety of datasets that span different species at various scales, regions of the nervous system and developmental stages. To further expedite research in connectomics, we provide an EM resource of ground truth annotation from four different animals and five datasets, amounting to around 180 h of expert annotations, yielding more than 1.2 GB of annotated EM images. In addition, we provide a set of four pre-trained networks for said datasets. All tools are available from https://lichtman.rc.fas.harvard.edu/mEMbrain/. With our software, our hope is to provide a solution for lab-based neural reconstructions which does not require coding by the user, thus paving the way to affordable connectomics.

High-resolution volumetric imaging constrains compartmental models to explore synaptic integration and temporal processing by cochlear nucleus globular bushy cells

Globular bushy cells (GBCs) of the cochlear nucleus play central roles in the temporal processing of sound. Despite investigation over many decades, fundamental questions remain about their dendrite structure, afferent innervation, and integration of synaptic inputs. Here, we use volume electron microscopy (EM) of the mouse cochlear nucleus to construct synaptic maps that precisely specify convergence ratios and synaptic weights for auditory nerve innervation and accurate surface areas of all postsynaptic compartments. Detailed biophysically based compartmental models can help develop hypotheses regarding how GBCs integrate inputs to yield their recorded responses to sound. We established a pipeline to export a precise reconstruction of auditory nerve axons and their endbulb terminals together with high-resolution dendrite, soma, and axon reconstructions into biophysically detailed compartmental models that could be activated by a standard cochlear transduction model. With these constraints, the models predict auditory nerve input profiles whereby all endbulbs onto a GBC are subthreshold (coincidence detection mode), or one or two inputs are suprathreshold (mixed mode). The models also predict the relative importance of dendrite geometry, soma size, and axon initial segment length in setting action potential threshold and generating heterogeneity in sound-evoked responses, and thereby propose mechanisms by which GBCs may homeostatically adjust their excitability. Volume EM also reveals new dendritic structures and dendrites that lack innervation. This framework defines a pathway from subcellular morphology to synaptic connectivity, and facilitates investigation into the roles of specific cellular features in sound encoding. We also clarify the need for new experimental measurements to provide missing cellular parameters, and predict responses to sound for further in vivo studies, thereby serving as a template for investigation of other neuron classes.

Cross-modality supervised image restoration enables nanoscale tracking of synaptic plasticity in living mice

Learning is thought to involve changes in glutamate receptors at synapses, submicron structures that mediate communication between neurons in the central nervous system. Due to their small size and high density, synapses are difficult to resolve in vivo, limiting our ability to directly relate receptor dynamics to animal behavior. Here we developed a combination of computational and biological methods to overcome these challenges. First, we trained a deep-learning image-restoration algorithm that combines the advantages of ex vivo super-resolution and in vivo imaging modalities to overcome limitations specific to each optical system. When applied to in vivo images from transgenic mice expressing fluorescently labeled glutamate receptors, this restoration algorithm super-resolved synapses, enabling the tracking of behavior-associated synaptic plasticity with high spatial resolution. This method demonstrates the capabilities of image enhancement to learn from ex vivo data and imaging techniques to improve in vivo imaging resolution.

A unifying perspective on neural manifolds and circuits for cognition

Two different perspectives have informed efforts to explain the link between the brain and behaviour. One approach seeks to identify neural circuit elements that carry out specific functions, emphasizing connectivity between neurons as a substrate for neural computations. Another approach centres on neural manifolds - low-dimensional representations of behavioural signals in neural population activity - and suggests that neural computations are realized by emergent dynamics. Although manifolds reveal an interpretable structure in heterogeneous neuronal activity, finding the corresponding structure in connectivity remains a challenge. We highlight examples in which establishing the correspondence between low-dimensional activity and connectivity has been possible, unifying the neural manifold and circuit perspectives. This relationship is conspicuous in systems in which the geometry of neural responses mirrors their spatial layout in the brain, such as the fly navigational system. Furthermore, we describe evidence that, in systems in which neural responses are heterogeneous, the circuit comprises interactions between activity patterns on the manifold via low-rank connectivity. We suggest that unifying the manifold and circuit approaches is important if we are to be able to causally test theories about the neural computations that underlie behaviour.

The neuromechanics of animal locomotion: From biology to robotics and back

Robotics and neuroscience are sister disciplines that both aim to understand how agile, efficient, and robust locomotion can be achieved in autonomous agents. Robotics has already benefitted from neuromechanical principles discovered by investigating animals. These include the use of high-level commands to control low-level central pattern generator-like controllers, which, in turn, are informed by sensory feedback. Reciprocally, neuroscience has benefited from tools and intuitions in robotics to reveal how embodiment, physical interactions with the environment, and sensory feedback help sculpt animal behavior. We illustrate and discuss exemplar studies of this dialog between robotics and neuroscience. We also reveal how the increasing biorealism of simulations and robots is driving these two disciplines together, forging an integrative science of autonomous behavioral control with many exciting future opportunities.

Targeting operational regimes of interest in recurrent neural networks

Neural computations emerge from local recurrent neural circuits or computational units such as cortical columns that comprise hundreds to a few thousand neurons. Continuous progress in connectomics, electrophysiology, and calcium imaging require tractable spiking network models that can consistently incorporate new information about the network structure and reproduce the recorded neural activity features. However, for spiking networks, it is challenging to predict which connectivity configurations and neural properties can generate fundamental operational states and specific experimentally reported nonlinear cortical computations. Theoretical descriptions for the computational state of cortical spiking circuits are diverse, including the balanced state where excitatory and inhibitory inputs balance almost perfectly or the inhibition stabilized state (ISN) where the excitatory part of the circuit is unstable. It remains an open question whether these states can co-exist with experimentally reported nonlinear computations and whether they can be recovered in biologically realistic implementations of spiking networks. Here, we show how to identify spiking network connectivity patterns underlying diverse nonlinear computations such as XOR, bistability, inhibitory stabilization, supersaturation, and persistent activity. We establish a mapping between the stabilized supralinear network (SSN) and spiking activity which allows us to pinpoint the location in parameter space where these activity regimes occur. Notably, we find that biologically-sized spiking networks can have irregular asynchronous activity that does not require strong excitation-inhibition balance or large feedforward input and we show that the dynamic firing rate trajectories in spiking networks can be precisely targeted without error-driven training algorithms.

Correlated Somatosensory Input in Parvalbumin/Pyramidal Cells in Mouse Motor Cortex

In mammalian cortex, feedforward excitatory connections recruit feedforward inhibition. This is often carried by parvalbumin (PV+) interneurons, which may densely connect to local pyramidal (Pyr) neurons. Whether this inhibition affects all local excitatory cells indiscriminately or is targeted to specific subnetworks is unknown. Here, we test how feedforward inhibition is recruited by using two-channel circuit mapping to excite cortical and thalamic inputs to PV+ interneurons and Pyr neurons to mouse primary vibrissal motor cortex (M1). Single Pyr and PV+ neurons receive input from both cortex and thalamus. Connected pairs of PV+ interneurons and excitatory Pyr neurons receive correlated cortical and thalamic inputs. While PV+ interneurons are more likely to form local connections to Pyr neurons, Pyr neurons are much more likely to form reciprocal connections with PV+ interneurons that inhibit them. This suggests that Pyr and PV ensembles may be organized based on their local and long-range connections, an organization that supports the idea of local subnetworks for signal transduction and processing. Excitatory inputs to M1 can thus target inhibitory networks in a specific pattern which permits recruitment of feedforward inhibition to specific subnetworks within the cortical column.

mEMbrain: an interactive deep learning MATLAB tool for connectomic segmentation on commodity desktops

Connectomics is fundamental in propelling our understanding of the nervous system’s organization, unearthing cells and wiring diagrams reconstructed from volume electron microscopy (EM) datasets. Such reconstructions, on the one hand, have benefited from ever more precise automatic segmentation methods, which leverage sophisticated deep learning architectures and advanced machine learning algorithms. On the other hand, the field of neuroscience at large, and of image processing in particular, has manifested a need for user-friendly and open source tools which enable the community to carry out advanced analyses. In line with this second vein, here we propose mEMbrain, an interactive MATLAB-based software which wraps algorithms and functions that enable labeling and segmentation of electron microscopy datasets in a user-friendly user interface compatible with Linux and Windows. Through its integration as an API to the volume annotation and segmentation tool VAST, mEMbrain encompasses functions for ground truth generation, image preprocessing, training of deep neural networks, and on-the-fly predictions for proofreading and evaluation. The final goals of our tool are to expedite manual labeling efforts and to harness MATLAB users with an array of semi-automatic approaches for instance segmentation. We tested our tool on a variety of datasets that span different species at various scales, regions of the nervous system and developmental stages. To further expedite research in connectomics, we provide an EM resource of ground truth annotation from 4 different animals and 5 datasets, amounting to around 180 hours of expert annotations, yielding more than 1.2 GB of annotated EM images. In addition, we provide a set of 4 pre-trained networks for said datasets. All tools are available from https://lichtman.rc.fas.harvard.edu/mEMbrain/ . With our software, our hope is to provide a solution for lab-based neural reconstructions which does not require coding by the user, thus paving the way to affordable connectomics.