Brightness illusions drive a neuronal response in the primary visual cortex under top-down modulation

Brightness illusions are a powerful tool in studying vision, yet their neural correlates are poorly understood. Based on a human paradigm, we presented illusory drifting gratings to mice. Primary visual cortex (V1) neurons responded to illusory gratings, matching their direction selectivity for real gratings, and they tracked the spatial phase offset between illusory and real gratings. Illusion responses were delayed compared to real gratings, in line with the theory that processing illusions requires feedback from higher visual areas (HVAs). We provide support for this theory by showing a reduced V1 response to illusions, but not real gratings, following HVAs optogenetic inhibition. Finally, we used the pupil response (PR) as an indirect perceptual report and showed that the mouse PR matches the human PR to perceived luminance changes. Our findings resolve debates over whether V1 neurons are involved in processing illusions and highlight the involvement of feedback from HVAs.

Visuo-frontal interactions during social learning in freely moving macaques

Social interactions represent a ubiquitous aspect of our everyday life that we acquire by interpreting and responding to visual cues from conspecifics1. However, despite the general acceptance of this view, how visual information is used to guide the decision to cooperate is unknown. Here, we wirelessly recorded the spiking activity of populations of neurons in the visual and prefrontal cortex in conjunction with wireless recordings of oculomotor events while freely moving macaques engaged in social cooperation. As animals learned to cooperate, visual and executive areas refined the representation of social variables, such as the conspecific or reward, by distributing socially relevant information among neurons in each area. Decoding population activity showed that viewing social cues influences the decision to cooperate. Learning social events increased coordinated spiking between visual and prefrontal cortical neurons, which was associated with improved accuracy of neural populations to encode social cues and the decision to cooperate. These results indicate that the visual-frontal cortical network prioritizes relevant sensory information to facilitate learning social interactions while freely moving macaques interact in a naturalistic environment.

Cortical reactivations predict future sensory responses

Many theories of offline memory consolidation posit that the pattern of neurons activated during a salient sensory experience will be faithfully reactivated, thereby stabilizing the pattern1,2. However, sensory-evoked patterns are not stable but, instead, drift across repeated experiences3-6. Here, to investigate the relationship between reactivations and the drift of sensory representations, we imaged the calcium activity of thousands of excitatory neurons in the mouse lateral visual cortex. During the minute after a visual stimulus, we observed transient, stimulus-specific reactivations, often coupled with hippocampal sharp-wave ripples. Stimulus-specific reactivations were abolished by local cortical silencing during the preceding stimulus. Reactivations early in a session systematically differed from the pattern evoked by the previous stimulus-they were more similar to future stimulus response patterns, thereby predicting both within-day and across-day representational drift. In particular, neurons that participated proportionally more or less in early stimulus reactivations than in stimulus response patterns gradually increased or decreased their future stimulus responses, respectively. Indeed, we could accurately predict future changes in stimulus responses and the separation of responses to distinct stimuli using only the rate and content of reactivations. Thus, reactivations may contribute to a gradual drift and separation in sensory cortical response patterns, thereby enhancing sensory discrimination7.

Learnable latent embeddings for joint behavioural and neural analysis

Mapping behavioural actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioural data increases, there is growing interest in modelling neural dynamics during adaptive behaviours to probe neural representations1-3. In particular, although neural latent embeddings can reveal underlying correlates of behaviour, we lack nonlinear techniques that can explicitly and flexibly leverage joint behaviour and neural data to uncover neural dynamics3-5. Here, we fill this gap with a new encoding method, CEBRA, that jointly uses behavioural and neural data in a (supervised) hypothesis- or (self-supervised) discovery-driven manner to produce both consistent and high-performance latent spaces. We show that consistency can be used as a metric for uncovering meaningful differences, and the inferred latents can be used for decoding. We validate its accuracy and demonstrate our tool's utility for both calcium and electrophysiology datasets, across sensory and motor tasks and in simple or complex behaviours across species. It allows leverage of single- and multi-session datasets for hypothesis testing or can be used label free. Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, for the production of consistent latent spaces across two-photon and Neuropixels data, and can provide rapid, high-accuracy decoding of natural videos from visual cortex.

Temporal dynamics of the neural representation of hue and luminance polarity

Hue and luminance contrast are basic visual features. Here we use multivariate analyses of magnetoencephalography data to investigate the timing of the neural computations that extract them, and whether they depend on common neural circuits. We show that hue and luminance-contrast polarity can be decoded from MEG data and, with lower accuracy, both features can be decoded across changes in the other feature. These results are consistent with the existence of both common and separable neural mechanisms. The decoding time course is earlier and more temporally precise for luminance polarity than hue, a result that does not depend on task, suggesting that luminance contrast is an updating signal that separates visual events. Meanwhile, cross-temporal generalization is slightly greater for representations of hue compared to luminance polarity, providing a neural correlate of the preeminence of hue in perceptual grouping and memory. Finally, decoding of luminance polarity varies depending on the hues used to obtain training and testing data. The pattern of results is consistent with observations that luminance contrast is mediated by both L-M and S cone sub-cortical mechanisms.

Constrained brain volume in an efficient coding model explains the fraction of excitatory and inhibitory neurons in sensory cortices

The number of neurons in mammalian cortex varies by multiple orders of magnitude across different species. In contrast, the ratio of excitatory to inhibitory neurons (E:I ratio) varies in a much smaller range, from 3:1 to 9:1 and remains roughly constant for different sensory areas within a species. Despite this structure being important for understanding the function of neural circuits, the reason for this consistency is not yet understood. While recent models of vision based on the efficient coding hypothesis show that increasing the number of both excitatory and inhibitory cells improves stimulus representation, the two cannot increase simultaneously due to constraints on brain volume. In this work, we implement an efficient coding model of vision under a constraint on the volume (using number of neurons as a surrogate) while varying the E:I ratio. We show that the performance of the model is optimal at biologically observed E:I ratios under several metrics. We argue that this happens due to trade-offs between the computational accuracy and the representation capacity for natural stimuli. Further, we make experimentally testable predictions that 1) the optimal E:I ratio should be higher for species with a higher sparsity in the neural activity and 2) the character of inhibitory synaptic distributions and firing rates should change depending on E:I ratio. Our findings, which are supported by our new preliminary analyses of publicly available data, provide the first quantitative and testable hypothesis based on optimal coding models for the distribution of excitatory and inhibitory neural types in the mammalian sensory cortices.

Neurons in the brain come in two main types: excitatory and inhibitory. The interplay between them shapes neural computation. Despite brain sizes varying by several orders of magnitude across species, the ratio of excitatory and inhibitory sub-populations (E:I ratio) remains relatively constant, and we don’t know why. Simulations of theoretical models of the brain can help answer such questions, especially when experiments are prohibitive or impossible. Here we placed one such theoretical model of sensory coding (’sparse coding’ that minimizes the simultaneously active neurons) under a biophysical ‘volume’ constraint that fixes the total number of neurons available. We vary the E:I ratio in the model (which cannot be done in experiments), and reveal an optimal E:I ratio where the representation of sensory stimulus and energy consumption within the circuit are concurrently optimal. We also show that varying the population sparsity changes the optimal E:I ratio, spanning the relatively narrow ranges observed in biology. Crucially, this minimally parameterized theoretical model makes predictions about structure (recurrent connectivity) and activity (population sparsity) in neural circuits with different E:I ratios (i.e., different species), of which we verify the latter in a first-of-its-kind inter-species comparison using newly publicly available data.

Vision-dependent specification of cell types and function in the developing cortex

The role of postnatal experience in sculpting cortical circuitry, while long appreciated, is poorly understood at the level of cell types. We explore this in the mouse primary visual cortex (V1) using single-nucleus RNA sequencing, visual deprivation, genetics, and functional imaging. We find that vision selectively drives the specification of glutamatergic cell types in upper layers (L) (L2/3/4), while deeper-layer glutamatergic, GABAergic, and non-neuronal cell types are established prior to eye opening. L2/3 cell types form an experience-dependent spatial continuum defined by the graded expression of ∼200 genes, including regulators of cell adhesion and synapse formation. One of these genes, Igsf9b, a vision-dependent gene encoding an inhibitory synaptic cell adhesion molecule, is required for the normal development of binocular responses in L2/3. In summary, vision preferentially regulates the development of upper-layer glutamatergic cell types through the regulation of cell-type-specific gene expression programs.

Fast optical recording of neuronal activity by three-dimensional custom-access serial holography

Optical recording of neuronal activity in three-dimensional (3D) brain circuits at cellular and millisecond resolution in vivo is essential for probing information flow in the brain. While random-access multiphoton microscopy permits fast optical access to neuronal targets in three dimensions, the method is challenged by motion artifacts when recording from behaving animals. Therefore, we developed three-dimensional custom-access serial holography (3D-CASH). Built on a fast acousto-optic light modulator, 3D-CASH performs serial sampling at 40 kHz from neurons at freely selectable 3D locations. Motion artifacts are eliminated by targeting each neuron with a size-optimized pattern of excitation light covering the cell body and its anticipated displacement field. Spike rates inferred from GCaMP6f recordings in visual cortex of awake mice tracked the phase of a moving bar stimulus with higher spike correlation between intra compared to interlaminar neuron pairs. 3D-CASH offers access to the millisecond correlation structure of in vivo neuronal activity in 3D microcircuits.

Visual prototypes in the ventral stream are attuned to complexity and gaze behavior

Early theories of efficient coding suggested the visual system could compress the world by learning to represent features where information was concentrated, such as contours. This view was validated by the discovery that neurons in posterior visual cortex respond to edges and curvature. Still, it remains unclear what other information-rich features are encoded by neurons in more anterior cortical regions (e.g., inferotemporal cortex). Here, we use a generative deep neural network to synthesize images guided by neuronal responses from across the visuocortical hierarchy, using floating microelectrode arrays in areas V1, V4 and inferotemporal cortex of two macaque monkeys. We hypothesize these images ("prototypes") represent such predicted information-rich features. Prototypes vary across areas, show moderate complexity, and resemble salient visual attributes and semantic content of natural images, as indicated by the animals' gaze behavior. This suggests the code for object recognition represents compressed features of behavioral relevance, an underexplored aspect of efficient coding.

Dynamic divisive normalization circuits explain and predict change detection in monkey area MT

Sudden changes in visual scenes often indicate important events for behavior. For their quick and reliable detection, the brain must be capable to process these changes as independently as possible from its current activation state. In motion-selective area MT, neurons respond to instantaneous speed changes with pronounced transients, often far exceeding the expected response as derived from their speed tuning profile. We here show that this complex, non-linear behavior emerges from the combined temporal dynamics of excitation and divisive inhibition, and provide a comprehensive mathematical analysis. A central prediction derived from this investigation is that attention increases the steepness of the transient response irrespective of the activation state prior to a stimulus change, and irrespective of the sign of the change (i.e. irrespective of whether the stimulus is accelerating or decelerating). Extracellular recordings of attention-dependent representation of both speed increments and decrements confirmed this prediction and suggest that improved change detection derives from basic computations in a canonical cortical circuitry.

Tracking calcium dynamics from individual neurons in behaving animals

Measuring the activity of neuronal populations with calcium imaging can capture emergent functional properties of neuronal circuits with single cell resolution. However, the motion of freely behaving animals, together with the intermittent detectability of calcium sensors, can hinder automatic monitoring of neuronal activity and their subsequent functional characterization. We report the development and open-source implementation of a multi-step cellular tracking algorithm (Elastic Motion Correction and Concatenation or EMC2) that compensates for the intermittent disappearance of moving neurons by integrating local deformation information from detectable neurons. We demonstrate the accuracy and versatility of our algorithm using calcium imaging data from two-photon volumetric microscopy in visual cortex of awake mice, and from confocal microscopy in behaving Hydra, which experiences major body deformation during its contractions. We quantify the performance of our algorithm using ground truth manual tracking of neurons, along with synthetic time-lapse sequences, covering a wide range of particle motions and detectability parameters. As a demonstration of the utility of the algorithm, we monitor for several days calcium activity of the same neurons in layer 2/3 of mouse visual cortex in vivo, finding significant turnover within the active neurons across days, with only few neurons that remained active across days. Also, combining automatic tracking of single neuron activity with statistical clustering, we characterize and map neuronal ensembles in behaving Hydra, finding three major non-overlapping ensembles of neurons (CB, RP1 and RP2) whose activity correlates with contractions and elongations. Our results show that the EMC2 algorithm can be used as a robust and versatile platform for neuronal tracking in behaving animals.

Rats spontaneously perceive global motion direction of drifting plaids

Computing global motion direction of extended visual objects is a hallmark of primate high-level vision. Although neurons selective for global motion have also been found in mouse visual cortex, it remains unknown whether rodents can combine multiple motion signals into global, integrated percepts. To address this question, we trained two groups of rats to discriminate either gratings (G group) or plaids (i.e., superpositions of gratings with different orientations; P group) drifting horizontally along opposite directions. After the animals learned the task, we applied a visual priming paradigm, where presentation of the target stimulus was preceded by the brief presentation of either a grating or a plaid. The extent to which rat responses to the targets were biased by such prime stimuli provided a measure of the spontaneous, perceived similarity between primes and targets. We found that gratings and plaids, when used as primes, were equally effective at biasing the perception of plaid direction for the rats of the P group. Conversely, for the G group, only the gratings acted as effective prime stimuli, while the plaids failed to alter the perception of grating direction. To interpret these observations, we simulated a decision neuron reading out the representations of gratings and plaids, as conveyed by populations of either component or pattern cells (i.e., local or global motion detectors). We concluded that the findings for the P group are highly consistent with the existence of a population of pattern cells, playing a functional role similar to that demonstrated in primates. We also explored different scenarios that could explain the failure of the plaid stimuli to elicit a sizable priming magnitude for the G group. These simulations yielded testable predictions about the properties of motion representations in rodent visual cortex at the single-cell and circuitry level, thus paving the way to future neurophysiology experiments.

Bridging neuronal correlations and dimensionality reduction

Two commonly used approaches to study interactions among neurons are spike count correlation, which describes pairs of neurons, and dimensionality reduction, applied to a population of neurons. Although both approaches have been used to study trial-to-trial neuronal variability correlated among neurons, they are often used in isolation and have not been directly related. We first established concrete mathematical and empirical relationships between pairwise correlation and metrics of population-wide covariability based on dimensionality reduction. Applying these insights to macaque V4 population recordings, we found that the previously reported decrease in mean pairwise correlation associated with attention stemmed from three distinct changes in population-wide covariability. Overall, our work builds the intuition and formalism to bridge between pairwise correlation and population-wide covariability and presents a cautionary tale about the inferences one can make about population activity by using a single statistic, whether it be mean pairwise correlation or dimensionality.

The development of active binocular vision under normal and alternate rearing conditions

The development of binocular vision is an active learning process comprising the development of disparity tuned neurons in visual cortex and the establishment of precise vergence control of the eyes. We present a computational model for the learning and self-calibration of active binocular vision based on the Active Efficient Coding framework, an extension of classic efficient coding ideas to active perception. Under normal rearing conditions with naturalistic input, the model develops disparity tuned neurons and precise vergence control, allowing it to correctly interpret random dot stereograms. Under altered rearing conditions modeled after neurophysiological experiments, the model qualitatively reproduces key experimental findings on changes in binocularity and disparity tuning. Furthermore, the model makes testable predictions regarding how altered rearing conditions impede the learning of precise vergence control. Finally, the model predicts a surprising new effect that impaired vergence control affects the statistics of orientation tuning in visual cortical neurons.

A direct interareal feedback-to-feedforward circuit in primate visual cortex

The mammalian sensory neocortex consists of hierarchically organized areas reciprocally connected via feedforward (FF) and feedback (FB) circuits. Several theories of hierarchical computation ascribe the bulk of the computational work of the cortex to looped FF-FB circuits between pairs of cortical areas. However, whether such corticocortical loops exist remains unclear. In higher mammals, individual FF-projection neurons send afferents almost exclusively to a single higher-level area. However, it is unclear whether FB-projection neurons show similar area-specificity, and whether they influence FF-projection neurons directly or indirectly. Using viral-mediated monosynaptic circuit tracing in macaque primary visual cortex (V1), we show that V1 neurons sending FF projections to area V2 receive monosynaptic FB inputs from V2, but not other V1-projecting areas. We also find monosynaptic FB-to-FB neuron contacts as a second motif of FB connectivity. Our results support the existence of FF-FB loops in primate cortex, and suggest that FB can rapidly and selectively influence the activity of incoming FF signals.

Long-term stability of cortical ensembles

Neuronal ensembles, coactive groups of neurons found in spontaneous and evoked cortical activity, are causally related to memories and perception, but it is still unknown how stable or flexible they are over time. We used two-photon multiplane calcium imaging to track over weeks the activity of the same pyramidal neurons in layer 2/3 of the visual cortex from awake mice and recorded their spontaneous and visually evoked responses. Less than half of the neurons remained active across any two imaging sessions. These stable neurons formed ensembles that lasted weeks, but some ensembles were also transient and appeared only in one single session. Stable ensembles preserved most of their neurons for up to 46 days, our longest imaged period, and these 'core' cells had stronger functional connectivity. Our results demonstrate that neuronal ensembles can last for weeks and could, in principle, serve as a substrate for long-lasting representation of perceptual states or memories.

Reconciling functional differences in populations of neurons recorded with two-photon imaging and electrophysiology

Extracellular electrophysiology and two-photon calcium imaging are widely used methods for measuring physiological activity with single-cell resolution across large populations of cortical neurons. While each of these two modalities has distinct advantages and disadvantages, neither provides complete, unbiased information about the underlying neural population. Here, we compare evoked responses in visual cortex recorded in awake mice under highly standardized conditions using either imaging of genetically expressed GCaMP6f or electrophysiology with silicon probes. Across all stimulus conditions tested, we observe a larger fraction of responsive neurons in electrophysiology and higher stimulus selectivity in calcium imaging, which was partially reconciled by applying a spikes-to-calcium forward model to the electrophysiology data. However, the forward model could only reconcile differences in responsiveness when restricted to neurons with low contamination and an event rate above a minimum threshold. This work established how the biases of these two modalities impact functional metrics that are fundamental for characterizing sensory-evoked responses.

Modeling the role of gap junctions between excitatory neurons in the developing visual cortex

Recent experiments in the developing mammalian visual cortex have revealed that gap junctions couple excitatory cells and potentially influence the formation of chemical synapses. In particular, cells that were coupled by a gap junction during development tend to share an orientation preference and are preferentially coupled by a chemical synapse in the adult cortex, a property that is diminished when gap junctions are blocked. In this work, we construct a simplified model of the developing mouse visual cortex including spike-timing-dependent plasticity of both the feedforward synaptic inputs and recurrent cortical synapses. We use this model to show that synchrony among gap-junction-coupled cells underlies their preference to form strong recurrent synapses and develop similar orientation preference; this effect decreases with an increase in coupling density. Additionally, we demonstrate that gap-junction coupling works, together with the relative timing of synaptic development of the feedforward and recurrent synapses, to determine the resulting cortical map of orientation preference.

Gap junctions, or sites of direct electrical connections between neurons, have a significant presence in the cortex, both during development and in adulthood. Their primary function during either of these periods, however, is still poorly understood. In the adult cortex, gap junctions between local, inhibitory neurons have been shown to promote synchronous firing, a network characteristic thought to be important for learning, attention, and memory. During development, gap junctions between excitatory, pyramidal cells, have been conjectured to play a role in synaptic plasticity and the formation of cortical circuits. In the visual cortex, where neurons exhibit tuned responses to properties of visual input such as orientation and direction, recent experiments show that excitatory cells are coupled by gap junctions during the first postnatal week and are replaced by chemical synapses during the second week. In this work, we explore the possible contribution of gap-junction coupling during development to the formation of chemical synapses between the visual cortex from the thalamus and between cortical cells within the visual cortex. Specifically, using a mathematical model of the visual cortex during development, we identify the response properties of gap-junction-coupled cells and their influence on the formation of the cortical map of orientation preference.

Coordination of top-down influence on V1 responses by interneurons and brain rhythms

Top-down processing in neocortex underlies functions such as prediction, expectation, and attention. Visual systems have much feedback connection that carries information of behavioral context. Top-down signals along feedback pathways modulate the representation of visual information in early visual areas such as primary visual cortex (V1). Recent studies have shown further that beta rhythms are responsible for the transmission of behavioral-context information to lower visual areas. However, the mechanism underlying top-down influence and the role of brain rhythms in top-down processing are poorly understood. To address these issues, we focus on experimental studies on top-down influence in visual perceptual tasks. We develop a model of visual system, in which early visual areas are subjected to top-down influence from a recognition area. We show that task-relevant information in early visual areas is regulated by a push-pull effect, produced by somatostatin-expressing interneurons and top-down signal. We also show that task-context information is coordinated by the phase-phase coupling of beta rhythms, while the local, task-relevant stimulus features are enhanced by the phase-amplitude coupling of beta and gamma rhythms. Furthermore, the feedback from a higher visual area such as secondary visual area facilitates the gating of task-relevant information in V1. The results provide insights to understanding the roles of inhibitory interneurons and brain rhythms in top-down influence on information processing in early visual areas.

Creative Destruction: A Basic Computational Model of Cortical Layer Formation

One of the most characteristic properties of many vertebrate neural systems is the layered organization of different cell types. This cytoarchitecture exists in the cortex, the retina, the hippocampus, and many other parts of the central nervous system. The developmental mechanisms of neural layer formation have been subject to substantial experimental efforts. Here, we provide a general computational model for cortical layer formation in 3D physical space. We show that this multiscale, agent-based model, comprising two distinct stages of apoptosis, can account for the wide range of neuronal numbers encountered in different cortical areas and species. Our results demonstrate the phenotypic richness of a basic state diagram structure. Importantly, apoptosis allows for changing the thickness of one layer without automatically affecting other layers. Therefore, apoptosis increases the flexibility for evolutionary change in layer architecture. Notably, slightly changed gene regulatory dynamics recapitulate the characteristic properties observed in neurodevelopmental diseases. Overall, we propose a novel computational model using gene-type rules, exhibiting many characteristics of normal and pathological cortical development.

Cell segmentation-free inference of cell types from in situ transcriptomics data

Multiplexed fluorescence in situ hybridization techniques have enabled cell-type identification, linking transcriptional heterogeneity with spatial heterogeneity of cells. However, inaccurate cell segmentation reduces the efficacy of cell-type identification and tissue characterization. Here, we present a method called Spot-based Spatial cell-type Analysis by Multidimensional mRNA density estimation (SSAM), a robust cell segmentation-free computational framework for identifying cell-types and tissue domains in 2D and 3D. SSAM is applicable to a variety of in situ transcriptomics techniques and capable of integrating prior knowledge of cell types. We apply SSAM to three mouse brain tissue images: the somatosensory cortex imaged by osmFISH, the hypothalamic preoptic region by MERFISH, and the visual cortex by multiplexed smFISH. Here, we show that SSAM detects regions occupied by known cell types that were previously missed and discovers new cell types.

Diminished Cortical Excitation and Elevated Inhibition During Perceptual Impairments in a Mouse Model of Autism

Sensory impairments are a core feature of autism spectrum disorder (ASD). These impairments affect visual perception and have been hypothesized to arise from imbalances in cortical excitatory and inhibitory activity. There is conflicting evidence for this hypothesis from several recent studies of transgenic mouse models of ASD; crucially, none have measured activity from identified excitatory and inhibitory neurons during simultaneous impairments of sensory perception. Here, we directly recorded putative excitatory and inhibitory population spiking in primary visual cortex (V1) while simultaneously measuring visual perceptual behavior in CNTNAP2-/- knockout (KO) mice. We observed quantitative impairments in the speed, accuracy, and contrast sensitivity of visual perception in KO mice. During these perceptual impairments, stimuli evoked more firing of inhibitory neurons and less firing of excitatory neurons, with reduced neural sensitivity to contrast. In addition, pervasive 3-10 Hz oscillations in superficial cortical layers 2/3 (L2/3) of KO mice degraded predictions of behavioral performance from neural activity. Our findings show that perceptual deficits relevant to ASD may be associated with elevated cortical inhibitory activity along with diminished and aberrant excitatory population activity in L2/3, a major source of feedforward projections to higher cortical regions.

Modular Network between Postrhinal Visual Cortex, Amygdala, and Entorhinal Cortex

The postrhinal area (POR) is a known center for integrating spatial with nonspatial visual information and a possible hub for influencing landmark navigation by affective input from the amygdala. This may involve specific circuits within muscarinic acetylcholine receptor 2 (M2)-positive (M2+) or M2- modules of POR that associate inputs from the thalamus, cortex, and amygdala, and send outputs to the entorhinal cortex. Using anterograde and retrograde labeling with conventional and viral tracers in male and female mice, we found that all higher visual areas of the ventral cortical stream project to the amygdala, while such inputs are absent from primary visual cortex and dorsal stream areas. Unexpectedly for the presumed salt-and-pepper organization of mouse extrastriate cortex, tracing results show that inputs from the dorsal lateral geniculate nucleus and lateral posterior nucleus were spatially clustered in layer 1 (L1) and overlapped with M2+ patches of POR. In contrast, input from the amygdala to L1 of POR terminated in M2- interpatches. Importantly, the amygdalocortical input to M2- interpatches in L1 overlapped preferentially with spatially clustered apical dendrites of POR neurons projecting to amygdala and entorhinal area lateral, medial (ENTm). The results suggest that subnetworks in POR, used to build spatial maps for navigation, do not receive direct thalamocortical M2+ patch-targeting inputs. Instead, they involve local networks of M2- interpatches, which are influenced by affective information from the amygdala and project to ENTm, whose cells respond to visual landmark cues for navigation.SIGNIFICANCE STATEMENT A central purpose of visual object recognition is identifying the salience of objects and approaching or avoiding them. However, it is not currently known how the visual cortex integrates the multiple streams of information, including affective and navigational cues, which are required to accomplish this task. We find that in a higher visual area, the postrhinal cortex, the cortical sheet is divided into interdigitating modules receiving distinct inputs from visual and emotion-related sources. One of these modules is preferentially connected with the amygdala and provides outputs to entorhinal cortex, constituting a processing stream that may assign emotional salience to objects and landmarks for the guidance of goal-directed navigation.

Embedding optimization reveals long-lasting history dependence in neural spiking activity

Information processing can leave distinct footprints on the statistics of neural spiking. For example, efficient coding minimizes the statistical dependencies on the spiking history, while temporal integration of information may require the maintenance of information over different timescales. To investigate these footprints, we developed a novel approach to quantify history dependence within the spiking of a single neuron, using the mutual information between the entire past and current spiking. This measure captures how much past information is necessary to predict current spiking. In contrast, classical time-lagged measures of temporal dependence like the autocorrelation capture how long-potentially redundant-past information can still be read out. Strikingly, we find for model neurons that our method disentangles the strength and timescale of history dependence, whereas the two are mixed in classical approaches. When applying the method to experimental data, which are necessarily of limited size, a reliable estimation of mutual information is only possible for a coarse temporal binning of past spiking, a so-called past embedding. To still account for the vastly different spiking statistics and potentially long history dependence of living neurons, we developed an embedding-optimization approach that does not only vary the number and size, but also an exponential stretching of past bins. For extra-cellular spike recordings, we found that the strength and timescale of history dependence indeed can vary independently across experimental preparations. While hippocampus indicated strong and long history dependence, in visual cortex it was weak and short, while in vitro the history dependence was strong but short. This work enables an information-theoretic characterization of history dependence in recorded spike trains, which captures a footprint of information processing that is beyond time-lagged measures of temporal dependence. To facilitate the application of the method, we provide practical guidelines and a toolbox.

Neurovascular coupling and oxygenation are decreased in hippocampus compared to neocortex because of microvascular differences

The hippocampus is essential for spatial and episodic memory but is damaged early in Alzheimer's disease and is very sensitive to hypoxia. Understanding how it regulates its oxygen supply is therefore key for designing interventions to preserve its function. However, studies of neurovascular function in the hippocampus in vivo have been limited by its relative inaccessibility. Here we compared hippocampal and visual cortical neurovascular function in awake mice, using two photon imaging of individual neurons and vessels and measures of regional blood flow and haemoglobin oxygenation. We show that blood flow, blood oxygenation and neurovascular coupling were decreased in the hippocampus compared to neocortex, because of differences in both the vascular network and pericyte and endothelial cell function. Modelling oxygen diffusion indicates that these features of the hippocampal vasculature may restrict oxygen availability and could explain its sensitivity to damage during neurological conditions, including Alzheimer's disease, where the brain's energy supply is decreased.

Transcranial alternating current stimulation entrains alpha oscillations by preferential phase synchronization of fast-spiking cortical neurons to stimulation waveform

Computational modeling and human studies suggest that transcranial alternating current stimulation (tACS) modulates alpha oscillations by entrainment. Yet, a direct examination of how tACS interacts with neuronal spiking activity that gives rise to the alpha oscillation in the thalamo-cortical system has been lacking. Here, we demonstrate how tACS entrains endogenous alpha oscillations in head-fixed awake ferrets. We first show that endogenous alpha oscillations in the posterior parietal cortex drive the primary visual cortex and the higher-order visual thalamus. Spike-field coherence is largest for the alpha frequency band, and presumed fast-spiking inhibitory interneurons exhibit strongest coupling to this oscillation. We then apply alpha-tACS that results in a field strength comparable to what is commonly used in humans (<0.5 mV/mm). Both in these ferret experiments and in a computational model of the thalamo-cortical system, tACS entrains alpha oscillations by following the theoretically predicted Arnold tongue. Intriguingly, the fast-spiking inhibitory interneurons exhibit a stronger entrainment response to tACS in both the ferret experiments and the computational model, likely due to their stronger endogenous coupling to the alpha oscillation. Our findings demonstrate the in vivo mechanism of action for the modulation of the alpha oscillation by tACS.

Audiovisual integration in macaque face patch neurons

Primate social communication depends on the perceptual integration of visual and auditory cues, reflected in the multimodal mixing of sensory signals in certain cortical areas. The macaque cortical face patch network, identified through visual, face-selective responses measured with fMRI, is assumed to contribute to visual social interactions. However, whether face patch neurons are also influenced by acoustic information, such as the auditory component of a natural vocalization, remains unknown. Here, we recorded single-unit activity in the anterior fundus (AF) face patch, in the superior temporal sulcus, and anterior medial (AM) face patch, on the undersurface of the temporal lobe, in macaques presented with audiovisual, visual-only, and auditory-only renditions of natural movies of macaques vocalizing. The results revealed that 76% of neurons in face patch AF were significantly influenced by the auditory component of the movie, most often through enhancement of visual responses but sometimes in response to the auditory stimulus alone. By contrast, few neurons in face patch AM exhibited significant auditory responses or modulation. Control experiments in AF used an animated macaque avatar to demonstrate, first, that the structural elements of the face were often essential for audiovisual modulation and, second, that the temporal modulation of the acoustic stimulus was more important than its frequency spectrum. Together, these results identify a striking contrast between two face patches and specifically identify AF as playing a potential role in the integration of audiovisual cues during natural modes of social communication.

Two-photon GCaMP6f imaging of infrared neural stimulation evoked calcium signals in mouse cortical neurons in vivo

Infrared neural stimulation is a promising tool for stimulating the brain because it can be used to excite with high spatial precision without the need of delivering or inserting any exogenous agent into the tissue. Very few studies have explored its use in the brain, as most investigations have focused on sensory or motor nerve stimulation. Using intravital calcium imaging with the genetically encoded calcium indicator GCaMP6f, here we show that the application of infrared neural stimulation induces intracellular calcium signals in Layer 2/3 neurons in mouse cortex in vivo. The number of neurons exhibiting infrared-induced calcium response as well as the amplitude of those signals are shown to be both increasing with the energy density applied. By studying as well the spatial extent of the stimulation, we show that reproducibility of the stimulation is achieved mainly in the central part of the infrared beam path. Stimulating in vivo at such a degree of precision and without any exogenous chromophores enables multiple applications, from mapping the brain's connectome to applications in systems neuroscience and the development of new therapeutic tools for investigating the pathological brain.

A biomimetic neural encoder for spiking neural network

Spiking neural networks (SNNs) promise to bridge the gap between artificial neural networks (ANNs) and biological neural networks (BNNs) by exploiting biologically plausible neurons that offer faster inference, lower energy expenditure, and event-driven information processing capabilities. However, implementation of SNNs in future neuromorphic hardware requires hardware encoders analogous to the sensory neurons, which convert external/internal stimulus into spike trains based on specific neural algorithm along with inherent stochasticity. Unfortunately, conventional solid-state transducers are inadequate for this purpose necessitating the development of neural encoders to serve the growing need of neuromorphic computing. Here, we demonstrate a biomimetic device based on a dual gated MoS2 field effect transistor (FET) capable of encoding analog signals into stochastic spike trains following various neural encoding algorithms such as rate-based encoding, spike timing-based encoding, and spike count-based encoding. Two important aspects of neural encoding, namely, dynamic range and encoding precision are also captured in our demonstration. Furthermore, the encoding energy was found to be as frugal as ≈1-5 pJ/spike. Finally, we show fast (≈200 timesteps) encoding of the MNIST data set using our biomimetic device followed by more than 91% accurate inference using a trained SNN.

Single trial neuronal activity dynamics of attentional intensity in monkey visual area V4

Understanding how activity of visual neurons represents distinct components of attention and their dynamics that account for improved visual performance remains elusive because single-unit experiments have not isolated the intensive aspect of attention from attentional selectivity. We isolated attentional intensity and its single trial dynamics as determined by spatially non-selective attentional performance in an orientation discrimination task while recording from neurons in monkey visual area V4. We found that attentional intensity is a distinct cognitive signal that can be distinguished from spatial selectivity, reward expectations and motor actions. V4 spiking on single trials encodes a combination of sensory and cognitive signals on different time scales. Attentional intensity and the detection of behaviorally relevant sensory signals are well represented, but immediate reward expectation and behavioral choices are poorly represented in V4 spiking. These results provide a detailed representation of perceptual and cognitive signals in V4 that are crucial for attentional performance.

Analysis of spiking synchrony in visual cortex reveals distinct types of top-down modulation signals for spatial and object-based attention

The activity of a border ownership selective (BOS) neuron indicates where a foreground object is located relative to its (classical) receptive field (RF). A population of BOS neurons thus provides an important component of perceptual grouping, the organization of the visual scene into objects. In previous theoretical work, it has been suggested that this grouping mechanism is implemented by a population of dedicated grouping (“G”) cells that integrate the activity of the distributed feature cells representing an object and, by feedback, modulate the same cells, thus making them border ownership selective. The feedback modulation by G cells is thought to also provide the mechanism for object-based attention. A recent modeling study showed that modulatory common feedback, implemented by synapses with N-methyl-D-aspartate (NMDA)-type glutamate receptors, accounts for the experimentally observed synchrony in spike trains of BOS neurons and the shape of cross-correlations between them, including its dependence on the attentional state. However, that study was limited to pairs of BOS neurons with consistent border ownership preferences, defined as two neurons tuned to respond to the same visual object, in which attention decreases synchrony. But attention has also been shown to increase synchrony in neurons with inconsistent border ownership selectivity. Here we extend the computational model from the previous study to fully understand these effects of attention. We postulate the existence of a second type of G-cell that represents spatial attention by modulating the activity of all BOS cells in a spatially defined area. Simulations of this model show that a combination of spatial and object-based mechanisms fully accounts for the observed pattern of synchrony between BOS neurons. Our results suggest that modulatory feedback from G-cells may underlie both spatial and object-based attention.

Vision allows us to make sense out of a very complex signal, the patterns of light rays reaching our eyes. Two mechanisms are essential for this: perceptual organization which structures the input into meaningful visual objects, and attention which selects only the most important parts in the input. Prior work suggests that both of these mechanisms are implemented by neurons called grouping cells. These organize the object features into coherent entities (perceptual grouping) and access them as needed (selective attention). For technical reasons it is difficult to observe grouping cells but their effect can be seen in the influence they have on responses of other classes of cells. These responses have been measured experimentally and it was found that they depend in unexpected ways on where the subject was attending. Using a computational model, we here demonstrate that the responses can be understood in terms of the interaction between two kinds of selective attention, both of which are known to occur in primate perception. One is attention to a specific area in the environment, the other is to specific objects. A model including both of these attentional mechanisms generates neuronal responses in agreement with the observed patterns of neural activity.

Learning receptive field properties of complex cells in V1

There are two distinct classes of cells in the primary visual cortex (V1): simple cells and complex cells. One defining feature of complex cells is their spatial phase invariance; they respond strongly to oriented grating stimuli with a preferred orientation but with a wide range of spatial phases. A classical model of complete spatial phase invariance in complex cells is the energy model, in which the responses are the sum of the squared outputs of two linear spatially phase-shifted filters. However, recent experimental studies have shown that complex cells have a diverse range of spatial phase invariance and only a subset can be characterized by the energy model. While several models have been proposed to explain how complex cells could learn to be selective to orientation but invariant to spatial phase, most existing models overlook many biologically important details. We propose a biologically plausible model for complex cells that learns to pool inputs from simple cells based on the presentation of natural scene stimuli. The model is a three-layer network with rate-based neurons that describes the activities of LGN cells (layer 1), V1 simple cells (layer 2), and V1 complex cells (layer 3). The first two layers implement a recently proposed simple cell model that is biologically plausible and accounts for many experimental phenomena. The neural dynamics of the complex cells is modeled as the integration of simple cells inputs along with response normalization. Connections between LGN and simple cells are learned using Hebbian and anti-Hebbian plasticity. Connections between simple and complex cells are learned using a modified version of the Bienenstock, Cooper, and Munro (BCM) rule. Our results demonstrate that the learning rule can describe a diversity of complex cells, similar to those observed experimentally.

Many cortical functions originate from the learning ability of the brain. How the properties of cortical cells are learned is vital for understanding how the brain works. There are many models that explain how V1 simple cells can be learned. However, how V1 complex cells are learned still remains unclear. In this paper, we propose a model of learning in complex cells based on the Bienenstock, Cooper, and Munro (BCM) rule. We demonstrate that properties of receptive fields of complex cells can be learned using this biologically plausible learning rule. Quantitative comparisons between the model and experimental data are performed. Results show that model complex cells can account for the diversity of complex cells found in experimental studies. In summary, this study provides a plausible explanation for how complex cells can be learned using biologically plausible plasticity mechanisms. Our findings help us to better understand biological vision processing and provide us with insights into the general signal processing principles that the visual cortex employs to process visual information.

Fast and precise single-cell data analysis using a hierarchical autoencoder

A primary challenge in single-cell RNA sequencing (scRNA-seq) studies comes from the massive amount of data and the excess noise level. To address this challenge, we introduce an analysis framework, named single-cell Decomposition using Hierarchical Autoencoder (scDHA), that reliably extracts representative information of each cell. The scDHA pipeline consists of two core modules. The first module is a non-negative kernel autoencoder able to remove genes or components that have insignificant contributions to the part-based representation of the data. The second module is a stacked Bayesian autoencoder that projects the data onto a low-dimensional space (compressed). To diminish the tendency to overfit of neural networks, we repeatedly perturb the compressed space to learn a more generalized representation of the data. In an extensive analysis, we demonstrate that scDHA outperforms state-of-the-art techniques in many research sub-fields of scRNA-seq analysis, including cell segregation through unsupervised learning, visualization of transcriptome landscape, cell classification, and pseudo-time inference.

Laminar-specific cortico-cortical loops in mouse visual cortex

Many theories propose recurrent interactions across the cortical hierarchy, but it is unclear if cortical circuits are selectively wired to implement looped computations. Using subcellular channelrhodopsin-2-assisted circuit mapping in mouse visual cortex, we compared feedforward (FF) or feedback (FB) cortico-cortical (CC) synaptic input to cells projecting back to the input source (looped neurons) with cells projecting to a different cortical or subcortical area. FF and FB afferents showed similar cell-type selectivity, making stronger connections with looped neurons than with other projection types in layer (L)5 and L6, but not in L2/3, resulting in selective modulation of activity in looped neurons. In most cases, stronger connections in looped L5 neurons were located on their apical tufts, but not on their perisomatic dendrites. Our results reveal that CC connections are selectively wired to form monosynaptic excitatory loops and support a differential role of supragranular and infragranular neurons in hierarchical recurrent computations.

Cortical response selectivity derives from strength in numbers of synapses

Single neocortical neurons are driven by populations of excitatory inputs, which form the basis of neuronal selectivity to features of sensory input. Excitatory connections are thought to mature during development through activity-dependent Hebbian plasticity1, whereby similarity between presynaptic and postsynaptic activity selectively strengthens some synapses and weakens others2. Evidence in support of this process includes measurements of synaptic ultrastructure and in vitro and in vivo physiology and imaging studies3-8. These corroborating lines of evidence lead to the prediction that a small number of strong synaptic inputs drive neuronal selectivity, whereas weak synaptic inputs are less correlated with the somatic output and modulate activity overall6,7. Supporting evidence from cortical circuits, however, has been limited to measurements of neighbouring, connected cell pairs, raising the question of whether this prediction holds for a broad range of synapses converging onto cortical neurons. Here we measure the strengths of functionally characterized excitatory inputs contacting single pyramidal neurons in ferret primary visual cortex (V1) by combining in vivo two-photon synaptic imaging and post hoc electron microscopy. Using electron microscopy reconstruction of individual synapses as a metric of strength, we find no evidence that strong synapses have a predominant role in the selectivity of cortical neuron responses to visual stimuli. Instead, selectivity appears to arise from the total number of synapses activated by different stimuli. Moreover, spatial clustering of co-active inputs appears to be reserved for weaker synapses, enhancing the contribution of weak synapses to somatic responses. Our results challenge the role of Hebbian mechanisms in shaping neuronal selectivity in cortical circuits, and suggest that selectivity reflects the co-activation of large populations of presynaptic neurons with similar properties and a mixture of strengths.

Oxytocin Shapes Spontaneous Activity Patterns in the Developing Visual Cortex by Activating Somatostatin Interneurons

Spontaneous network activity shapes emerging neuronal circuits during early brain development prior to sensory perception. However, how neuromodulation influences this activity is not fully understood. Here, we report that the neuromodulator oxytocin differentially shapes spontaneous activity patterns across sensory cortices. In vivo, oxytocin strongly decreased the frequency and pairwise correlations of spontaneous activity events in the primary visual cortex (V1), but it did not affect the frequency of spontaneous network events in the somatosensory cortex (S1). Patch-clamp recordings in slices and RNAscope showed that oxytocin affects S1 excitatory and inhibitory neurons similarly, whereas in V1, oxytocin targets only inhibitory neurons. Somatostatin-positive (SST+) interneurons expressed the oxytocin receptor and were activated by oxytocin in V1. Accordingly, pharmacogenetic silencing of V1 SST+ interneurons fully blocked oxytocin's effect on inhibition in vitro as well its effect on spontaneous activity patterns in vivo. Thus, oxytocin decreases the excitatory/inhibitory (E/I) ratio by recruiting SST+ interneurons and modulates specific features of V1 spontaneous activity patterns that are crucial for the wiring and refining of developing sensory circuits.

Dynamic Suppression of Average Facial Structure Shapes Neural Tuning in Three Macaque Face Patches

The visual perception of identity in humans and other primates is thought to draw upon cortical areas specialized for the analysis of facial structure. A prominent theory of face recognition holds that the brain computes and stores average facial structure, which it then uses to efficiently determine individual identity, though the neural mechanisms underlying this process are controversial. Here, we demonstrate that the dynamic suppression of average facial structure plays a prominent role in the responses of neurons in three fMRI-defined face patches of the macaque. Using photorealistic face stimuli that systematically varied in identity level according to a psychophysically based face space, we found that single units in the AF, AM, and ML face patches exhibited robust tuning around average facial structure. This tuning emerged after the initial excitatory response to the face and was expressed as the selective suppression of sustained responses to low-identity faces. The coincidence of this suppression with increased spike timing synchrony across the population suggests a mechanism of active inhibition underlying this effect. Control experiments confirmed that the diminished responses to low-identity faces were not due to short-term adaptation processes. We propose that the brain's neural suppression of average facial structure facilitates recognition by promoting the extraction of distinctive facial characteristics and suppressing redundant or irrelevant responses across the population.

Cortical State Fluctuations during Sensory Decision Making

In many behavioral tasks, cortex enters a desynchronized state where low-frequency fluctuations in population activity are suppressed. The precise behavioral correlates of desynchronization and its global organization are unclear. One hypothesis holds that desynchronization enhances stimulus coding in the relevant sensory cortex. Another hypothesis holds that desynchronization reflects global arousal, such as task engagement. Here, we trained mice on tasks where task engagement could be distinguished from sensory accuracy. Using widefield calcium imaging, we found that performance-related desynchronization was global and correlated better with engagement than with accuracy. Consistent with this link between desynchronization and engagement, rewards had a long-lasting desynchronizing effect. To determine whether engagement-related state changes depended on the relevant sensory modality, we trained mice on visual and auditory tasks and found that in both cases desynchronization was global, including regions such as somatomotor cortex. We conclude that variations in low-frequency fluctuations are predominately global and related to task engagement.

Mouse Higher Visual Areas Provide Both Distributed and Specialized Contributions to Visually Guided Behaviors

Cortical parallel processing streams segregate many diverse features of a sensory scene. However, some features are distributed across streams, begging the question of whether and how such distributed representations contribute to perception. We determined the necessity of the primary visual cortex (V1) and three key higher visual areas (lateromedial [LM], anterolateral [AL], and posteromedial [PM]) for perception of orientation and contrast, two features that are robustly encoded across all four areas. Suppressing V1, LM, or AL decreased sensitivity for both orientation discrimination and contrast detection, consistent with a role for these areas in sensory perception. In comparison, suppressing PM selectively increased false alarm (FA) rates during contrast detection, without any effect on orientation discrimination. This effect was not retinotopically specific, suggesting that suppression of PM altered sensory integration or the decision-making process rather than processing of local visual features. Thus, we find that distributed representations in the visual system can nonetheless support specialized perceptual roles for higher visual cortical areas.

Low rank mechanisms underlying flexible visual representations

Neuronal population responses to sensory stimuli are remarkably flexible. The responses of neurons in visual cortex have heterogeneous dependence on stimulus properties (e.g., contrast), processes that affect all stages of visual processing (e.g., adaptation), and cognitive processes (e.g., attention or task switching). Understanding whether these processes affect similar neuronal populations and whether they have similar effects on entire populations can provide insight into whether they utilize analogous mechanisms. In particular, it has recently been demonstrated that attention has low rank effects on the covariability of populations of visual neurons, which impacts perception and strongly constrains mechanistic models. We hypothesized that measuring changes in population covariability associated with other sensory and cognitive processes could clarify whether they utilize similar mechanisms or computations. Our experimental design included measurements in multiple visual areas using four distinct sensory and cognitive processes. We found that contrast, adaptation, attention, and task switching affect the variability of responses of populations of neurons in primate visual cortex in a similarly low rank way. These results suggest that a given circuit may use similar mechanisms to perform many forms of modulation and likely reflects a general principle that applies to a wide range of brain areas and sensory, cognitive, and motor processes.

Common schizophrenia risk variants are enriched in open chromatin regions of human glutamatergic neurons

The chromatin landscape of human brain cells encompasses key information to understanding brain function. Here we use ATAC-seq to profile the chromatin structure in four distinct populations of cells (glutamatergic neurons, GABAergic neurons, oligodendrocytes, and microglia/astrocytes) from three different brain regions (anterior cingulate cortex, dorsolateral prefrontal cortex, and primary visual cortex) in human postmortem brain samples. We find that chromatin accessibility varies greatly by cell type and, more moderately, by brain region, with glutamatergic neurons showing the largest regional variability. Transcription factor footprinting implicates cell-specific transcriptional regulators and infers cell-specific regulation of protein-coding genes, long intergenic noncoding RNAs and microRNAs. In vivo transgenic mouse experiments validate the cell type specificity of several of these human-derived regulatory sequences. We find that open chromatin regions in glutamatergic neurons are enriched for neuropsychiatric risk variants, particularly those associated with schizophrenia. Integration of cell-specific chromatin data with a bulk tissue study of schizophrenia brains increases statistical power and confirms that glutamatergic neurons are most affected. These findings illustrate the utility of studying the cell-type-specific epigenome in complex tissues like the human brain, and the potential of such approaches to better understand the genetic basis of human brain function.

Model-based decoupling of evoked and spontaneous neural activity in calcium imaging data

The pattern of neural activity evoked by a stimulus can be substantially affected by ongoing spontaneous activity. Separating these two types of activity is particularly important for calcium imaging data given the slow temporal dynamics of calcium indicators. Here we present a statistical model that decouples stimulus-driven activity from low dimensional spontaneous activity in this case. The model identifies hidden factors giving rise to spontaneous activity while jointly estimating stimulus tuning properties that account for the confounding effects that these factors introduce. By applying our model to data from zebrafish optic tectum and mouse visual cortex, we obtain quantitative measurements of the extent that neurons in each case are driven by evoked activity, spontaneous activity, and their interaction. By not averaging away potentially important information encoded in spontaneous activity, this broadly applicable model brings new insight into population-level neural activity within single trials.

Refractory density model of cortical direction selectivity: Lagged-nonlagged, transient-sustained, and On-Off thalamic neuron-based mechanisms and intracortical amplification

A biophysically detailed description of the mechanisms of the primary vision is still being developed. We have incorporated a simplified, filter-based description of retino-thalamic visual signal processing into the detailed, conductance-based refractory density description of the neuronal population activity of the primary visual cortex. We compared four mechanisms of the direction selectivity (DS), three of them being based on asymmetrical projections of different types of thalamic neurons to the cortex, distinguishing between (i) lagged and nonlagged, (ii) transient and sustained, and (iii) On and Off neurons. The fourth mechanism implies a lack of subcortical bias and is an epiphenomenon of intracortical interactions between orientation columns. The simulations of the cortical response to moving gratings have verified that first three mechanisms provide DS to an extent compared with experimental data and that the biophysical model realistically reproduces characteristics of the visual cortex activity, such as membrane potential, firing rate, and synaptic conductances. The proposed model reveals the difference between the mechanisms of both the intact and the silenced cortex, favoring the second mechanism. In the fourth case, DS is weaker but significant; it completely vanishes in the silenced cortex.DS in the On-Off mechanism derives from the nonlinear interactions within the orientation map. Results of simulations can help to identify a prevailing mechanism of DS in V1. This is a step towards a comprehensive biophysical modeling of the primary visual system in the frameworks of the population rate coding concept.

A major mechanism that underlies tuning of cortical neurons to the direction of a moving stimulus is still debated. Considering the visual cortex structured with orientation-selective columns, we have realized and compared in our biophysically detailed mathematical model four hypothetical mechanisms of the direction selectivity (DS) known from experiments. The present model accomplishes our previous model that was tuned to experimental data on excitability in slices and reproduces orientation tuning effects in vivo. In simulations, we have found that the convergence of inputs from so-called transient and sustained (or lagged and nonlagged) thalamic neurons in the cortex provides an initial bias for DS, whereas cortical interactions amplify the tuning. In the absence of any bias, DS emerges as an epiphenomenon of the orientation map. In the case of a biased convergence of On- and Off- thalamic inputs, DS emerges with the help of the intracortical interactions on the orientation map, also. Thus, we have proposed a comprehensive description of the primary vision and revealed characteristic features of different mechanisms of DS in the visual cortex with columnar structure.

A recurrent circuit implements normalization, simulating the dynamics of V1 activity

The normalization model has been applied to explain neural activity in diverse neural systems including primary visual cortex (V1). The model's defining characteristic is that the response of each neuron is divided by a factor that includes a weighted sum of activity of a pool of neurons. Despite the success of the normalization model, there are three unresolved issues. 1) Experimental evidence supports the hypothesis that normalization in V1 operates via recurrent amplification, i.e., amplifying weak inputs more than strong inputs. It is unknown how normalization arises from recurrent amplification. 2) Experiments have demonstrated that normalization is weighted such that each weight specifies how one neuron contributes to another's normalization pool. It is unknown how weighted normalization arises from a recurrent circuit. 3) Neural activity in V1 exhibits complex dynamics, including gamma oscillations, linked to normalization. It is unknown how these dynamics emerge from normalization. Here, a family of recurrent circuit models is reported, each of which comprises coupled neural integrators to implement normalization via recurrent amplification with arbitrary normalization weights, some of which can recapitulate key experimental observations of the dynamics of neural activity in V1.

Direction-selective motion discrimination by traveling waves in visual cortex

The majority of neurons in primary visual cortex respond selectively to bars of light that have a specific orientation and move in a specific direction. The spatial and temporal responses of such neurons are non-separable. How neurons accomplish that computational feat without resort to explicit time delays is unknown. We propose a novel neural mechanism whereby visual cortex computes non-separable responses by generating endogenous traveling waves of neural activity that resonate with the space-time signature of the visual stimulus. The spatiotemporal characteristics of the response are defined by the local topology of excitatory and inhibitory lateral connections in the cortex. We simulated the interaction between endogenous traveling waves and the visual stimulus using spatially distributed populations of excitatory and inhibitory neurons with Wilson-Cowan dynamics and inhibitory-surround coupling. Our model reliably detected visual gratings that moved with a given speed and direction provided that we incorporated neural competition to suppress false motion signals in the opposite direction. The findings suggest that endogenous traveling waves in visual cortex can impart direction-selectivity on neural responses without resort to explicit time delays. They also suggest a functional role for motion opponency in eliminating false motion signals.

It is well established that the so-called ‘simple cells’ of the primary visual cortex respond preferentially to oriented bars of light that move across the visual field with a particular speed and direction. The spatiotemporal responses of such neurons are said to be non-separable because they cannot be constructed from independent spatial and temporal neural mechanisms. Contemporary theories of how neurons compute non-separable responses typically rely on finely tuned transmission delays between signals from disparate regions of the visual field. However the existence of such delays is controversial. We propose an alternative neural mechanism for computing non-separable responses that does not require transmission delays. It instead relies on the predisposition of the cortical tissue to spontaneously generate spatiotemporal waves of neural activity that travel with a particular speed and direction. We propose that the endogenous wave activity resonates with the visual stimulus to elicit direction-selective neural responses to visual motion. We demonstrate the principle in computer models and show that competition between opposing neurons robustly enhances their ability to discriminate between visual gratings that move in opposite directions.

Learning prediction error neurons in a canonical interneuron circuit

Sensory systems constantly compare external sensory information with internally generated predictions. While neural hallmarks of prediction errors have been found throughout the brain, the circuit-level mechanisms that underlie their computation are still largely unknown. Here, we show that a well-orchestrated interplay of three interneuron types shapes the development and refinement of negative prediction-error neurons in a computational model of mouse primary visual cortex. By balancing excitation and inhibition in multiple pathways, experience-dependent inhibitory plasticity can generate different variants of prediction-error circuits, which can be distinguished by simulated optogenetic experiments. The experience-dependence of the model circuit is consistent with that of negative prediction-error circuits in layer 2/3 of mouse primary visual cortex. Our model makes a range of testable predictions that may shed light on the circuitry underlying the neural computation of prediction errors.

Aversive perception in a threat context: Separate and independent neural activation

Unpleasant, compared to neutral, scenes reliably prompt enhanced functional brain activity in the amygdala and inferotemporal cortex. Considering data from psychophysiological studies in which defensive reactivity is further enhanced when viewing unpleasant scenes under threat of shock (compared to safety), the current study investigates functional activation in the amygdala-inferotemporal circuit when unpleasant (or neutral) scenes are viewed under threat of shock or safety. In this paradigm, a cue signaling threat or safety was presented in conjunction with either an unpleasant or neutral picture. Replicating previous studies, unpleasant, compared to neutral, scenes reliably enhanced activation in the amygdala and inferotemporal cortex. Functional activity in these regions, however, did not differ whether scenes were presented in a context threatening shock exposure, compared to safety, which instead activated regions of the anterior insula and cingulate cortex. Taken together, the data support a view in which neural regions activated in different defensive situations act independently.

Feedback generates a second receptive field in neurons of the visual cortex

Animals sense the environment through pathways that link sensory organs to the brain. In the visual system, these feedforward pathways define the classical feedforward receptive field (ffRF), the area in space in which visual stimuli excite a neuron1. The visual system also uses visual context-the visual scene surrounding a stimulus-to predict the content of the stimulus2, and accordingly, neurons have been identified that are excited by stimuli outside their ffRF3-8. However, the mechanisms that generate excitation to stimuli outside the ffRF are unclear. Here we show that feedback projections onto excitatory neurons in the mouse primary visual cortex generate a second receptive field that is driven by stimuli outside the ffRF. The stimulation of this feedback receptive field (fbRF) elicits responses that are slower and are delayed in comparison with those resulting from the stimulation of the ffRF. These responses are preferentially reduced by anaesthesia and by silencing higher visual areas. Feedback inputs from higher visual areas have scattered receptive fields relative to their putative targets in the primary visual cortex, which enables the generation of the fbRF. Neurons with fbRFs are located in cortical layers that receive strong feedback projections and are absent in the main input layer, which is consistent with a laminar processing hierarchy. The observation that large, uniform stimuli-which cover both the fbRF and the ffRF-suppress these responses indicates that the fbRF and the ffRF are mutually antagonistic. Whereas somatostatin-expressing inhibitory neurons are driven by these large stimuli, inhibitory neurons that express parvalbumin and vasoactive intestinal peptide have mutually antagonistic fbRF and ffRF, similar to excitatory neurons. Feedback projections may therefore enable neurons to use context to estimate information that is missing from the ffRF and to report differences in stimulus features across visual space, regardless of whether excitation occurs inside or outside the ffRF. By complementing the ffRF, the fbRF that we identify here could contribute to predictive processing.

Anatomy and Physiology of Macaque Visual Cortical Areas V1, V2, and V5/MT: Bases for Biologically Realistic Models

The cerebral cortex of primates encompasses multiple anatomically and physiologically distinct areas processing visual information. Areas V1, V2, and V5/MT are conserved across mammals and are central for visual behavior. To facilitate the generation of biologically accurate computational models of primate early visual processing, here we provide an overview of over 350 published studies of these three areas in the genus Macaca, whose visual system provides the closest model for human vision. The literature reports 14 anatomical connection types from the lateral geniculate nucleus of the thalamus to V1 having distinct layers of origin or termination, and 194 connection types between V1, V2, and V5, forming multiple parallel and interacting visual processing streams. Moreover, within V1, there are reports of 286 and 120 types of intrinsic excitatory and inhibitory connections, respectively. Physiologically, tuning of neuronal responses to 11 types of visual stimulus parameters has been consistently reported. Overall, the optimal spatial frequency (SF) of constituent neurons decreases with cortical hierarchy. Moreover, V5 neurons are distinct from neurons in other areas for their higher direction selectivity, higher contrast sensitivity, higher temporal frequency tuning, and wider SF bandwidth. We also discuss currently unavailable data that could be useful for biologically accurate models.

Integrative analysis of in vivo recording with single-cell RNA-seq data reveals molecular properties of light-sensitive neurons in mouse V1

Vision formation is classically based on projections from retinal ganglion cells (RGC) to the lateral geniculate nucleus (LGN) and the primary visual cortex (V1). Neurons in the mouse V1 are tuned to light stimuli. Although the cellular information of the retina and the LGN has been widely studied, the transcriptome profiles of single light-stimulated neuron in V1 remain unknown. In our study, in vivo calcium imaging and whole-cell electrophysiological patch-clamp recording were utilized to identify 53 individual cells from layer 2/3 of V1 as light-sensitive (LS) or non-light-sensitive (NS) by single-cell light-evoked calcium evaluation and action potential spiking. The contents of each cell after functional tests were aspirated in vivo through a patch-clamp pipette for mRNA sequencing. Moreover, the three-dimensional (3-D) morphological characterizations of the neurons were reconstructed in a live mouse after the whole-cell recordings. Our sequencing results indicated that V1 neurons with a high expression of genes related to transmission regulation, such as Rtn4r and Rgs7, and genes involved in membrane transport, such as Na⁺/K⁺ ATPase and NMDA-type glutamatergic receptors, preferentially responded to light stimulation. Furthermore, an antagonist that blocks Rtn4r signals could inactivate the neuronal responses to light stimulation in live mice. In conclusion, our findings of the vivo-seq analysis indicate the key role of the strength of synaptic transmission possesses neurons in V1 of light sensory.