Dynamically modulated spike correlation in monkey inferior temporal cortex depending on the feature configuration within a whole object

Dependence of Contextual Modulation in Macaque V1 on Interlaminar Signal Flow

In visual cortex, neural correlates of subjective perception can be generated by modulation of activity from beyond the classical receptive field (CRF). In macaque V1, activity generated by nonclassical receptive field (nCRF) stimulation involves different intracortical circuitry than activity generated by CRF stimulation, suggesting that interactions between neurons across V1 layers differ under CRF and nCRF stimulus conditions. Using Neuropixels probes, we measured border ownership modulation within large, local populations of V1 neurons. We found that neurons in single columns preferred the same side of objects located outside of the CRF. In addition, we found that cross-correlations between pairs of neurons situated across feedback/horizontal and input layers differed between CRF and nCRF stimulation. Furthermore, independent of the comparison with CRF stimulation, we observed that the magnitude of border ownership modulation increased with the proportion of information flow from feedback/horizontal layers to input layers. These results demonstrate that the flow of signals between layers covaries with the degree to which neurons integrate information from beyond the CRF.

Multiscale chemogenetic dissection of fronto-temporal top-down regulation for object memory in primates

Visual object memory is a fundamental element of various cognitive abilities, and the underlying neural mechanisms have been extensively examined especially in the anterior temporal cortex of primates. However, both macroscopic large-scale functional network in which this region is embedded and microscopic neuron-level dynamics of top-down regulation it receives for object memory remains elusive. Here, we identified the orbitofrontal node as a critical partner of the anterior temporal node for object memory by combining whole-brain functional imaging during rest and a short-term object memory task in male macaques. Focal chemogenetic silencing of the identified orbitofrontal node downregulated both the local orbitofrontal and remote anterior temporal nodes during the task, in association with deteriorated mnemonic, but not perceptual, performance. Furthermore, imaging-guided neuronal recordings in the same monkeys during the same task causally revealed that orbitofrontal top-down modulation enhanced stimulus-selective mnemonic signal in individual anterior temporal neurons while leaving bottom-up perceptual signal unchanged. Furthermore, similar activity difference was also observed between correct and mnemonic error trials before silencing, suggesting its behavioral relevance. These multifaceted but convergent results provide a multiscale causal understanding of dynamic top-down regulation of the anterior temporal cortex along the ventral fronto-temporal network underpinning short-term object memory in primates.

Protocol to measure monosynaptic connections between different cortical regions in mice using cell-pair cross correlogram of spike events

Neuromodulation can facilitate interactions between neurons to rescue impaired brain function after stroke. Here, we present a protocol for measuring putative monosynaptic connections between different cortical regions. We detail procedures for tetrode fabrication, implantation surgery, stroke induction in mice, multi-site in vivo electrophysiological recording, units clustering, principal neuron/interneuron classification, and functional connection analysis. This protocol allows us to understand the mechanisms of stroke recovery. For complete details on the use and execution of this protocol, please refer to Wang et al.1.

Spike synchrony as a measure of Gestalt structure

The function of spike synchrony is debatable: some researchers view it as a mechanism for binding perceptual features, others - as a byproduct of brain activity. We argue for an alternative computational role: synchrony can estimate the prior probability of incoming stimuli. In V1, this can be achieved by comparing input with previously acquired visual experience, which is encoded in plastic horizontal intracortical connections. V1 connectivity structure can encode the acquired visual experience in the form of its aggregate statistics. Since the aggregate statistics of natural images tend to follow the Gestalt principles, we can assume that V1 is more often exposed to Gestalt-like stimuli, and this is manifested in its connectivity structure. At the same time, the connectivity structure has an impact on spike synchrony in V1. We used a spiking model with V1-like connectivity to demonstrate that spike synchrony reflects the Gestalt structure of the stimulus. We conducted simulation experiments with three Gestalt laws: proximity, similarity, and continuity, and found substantial differences in firing synchrony for stimuli with varying degrees of Gestalt-likeness. This allows us to conclude that spike synchrony indeed reflects the Gestalt structure of the stimulus, which can be interpreted as a mechanism for prior probability estimation.

Combinatorial Language parent-report Scores Differ Significantly Between Typically Developing Children and Those with Autism Spectrum Disorders

Prefrontal synthesis (PFS) is a component of constructive imagination. It is defined as the process of mentally juxtaposing objects into novel combinations. For example, to comprehend the instruction "put the cat under the dog and above the monkey," it is necessary to use PFS in order to correctly determine the spatial arrangement of the cat, dog, and monkey with relation to one another. The acquisition of PFS hinges on the use of combinatorial language during early childhood development. Accordingly, children with developmental delays exhibit a deficit in PFS, and frequent assessments are recommended for such individuals. In 2018, we developed the Mental Synthesis Evaluation Checklist (MSEC), a parent-reported evaluation designed to assess PFS and combinatorial language comprehension. In this manuscript we use MSEC to identify differences in combinatorial language acquisition between ASD (N = 29,138) and neurotypical (N = 111) children. Results emphasize the utility of the MSEC in distinguishing language deficits in ASD from typical development as early as 2 years of age (p < 0.0001).

Local features drive identity responses in macaque anterior face patches

Humans and other primates recognize one another in part based on unique structural details of the face, including both local features and their spatial configuration within the head and body. Visual analysis of the face is supported by specialized regions of the primate cerebral cortex, which in macaques are commonly known as face patches. Here we ask whether the responses of neurons in anterior face patches, thought to encode face identity, are more strongly driven by local or holistic facial structure. We created stimuli consisting of recombinant photorealistic images of macaques, where we interchanged the eyes, mouth, head, and body between individuals. Unexpectedly, neurons in the anterior medial (AM) and anterior fundus (AF) face patches were predominantly tuned to local facial features, with minimal neural selectivity for feature combinations. These findings indicate that the high-level structural encoding of face identity rests upon populations of neurons specialized for local features.

Anterior face patches in the macaque have been assumed to represent face identity in a holistic manner. Here the authors show that the neural encoding of face identity in the anterior medial and anterior fundus face patches are instead driven principally by local features.

Language evolution is not limited to speech acquisition: a large study of language development in children with language deficits highlights the importance of the voluntary imagination component of language

Did the boy bite the cat or was it the other way around? When processing a sentence with several objects, one has to establish ‘who did what to whom’. When a sentence cannot be interpreted by recalling an image from memory, we rely on the special type of voluntary constructive imagination called Prefrontal synthesis (PFS). PFS is defined as the ability to juxtapose mental visuospatial objects at will. We hypothesised that PFS has fundamental importance for language acquisition. To test this hypothesis, we designed a PFS-targeting intervention and administered it to 6,454 children with language deficiencies (age 2 to 12 years). The results from the three-year-long study demonstrated that children who engaged with the PFS intervention showed 2.2-fold improvement in combinatorial language comprehension compared to children with similar initial evaluations. These findings suggest that language can be improved by training the PFS and exposes the importance of the visuospatial component of language. This manuscript reflects on the experimental findings from the point of view of human language evolution. When used as a proxy for evolutionary language acquisition, the study results suggest a dichotomy of language evolution, with its speech component and its visuospatial component developing in parallel. The study highlights the radical idea that evolutionary acquisition of language was driven primarily by improvements of voluntary imagination rather than by improvements in the speech apparatus.

Operating principles of the cerebral cortex as a six-layered network in primates: beyond the classic canonical circuit model

The cerebral cortex performs its computations with many six-layered fundamental units, collectively spreading along the cortical sheet. What is the local network structure and the operating dynamics of such a fundamental unit? Previous investigations of primary sensory areas revealed a classic "canonical" circuit model, leading to an expectation of similar circuit organization and dynamics throughout the cortex. This review clarifies the different circuit dynamics at play in the higher association cortex of primates that implements computation for high-level cognition such as memory and attention. Instead of feedforward processing of response selectivity through Layers 4 to 2/3 that the classic canonical circuit stipulates, memory recall in primates occurs in Layer 5/6 with local backward projection to Layer 2/3, after which the retrieved information is sent back from Layer 6 to lower-level cortical areas for further retrieval of nested associations of target attributes. In this review, a novel "dynamic multimode module (D3M)" in the primate association cortex is proposed, as a new "canonical" circuit model performing this operation.

Concept neurons in the human medial temporal lobe flexibly represent abstract relations between concepts

Concept neurons in the medial temporal lobe respond to semantic features of presented stimuli. Analyzing 61 concept neurons recorded from twelve patients who underwent surgery to treat epilepsy, we show that firing patterns of concept neurons encode relations between concepts during a picture comparison task. Thirty-three of these responded to non-preferred stimuli with a delayed but well-defined onset whenever the task required a comparison to a response-eliciting concept, but not otherwise. Supporting recent theories of working memory, concept neurons increased firing whenever attention was directed towards this concept and could be reactivated after complete activity silence. Population cross-correlations of pairs of concept neurons exhibited order-dependent asymmetric peaks specifically when their response-eliciting concepts were to be compared. Our data are consistent with synaptic mechanisms that support reinstatement of concepts and their relations after activity silence, flexibly induced through task-specific sequential activation. This way arbitrary contents of experience could become interconnected in both working and long-term memory.

It is unclear how distinct concepts are processed in the brain. Here, the authors recorded from concept cells in human subjects with epilepsy and found that a subset of concept cells responded to non-preferred concepts if those non-preferred concepts required comparison to a preferred concept.

Recurrent dynamics in the cerebral cortex: Integration of sensory evidence with stored knowledge

Current concepts of sensory processing in the cerebral cortex emphasize serial extraction and recombination of features in hierarchically structured feed-forward networks in order to capture the relations among the components of perceptual objects. These concepts are implemented in convolutional deep learning networks and have been validated by the astounding similarities between the functional properties of artificial systems and their natural counterparts. However, cortical architectures also display an abundance of recurrent coupling within and between the layers of the processing hierarchy. This massive recurrence gives rise to highly complex dynamics whose putative function is poorly understood. Here a concept is proposed that assigns specific functions to the dynamics of cortical networks and combines, in a unifying approach, the respective advantages of recurrent and feed-forward processing. It is proposed that the priors about regularities of the world are stored in the weight distributions of feed-forward and recurrent connections and that the high-dimensional, dynamic space provided by recurrent interactions is exploited for computations. These comprise the ultrafast matching of sensory evidence with the priors covertly represented in the correlation structure of spontaneous activity and the context-dependent grouping of feature constellations characterizing natural objects. The concept posits that information is encoded not only in the discharge frequency of neurons but also in the precise timing relations among the discharges. Results of experiments designed to test the predictions derived from this concept support the hypothesis that cerebral cortex exploits the high-dimensional recurrent dynamics for computations serving predictive coding.

Cellular and Network Mechanisms for Temporal Signal Propagation in a Cortical Network Model

The mechanisms underlying an effective propagation of high intensity information over a background of irregular firing and response latency in cognitive processes remain unclear. Here we propose a SSCCPI circuit to address this issue. We hypothesize that when a high-intensity thalamic input triggers synchronous spike events (SSEs), dense spikes are scattered to many receiving neurons within a cortical column in layer IV, many sparse spike trains are propagated in parallel along minicolumns at a substantially high speed and finally integrated into an output spike train toward or in layer Va. We derive the sufficient conditions for an effective (fast, reliable, and precise) SSCCPI circuit: (i) SSEs are asynchronous (near synchronous); (ii) cortical columns prevent both repeatedly triggering SSEs and incorrectly synaptic connections between adjacent columns; and (iii) the propagator in interneurons is temporally complete fidelity and reliable. We encode the membrane potential responses to stimuli using the non-linear autoregressive integrated process derived by applying Newton's second law to stochastic resilience systems. We introduce a multithreshold decoder to correct encoding errors. Evidence supporting an effective SSCCPI circuit includes that for the condition, (i) time delay enhances SSEs, suggesting that response latency induces SSEs in high-intensity stimuli; irregular firing causes asynchronous SSEs; asynchronous SSEs relate to healthy neurons; and rigorous SSEs relate to brain disorders. For the condition (ii) neurons within a given minicolumn are stereotypically interconnected in the vertical dimension, which prevents repeated triggering SSEs and ensures signal parallel propagation; columnar segregation avoids incorrect synaptic connections between adjacent columns; and signal propagation across layers overwhelmingly prefers columnar direction. For the condition (iii), accumulating experimental evidence supports temporal transfer precision with millisecond fidelity and reliability in interneurons; homeostasis supports a stable fixed-point encoder by regulating changes to synaptic size, synaptic strength, and ion channel function in the membrane; together all-or-none modulation, active backpropagation, additive effects of graded potentials, and response variability functionally support the multithreshold decoder; our simulations demonstrate that the encoder-decoder is temporally complete fidelity and reliable in special intervals contained within the stable fixed-point range. Hence, the SSCCPI circuit provides a possible mechanism of effective signal propagation in cortical networks.

High-order coordination of cortical spiking activity modulates perceptual accuracy

The accurate relay of electrical signals within cortical networks is key to perception and cognitive function. Theoretically, it has long been proposed that temporal coordination of neuronal spiking activity controls signal transmission and behavior. However, whether and how temporally precise neuronal coordination in population activity influences perception are unknown. Here, we recorded populations of neurons in early and mid-level visual cortex (areas V1 and V4) simultaneously to discover that the precise temporal coordination between the spiking activity of three or more cells carries information about visual perception in the absence of firing rate modulation. The accuracy of perceptual responses correlated with high-order spiking coordination within V4, but not V1, and with feedforward coordination between V1 and V4. These results indicate that while visual stimuli are encoded in the discharge rates of neurons, perceptual accuracy is related to temporally precise spiking coordination within and between cortical networks.

The Magnitude, But Not the Sign, of MT Single-Trial Spike-Time Correlations Predicts Motion Detection Performance

Spike-time correlations capture the short timescale covariance between the activity of neurons on a single trial. These correlations can significantly vary in magnitude and sign from trial to trial, and have been proposed to contribute to information encoding in visual cortex. While monkeys performed a motion-pulse detection task, we examined the behavioral impact of both the magnitude and sign of single-trial spike-time correlations between two nonoverlapping pools of middle temporal (MT) neurons. We applied three single-trial measures of spike-time correlation between our multiunit MT spike trains (Pearson's, absolute value of Pearson's, and mutual information), and examined the degree to which they predicted a subject's performance on a trial-by-trial basis. We found that on each trial, positive and negative spike-time correlations were almost equally likely, and, once the correlational sign was accounted for, all three measures were similarly predictive of behavior. Importantly, just before the behaviorally relevant motion pulse occurred, single-trial spike-time correlations were as predictive of the performance of the animal as single-trial firing rates. While firing rates were positively associated with behavioral outcomes, the presence of either strong positive or negative correlations had a detrimental effect on behavior. These correlations occurred on short timescales, and the strongest positive and negative correlations modulated behavioral performance by ∼9%, compared with trials with no correlations. We suggest a model where spike-time correlations are associated with a common noise source for the two MT pools, which in turn decreases the signal-to-noise ratio of the integrated signals that drive motion detection.SIGNIFICANCE STATEMENT Previous work has shown that spike-time correlations occurring on short timescales can affect the encoding of visual inputs. Although spike-time correlations significantly vary in both magnitude and sign across trials, their impact on trial-by-trial behavior is not fully understood. Using neural recordings from area MT (middle temporal) in monkeys performing a motion-detection task using a brief stimulus, we found that both positive and negative spike-time correlations predicted behavioral responses as well as firing rate on a trial-by-trial basis. We propose that strong positive and negative spike-time correlations decreased behavioral performance by reducing the signal-to-noise ratio of integrated MT neural signals.

An interplay of fusiform gyrus and hippocampus enables prototype- and exemplar-based category learning

The aim of the present study was to examine the contributions of different brain structures to prototype- and exemplar-based category learning using functional magnetic resonance imaging (fMRI). Twenty-eight subjects performed a categorization task in which they had to assign prototypes and exceptions to two different families. This test procedure usually produces different learning curves for prototype and exception stimuli. Our behavioral data replicated these previous findings by showing an initially superior performance for prototypes and typical stimuli and a switch from a prototype-based to an exemplar-based categorization for exceptions in the later learning phases. Since performance varied, we divided participants into learners and non-learners. Analysis of the functional imaging data revealed that the interaction of group (learners vs. non-learners) and block (Block 5 vs. Block 1) yielded an activation of the left fusiform gyrus for the processing of prototypes, and an activation of the right hippocampus for exceptions after learning the categories. Thus, successful prototype- and exemplar-based category learning is associated with activations of complementary neural substrates that constitute object-based processes of the ventral visual stream and their interaction with unique-cue representations, possibly based on sparse coding within the hippocampus.

Network Anisotropy Trumps Noise for Efficient Object Coding in Macaque Inferior Temporal Cortex

How neuronal ensembles compute information is actively studied in early visual cortex. Much less is known about how local ensembles function in inferior temporal (IT) cortex, the last stage of the ventral visual pathway that supports visual recognition. Previous reports suggested that nearby neurons carry information mostly independently, supporting efficient processing (Barlow, 1961). However, others postulate that noise covariation effects may depend on network anisotropy/homogeneity and on how the covariation relates to representation. Do slow trial-by-trial noise covariations increase or decrease IT's object coding capability, how does encoding capability relate to correlational structure (i.e., the spatial pattern of signal and noise redundancy/homogeneity across neurons), and does knowledge of correlational structure matter for decoding? We recorded simultaneously from ∼80 spiking neurons in ∼1 mm(3) of macaque IT under light neurolept anesthesia. Noise correlations were stronger for neurons with correlated tuning, and noise covariations reduced object encoding capability, including generalization across object pose and illumination. Knowledge of noise covariations did not lead to better decoding performance. However, knowledge of anisotropy/homogeneity improved encoding and decoding efficiency by reducing the number of neurons needed to reach a given performance level. Such correlated neurons were found mostly in supragranular and infragranular layers, supporting theories that link recurrent circuitry to manifold representation. These results suggest that redundancy benefits manifold learning of complex high-dimensional information and that subsets of neurons may be more immune to noise covariation than others.

SIGNIFICANCE STATEMENT

How noise affects neuronal population coding is poorly understood. By sampling densely from local populations supporting visual object recognition, we show that recurrent circuitry supports useful representations and that subsets of neurons may be more immune to noise covariation than others.

Population coding in area V4 during rapid shape detections

While previous studies have suggested that neuronal correlations are common in visual cortex over a range of timescales, the effect of correlations on rapid visually based decisions has received little attention. We trained Macaca mulatta to saccade to a peripherally presented shape embedded in dynamic noise as soon as the shape appeared. While the monkeys performed the task, we recorded from neuronal populations (5-29 cells) using a microelectrode array implanted in area V4, a visual area thought to be involved in form perception. While modest correlations were present between cells during visual stimulation, their magnitude did not change significantly subsequent to the appearance of a shape. We quantified the reliability and temporal precision with which neuronal populations signaled the appearance of the shape and predicted the animals' choices using mutual information analyses. To study the impact of correlations, we shuffled the activity from each cell across observations while retaining stimulus-dependent modulations in firing rate. We found that removing correlations by shuffling across trials minimally affected the reliability or timing with which pairs, or larger groups of cells, signaled the presence of a shape. To assess the downstream impact of correlations, we also studied how shuffling affected the ability of V4 populations to predict behavioral choices. Surprisingly, shuffling created a modest increase in the accuracy of such predictions, suggesting that the reliability of downstream neurons is slightly compromised by activity correlations. Our findings are consistent with neuronal correlations having a minimal effect on the reliability and timing of rapid perceptual decisions.

Tuning and spontaneous spike time synchrony share a common structure in macaque inferior temporal cortex

Investigating the relationship between tuning and spike timing is necessary to understand how neuronal populations in anterior visual cortex process complex stimuli. Are tuning and spontaneous spike time synchrony linked by a common spatial structure (do some cells covary more strongly, even in the absence of visual stimulation?), and what is the object coding capability of this structure? Here, we recorded from spiking populations in macaque inferior temporal (IT) cortex under neurolept anesthesia. We report that, although most nearby IT neurons are weakly correlated, neurons with more similar tuning are also more synchronized during spontaneous activity. This link between tuning and synchrony was not simply due to cell separation distance. Instead, it expands on previous reports that neurons along an IT penetration are tuned to similar but slightly different features. This constraint on possible population firing rate patterns was consistent across stimulus sets, including animate vs. inanimate object categories. A classifier trained on this structure was able to generalize category "read-out" to untrained objects using only a few dimensions (a few patterns of site weightings per electrode array). We suggest that tuning and spike synchrony are linked by a common spatial structure that is highly efficient for object representation.

Organization of local horizontal functional interactions between neurons in the inferior temporal cortex of macaque monkeys

Detailed knowledge of neuronal circuitry is necessary for understanding the mechanisms underlying information processing in the brain. We investigated the organization of horizontal functional interactions in the inferior temporal cortex of macaque monkeys, which plays important roles in visual object recognition. Neuronal activity was recorded from the inferior temporal cortex using an array of eight tetrodes, with spatial separation between paired neurons up to 1.4 mm. We evaluated functional interactions on a time scale of milliseconds using cross-correlation analysis of neuronal activity of the paired neurons. Visual response properties of neurons were evaluated using responses to a set of 100 visual stimuli. Adjacent neuron pairs tended to show strong functional interactions compared with more distant neuron pairs, and neurons with similar stimulus preferences tended to show stronger functional interactions than neurons with different stimulus preferences. Thus horizontal functional interactions in the inferior temporal cortex appear to be organized according to both cortical distances and similarity in stimulus preference between neurons. Furthermore, the relationship between strength of functional interactions and similarity in stimulus preference observed in distant neuron pairs was more prominent than in adjacent pairs. The results suggest that functional circuitry is specifically organized, depending on the horizontal distances between neurons. Such specificity endows each circuit with unique functions.

Method for stationarity-segmentation of spike train data with application to the Pearson cross-correlation

Correlations among neurons are supposed to play an important role in computation and information coding in the nervous system. Empirically, functional interactions between neurons are most commonly assessed by cross-correlation functions. Recent studies have suggested that pairwise correlations may indeed be sufficient to capture most of the information present in neural interactions. Many applications of correlation functions, however, implicitly tend to assume that the underlying processes are stationary. This assumption will usually fail for real neurons recorded in vivo since their activity during behavioral tasks is heavily influenced by stimulus-, movement-, or cognition-related processes as well as by more general processes like slow oscillations or changes in state of alertness. To address the problem of nonstationarity, we introduce a method for assessing stationarity empirically and then “slicing” spike trains into stationary segments according to the statistical definition of weak-sense stationarity. We examine pairwise Pearson cross-correlations (PCCs) under both stationary and nonstationary conditions and identify another source of covariance that can be differentiated from the covariance of the spike times and emerges as a consequence of residual nonstationarities after the slicing process: the covariance of the firing rates defined on each segment. Based on this, a correction of the PCC is introduced that accounts for the effect of segmentation. We probe these methods both on simulated data sets and on in vivo recordings from the prefrontal cortex of behaving rats. Rather than for removing nonstationarities, the present method may also be used for detecting significant events in spike trains.

Circuit mechanisms revealed by spike-timing correlations in macaque area MT

We recorded simultaneously from pairs of motion-sensitive neurons in the middle temporal cortex (MT) of macaque monkeys and used cross-correlations in the timing of spikes between neurons to gain insights into cortical circuitry. We characterized the time course and stimulus dependency of the cross-correlogram (CCG) for each pair of neurons and of the auto-correlogram (ACG) of the individual neurons. For some neuron pairs, the CCG showed negative flanks that emerged next to the central peak during stimulus-driven responses. Similar negative flanks appeared in the ACG of many neurons. Negative flanks were most prevalent and deepest when the neurons were driven to high rates by visual stimuli that moved in the neurons' preferred directions. The temporal development of the negative flanks in the CCG coincided with a parallel, modest reduction of the noise correlation between the spike counts of the neurons. Computational analysis of a model cortical circuit suggested that negative flanks in the CCG arise from the excitation-triggered mutual cross-inhibition between pairs of excitatory neurons. Intracortical recurrent inhibition and afterhyperpolarization caused by intrinsic outward currents, such as the calcium-activated potassium current of small conductance, can both contribute to the negative flanks in the ACG. In the model circuit, stronger intracortical inhibition helped to maintain the temporal precision between the spike trains of pairs of neurons and led to weaker noise correlations. Our results suggest a neural circuit architecture that can leverage activity-dependent intracortical inhibition to adaptively modulate both the synchrony of spike timing and the correlations in response variability.

Neural mechanism of dynamic responses of neurons in inferior temporal cortex in face perception

Understanding the neural mechanisms of object and face recognition is one of the fundamental challenges of visual neuroscience. The neurons in inferior temporal (IT) cortex have been reported to exhibit dynamic responses to face stimuli. However, little is known about how the dynamic properties of IT neurons emerge in the face information processing. To address this issue, we made a model of IT cortex, which performs face perception via an interaction between different IT networks. The model was based on the face information processed by three resolution maps in early visual areas. The network model of IT cortex consists of four kinds of networks, in which the information about a whole face is combined with the information about its face parts and their arrangements. We show here that the learning of face stimuli makes the functional connections between these IT networks, causing a high spike correlation of IT neuron pairs. A dynamic property of subthreshold membrane potential of IT neuron, produced by Hodgkin-Huxley model, enables the coordination of temporal information without changing the firing rate, providing the basis of the mechanism underlying face perception. We show also that the hierarchical processing of face information allows IT cortex to perform a "coarse-to-fine" processing of face information. The results presented here seem to be compatible with experimental data about dynamic properties of IT neurons.

The fusiform face area is engaged in holistic, not parts-based, representation of faces

Numerous studies with functional magnetic resonance imaging have shown that the fusiform face area (FFA) in the human brain plays a key role in face perception. Recent studies have found that both the featural information of faces (e.g., eyes, nose, and mouth) and the configural information of faces (i.e., spatial relation among features) are encoded in the FFA. However, little is known about whether the featural information is encoded independent of or combined with the configural information in the FFA. Here we used multi-voxel pattern analysis to examine holistic representation of faces in the FFA by correlating spatial patterns of activation with behavioral performance in discriminating face parts with face configurations either present or absent. Behaviorally, the absence of face configurations (versus presence) impaired discrimination of face parts, suggesting a holistic representation in the brain. Neurally, spatial patterns of activation in the FFA were more similar among correct than incorrect trials only when face parts were presented in a veridical face configuration. In contrast, spatial patterns of activation in the occipital face area, as well as the object-selective lateral occipital complex, were more similar among correct than incorrect trials regardless of the presence of veridical face configurations. This finding suggests that in the FFA faces are represented not on the basis of individual parts but in terms of the whole that emerges from the parts.

Conditional modeling and the jitter method of spike resampling

The existence and role of fine-temporal structure in the spiking activity of central neurons is the subject of an enduring debate among physiologists. To a large extent, the problem is a statistical one: what inferences can be drawn from neurons monitored in the absence of full control over their presynaptic environments? In principle, properly crafted resampling methods can still produce statistically correct hypothesis tests. We focus on the approach to resampling known as jitter. We review a wide range of jitter techniques, illustrated by both simulation experiments and selected analyses of spike data from motor cortical neurons. We rely on an intuitive and rigorous statistical framework known as conditional modeling to reveal otherwise hidden assumptions and to support precise conclusions. Among other applications, we review statistical tests for exploring any proposed limit on the rate of change of spiking probabilities, exact tests for the significance of repeated fine-temporal patterns of spikes, and the construction of acceptance bands for testing any purported relationship between sensory or motor variables and synchrony or other fine-temporal events.

Visual stimulation synchronizes or desynchronizes the activity of neuron pairs between the caudate nucleus and the posterior thalamus

Recent morphological and physiological studies have suggested a strong relationship between the suprageniculate nucleus (Sg) of the posterior thalamus and the input structure of the basal ganglia, the caudate nucleus (CN) of the feline brain. Accordingly, to clarify if there is a real functional relationship between Sg and CN during visual information processing, we investigated the temporal relations of simultaneously recorded neuronal spike trains of these two structures, looking for any significant cross-correlation between the spiking of the simultaneously recorded neurons. For the purposes of statistical analysis, we used the shuffle and jittering resampling methods. Of the recorded 288 Sg-CN neuron pairs, 26 (9.2%) showed significantly correlated spontaneous activity. Nineteen pairs (6.7%) showed correlated activity during stationary visual stimulation, while 21 (7.4%) pairs during stimulus movement. There was no overlap between the neuron pairs that showed cross-correlated spontaneous activity and the pairs that synchronized their activity during visual stimulation. Thus visual stimulation seems to have been able to synchronize, and also, by other neuron pairs, desynchronize the activity of CN and Sg. In about half of the cases, the activation of Sg preceded the activation of CN by a few milliseconds, while in the other half, CN was activated earlier. Our results provide the first piece of evidence for the existence of a functional cooperation between Sg and CN. We argue that either a monosynaptic bidirectional direct connection should exist between these structures, or a common input comprising of parallel pathways synchronizing them.

Role of temporal processing stages by inferior temporal neurons in facial recognition

In this review, we focus on the role of temporal stages of encoded facial information in the visual system, which might enable the efficient determination of species, identity, and expression. Facial recognition is an important function of our brain and is known to be processed in the ventral visual pathway, where visual signals are processed through areas V1, V2, V4, and the inferior temporal (IT) cortex. In the IT cortex, neurons show selective responses to complex visual images such as faces, and at each stage along the pathway the stimulus selectivity of the neural responses becomes sharper, particularly in the later portion of the responses. In the IT cortex of the monkey, facial information is represented by different temporal stages of neural responses, as shown in our previous study: the initial transient response of face-responsive neurons represents information about global categories, i.e., human vs. monkey vs. simple shapes, whilst the later portion of these responses represents information about detailed facial categories, i.e., expression and/or identity. This suggests that the temporal stages of the neuronal firing pattern play an important role in the coding of visual stimuli, including faces. This type of coding may be a plausible mechanism underlying the temporal dynamics of recognition, including the process of detection/categorization followed by the identification of objects. Recent single-unit studies in monkeys have also provided evidence consistent with the important role of the temporal stages of encoded facial information. For example, view-invariant facial identity information is represented in the response at a later period within a region of face-selective neurons. Consistent with these findings, temporally modulated neural activity has also been observed in human studies. These results suggest a close correlation between the temporal processing stages of facial information by IT neurons and the temporal dynamics of face recognition.

Higher Order Spike Synchrony in Prefrontal Cortex during Visual Memory

Precise temporal synchrony of spike firing has been postulated as an important neuronal mechanism for signal integration and the induction of plasticity in neocortex. As prefrontal cortex plays an important role in organizing memory and executive functions, the convergence of multiple visual pathways onto PFC predicts that neurons should preferentially synchronize their spiking when stimulus information is processed. Furthermore, synchronous spike firing should intensify if memory processes require the induction of neuronal plasticity, even if this is only for short-term. Here we show with multiple simultaneously recorded units in ventral prefrontal cortex that neurons participate in 3 ms precise synchronous discharges distributed across multiple sites separated by at least 500 μm. The frequency of synchronous firing is modulated by behavioral performance and is specific for the memorized visual stimuli. In particular, during the memory period in which activity is not stimulus driven, larger groups of up to seven sites exhibit performance dependent modulation of their spike synchronization.

Neural syntax: cell assemblies, synapsembles, and readers

A widely discussed hypothesis in neuroscience is that transiently active ensembles of neurons, known as "cell assemblies," underlie numerous operations of the brain, from encoding memories to reasoning. However, the mechanisms responsible for the formation and disbanding of cell assemblies and temporal evolution of cell assembly sequences are not well understood. I introduce and review three interconnected topics, which could facilitate progress in defining cell assemblies, identifying their neuronal organization, and revealing causal relationships between assembly organization and behavior. First, I hypothesize that cell assemblies are best understood in light of their output product, as detected by "reader-actuator" mechanisms. Second, I suggest that the hierarchical organization of cell assemblies may be regarded as a neural syntax. Third, constituents of the neural syntax are linked together by dynamically changing constellations of synaptic weights ("synapsembles"). The existing support for this tripartite framework is reviewed and strategies for experimental testing of its predictions are discussed.

Ambiguous figures and binding: EEG frequency modulations during multistable perception

Ambiguous figures induce sudden transitions between rivaling percepts. We investigated electroencephalogram frequency modulations of accompanying change-related de- and rebinding processes. Presenting the stimuli discontinously, we synchronized perceptual reversals with stimulus onset, which served as a time reference for averaging. The resultant gain in temporal resolution revealed a sequence of time-frequency correlates of the reversal process. Most conspicuous was a transient right-hemispheric gamma modulation preceding endogenous reversals by at least 200 ms. No such modulation occurred with exogenously induced reversals of unambiguous stimulus variants. Post-onset components were delayed for ambiguous compared to unambiguous stimuli. The time course of oscillatory activity differed in several respects from predictions based on binding-related hypotheses. The gamma modulation preceding endogenous reversals may indicate an unstable brain state, ready to switch.

Triphasic dynamics of stimulus-dependent information flow between single neurons in macaque inferior temporal cortex

The functional connectivity between cortical neurons is not static and is known to exhibit contextual modulations in terms of the coupling strength. Here we hypothesized that the information flow in a cortical local circuit exhibits complex forward-and-back dynamics, and conducted Granger causality analysis between the neuronal spike trains that were simultaneously recorded from macaque inferior temporal (IT) cortex while the animals performed a visual object discrimination task. Spikes from neuron pairs with a displaced peak on the cross-correlogram (CCG) showed Granger causality in the gamma-frequency range (30-80 Hz) with the dominance in the direction consistent with the CCG peak (forward direction). Although, in a classical view, the displaced CCG peak has been interpreted as an indicative of a pauci-synaptic serial linkage, temporal dynamics of the gamma Granger causality after stimulus onset exhibited a more complex triphasic pattern, with a transient forward component followed by a slowly developing backward component and subsequent reappearance of the forward component. These triphasic dynamics of causality were not explained by the firing rate dynamics and were not observed for cell pairs that exhibited a center peak on the CCG. Furthermore, temporal dynamics of Granger causality depended on the feature configuration within the presented object. Together, these results demonstrate that the classical view of functional connectivity could be expanded to incorporate more complex forward-and-back dynamics and also imply that multistage processing in the recognition of visual objects might be implemented by multiphasic dynamics of directional information flow between single neurons in a local circuit in the IT cortex.

Shift from hippocampal to neocortical centered retrieval network with consolidation

The standard model of system-level consolidation posits that the hippocampus is part of a retrieval network for recent memories. According to this theory, the memories are gradually transferred to neocortical circuits with consolidation, where the connections within this circuit grow stronger and reorganized so that redundant and/or contextual details may be lost. Thus, remote memories are based on neocortical networks and can be retrieved independently of the hippocampus. To test this model, we measured regional brain activity and connectivity during retrieval with functional magnetic resonance imaging. Subjects were trained on two sets of face-location association and were tested with two different delays, 15 min and 24 h including a whole night of sleep. We hypothesized that memory traces of the locations associated with specific faces will be linked through the hippocampus for the retrieval of recently learned association, but with consolidation, the activity and the functional connectivity between the neocortical areas will increase. We show that posterior hippocampal activity related to high-confidence retrieval decreased and neocortical activity increased with consolidation. Moreover, the connectivity between the hippocampus and the neocortical regions decreased and in turn, cortico-cortical connectivity between the representational areas increased. The results provide mechanistic support for a two-level process of the declarative memory system, involving initial representation of new associations in a network including the hippocampus and subsequent consolidation into a predominantly neocortical network.

Neural synchrony in cortical networks: history, concept and current status

Following the discovery of context-dependent synchronization of oscillatory neuronal responses in the visual system, the role of neural synchrony in cortical networks has been expanded to provide a general mechanism for the coordination of distributed neural activity patterns. In the current paper, we present an update of the status of this hypothesis through summarizing recent results from our laboratory that suggest important new insights regarding the mechanisms, function and relevance of this phenomenon. In the first part, we present recent results derived from animal experiments and mathematical simulations that provide novel explanations and mechanisms for zero and nero-zero phase lag synchronization. In the second part, we shall discuss the role of neural synchrony for expectancy during perceptual organization and its role in conscious experience. This will be followed by evidence that indicates that in addition to supporting conscious cognition, neural synchrony is abnormal in major brain disorders, such as schizophrenia and autism spectrum disorders. We conclude this paper with suggestions for further research as well as with critical issues that need to be addressed in future studies.

Dynamic synchrony of local cell assembly

In the present paper, we focus on the problem of the dynamic size of a cell assembly and discuss how we can detect synchronized firing of a local cell assembly consisting of closely neighboring neurons in the working brain. A local cell assembly is difficult to detect because of the problem of spike overlapping of neighboring neurons, which cannot be overcome by ordinary spike-sorting techniques. We introduce a unique technique of spike-sorting that combines independent component analysis (ICA) and an ordinary sorting method to separate individual neighboring neurons and analyze their firing synchrony in behaving animals. One of our experiments employing this method showed that some closely neighboring neurons in the monkey prefrontal cortex have dynamic and sharp synchrony of firing reflecting local cell assemblies during working-memory processes. Another experiment showed that our other method (ICSort) of novel spike-sorting by ICA using special electrodes (dodecatrodes) can distinguish firing signals from the soma and those from the dendrites of individual neurons in behaving rats and suggests that the somatic and dendritic signals have different roles in information processing. This indicates that functional connectivity among neurons may be more dynamic and complex and spikes from the soma and dendrites of individual neurons should be considered in the investigation of the activity of local cell assemblies. We finally propose that detailed and real features of a local cell assembly consisting of closely neighboring neurons should be examined further and detection of local cell assemblies could be applied to the development of neuronal prosthetic devices, that is, brain-machine interfaces (BMIs).

Spatial and temporal scales of neuronal correlation in primary visual cortex

The spiking activity of cortical neurons is correlated. For instance, trial-to-trial fluctuations in response strength are shared between neurons, and spikes often occur synchronously. Understanding the properties and mechanisms that generate these forms of correlation is critical for determining their role in cortical processing. We therefore investigated the spatial extent and functional specificity of correlated spontaneous and evoked activity. Because feedforward, recurrent, and feedback pathways have distinct extents and specificity, we reasoned that these measurements could elucidate the contribution of each type of input. We recorded single unit activity with microelectrode arrays which allowed us to measure correlation in many hundreds of pairings, across a large range of spatial scales. Our data show that correlated evoked activity is generated by two mechanisms that link neurons with similar orientation preferences on different spatial scales: one with high temporal precision and a limited spatial extent (approximately 3 mm), and a second that gives rise to correlation on a slow time scale and extends as far as we were able to measure (10 mm). The former is consistent with common input provided by horizontal connections; the latter likely involves feedback from extrastriate cortex. Spontaneous activity was correlated over a similar spatial extent, but approximately twice as strongly as evoked activity. Visual stimuli thus caused a substantial decrease in correlation, particularly at response onset. These properties and the circuit mechanism they imply provide new constraints on the functional role that correlation may play in visual processing.

Towards understanding of the cortical network underlying associative memory

Declarative knowledge and experiences are represented in the association cortex and are recalled by reactivation of the neural representation. Electrophysiological experiments have revealed that associations between semantically linked visual objects are formed in neural representations in the temporal and limbic cortices. Memory traces are created by the reorganization of neural circuits. These regions are reactivated during retrieval and contribute to the contents of a memory. Two different types of retrieval signals are suggested as follows: automatic and active. One flows backward from the medial temporal lobe during the automatic retrieval process, whereas the other is conveyed as a top-down signal from the prefrontal cortex to the temporal cortex during the active retrieval process. By sending the top-down signal, the prefrontal cortex manipulates and organizes to-be-remembered information, devises strategies for retrieval and monitors the outcome. To further understand the neural mechanism of memory, the following two complementary views are needed: how the multiple cortical areas in the brain-wide network interact to orchestrate cognitive functions and how the properties of single neurons and their synaptic connections with neighbouring neurons combine to form local circuits and to exhibit the function of each cortical area. We will discuss some new methodological innovations that tackle these challenges.

Behavior-dependent short-term assembly dynamics in the medial prefrontal cortex

Although short-term plasticity is believed to play a fundamental role in cortical computation, empirical evidence bearing on its role during behavior is scarce. Here we looked for the signature of short-term plasticity in the fine-timescale spiking relationships of a simultaneously recorded population of physiologically identified pyramidal cells and interneurons, in the medial prefrontal cortex of the rat, in a working memory task. On broader timescales, sequentially organized and transiently active neurons reliably differentiated between different trajectories of the rat in the maze. On finer timescales, putative monosynaptic interactions reflected short-term plasticity in their dynamic and predictable modulation across various aspects of the task, beyond a statistical accounting for the effect of the neurons' co-varying firing rates. Seeking potential mechanisms for such effects, we found evidence for both firing pattern-dependent facilitation and depression, as well as for a supralinear effect of presynaptic coincidence on the firing of postsynaptic targets.

Information measure for analyzing specific spiking patterns and applications to LGN bursts

Neural spiking responses can include a variety of spiking patterns. However, neither the mere presence of the patterns nor the pattern's frequency indicates that the pattern conveys distinct stimulus information. Here, we present an in-depth analysis of a Pattern Information measure, which quantifies how informative it is to distinguish a particular pattern of spikes from either a single spike or an another pattern. (1) We show how a shuffle-controlled estimation method minimizes the impact of sampling bias. (2) We describe how the Pattern Information could arise from time-varying firing rates, and we demonstrate an analysis to determine whether Pattern Information associated with a particular pattern captures structure not contained in the time-varying firing rate. (3) Because patterns may contain several spikes or inter-spike intervals, we extend the Pattern Information measure to determine whether the complete pattern carries information distinct from sub-patterns containing only a fraction of these spikes or intervals. (4) The Pattern Information is applied to determine whether a plurality of patterns carry distinct stimulus information from one another. In particular, we demonstrate these concepts using data from cells of the lateral geniculate nucleus (LGN), thereby extending previous analysis demonstrating that distinguishes between bursts of spikes and single spikes providing visual information.

Neuron participation in a synchrony-encoding assembly

BACKGROUND

Synchronization of action potentials between neurons is considered to be an encoding process that allows the grouping of various and multiple features of an image leading to a coherent perception. How this coding neuronal assembly is configured is debated. We have previously shown that the magnitude of synchronization between excited neurons is stimulus-dependent. In the present investigation we compare the levels of synchronization between synchronizing individual neurons and the synchronizing pool of cells to which they belong.

RESULTS

Even though neurons belonged to their respective pools, some cells synchronized for all presented stimuli while others were rather selective and only a few stimulating conditions produced a significant synchronization. In addition the experiments show that one synchronizing pair rarely replicates the level of synchrony between corresponding groups of units. But when synchronizing clusters of neurons increase in number, the correlation (measured as a coefficient of determination) between unit synchronization and the synchronization between the entire pools of cells to which individual neurons belong improves.

CONCLUSION

These results prompt the hypothesis that random or spontaneous synchronization becomes progressively less important, whereas coincident spikes related to encoding properties of targets gain significance because a particular configuration of an image biases the excitatory inputs in favor of connections driven by the applied features of the stimulus.

A Synchronization-Desynchronization Code for Natural Communication Signals

Synchronous spiking of neural populations is hypothesized to play important computational roles in forming neural assemblies and solving the binding problem. Although the opposite phenomenon of desynchronization is well known from EEG studies, it is largely neglected on the neuronal level. We here provide an example of in vivo recordings from weakly electric fish demonstrating that, depending on the social context, different types of natural communication signals elicit transient desynchronization as well as synchronization of the electroreceptor population without changing the mean firing rate. We conclude that, in general, both positive and negative changes in the degree of synchrony can be the relevant signals for neural information processing.