Spatial phase and the temporal structure of the response to gratings in V1

Receptive-field nonlinearities in primary auditory cortex: a comparative perspective

Cortical processing of auditory information can be affected by interspecies differences as well as brain states. Here we compare multifeature spectro-temporal receptive fields (STRFs) and associated input/output functions or nonlinearities (NLs) of neurons in primary auditory cortex (AC) of four mammalian species. Single-unit recordings were performed in awake animals (female squirrel monkeys, female, and male mice) and anesthetized animals (female squirrel monkeys, rats, and cats). Neuronal responses were modeled as consisting of two STRFs and their associated NLs. The NLs for the STRF with the highest information content show a broad distribution between linear and quadratic forms. In awake animals, we find a higher percentage of quadratic-like NLs as opposed to more linear NLs in anesthetized animals. Moderate sex differences of the shape of NLs were observed between male and female unanesthetized mice. This indicates that the core AC possesses a rich variety of potential computations, particularly in awake animals, suggesting that multiple computational algorithms are at play to enable the auditory system's robust recognition of auditory events.

Attention-Based Deep Spiking Neural Networks for Temporal Credit Assignment Problems

The temporal credit assignment (TCA) problem, which aims to detect predictive features hidden in distracting background streams, remains a core challenge in biological and machine learning. Aggregate-label (AL) learning is proposed by researchers to resolve this problem by matching spikes with delayed feedback. However, the existing AL learning algorithms only consider the information of a single timestep, which is inconsistent with the real situation. Meanwhile, there is no quantitative evaluation method for TCA problems. To address these limitations, we propose a novel attention-based TCA (ATCA) algorithm and a minimum editing distance (MED)-based quantitative evaluation method. Specifically, we define a loss function based on the attention mechanism to deal with the information contained within the spike clusters and use MED to evaluate the similarity between the spike train and the target clue flow. Experimental results on musical instrument recognition (MedleyDB), speech recognition (TIDIGITS), and gesture recognition (DVS128-Gesture) show that the ATCA algorithm can reach the state-of-the-art (SOTA) level compared with other AL learning algorithms.

Clustering of Neural Activity: A Design Principle for Population Codes

We propose that correlations among neurons are generically strong enough to organize neural activity patterns into a discrete set of clusters, which can each be viewed as a population codeword. Our reasoning starts with the analysis of retinal ganglion cell data using maximum entropy models, showing that the population is robustly in a frustrated, marginally sub-critical, or glassy, state. This leads to an argument that neural populations in many other brain areas might share this structure. Next, we use latent variable models to show that this glassy state possesses well-defined clusters of neural activity. Clusters have three appealing properties: (i) clusters exhibit error correction, i.e., they are reproducibly elicited by the same stimulus despite variability at the level of constituent neurons; (ii) clusters encode qualitatively different visual features than their constituent neurons; and (iii) clusters can be learned by downstream neural circuits in an unsupervised fashion. We hypothesize that these properties give rise to a "learnable" neural code which the cortical hierarchy uses to extract increasingly complex features without supervision or reinforcement.

A Spike Train Distance Robust to Firing Rate Changes Based on the Earth Mover's Distance

Neural spike train analysis methods are mainly used for understanding the temporal aspects of neural information processing. One approach is to measure the dissimilarity between the spike trains of a pair of neurons, often referred to as the spike train distance. The spike train distance has been often used to classify neuronal units with similar temporal patterns. Several methods to compute spike train distance have been developed so far. Intuitively, a desirable distance should be the shortest length between two objects. The Earth Mover’s Distance (EMD) can compute spike train distance by measuring the shortest length between two spike trains via shifting a fraction of spikes from one spike train to another. The EMD could accurately measure spike timing differences, temporal similarity, and spikes time synchrony. It is also robust to firing rate changes. Victor and Purpura (1996) distance measures the minimum cost between two spike trains. Although it also measures the shortest path between spike trains, its output can vary with the time-scale parameter. In contrast, the EMD measures distance in a unique way by calculating the genuine shortest length between spike trains. The EMD also outperforms other existing spike train distance methods in measuring various aspects of the temporal characteristics of spike trains and in robustness to firing rate changes. The EMD can effectively measure the shortest length between spike trains without being considerably affected by the overall firing rate difference between them. Hence, it is suitable for pure temporal coding exclusively, which is a predominant premise underlying the present study.

Noise-Robust Modes of the Retinal Population Code Have the Geometry of "Ridges" and Correspond to Neuronal Communities

An appealing new principle for neural population codes is that correlations among neurons organize neural activity patterns into a discrete set of clusters, which can each be viewed as a noise-robust population codeword. Previous studies assumed that these codewords corresponded geometrically with local peaks in the probability landscape of neural population responses. Here, we analyze multiple data sets of the responses of approximately 150 retinal ganglion cells and show that local probability peaks are absent under broad, nonrepeated stimulus ensembles, which are characteristic of natural behavior. However, we find that neural activity still forms noise-robust clusters in this regime, albeit clusters with a different geometry. We start by defining a soft local maximum, which is a local probability maximum when constrained to a fixed spike count. Next, we show that soft local maxima are robustly present and can, moreover, be linked across different spike count levels in the probability landscape to form a ridge. We found that these ridges comprise combinations of spiking and silence in the neural population such that all of the spiking neurons are members of the same neuronal community, a notion from network theory. We argue that a neuronal community shares many of the properties of Donald Hebb's classic cell assembly and show that a simple, biologically plausible decoding algorithm can recognize the presence of a specific neuronal community.

Error-Robust Modes of the Retinal Population Code

Across the nervous system, certain population spiking patterns are observed far more frequently than others. A hypothesis about this structure is that these collective activity patterns function as population codewords–collective modes–carrying information distinct from that of any single cell. We investigate this phenomenon in recordings of ∼150 retinal ganglion cells, the retina’s output. We develop a novel statistical model that decomposes the population response into modes; it predicts the distribution of spiking activity in the ganglion cell population with high accuracy. We found that the modes represent localized features of the visual stimulus that are distinct from the features represented by single neurons. Modes form clusters of activity states that are readily discriminated from one another. When we repeated the same visual stimulus, we found that the same mode was robustly elicited. These results suggest that retinal ganglion cells’ collective signaling is endowed with a form of error-correcting code–a principle that may hold in brain areas beyond retina.

Neurons in most parts of the nervous system represent and process information in a collective fashion, yet the nature of this collective code is poorly understood. An important constraint placed on any such collective processing comes from the fact that individual neurons’ signaling is prone to corruption by noise. The information theory and engineering literatures have studied error-correcting codes that allow individual noise-prone coding units to “check” each other, forming an overall representation that is robust to errors. In this paper, we have analyzed the population code of one of the best-studied neural systems, the retina, and found that it is structured in a manner analogous to error-correcting schemes. Indeed, we found that the complex activity patterns over ~150 retinal ganglion cells, the output neurons of the retina, could be mapped onto collective code words, and that these code words represented precise visual information while suppressing noise. In order to analyze this coding scheme, we introduced a novel quantitative model of the retinal output that predicted neural activity patterns more accurately than existing state-of-the-art approaches.

Neural coding of image structure and contrast polarity of Cartesian, hyperbolic, and polar gratings in the primary and secondary visual cortex of the tree shrew

We comprehensively characterize spiking and visual evoked potential (VEP) activity in tree shrew V1 and V2 using Cartesian, hyperbolic, and polar gratings. Neural selectivity to structure of Cartesian gratings was higher than other grating classes in both visual areas. From V1 to V2, structure selectivity of spiking activity increased, whereas corresponding VEP values tended to decrease, suggesting that single-neuron coding of Cartesian grating attributes improved while the cortical columnar organization of these neurons became less precise from V1 to V2. We observed that neurons in V2 generally exhibited similar selectivity for polar and Cartesian gratings, suggesting that structure of polar-like stimuli might be encoded as early as in V2. This hypothesis is supported by the preference shift from V1 to V2 toward polar gratings of higher spatial frequency, consistent with the notion that V2 neurons encode visual scene borders and contours. Neural sensitivity to modulations of polarity of hyperbolic gratings was highest among all grating classes and closely related to the visual receptive field (RF) organization of ON- and OFF-dominated subregions. We show that spatial RF reconstructions depend strongly on grating class, suggesting that intracortical contributions to RF structure are strongest for Cartesian and polar gratings. Hyperbolic gratings tend to recruit least cortical elaboration such that the RF maps are similar to those generated by sparse noise, which most closely approximate feedforward inputs. Our findings complement previous literature in primates, rodents, and carnivores and highlight novel aspects of shape representation and coding occurring in mammalian early visual cortex.

Computational identification of receptive fields

Natural stimuli elicit robust responses of neurons throughout sensory pathways, and therefore their use provides unique opportunities for understanding sensory coding. This review describes statistical methods that can be used to characterize neural feature selectivity, focusing on the case of natural stimuli. First, we discuss how such classic methods as reverse correlation/spike-triggered average and spike-triggered covariance can be generalized for use with natural stimuli to find the multiple relevant stimulus features that affect the responses of a given neuron. Second, ways to characterize neural feature selectivity while assuming that the neural responses exhibit a certain type of invariance, such as position invariance for visual neurons, are discussed. Finally, we discuss methods that do not require one to make an assumption of invariance and instead can determine the type of invariance by analyzing relationships between the multiple stimulus features that affect the neural responses.

Low error discrimination using a correlated population code

We explored the manner in which spatial information is encoded by retinal ganglion cell populations. We flashed a set of 36 shape stimuli onto the tiger salamander retina and used different decoding algorithms to read out information from a population of 162 ganglion cells. We compared the discrimination performance of linear decoders, which ignore correlation induced by common stimulation, with nonlinear decoders, which can accurately model these correlations. Similar to previous studies, decoders that ignored correlation suffered only a modest drop in discrimination performance for groups of up to ∼30 cells. However, for more realistic groups of 100+ cells, we found order-of-magnitude differences in the error rate. We also compared decoders that used only the presence of a single spike from each cell with more complex decoders that included information from multiple spike counts and multiple time bins. More complex decoders substantially outperformed simpler decoders, showing the importance of spike timing information. Particularly effective was the first spike latency representation, which allowed zero discrimination errors for the majority of shape stimuli. Furthermore, the performance of nonlinear decoders showed even greater enhancement compared with linear decoders for these complex representations. Finally, decoders that approximated the correlation structure in the population by matching all pairwise correlations with a maximum entropy model fit to all 162 neurons were quite successful, especially for the spike latency representation. Together, these results suggest a picture in which linear decoders allow a coarse categorization of shape stimuli, whereas nonlinear decoders, which take advantage of both correlation and spike timing, are needed to achieve high-fidelity discrimination.

What can spike train distances tell us about the neural code?

Time scale parametric spike train distances like the Victor and the van Rossum distances are often applied to study the neural code based on neural stimuli discrimination. Different neural coding hypotheses, such as rate or coincidence coding, can be assessed by combining a time scale parametric spike train distance with a classifier in order to obtain the optimal discrimination performance. The time scale for which the responses to different stimuli are distinguished best is assumed to be the discriminative precision of the neural code. The relevance of temporal coding is evaluated by comparing the optimal discrimination performance with the one achieved when assuming a rate code. We here characterize the measures quantifying the discrimination performance, the discriminative precision, and the relevance of temporal coding. Furthermore, we evaluate the information these quantities provide about the neural code. We show that the discriminative precision is too unspecific to be interpreted in terms of the time scales relevant for encoding. Accordingly, the time scale parametric nature of the distances is mainly an advantage because it allows maximizing the discrimination performance across a whole set of measures with different sensitivities determined by the time scale parameter, but not due to the possibility to examine the temporal properties of the neural code.

Low-frequency local field potentials and spikes in primary visual cortex convey independent visual information

Local field potentials (LFPs) reflect subthreshold integrative processes that complement spike train measures. However, little is yet known about the differences between how LFPs and spikes encode rich naturalistic sensory stimuli. We addressed this question by recording LFPs and spikes from the primary visual cortex of anesthetized macaques while presenting a color movie. We then determined how the power of LFPs and spikes at different frequencies represents the visual features in the movie. We found that the most informative LFP frequency ranges were 1-8 and 60-100 Hz. LFPs in the range of 12-40 Hz carried little information about the stimulus, and may primarily reflect neuromodulatory inputs. Spike power was informative only at frequencies <12 Hz. We further quantified "signal correlations" (correlations in the trial-averaged power response to different stimuli) and "noise correlations" (trial-by-trial correlations in the fluctuations around the average) of LFPs and spikes recorded from the same electrode. We found positive signal correlation between high-gamma LFPs (60-100 Hz) and spikes, as well as strong positive signal correlation within high-gamma LFPs, suggesting that high-gamma LFPs and spikes are generated within the same network. LFPs <24 Hz shared strong positive noise correlations, indicating that they are influenced by a common source, such as a diffuse neuromodulatory input. LFPs <40 Hz showed very little signal and noise correlations with LFPs >40 Hz and with spikes, suggesting that low-frequency LFPs reflect neural processes that in natural conditions are fully decoupled from those giving rise to spikes and to high-gamma LFPs.

Precision, reliability, and information-theoretic analysis of visual thalamocortical neurons

High-order statistics of neural responses allow one to gain insight into neural function that may not be evident from firing rate alone. In this study, we compared the precision, reliability, and information content of spike trains from X- and Y-cells in the lateral geniculate nucleus (LGN) and layer IV simple cells of area 17 in the cat. To a stochastic, contrast-modulated Gabor patch, layer IV simple cells responded as precisely as their primary inputs, LGN X-cells, but less reliably. LGN Y-cells were more precise and reliable than LGN X-cells. Also, within each LGN cell type, 1) responses to the same stimulus were nearly identical if they shared the same center sign and 2) responses of neurons with the same center sign were nearly identical to the responses of neurons of opposite center sign if the stimulus' contrasts were inverted. These results suggest simple cells receive highly precise and synchronous LGN input, resulting in precise responses. Nonetheless, the response precision of simple cells was greater than expected. Finally, information-theoretic calculations of our cell responses revealed that 1) LGN X-cells encoded information at half the rate of LGN Y-cells but 2.5 times the rate of layer IV simple cells; 2) LGN cells encoded information in their responses using temporal patterns, whereas simple cells did not; and 3) simple cells used more of their information capacity than LGN X-cells. We propose mechanisms that simple cells might use to ensure high precision.

Stochastic emergence of repeating cortical motifs in spontaneous membrane potential fluctuations in vivo

It was recently discovered that subthreshold membrane potential fluctuations of cortical neurons can precisely repeat during spontaneous activity, seconds to minutes apart, both in brain slices and in anesthetized animals. These repeats, also called cortical motifs, were suggested to reflect a replay of sequential neuronal firing patterns. We searched for motifs in spontaneous activity, recorded from the rat barrel cortex and from the cat striate cortex of anesthetized animals, and found numerous repeating patterns of high similarity and repetition rates. To test their significance, various statistics were compared between physiological data and three different types of stochastic surrogate data that preserve dynamical characteristics of the recorded data. We found no evidence for the existence of deterministically generated cortical motifs. Rather, the stochastic properties of cortical motifs suggest that they appear by chance, as a result of the constraints imposed by the coarse dynamics of subthreshold ongoing activity.

Temporally precise cortical firing patterns are associated with distinct action segments

Despite many reports indicating the existence of precise firing sequences in cortical activity, serious objections have been raised regarding the statistics used to detect them and the relations of these sequences to behavior. We show that in behaving monkeys, pairs of spikes from different neurons tend to prefer certain time delays when measured in relation to a specific behavior. Single-unit activity was recorded from eight microelectrodes inserted into the motor and premotor cortices of two monkeys while they were performing continuous drawinglike hand movements. Repeated scribbling paths, termed drawing components, were extracted by data-mining techniques. The set of the least predictable relations between drawing components and pairs of neurons was determined and represented by one statistic termed the relations score. The chance probability of the relations score was evaluated by teetering the spike times: 1,000 surrogates were generated by randomly teetering the original time of each spike in a small window. In nine of 13 experimental days the precision was better than 12 ms and, in the best case, spike precision reached 0.5 ms.

The integration of multiple stimulus features by V1 neurons

We investigated how V1 neurons integrate two stimulus features by presenting stimuli from a stimulus set made up of all combinations of eight different directions of motion and nine binocular disparities. We investigated the occurrence and shape of the resulting joint tuning function. Among V1 neurons, approximately 80% were jointly tuned for disparity with orientation or direction. The joint tuning function of all jointly tuned neurons was separable into distinct tuning for disparity on the one hand, and orientation or direction tuning on the other. The degree of separability and the mutual information between the stimulus and the firing rates were strongly correlated. The mutual information of jointly tuned neurons when both features were decoded together was highly correlated with the mutual information when the two features were decoded separately, and the information was then summed. Jointly tuned neurons were just as good at representing information about single features as neurons tuned for only a single feature. The tuning properties of most jointly tuned neurons did not dynamically evolve over time, nor did jointly tuned neurons respond earlier than neurons tuned for only a single feature. The response selectivity of V1 neurons is low and decreases the information that a neuron represents about a stimulus. Together these results suggest that distinct stimulus features are integrated very early in visual processing. Furthermore, V1 generates a distributed representation through low response selectivity that avoids the curse of dimensionality by using separable joint tuning functions.

Precision of spike trains in primate retinal ganglion cells

Recent studies have revealed striking precision in the spike trains of retinal ganglion cells in several species and suggested that this precision could be an important aspect of visual signaling. However, the precision of spike trains has not yet been described in primate retina. The spike time and count variability of parasol (magnocellular-projecting) retinal ganglion cells was examined in isolated macaque monkey retinas stimulated with repeated presentations of high contrast, spatially uniform intensity modulation. At the onset of clearly delineated periods of firing, retinal ganglion cells fired spikes time-locked to the stimulus with a variability across trials as low as 1 ms. Spike count variance across trials was much lower than the mean and sometimes approached the minimum variance possible with discrete counts, inconsistent with Poisson statistics expected from independently generated spikes. Spike time and count variability decreased systematically with stimulus strength. These findings were consistent with a model in which firing probability was determined by a stimulus-driven free firing rate modulated by a recovery function representing the action potential absolute and relative refractory period.

Phosphene threshold as a function of contrast of external visual stimuli

Transcranial magnetic stimulation (TMS) of the occipital lobe is frequently used to induce visual percepts by direct stimulation of visual cortex. The threshold magnetic field strength necessary to elicit a visual percept is often regarded as a measure of electrical excitability of visual cortex. Using single-pulse TMS during visual motion stimulus presentation, we investigated the relationship between different degrees of visual cortical preactivation and cortical phosphene threshold (PT). The two possible, mutually exclusive, predictions on the outcome of this experiment were that a) PT increases with stronger preactivation because of a decrease in the signal-to-noise ratio, or b) that PT decreases with increased preactivation because of the increase in neuronal response towards some threshold. PTs for single-pulse stimulation of the occipital lobe were determined for eight subjects while they passively viewed a horizontally drifting luminance-modulated sinewave grating. Gratings used were of four different luminance contrasts while the spatial and temporal frequencies remained constant. PTs were shown to increase significantly as the background grating increased in contrast. These results suggest that the neural activity underlying the perception of a phosphene can be considered a type of signal that can be partially masked by another signal, in this case the visual cortical activation produced by passive viewing of drifting gratings.

From another angle: Differences in cortical coding between fine and coarse discrimination of orientation

We measured the information available for orientation discrimination from metric distances for 24 cells in area 17 of cats that were paralyzed and anesthetized with Propofol and N(2)O. The metric distance information confirms fundamental coding differences for discrimination between fine (<10 degrees ) and coarse (>10 degrees ) orientation differences. The information for discriminating larger orientation differences is contained mainly in the firing rate, with minor enhancements from the coarse (30-70 ms) temporal structure in the firing rate. Both precise spike timing (9.2 ms) and intervals (6.8 ms) sustained over the stimulus presentation provide information for fine discrimination of orientation, where almost no reliable information is provided by the spike count. We compare and confirm the results (using the same data set) to vector distances based on classification theory. The results support a dynamic spiking mechanism where coordinated activity could provide fast and reliable information about detailed angle and/or direction information in the region of the preferred orientation.

Neural noise and movement-related codes in the macaque supplementary motor area

We analyzed the variability of spike counts and the coding capacity of simultaneously recorded pairs of neurons in the macaque supplementary motor area (SMA). We analyzed the mean-variance functions for single neurons, as well as signal and noise correlations between pairs of neurons. All three statistics showed a strong dependence on the bin width chosen for analysis. Changes in the correlation structure of single neuron spike trains over different bin sizes affected the mean-variance function, and signal and noise correlations between pairs of neurons were much smaller at small bin widths, increasing monotonically with the width of the bin. Analyses in the frequency domain showed that the noise between pairs of neurons, on average, was most strongly correlated at low frequencies, which explained the increase in noise correlation with increasing bin width. The coding performance was analyzed to determine whether the temporal precision of spike arrival times and the interactions within and between neurons could improve the prediction of the upcoming movement. We found that in approximately 62% of neuron pairs, the arrival times of spikes at a resolution between 66 and 40 msec carried more information than spike counts in a 200 msec bin. In addition, in 19% of neuron pairs, inclusion of within (11%)- or between-neuron (8%) correlations in spike trains improved decoding accuracy. These results suggest that in some SMA neurons elements of the spatiotemporal pattern of activity may be relevant for neural coding.

Neural coding of spatial phase in V1 of the macaque monkey

We examine the responses of single neurons and pairs of neurons, simultaneously recorded with a single tetrode in the primary visual cortex of the anesthetized macaque monkey, to transient presentations of stationary gratings of varying spatial phase. Such simultaneously recorded neurons tended to have similar tuning to the phase of the grating. To determine the response features that reliably discriminate these stimuli, we use the metric-space approach extended to pairs of neurons. We find that paying attention to the times of individual spikes, at a resolution of approximately 30 ms, and keeping track of which neuron fires which spike rather than just the summed local activity contribute substantially to phase coding. The contribution is both quantitative (increasing the fidelity of phase coding) and qualitative (enabling a 2-dimensional "response space" that corresponds to the spatial phase cycle). We use a novel approach, the extraction of "temporal profiles" from the metric space analysis, to interpret and compare temporal coding across neurons. Temporal profiles were remarkably consistent across a large subset of neurons. This consistency indicates that simple mechanisms (e.g., comparing the size of the transient and sustained components of the response) allow the temporal contribution to phase coding to be decoded.

Spatial organization of receptive fields of V1 neurons of alert monkeys: comparison with responses to gratings

We studied the spatial organization of receptive fields and the responses to gratings of neurons in parafoveal V1 of alert monkeys. Activating regions (ARs) of 228 cells were mapped with increment and decrement bars while compensating for fixational eye movements. For cells with two or more ARs, the overlap between ARs responsive to increments (INC) and ARs responsive to decrements (DEC) was characterized by a quantitative overlap index (OI). The distribution of overlap indices was bimodal. The larger group (78% of cells) was composed of complex cells with strongly overlapping ARs (OI >/= 0.5). The smaller group (14%) was composed of simple cells with minimal spatial overlap of ARs (OI </= 0.3). Simple cells were preferentially located in layers dominated by the magnocellular pathway. A third group of neurons, the monocontrast cells (8%), responded only to one sign of contrast and had more sustained responses to flashed stimuli than other cells. One hundred fourteen neurons were also studied with drifting sinusoidal gratings of various spatial frequencies and window widths. For complex cells, the relative modulation (RM, the ratio of the 1st harmonic to the mean firing rate), ranged from 0.6 +/- 0.4 to 1.1 +/- 0.5 (mean +/- SD), depending on the stimulus conditions and the mode of correction for eye movements. RM was not correlated with the degree of overlap of ARs, indicating that the spatial organization of receptive fields cannot reliably be predicted from RM values. In fact, a subset of complex cells had RM > 1, the traditional criterion for identifying simple cells. However, unlike simple cells, even those complex cells with high RM could exhibit diverse nonlinear responses when the spatial frequency or window size was changed. Furthermore, the responses of complex cells to counterphase gratings were predominantly nonlinear even harmonics. These results show that RM is not a robust test of linearity. Our results indicate that complex cells are the most frequently encountered neurons in primate V1, and their behavior needs to receive more emphasis in models of visual function.

Detection and discrimination of relative spatial phase by V1 neurons

Edge-like and line-like features result from spatial phase congruence, the local phase agreement between harmonic components of a spatial waveform. Psychophysical observations and models of early visual processing suggest that human visual feature detectors are specialized for edge-like and line-like phase congruence. To test whether primary visual cortex (V1) neurons account for such specificity, we made tetrode recordings in anesthetized macaque monkeys. Stimuli were drifting equal-energy compound gratings composed of four sinusoidal components. Eight congruence phases (one-dimensional features) were tested, including line-like and edge-like waveforms. Many of the 137 single V1 neurons (recorded at 45 sites) could reliably signal phase congruence by any of several response measures. Across neurons, the preferred spatial feature had only a modest bias for line-like waveforms. Information-theoretic analysis showed that congruence phase was temporally encoded in the frequency band present in the stimuli. The most sensitive neurons had feature discrimination thresholds that approached psychophysical levels, but typical neurons were substantially less sensitive. In single V1 neurons, feature discrimination exhibited various dependences on the congruence phase of the reference waveform. Simple cells were over-represented among the most sensitive neurons and on average carried twice as much feature information as complex cells. However, the distribution of the indices of optimal tuning and discrimination of relative phase was indistinguishable in simple and complex cells. Our results suggest that phase-sensitive pooling of responses is required to account for human psychophysical performance, although variation in feature selectivity among nearby neurons is considerable.

Visual cortex neurons of monkeys and cats: temporal dynamics of the contrast response function

Cortical neurons display two fundamental nonlinear response characteristics: contrast-set gain control (also termed contrast normalization) and response expansion (also termed half-squaring). These nonlinearities could play an important role in forming and maintaining stimulus selectivity during natural viewing, but only if they operate well within the time frame of a single fixation. To analyze the temporal dynamics of these nonlinearities, we measured the responses of individual neurons, recorded from the primary visual cortex of monkeys and cats, as a function of the contrast of transient stationary gratings that were presented for a brief interval (200 ms). We then examined 1) the temporal response profile (i.e., the post stimulus time histogram) as a function of contrast and 2) the contrast response function throughout the course of the temporal response. We found that the shape and complexity of the temporal response profile varies considerably from cell to cell. However, within a given cell, the shape remains relatively invariant as a function of contrast and appears to be simply scaled and shifted. Stated quantitatively, approximately 95% of the variation in the temporal responses as a function of contrast could be accounted for by scaling and shifting the average poststimulus time histogram. Equivalently, we found that the overall shape of the contrast response function (measured every 2 ms) remains relatively invariant from the onset through the entire temporal response. Further, the contrast-set gain control and the response expansion are fully expressed within the first 10 ms after the onset of the response. Stated quantitatively, the same, scaled Naka-Rushton equation (with the same half-saturation contrast and expansive response exponent) provides a good fit to the contrast response function from the first 10 ms through the last 10 ms of the temporal response. Based upon these measurements, it appears as though the two nonlinear properties, contrast-set gain control and response expansion, are present in full strength, virtually instantaneously, at the onset of the response. This observation suggests that response expansion and contrast-set gain control can influence the performance of visual cortex neurons very early in a single fixation, based on the contrast within that fixation. In the DISCUSSION, we consider the implications of the results within the context of 1) slower types of contrast gain control, 2) discrimination performance, 3) drifting steady-state measurements, 4) functional models that incorporate response expansion and contrast normalization, and 5) structural models of the biochemical and biophysical neural mechanisms.

Nonlinear spectrotemporal sound analysis by neurons in the auditory midbrain

The auditory system of humans and animals must process information from sounds that dynamically vary along multiple stimulus dimensions, including time, frequency, and intensity. Therefore, to understand neuronal mechanisms underlying acoustic processing in the central auditory pathway, it is essential to characterize how spectral and temporal acoustic dimensions are jointly processed by the brain. We use acoustic signals with a structurally rich time-varying spectrum to study linear and nonlinear spectrotemporal interactions in the central nucleus of the inferior colliculus (ICC). Our stimuli, the dynamic moving ripple (DMR) and ripple noise (RN), allow us to systematically characterize response attributes with the spectrotemporal receptive field (STRF) methods to a rich and dynamic stimulus ensemble. Theoretically, we expect that STRFs derived with DMR and RN would be identical for a linear integrating neuron, and we find that approximately 60% of ICC neurons meet this basic requirement. We find that the remaining neurons are distinctly nonlinear; these could either respond selectively to DMR or produce no STRFs despite selective activation to spectrotemporal acoustic attributes. Our findings delineate rules for spectrotemporal integration in the ICC that cannot be accounted for by conventional linear-energy integration models.

A unified approach to the study of temporal, correlational, and rate coding

We demonstrate that the information contained in the spike occurrence times of a population of neurons can be broken up into a series of terms, each reflecting something about potential coding mechanisms. This is possible in the coding regime in which few spikes are emitted in the relevant time window. This approach allows us to study the additional information contributed by spike timing beyond that present in the spike counts and to examine the contributions to the whole information of different statistical properties of spike trains, such as firing rates and correlation functions. It thus forms the basis for a new quantitative procedure for analyzing simultaneous multiple neuron recordings and provides theoretical constraints on neural coding strategies. We find a transition between two coding regimes, depending on the size of the relevant observation timescale. For time windows shorter than the timescale of the stimulus-induced response fluctuations, there exists a spike count coding phase, in which the purely temporal information is of third order in time. For time windows much longer than the characteristic timescale, there can be additional timing information of first order, leading to a temporal coding phase in which timing information may affect the instantaneous information rate. In this new framework, we study the relative contributions of the dynamic firing rate and correlation variables to the full temporal information, the interaction of signal and noise correlations in temporal coding, synergy between spikes and between cells, and the effect of refractoriness. We illustrate the utility of the technique by analyzing a few cells from the rat barrel cortex.

The role of spike timing in the coding of stimulus location in rat somatosensory cortex

Although the timing of single spikes is known to code for time-varying features of a sensory stimulus, it remains unclear whether time is also exploited in the neuronal coding of the spatial structure of the environment, where nontemporal stimulus features are fundamental. This report demonstrates that, in the whisker representation of rat cortex, precise spike timing of single neurons increases the information transmitted about stimulus location by 44%, compared to that transmitted only by the total number of spikes. Crucial to this code is the timing of the first spike after whisker movement. Complex, single neuron spike patterns play a smaller, synergistic role. Timing permits very few spikes to transmit high quantities of information about a behaviorally significant, spatial stimulus.

Temporal coding of contrast in primary visual cortex: when, what, and why

How do neurons in the primary visual cortex (V1) encode the contrast of a visual stimulus? In this paper, the information that V1 responses convey about the contrast of static visual stimuli is explicitly calculated. These responses often contain several easily distinguished temporal components, which will be called latency, transient, tonic, and off. Calculating the information about contrast conveyed in each component and in groups of components makes it possible to delineate aspects of the temporal structure that may be relevant for contrast encoding. The results indicate that as much or more contrast-related information is encoded into the temporal structure of spike train responses as into the firing rate and that the temporally coded information is manifested most strongly in the latency to response onset. Transient, tonic, and off responses contribute relatively little. The results also reveal that temporal coding is important for distinguishing subtle contrast differences, whereas firing rates are useful for gross discrimination. This suggests that the temporal structure of neurons' responses may extend the dynamic range for contrast encoding in the primate visual system.

Formal and attribute-specific information in primary visual cortex

We estimate the rates at which neurons in the primary visual cortex (V1) of anesthetized macaque monkeys transmit stimulus-related information in response to three types of visual stimulus. The stimuli-randomly modulated checkerboard patterns, stationary sinusoidal gratings, and drifting sinusoidal gratings-have very different spatiotemporal structures. We obtain the overall rate of information transmission, which we call formal information, by a direct method. We find the highest information rates in the responses of simple cells to drifting gratings (median: 10.3 bits/s, 0.92 bits/spike); responses to randomly modulated stimuli and stationary gratings transmit information at significantly lower rates. In general, simple cells transmit information at higher rates, and over a larger range, than do complex cells. Thus in the responses of V1 neurons, stimuli that are rapidly modulated do not necessarily evoke higher information rates, as might be the case with motion-sensitive neurons in area MT. By an extension of the direct method, we parse the formal information into attribute-specific components, which provide estimates of the information transmitted about contrast and spatiotemporal pattern. We find that contrast-specific information rates vary across neurons-about 0.3 to 2.1 bits/s or 0.05 to 0.22 bits/spike-but depend little on stimulus type. Spatiotemporal pattern-specific information rates, however, depend strongly on the type of stimulus and neuron (simple or complex). The remaining information rate, typically between 10 and 32% of the formal information rate for each neuron, cannot be unambiguously assigned to either contrast or spatiotemporal pattern. This indicates that some information concerning these two stimulus attributes is confounded in the responses of single neurons in V1. A model that considers a simple cell to consist of a linear spatiotemporal filter followed by a static rectifier predicts higher information rates than are found in real neurons and completely fails to replicate the performance of real cells in generating the confounded information.

How the brain uses time to represent and process visual information(1)

Information theory provides a theoretical framework for addressing fundamental questions concerning the nature of neural codes. Harnessing its power is not straightforward, because of the differences between mathematical abstractions and laboratory reality. We describe an approach to the analysis of neural codes that seeks to identify the informative features of neural responses, rather than to estimate the information content of neural responses per se. Our analysis, applied to neurons in primary visual cortex (V1), demonstrates that the informative precision of spike times varies with the stimulus modality being represented. Contrast is represented by spike times on the shortest time scale, and different kinds of pattern information are represented on longer time scales. The interspike interval distribution has a structure that is unanticipated from the firing rate. The significance of this structure is not that it contains additional information, but rather, that it may provide a means for simple synaptic mechanisms to decode the information that is multiplexed within a spike train. Extensions of this analysis to the simultaneous responses of pairs of neurons indicate that neighboring neurons convey largely independent information, if the decoding process is sensitive to the neuron of origin and not just the average firing rate. In summary, stimulus-related information is encoded into the precise times of spikes fired by V1 neurons. Much of this information would be obscured if individual spikes were merely taken to be estimators of the firing rate. Additional information would be lost by averaging across the responses of neurons in a local population. We propose that synaptic mechanisms sensitive to interspike intervals and dendritic processing beyond simple summation exist at least in part to enable the brain to take advantage of this extra information.

Influence of temporal correlation of synaptic input on the rate and variability of firing in neurons

The spike trains that transmit information between neurons are stochastic. We used the theory of random point processes and simulation methods to investigate the influence of temporal correlation of synaptic input current on firing statistics. The theory accounts for two sources for temporal correlation: synchrony between spikes in presynaptic input trains and the unitary synaptic current time course. Simulations show that slow temporal correlation of synaptic input leads to high variability in firing. In a leaky integrate-and-fire neuron model with spike afterhyperpolarization the theory accurately predicts the firing rate when the spike threshold is higher than two standard deviations of the membrane potential fluctuations. For lower thresholds the spike afterhyperpolarization reduces the firing rate below the theory's predicted level when the synaptic correlation decays rapidly. If the synaptic correlation decays slower than the spike afterhyperpolarization, spike bursts can occur during single broad peaks of input fluctuations, increasing the firing rate over the prediction. Spike bursts lead to a coefficient of variation for the interspike intervals that can exceed one, suggesting an explanation of high coefficient of variation for interspike intervals observed in vivo.

Interspike intervals, receptive fields, and information encoding in primary visual cortex

In the primate primary visual cortex (V1), the significance of individual action potentials has been difficult to determine, particularly in light of the considerable trial-to-trial variability of responses to visual stimuli. We show here that the information conveyed by an action potential depends on the duration of the immediately preceding interspike interval (ISI). The interspike intervals can be grouped into several different classes on the basis of reproducible features in the interspike interval histograms. Spikes in different classes bear different relationships to the visual stimulus, both qualitatively (in terms of the average stimulus preceding each spike) and quantitatively (in terms of the amount of information encoded per spike and per second). Spikes preceded by very short intervals (3 msec or less) convey information most efficiently and contribute disproportionately to the overall receptive-field properties of the neuron. Overall, V1 neurons can transmit between 5 and 30 bits of information per second in response to rapidly varying, pseudorandom stimuli, with an efficiency of approximately 25%. Although some (but not all) of our results would be expected from neurons that use a firing-rate code to transmit information, the evidence suggests that visual neurons are well equipped to decode stimulus-related information on the basis of relative spike timing and ISI duration.

Gating of local network signals appears as stimulus-dependent activity envelopes in striate cortex

Neuronal activity often is treated as a composition of a stimulus-driven component and a second component that corrupts the signal, adding or deleting spikes at random. Standard quantitative methods such as peristimulus histograms and Fourier analysis use stimulus-locked averaging to enhance detection of the driven component of neuronal responses and de-emphasize the "noise." However, neural activity also includes bursts, oscillations, and other episodic events that standard averaging methods overlook. If this activity is stimulus independent, it can be characterized by standard power spectral analysis (or autocorrelation). But activity that is excited by (but not temporally locked to) the visual stimulus cannot be characterized by averaging or standard spectral analysis. Phase-locked spectral analysis (PLSA) is a new method that examines this "residual" activity-the difference between the individual responses to each cycle of a periodic stimulus and their average. With PLSA, residual activity is characterized in terms of temporal envelopes and their carriers. Previously, PLSA demonstrated broadband interactions between periodic visual stimuli and fluctuations in the local field potential of macaque V1. In the present study, single-unit responses (SUA) from parafoveal V1 in anesthetized macaque monkey are examined with this technique. Recordings were made from 21 neurons, 6 of which were recorded in pairs along with multiunit activity (MUA) from separate electrodes and 8 of which were recorded along with MUA from the same electrode. PLSA was applied to responses to preferred (orientation, direction, and spatial frequency) and nonpreferred drifting gratings. For preferred stimuli, all cells demonstrated broadband (1-10 Hz and higher) residual activity that waxed and waned with the stimulus cycle, suggesting that changes in the residual activity are introduced routinely by visual stimulation. Moreover, some reconstructed envelopes indicate that the residual activity was sharply gated by the stimulus cycle. Oscillations occasionally were seen in the power spectrum of single units. Phase-locked cross-spectra were determined for 3 SUA/SUA pairs and 11 SUA/MUA pairs. Residual activity in the cross-spectra was generally much less than the residual activity determined separately from each neuron. The reduction in the residual activity in the cross-spectra suggests that nearby neurons may gate inputs from distinct and relatively independent neuronal subpopulations that together generate the background rhythms of striate cortex.

Spike timing in the mammalian visual system

Evidence is accumulating for the existence of mechanisms that create and detect synchrony among action potentials on short time scales both within and between neurons. Progress is most rapid in the retina, the lateral geniculate nucleus, and cortical slices, where signal flow is better understood or more manipulable. The debate over the functional relevance of spike timing in cortex has gained substance from new computational models but remains unresolved.

Robust temporal coding of contrast by V1 neurons for transient but not for steady-state stimuli

We show that spike timing adds to the information content of spike trains for transiently presented stimuli but not for comparable steady-state stimuli, even if the latter elicit transient responses. Contrast responses of 22 single neurons in macaque V1 to periodic presentation of steady-state stimuli (drifting sinusoidal gratings) and transient stimuli (drifting edges) of optimal spatiotemporal parameters were recorded extracellularly. The responses were analyzed for contrast-dependent clustering in spaces determined by metrics sensitive to the temporal structure of spike trains. Two types of metrics, cost-based spike time metrics and metrics based on Fourier harmonics of the response, were used. With both families of metrics, temporal coding of contrast is lacking in responses to drifting sinusoidal gratings of most (simple and complex) V1 neurons. However, two-thirds of all neurons, mostly complex cells, displayed significant temporal coding of contrast for edge stimuli. The Fourier metrics indicated that different response harmonics are partially independent, and their combined use increases information about transient stimuli. Our results demonstrate the importance of stimulus transience for temporal coding. This finding is significant for natural vision because moving edges, which are present in moving object boundaries, and saccades induce transients. We think that an abrupt change in the adapted state of the local visual circuitry triggers the temporal structuring of spike trains in V1 neurons.