Noise shaping in populations of coupled model neurons

A Spiking Neuron and Population Model Based on the Growth Transform Dynamical System

In neuromorphic engineering, neural populations are generally modeled in a bottom-up manner, where individual neuron models are connected through synapses to form large-scale spiking networks. Alternatively, a top-down approach treats the process of spike generation and neural representation of excitation in the context of minimizing some measure of network energy. However, these approaches usually define the energy functional in terms of some statistical measure of spiking activity (ex. firing rates), which does not allow independent control and optimization of neurodynamical parameters. In this paper, we introduce a new spiking neuron and population model where the dynamical and spiking responses of neurons can be derived directly from a network objective or energy functional of continuous-valued neural variables like the membrane potential. The key advantage of the model is that it allows for independent control over three neuro-dynamical properties: (a) control over the steady-state population dynamics that encodes the minimum of an exact network energy functional; (b) control over the shape of the action potentials generated by individual neurons in the network without affecting the network minimum; and (c) control over spiking statistics and transient population dynamics without affecting the network minimum or the shape of action potentials. At the core of the proposed model are different variants of Growth Transform dynamical systems that produce stable and interpretable population dynamics, irrespective of the network size and the type of neuronal connectivity (inhibitory or excitatory). In this paper, we present several examples where the proposed model has been configured to produce different types of single-neuron dynamics as well as population dynamics. In one such example, the network is shown to adapt such that it encodes the steady-state solution using a reduced number of spikes upon convergence to the optimal solution. In this paper, we use this network to construct a spiking associative memory that uses fewer spikes compared to conventional architectures, while maintaining high recall accuracy at high memory loads.

How single neuron properties shape chaotic dynamics and signal transmission in random neural networks

While most models of randomly connected neural networks assume single-neuron models with simple dynamics, neurons in the brain exhibit complex intrinsic dynamics over multiple timescales. We analyze how the dynamical properties of single neurons and recurrent connections interact to shape the effective dynamics in large randomly connected networks. A novel dynamical mean-field theory for strongly connected networks of multi-dimensional rate neurons shows that the power spectrum of the network activity in the chaotic phase emerges from a nonlinear sharpening of the frequency response function of single neurons. For the case of two-dimensional rate neurons with strong adaptation, we find that the network exhibits a state of "resonant chaos", characterized by robust, narrow-band stochastic oscillations. The coherence of stochastic oscillations is maximal at the onset of chaos and their correlation time scales with the adaptation timescale of single units. Surprisingly, the resonance frequency can be predicted from the properties of isolated neurons, even in the presence of heterogeneity in the adaptation parameters. In the presence of these internally-generated chaotic fluctuations, the transmission of weak, low-frequency signals is strongly enhanced by adaptation, whereas signal transmission is not influenced by adaptation in the non-chaotic regime. Our theoretical framework can be applied to other mechanisms at the level of single neurons, such as synaptic filtering, refractoriness or spike synchronization. These results advance our understanding of the interaction between the dynamics of single units and recurrent connectivity, which is a fundamental step toward the description of biologically realistic neural networks.

Interspike interval correlations in networks of inhibitory integrate-and-fire neurons

We study temporal correlations of interspike intervals, quantified by the network-averaged serial correlation coefficient (SCC), in networks of both current- and conductance-based purely inhibitory integrate-and-fire neurons. Numerical simulations reveal transitions to negative SCCs at intermediate values of bias current drive and network size. As bias drive and network size are increased past these values, the SCC returns to zero. The SCC is maximally negative at an intermediate value of the network oscillation strength. The dependence of the SCC on two canonical schemes for synaptic connectivity is studied, and it is shown that the results occur robustly in both schemes. For conductance-based synapses, the SCC becomes negative at the onset of both a fast and slow coherent network oscillation. We then show by means of offline simulations using prerecorded network activity that a neuron's SCC is highly sensitive to its number of presynaptic inputs. Finally, we devise a noise-reduced diffusion approximation for current-based networks that accounts for the observed temporal correlation transitions.

Theory of nonstationary Hawkes processes

We expand the theory of Hawkes processes to the nonstationary case, in which the mutually exciting point processes receive time-dependent inputs. We derive an analytical expression for the time-dependent correlations, which can be applied to networks with arbitrary connectivity, and inputs with arbitrary statistics. The expression shows how the network correlations are determined by the interplay between the network topology, the transfer functions relating units within the network, and the pattern and statistics of the external inputs. We illustrate the correlation structure using several examples in which neural network dynamics are modeled as a Hawkes process. In particular, we focus on the interplay between internally and externally generated oscillations and their signatures in the spike and rate correlation functions.

The effect of positive interspike interval correlations on neuronal information transmission

Experimentally it is known that some neurons encode preferentially information about low-frequency (slow) components of a time-dependent stimulus while others prefer intermediate or high-frequency (fast) components. Accordingly, neurons can be categorized as low-pass, band-pass or high-pass information filters. Mechanisms of information filtering at the cellular and the network levels have been suggested. Here we propose yet another mechanism, based on noise shaping due to spontaneous non-renewal spiking statistics. We compare two integrate-and-fire models with threshold noise that differ solely in their interspike interval (ISI) correlations: the renewal model generates independent ISIs, whereas the non-renewal model exhibits positive correlations between adjacent ISIs. For these simplified neuron models we analytically calculate ISI density and power spectrum of the spontaneous spike train as well as approximations for input-output cross-spectrum and spike-train power spectrum in the presence of a broad-band Gaussian stimulus. This yields the spectral coherence, an approximate frequency-resolved measure of information transmission. We demonstrate that for low spiking variability the renewal model acts as a low-pass filter of information (coherence has a global maximum at zero frequency), whereas the non-renewal model displays a pronounced maximum of the coherence at non-vanishing frequency and thus can be regarded as a band-pass filter of information.

A Markov model for the temporal dynamics of balanced random networks of finite size

The balanced state of recurrent networks of excitatory and inhibitory spiking neurons is characterized by fluctuations of population activity about an attractive fixed point. Numerical simulations show that these dynamics are essentially nonlinear, and the intrinsic noise (self-generated fluctuations) in networks of finite size is state-dependent. Therefore, stochastic differential equations with additive noise of fixed amplitude cannot provide an adequate description of the stochastic dynamics. The noise model should, rather, result from a self-consistent description of the network dynamics. Here, we consider a two-state Markovian neuron model, where spikes correspond to transitions from the active state to the refractory state. Excitatory and inhibitory input to this neuron affects the transition rates between the two states. The corresponding nonlinear dependencies can be identified directly from numerical simulations of networks of leaky integrate-and-fire neurons, discretized at a time resolution in the sub-millisecond range. Deterministic mean-field equations, and a noise component that depends on the dynamic state of the network, are obtained from this model. The resulting stochastic model reflects the behavior observed in numerical simulations quite well, irrespective of the size of the network. In particular, a strong temporal correlation between the two populations, a hallmark of the balanced state in random recurrent networks, are well represented by our model. Numerical simulations of such networks show that a log-normal distribution of short-term spike counts is a property of balanced random networks with fixed in-degree that has not been considered before, and our model shares this statistical property. Furthermore, the reconstruction of the flow from simulated time series suggests that the mean-field dynamics of finite-size networks are essentially of Wilson-Cowan type. We expect that this novel nonlinear stochastic model of the interaction between neuronal populations also opens new doors to analyze the joint dynamics of multiple interacting networks.

Emergence of dynamical complexity related to human heart rate variability

We apply the refined composite multiscale entropy (MSE) method to a one-dimensional directed small-world network composed of nodes whose states are binary and whose dynamics obey the majority rule. We find that the resulting fluctuating signal becomes dynamically complex. This dynamical complexity is caused (i) by the presence of both short-range connections and long-range shortcuts and (ii) by how well the system can adapt to the noisy environment. By tuning the adaptability of the environment and the long-range shortcuts we can increase or decrease the dynamical complexity, thereby modeling trends found in the MSE of a healthy human heart rate in different physiological states. When the shortcut and adaptability values increase, the complexity in the system dynamics becomes uncorrelated.

Fluctuations and information filtering in coupled populations of spiking neurons with adaptation

Finite-sized populations of spiking elements are fundamental to brain function but also are used in many areas of physics. Here we present a theory of the dynamics of finite-sized populations of spiking units, based on a quasirenewal description of neurons with adaptation. We derive an integral equation with colored noise that governs the stochastic dynamics of the population activity in response to time-dependent stimulation and calculate the spectral density in the asynchronous state. We show that systems of coupled populations with adaptation can generate a frequency band in which sensory information is preferentially encoded. The theory is applicable to fully as well as randomly connected networks and to leaky integrate-and-fire as well as to generalized spiking neurons with adaptation on multiple time scales.

Temporal whitening by power-law adaptation in neocortical neurons

Spike-frequency adaptation (SFA) is widespread in the CNS, but its function remains unclear. In neocortical pyramidal neurons, adaptation manifests itself by an increase in the firing threshold and by adaptation currents triggered after each spike. Combining electrophysiological recordings in mice with modeling, we found that these adaptation processes lasted for more than 20 s and decayed over multiple timescales according to a power law. The power-law decay associated with adaptation mirrored and canceled the temporal correlations of input current received in vivo at the somata of layer 2/3 somatosensory pyramidal neurons. These findings suggest that, in the cortex, SFA causes temporal decorrelation of output spikes (temporal whitening), an energy-efficient coding procedure that, at high signal-to-noise ratio, improves the information transfer.

Enhancing the resolution of a sensor via negative correlation: a biologically inspired approach

We demonstrate that a neuronal system, underpinned by "fire-then-reset" dynamics, can display an enhanced resolution R~T(ob)(-1) where T(ob) is the observation time of the measurement; this occurs when the interspike intervals are negatively correlated and T(ob)<Δ/ε, where ε is a parameter characterizing the level of correlation between interspike intervals and Δ is the average interspike interval. We also show that by introducing negative correlations into the time domain response of a nonlinear dynamical sensor it is possible to replicate this enhanced scaling of the resolution. Thus, we demonstrate the potential for designing a novel class of biomimetic sensors that afford improved signal resolution by functionally utilizing negative correlations.

Detection of M-sequences from spike sequence in neuronal networks

In circuit theory, it is well known that a linear feedback shift register (LFSR) circuit generates pseudorandom bit sequences (PRBS), including an M-sequence with the maximum period of length. In this study, we tried to detect M-sequences known as a pseudorandom sequence generated by the LFSR circuit from time series patterns of stimulated action potentials. Stimulated action potentials were recorded from dissociated cultures of hippocampal neurons grown on a multielectrode array. We could find several M-sequences from a 3-stage LFSR circuit (M3). These results show the possibility of assembling LFSR circuits or its equivalent ones in a neuronal network. However, since the M3 pattern was composed of only four spike intervals, the possibility of an accidental detection was not zero. Then, we detected M-sequences from random spike sequences which were not generated from an LFSR circuit and compare the result with the number of M-sequences from the originally observed raster data. As a result, a significant difference was confirmed: a greater number of “0–1” reversed the 3-stage M-sequences occurred than would have accidentally be detected. This result suggests that some LFSR equivalent circuits are assembled in neuronal networks.

Neuronal model with distributed delay: analysis and simulation study for gamma distribution memory kernel

A single neuronal model incorporating distributed delay (memory)is proposed. The stochastic model has been formulated as a Stochastic Integro-Differential Equation (SIDE) which results in the underlying process being non-Markovian. A detailed analysis of the model when the distributed delay kernel has exponential form (weak delay) has been carried out. The selection of exponential kernel has enabled the transformation of the non-Markovian model to a Markovian model in an extended state space. For the study of First Passage Time (FPT) with exponential delay kernel, the model has been transformed to a system of coupled Stochastic Differential Equations (SDEs) in two-dimensional state space. Simulation studies of the SDEs provide insight into the effect of weak delay kernel on the Inter-Spike Interval(ISI) distribution. A measure based on Jensen-Shannon divergence is proposed which can be used to make a choice between two competing models viz. distributed delay model vis-á-vis LIF model. An interesting feature of the model is that the behavior of (CV(t))((ISI)) (Coefficient of Variation) of the ISI distribution with respect to memory kernel time constant parameter η reveals that neuron can switch from a bursting state to non-bursting state as the noise intensity parameter changes. The membrane potential exhibits decaying auto-correlation structure with or without damped oscillatory behavior depending on the choice of parameters. This behavior is in agreement with empirically observed pattern of spike count in a fixed time window. The power spectral density derived from the auto-correlation function is found to exhibit single and double peaks. The model is also examined for the case of strong delay with memory kernel having the form of Gamma distribution. In contrast to fast decay of damped oscillations of the ISI distribution for the model with weak delay kernel, the decay of damped oscillations is found to be slower for the model with strong delay kernel.

Nonrenewal spike train statistics: causes and functional consequences on neural coding

Many neurons display significant patterning in their spike trains (e.g. oscillations, bursting), and there is accumulating evidence that information is contained in these patterns. In many cases, this patterning is caused by intrinsic mechanisms rather than external signals. In this review, we focus on spiking activity that displays nonrenewal statistics (i.e. memory that persists from one firing to the next). Such statistics are seen in both peripheral and central neurons and appear to be ubiquitous in the CNS. We review the principal mechanisms that can give rise to nonrenewal spike train statistics. These are separated into intrinsic mechanisms such as relative refractoriness and network mechanisms such as coupling with delayed inhibitory feedback. Next, we focus on the functional roles for nonrenewal spike train statistics. These can either increase or decrease information transmission. We also focus on how such statistics can give rise to an optimal integration timescale at which spike train variability is minimal and how this might be exploited by sensory systems to maximize the detection of weak signals. We finish by pointing out some interesting future directions for research in this area. In particular, we explore the interesting possibility that synaptic dynamics might be matched with the nonrenewal spiking statistics of presynaptic spike trains in order to further improve information transmission.

Enhanced stimulus encoding capabilities with spectral selectivity in inhibitory circuits by STDP

The ability to encode and transmit a signal is an essential property that must demonstrate many neuronal circuits in sensory areas in addition to any processing they may provide. It is known that an appropriate level of lateral inhibition, as observed in these areas, can significantly improve the encoding ability of a population of neurons. We show here a homeostatic mechanism by which a spike-timing-dependent plasticity (STDP) rule with a symmetric timing window (swSTDP) spontaneously drives the inhibitory coupling to a level that ensures accurate encoding in response to input signals within a certain frequency range. Interpreting these results mathematically, we find that this coupling level depends on the overlap of spectral information between stimulus and STDP window function. Generalization to arbitrary swSTDP and arbitrary stimuli reveals that the signals for which this improvement of encoding takes place can be finely selected on spectral criteria. We finally show that this spectral overlap principle holds for a variety of neuron types and network characteristics. The highly tunable frequency-power domain of efficiency of this mechanism, together with its ability to operate in very various neuronal contexts, suggest that it may be at work in most sensory areas.

Relative spike timing in stochastic oscillator networks of the Hermissenda eye

The role of relative spike timing on sensory coding and stochastic dynamics of small pulse-coupled oscillator networks is investigated physiologically and mathematically, based on the small biological eye network of the marine invertebrate Hermissenda. Without network interactions, the five inhibitory photoreceptors of the eye network exhibit quasi-regular rhythmic spiking; in contrast, within the active network, they display more irregular spiking but collective network rhythmicity. We investigate the source of this emergent network behavior first analyzing the role of relative input to spike-timing relationships in individual cells. We use a stochastic phase oscillator equation to model photoreceptor spike sequences in response to sequences of inhibitory current pulses. Although spike sequences can be complex and irregular in response to inputs, we show that spike timing is better predicted if relative timing of spikes to inputs is accounted for in the model. Further, we establish that greater noise levels in the model serve to destroy network phase-locked states that induce non-monotonic stimulus rate-coding, as predicted in Butson and Clark (J Neurophysiol 99:146-154, 2008a; J Neurophysiol 99:155-165, 2008b). Hence, rate-coding can function better in noisy spiking cells relative to non-noisy cells. We then study how relative input to spike-timing dynamics of single oscillators contribute to network-level dynamics. Relative timing interactions in the network sharpen the stimulus window that can trigger a spike, affecting stimulus encoding. Also, we derive analytical inter-spike interval distributions of cells in the model network, revealing that irregular Poisson-like spike emission and collective network rhythmicity are emergent properties of network dynamics, consistent with experimental observations. Our theoretical results generate experimental predictions about the nature of spike patterns in the Hermissenda eye.

Next-generation synthetic gene networks

Synthetic biology is focused on the rational construction of biological systems based on engineering principles. During the field's first decade of development, significant progress has been made in designing biological parts and assembling them into genetic circuits to achieve basic functionalities. These circuits have been used to construct proof-of-principle systems with promising results in industrial and medical applications. However, advances in synthetic biology have been limited by a lack of interoperable parts, techniques for dynamically probing biological systems and frameworks for the reliable construction and operation of complex, higher-order networks. As these challenges are addressed, synthetic biologists will be able to construct useful next-generation synthetic gene networks with real-world applications in medicine, biotechnology, bioremediation and bioenergy.

Noise Shaping in Neural Populations with Global Delayed Feedback

The interplay between intrinsic and network dynamics has been the focus of many investigations. Here we use a combination of theoretical and numerical approaches to study the effects of delayed global feedback on the information transmission properties of neural networks. Specifically, we compare networks of neurons that display intrinsic interspike interval correlations (nonrenewal) to networks that do not (renewal). We find that excitatory and inhibitory delays can tune information transmission by single neurons but not by the entire network. Most surprisingly, addition of a delay can change the dependence of the information on the coupling strength for renewal neurons and not for nonrenewal neurons. Our results show that intrinsic ISI correlations can have nontrivial interactions with network-induced phenomena.

Coupling regularizes individual units in noisy populations

The regularity of a noisy system can modulate in various ways. It is well known that coupling in a population can lower the variability of the entire network; the collective activity is more regular. Here, we show that diffusive (reciprocal) coupling of two simple Ornstein-Uhlenbeck (O-U) processes can regularize the individual, even when it is coupled to a noisier process. In cellular networks, the regularity of individual cells is important when a select few play a significant role. The regularizing effect of coupling surprisingly applies also to general nonlinear noisy oscillators. However, unlike with the O-U process, coupling-induced regularity is robust to different kinds of coupling. With two coupled noisy oscillators, we derive an asymptotic formula assuming weak noise and coupling for the variance of the period (i.e., spike times) that accurately captures this effect. Moreover, we find that reciprocal coupling can regularize the individual period of higher dimensional oscillators such as the Morris-Lecar and Brusselator models, even when coupled to noisier oscillators. Coupling can have a counterintuitive and beneficial effect on noisy systems. These results have implications for the role of connectivity with noisy oscillators and the modulation of variability of individual oscillators.

Electrical coupling promotes fidelity of responses in the networks of model neurons

We consider an integrate-and-fire element subject to randomly perturbed synaptic input and an electrically coupled ensemble of such elements. The latter is interpreted as either a model of electrically coupled population of neurons or a multicompartment model of a dendrite. Random fluctuations blur the input signal and cause false responses in the system dynamics. For instance, under the influence of noise, the system may respond with an action potential to a subthreshold stimulus. We show that the responses of the elements within the network are more reliable than the responses of the same elements in isolation. Specifically, we show that the variances of the stochastic processes generated by the coupled model can be made arbitrarily small (i.e., the network responses can be made arbitrarily accurate) by increasing the number of elements in the network and the strength of electrical coupling. Our results suggest that the organization of cells in electrically coupled groups on the network level, or the dendritic morphology on the cellular level, may be involved in the filtering noise and therefore may play an important role in the information processing mechanisms operating on the network or cellular level respectively.

Coherent stochastic oscillations enhance signal detection in spiking neurons

We study the effect of noisy oscillatory input on the signal discrimination by spontaneously firing neurons. Using analytically tractable model, we contrast signal detection in two situations: (i) when the neuron is driven by coherent oscillations and (ii) when the coherence of oscillations is destroyed. Analytical calculations revealed a region in the parameter space of the model where oscillations act to reduce the variability of neuronal firing and to enhance the discriminability of weak signals. These analytical results are employed to unveil a possible role of coherent oscillations in peripheral electrosensory system of paddlefish in improvement of detection of weak stimuli. The proposed mechanism may be relevant to a wide range of phenomena involving coherently driven oscillators.

Representing spike trains using constant sampling intervals

Sensory neurons encode external information by a series of times of action potentials, which is called a spike train. However, since it is a point process, it is hard to analyze. Here we propose a method for converting a spike train into a real-valued time series with a fixed sampling interval under the assumption of temporal codes. The proposed method yields time series that represent encoded signals. Especially when the method is applied to spike trains generated using integrate-and-fire models, the yielded time series look very similar to those of encoded information. The method works robustly even when a spike train is contaminated with noise. Since unlike filters it does not use its original signals for the conversion, the proposed method can be widely used for investigating spike train data in the real world.

Spike-rate coding and spike-time coding are affected oppositely by different adaptation mechanisms

Spike-frequency adaptation causes reduced spiking during prolonged stimulation, but the full impact of adaptation on neural coding is far more complex, especially if one takes into account the diversity of biophysical mechanisms mediating adaptation and the different ways in which neural information can be encoded. Here, we show that adaptation has opposite effects depending on the neural coding strategy and the biophysical mechanism responsible for adaptation. Under noisy conditions, calcium-activated K(+) current (I(AHP)) improved efficient spike-rate coding at the expense of spike-time coding by regularizing the spike train elicited by slow or constant inputs; noise power was increased at high frequencies but reduced at low frequencies, consistent with noise shaping that improves coding of low- frequency signals. In contrast, voltage-activated M-type K(+) current (I(M)) improved spike-time coding at the expense of spike-rate coding by stopping the neuron from spiking repetitively to slow inputs so that it could generate isolated, well timed spikes in response to fast inputs. Using dynamical systems analysis, we demonstrate how I(AHP) minimizes perturbation of the interspike interval caused by high- frequency noise, whereas I(M) minimizes disruption of spike-timing accuracy caused by repetitive spiking. The dichotomous outcomes are related directly to the distinct activation requirements for I(AHP) and I(M), which in turn dictate whether those currents mediate negative feedback onto spiking or membrane potential. Thus, based on their distinct activation properties, I(AHP) implements noise shaping that improves spike-rate coding of low-frequency signals, whereas I(M) implements high-pass filtering that improves spike-time coding of high- frequency signals.

Noise shaping in neural populations

Many neurons display intrinsic interspike interval correlations in their spike trains. However, the effects of such correlations on information transmission in neural populations are not well understood. We quantified signal processing using linear response theory supported by numerical simulations in networks composed of two different models: One model generates a renewal process where interspike intervals are not correlated while the other generates a nonrenewal process where subsequent interspike intervals are negatively correlated. Our results show that the fractional rate of increase in information rate as a function of network size and stimulus intensity is lower for the nonrenewal model than for the renewal one. We show that this is mostly due to the lower amount of effective noise in the nonrenewal model. We also show the surprising result that coupling has opposite effects in renewal and nonrenewal networks: Excitatory (inhibitory coupling) will decrease (increase) the information rate in renewal networks while inhibitory (excitatory coupling) will decrease (increase) the information rate in nonrenewal networks. We discuss these results and their applicability to other classes of excitable systems.

Testing a neural coding hypothesis using surrogate data

Determining how a particular neuron, or population of neurons, encodes information in their spike trains is not a trivial problem, because multiple coding schemes exist and are not necessarily mutually exclusive. Coding schemes generally fall into one of two broad categories, which we refer to as rate and temporal coding. In rate coding schemes, information is encoded in the variations of the average firing rate of the spike train. In contrast, in temporal coding schemes, information is encoded in the specific timing of the individual spikes that comprise the train. Here, we describe a method for testing the presence of temporal encoding of information. Suppose that a set of original spike trains is given. First, surrogate spike trains are generated by randomizing each of the original spike trains subject to the following constraints: the local average firing rate is approximately preserved, while the overall average firing rate and the distribution of primary interspike intervals are perfectly preserved. These constraints ensure that any rate coding of information present in the original spike trains is preserved in the members of the surrogate population. The null-hypothesis is rejected when additional information is found to be present in the original spike trains, implying that temporal coding is present. The method is validated using artificial data, and then demonstrated using real neuronal data.

Mechanisms of Noise-Induced Improvement in Light-Intensity Encoding inHermissendaPhotoreceptor Network

In a companion paper we showed that random channel and synaptic noise improve the ability of a biologically realistic, GENESIS-based computational model of the Hermissenda eye to encode light intensity. In this paper we explore mechanisms for noise-induced improvement by examining contextual spike-timing relationships among neurons in the photoreceptor network. In other systems, synaptically connected pairs of spiking cells can develop phase-locked spike-timing relationships at particular, well-defined frequencies. Consequently, domains of stability (DOS) emerge in which an increase in the frequency of inhibitory postsynaptic potentials can paradoxically increase, rather than decrease, the firing rate of the postsynaptic cell. We have extended this analysis to examine DOS as a function of noise amplitude in the exclusively inhibitory Hermissenda photoreceptor network. In noise-free simulations, DOS emerge at particular firing frequencies of type B and type A photoreceptors, thus producing a nonmonotonic relationship between their firing rates and light intensity. By contrast, in the noise-added conditions, an increase in noise amplitude leads to an increase in the variance of the interspike interval distribution for a given cell; in turn, this blocks the emergence of phase locking and DOS. These noise-induced changes enable the eye to better perform one of its basic tasks: encoding light intensity. This effect is independent of stochastic resonance, which is often used to describe perithreshold stimuli. The constructive role of noise in biological signal processing has implications both for understanding the dynamics of the nervous system and for the design of neural interface devices.

Random Noise Paradoxically Improves Light-Intensity Encoding in Hermissenda Photoreceptor Network

Neurons are notoriously noisy devices. Although the traditional view posits that noise degrades system performance, recent evidence suggests that noise may instead enhance neural information processing under certain conditions. Here we report that random channel and synaptic noise improve the ability of a biologically realistic computational model of the Hermissenda eye to encode light intensity. The model was created in GENESIS and is based on a previous model used to examine effects of changes in type B photoreceptor excitability, synaptic strength, and network architecture. The network consists of two type A and three type B multicompartmental photoreceptors. Each compartment contains a population of Hodgkin–Huxley-type ion channels and each cell is stimulated via artificial light currents. We found that the addition of random channel and synaptic noise yielded a significant improvement in the accuracy of the network's encoding of light intensity across eight light levels spanning 3.5 log units ( P < 0.001, modified Levene test). The benefits of noise remained after controlling for several consequences of randomness in the model. Additionally, improvements were not confined to perithreshold stimulus intensities. Finally, the effects of noise are not present in individual neurons, but rather are an emergent property of the synaptically connected network that is independent of stochastic resonance. These results suggest that noise plays a constructive role in neural information processing, a concept that could have important implications for understanding neural information processing or designing neural interface devices.

Threshold fatigue and information transfer

Neurons in vivo must process sensory information in the presence of significant noise. It is thus plausible to assume that neural systems have developed mechanisms to reduce this noise. Theoretical studies have shown that threshold fatigue (i.e. cumulative increases in the threshold during repetitive firing) could lead to noise reduction at certain frequencies bands and thus improved signal transmission as well as noise increases and decreased signal transmission at other frequencies: a phenomenon called noise shaping. There is, however, no experimental evidence that threshold fatigue actually occurs and, if so, that it will actually lead to noise shaping. We analyzed action potential threshold variability in intracellular recordings in vivo from pyramidal neurons in weakly electric fish and found experimental evidence for threshold fatigue: an increase in instantaneous firing rate was on average accompanied by an increase in action potential threshold. We show that, with a minor modification, the standard Hodgkin-Huxley model can reproduce this phenomenon. We next compared the performance of models with and without threshold fatigue. Our results show that threshold fatigue will lead to a more regular spike train as well as robustness to intrinsic noise via noise shaping. We finally show that the increased/reduced noise levels due to threshold fatigue correspond to decreased/increased information transmission at different frequencies.

Frequency-dependent response properties of adapting spiking neurons

The dynamics of a population of integrate and fire (IF) neurons with spike-frequency adaptation (SFA) is studied. Using a population density approach and assuming a slow dynamics for the variable driving SFA, an equation for the emission rate of a finite set of uncoupled neurons is derived. The system dynamics is then analyzed in the neighborhood of its stable fixed points by linearizing the emission rate equation. The information transfer properties are then probed by perturbing the system with a sinusoidal input current: despite the low-pass properties of the dynamical variable associated with SFA, the adapting IF neuron behaves as a band-pass device and a phase-lock condition appears at a frequency related to the characteristic time constants of both neuronal and SFA dynamics. When a finite set of neurons is considered, the power spectral density of the pooled firing rates shows for intermediate omega a rich pattern of resonances. Theoretical predictions are successfully compared to numerical simulations.

Analog VLSI implementation of resonate-and-fire neuron

We propose an analog integrated circuit that implements a resonate-and-fire neuron (RFN) model based on the Lotka-Volterra (LV) system. The RFN model is a spiking neuron model that has second-order membrane dynamics, and thus exhibits fast damped subthreshold oscillation, resulting in the coincidence detection, frequency preference, and post-inhibitory rebound. The RFN circuit has been derived from the LV system to mimic such dynamical behavior of the RFN model. Through circuit simulations, we demonstrate that the RFN circuit can act as a coincidence detector and a band-pass filter at circuit level even in the presence of additive white noise and background random activity. These results show that our circuit is expected to be useful for very large-scale integration (VLSI) implementation of functional spiking neural networks.

Envelope gating and noise shaping in populations of noisy neurons

Narrowband signals have fast and slow time scales. The transmission of narrowband signal features on both times cales, by spiking neurons, is demonstrated experimentally and theoretically. The interaction of the narrowband input and the threshold nonlinearity may create out-of-band interference, hindering the transmission of signals in a low-frequency range. The resultant out-of-band signal is the "envelope," or time-varying modulation of the narrowband signal. The levels of noise and nonlinearity intrinsic to the neuron gate transmission on the slow "envelope" time scale. When a narrowband and a distinct slow signal drive the neuron, the slow signal may be poorly transmitted. Increasing intrinsic noise in an averaging network removes the envelope in favor of the slow signal, paradoxically increasing the signal-to-noise ratio. These gating effects are generic for threshold and excitable systems.

Dual coding hypotheses for neural information representation

Information is represented and processed in neural systems in various ways. The rate coding, population coding, and temporal coding are typical examples of representation. It is a hot issue in neuroscience what kinds of coding is used in real neural systems. Different regions of the brain may resort to different coding strategies. Moreover, recent studies suggest the possibility of dual or multiple codes, in which different modes of information are embedded in one neural system. The present paper reviews various possibilities of neural codes focusing on dual codes.

Noise reduction of coincidence detector output by the inferior colliculus of the barn owl

A recurring theme in theoretical work is that integration over populations of similarly tuned neurons can reduce neural noise. However, there are relatively few demonstrations of an explicit noise reduction mechanism in a neural network. Here we demonstrate that the brainstem of the barn owl includes a stage of processing apparently devoted to increasing the signal-to-noise ratio in the encoding of the interaural time difference (ITD), one of two primary binaural cues used to compute the position of a sound source in space. In the barn owl, the ITD is processed in a dedicated neural pathway that terminates at the core of the inferior colliculus (ICcc). The actual locus of the computation of the ITD is before ICcc in the nucleus laminaris (NL), and ICcc receives no inputs carrying information that did not originate in NL. Unlike in NL, the rate-ITD functions of ICcc neurons require as little as a single stimulus presentation per ITD to show coherent ITD tuning. ICcc neurons also displayed a greater dynamic range with a maximal difference in ITD response rates approximately double that seen in NL. These results indicate that ICcc neurons perform a computation functionally analogous to averaging across a population of similarly tuned NL neurons.

Relation between single neuron and population spiking statistics and effects on network activity

To simplify theoretical analyses of neural networks, individual neurons are often modeled as Poisson processes. An implicit assumption is that even if the spiking activity of each neuron is non-Poissonian, the composite activity obtained by summing many spike trains limits to a Poisson process. Here, we show analytically and through simulations that this assumption is invalid. Moreover, we show with Fokker-Planck equations that the behavior of feedforward networks is reproduced accurately only if the tendency of neurons to fire periodically is incorporated by using colored noise whose autocorrelation has a negative component.

Stochastic resonance of collective variables in finite sets of interacting identical subsystems

We explore stochastic resonance effects in the response of a complex stochastic system formed by a finite number of interacting, identical subunits driven by a time-periodic force. The driving force alone cannot induce sustained oscillations between the different attractors of the dynamics in the absence of noise. We focus on a global stochastic variable defined as the arithmetic mean of the relevant stochastic variable of each subunit. We construct numerical approximations to its first two long time cumulant moments and its long time correlation function. We also compute the output signal-to-noise ratio and the stochastic resonance gain, for a wide range of parameter values and several types of driving forces. The coupling between the subsystems leads, within adequate ranges of the parameter values, to global outputs with very large signal-to-noise ratios. We have also observed gains larger than unity in the global response to subthreshold sinusoidal driving forces.

Theory of oscillatory firing induced by spatially correlated noise and delayed inhibitory feedback

A network of leaky integrate-and-fire neurons with global inhibitory feedback and under the influence of spatially correlated noise is studied. We calculate the spectral statistics of the network (power spectrum of the population activity, cross spectrum between spike trains of different neurons) as well as of a single neuron (power spectrum of spike train, cross spectrum between external noise and spike train) within the network. As shown by comparison with numerical simulations, our theory works well for arbitrary network size if the feedback is weak and the amount of external noise does not exceed that of the internal noise. By means of our analytical results we discuss the quality of the correlation-induced oscillation in a large network as a function of the transmission delay and the internal noise intensity. It is shown that the strongest oscillation is obtained in a system with zero internal noise and adiabatically long delay (i.e., the delay period is longer than any other time scale in the system). For a neuron with a strong intrinsic frequency, the oscillation becomes strongly anharmonic in the case of a long delay time. We also discuss briefly the kind of synchrony introduced by the feedback-induced oscillation.

Delayed excitatory and inhibitory feedback shape neural information transmission

Feedback circuitry with conduction and synaptic delays is ubiquitous in the nervous system. Yet the effects of delayed feedback on sensory processing of natural signals are poorly understood. This study explores the consequences of delayed excitatory and inhibitory feedback inputs on the processing of sensory information. We show, through numerical simulations and theory, that excitatory and inhibitory feedback can alter the firing frequency response of stochastic neurons in opposite ways by creating dynamical resonances, which in turn lead to information resonances (i.e., increased information transfer for specific ranges of input frequencies). The resonances are created at the expense of decreased information transfer in other frequency ranges. Using linear response theory for stochastically firing neurons, we explain how feedback signals shape the neural transfer function for a single neuron as a function of network size. We also find that balanced excitatory and inhibitory feedback can further enhance information tuning while maintaining a constant mean firing rate. Finally, we apply this theory to in vivo experimental data from weakly electric fish in which the feedback loop can be opened. We show that it qualitatively predicts the observed effects of inhibitory feedback. Our study of feedback excitation and inhibition reveals a possible mechanism by which optimal processing may be achieved over selected frequency ranges.

Integrate-and-fire neurons with threshold noise: a tractable model of how interspike interval correlations affect neuronal signal transmission

Many neurons exhibit interval correlations in the absence of input signals. We study the influence of these intrinsic interval correlations of model neurons on their signal transmission properties. For this purpose, we employ two simple firing models, one of which generates a renewal process, while the other leads to a nonrenewal process with negative interval correlations. Different methods to solve for spectral statistics in the presence of a weak stimulus (spike train power spectra, cross spectra, and coherence functions) are presented, and their range of validity is discussed. Using these analytical results, we explore a lower bound on the mutual information rate between output spike train and input stimulus as a function of the system's parameters. We demonstrate that negative correlations in the baseline activity can lead to enhanced information transfer of a weak signal by means of noise shaping of the background noise spectrum. We also show that an enhancement is not compulsory--for a stimulus with power exclusively at high frequencies, the renewal model can transfer more information than the nonrenewal model does. We discuss the application of our analytical results to other problems in neuroscience. Our results are also relevant to the general problem of how a signal affects the power spectrum of a nonlinear stochastic system.

Increased noise level of purkinje cell activities minimizes impact of their modulation during sensorimotor control

While firing rate is well established as a relevant parameter for encoding information exchanged between neurons, the significance of other parameters is more conjectural. Here, we show that regularity of neuronal spike activities affects sensorimotor processing in tottering mutants, which suffer from a mutation in P/Q-type voltage-gated calcium channels. While the modulation amplitude of the simple spike firing rate of their floccular Purkinje cells during optokinetic stimulation is indistinguishable from that of wild-types, the regularity of their firing is markedly disrupted. The gain and phase values of tottering's compensatory eye movements are indistinguishable from those of flocculectomized wild-types or from totterings with the flocculus treated with P/Q-type calcium channel blockers. Moreover, normal eye movements can be evoked in tottering when the flocculus is electrically stimulated with regular spike trains mimicking the firing pattern of normal simple spikes. This study demonstrates the importance of regularity of firing in Purkinje cells for neuronal information processing.

Electroreceptor neuron dynamics shape information transmission

The gymnotiform weakly electric fish Apteronotus leptorhynchus can capture prey using electrosensory cues that are dominated by low temporal frequencies. However, conventional tuning curves predict poor electroreceptor afferent responses to low-frequency stimuli. We compared conventional tuning curves with information tuning curves and found that the latter predicted substantially improved responses to these behaviorally relevant stimuli. Analysis of receptor afferent baseline activity showed that negative correlations reduced low-frequency noise levels, thereby increasing information transmission. Multiunit recordings from receptor afferents showed that this increased information transmission could persist at the population level. Finally, we verified that this increased low-frequency information is preserved in the spike trains of central neurons that receive receptor afferent input. Our results demonstrate that conventional tuning curves can be misleading when certain noise reduction strategies are used by the nervous system.

Emergence of complex dynamics in a simple model of signaling networks

Various physical, social, and biological systems generate complex fluctuations with correlations across multiple time scales. In physiologic systems, these long-range correlations are altered with disease and aging. Such correlated fluctuations in living systems have been attributed to the interaction of multiple control systems; however, the mechanisms underlying this behavior remain unknown. Here, we show that a number of distinct classes of dynamical behaviors, including correlated fluctuations characterized by 1/f scaling of their power spectra, can emerge in networks of simple signaling units. We found that, under general conditions, complex dynamics can be generated by systems fulfilling the following two requirements, (i) a "small-world" topology and (ii) the presence of noise. Our findings support two notable conclusions. First, complex physiologic-like signals can be modeled with a minimal set of components; and second, systems fulfilling conditions i and ii are robust to some degree of degradation (i.e., they will still be able to generate 1/f dynamics).

Finite-size dynamics of inhibitory and excitatory interacting spiking neurons

The dynamic mean-field approach we recently developed is extended to study the dynamics of population emission rates nu (t) for a finite network of coupled excitatory (E) and inhibitory (I) integrate-and-fire (IF) neurons. The power spectrum of nu (t) in an asynchronous state is computed and compared to simulations. We calculate the interpopulations transfer functions and show how synaptic interaction modulates the otherwise low-pass filter with resonances which go well beyond the filter's cut (omega approximately nu) , allowing efficient information transmission on very short time scales determined by spike transmission delays. The saddle-node instability of the asynchronous state is studied and a simple exact dependence of the stability condition on the current-to-rate gain functions is derived, by which self-couplings (EE and II) decrease stability while mutual interaction (EI and IE) favor stability.

Oscillatory activity in electrosensory neurons increases with the spatial correlation of the stochastic input stimulus

We present results from a novel experimental paradigm to investigate the influence of spatial correlations of stimuli on electrosensory neural network dynamics. Further, a new theoretical analysis for the dynamics of a model network of stochastic leaky integrate-and-fire neurons with delayed feedback is proposed. Experiment and theory for this system both establish that spatial correlations induce a network oscillation, the strength of which is proportional to the degree of stimulus correlation at constant total stimulus power.

Neural noise and movement-related codes in the macaque supplementary motor area

We analyzed the variability of spike counts and the coding capacity of simultaneously recorded pairs of neurons in the macaque supplementary motor area (SMA). We analyzed the mean-variance functions for single neurons, as well as signal and noise correlations between pairs of neurons. All three statistics showed a strong dependence on the bin width chosen for analysis. Changes in the correlation structure of single neuron spike trains over different bin sizes affected the mean-variance function, and signal and noise correlations between pairs of neurons were much smaller at small bin widths, increasing monotonically with the width of the bin. Analyses in the frequency domain showed that the noise between pairs of neurons, on average, was most strongly correlated at low frequencies, which explained the increase in noise correlation with increasing bin width. The coding performance was analyzed to determine whether the temporal precision of spike arrival times and the interactions within and between neurons could improve the prediction of the upcoming movement. We found that in approximately 62% of neuron pairs, the arrival times of spikes at a resolution between 66 and 40 msec carried more information than spike counts in a 200 msec bin. In addition, in 19% of neuron pairs, inclusion of within (11%)- or between-neuron (8%) correlations in spike trains improved decoding accuracy. These results suggest that in some SMA neurons elements of the spatiotemporal pattern of activity may be relevant for neural coding.

Neural noise and movement-related codes in the macaque supplementary motor area

We analyzed the variability of spike counts and the coding capacity of simultaneously recorded pairs of neurons in the macaque supplementary motor area (SMA). We analyzed the mean-variance functions for single neurons, as well as signal and noise correlations between pairs of neurons. All three statistics showed a strong dependence on the bin width chosen for analysis. Changes in the correlation structure of single neuron spike trains over different bin sizes affected the mean-variance function, and signal and noise correlations between pairs of neurons were much smaller at small bin widths, increasing monotonically with the width of the bin. Analyses in the frequency domain showed that the noise between pairs of neurons, on average, was most strongly correlated at low frequencies, which explained the increase in noise correlation with increasing bin width. The coding performance was analyzed to determine whether the temporal precision of spike arrival times and the interactions within and between neurons could improve the prediction of the upcoming movement. We found that in approximately 62% of neuron pairs, the arrival times of spikes at a resolution between 66 and 40 msec carried more information than spike counts in a 200 msec bin. In addition, in 19% of neuron pairs, inclusion of within (11%)- or between-neuron (8%) correlations in spike trains improved decoding accuracy. These results suggest that in some SMA neurons elements of the spatiotemporal pattern of activity may be relevant for neural coding.

Stochastic model for heart-rate fluctuations

A normal human heart rate shows complex fluctuations in time, which is natural, because the heart rate is controlled by a large number of different feedback control loops. These unpredictable fluctuations have been shown to display fractal dynamics, long-term correlations, and 1/f noise. These characterizations are statistical and they have been widely studied and used, but much less is known about the detailed time evolution (dynamics) of the heart-rate control mechanism. Here we show that a simple one-dimensional Langevin-type stochastic difference equation can accurately model the heart-rate fluctuations in a time scale from minutes to hours. The model consists of a deterministic nonlinear part and a stochastic part typical to Gaussian noise, and both parts can be directly determined from the measured heart-rate data. Studies of 27 healthy subjects reveal that in most cases, the deterministic part has a form typically seen in bistable systems: there are two stable fixed points and one unstable one.

Duality of rate coding and temporal coding in multilayered feedforward networks

A functional role for precise spike timing has been proposed as an alternative hypothesis to rate coding. We show in this article that both the synchronous firing code and the population rate code can be used dually in a common framework of a single neural network model. Furthermore, these two coding mechanisms are bridged continuously by several modulatable model parameters, including shared connectivity, feedback strength, membrane leak rate, and neuron heterogeneity. The rates of change of these parameters are closely related to the response time and the timescale of learning.