Entropy as an index of the informational state of neurons

Neuronal spike train entropy estimation by history clustering

Neurons send signals to each other by means of sequences of action potentials (spikes). Ignoring variations in spike amplitude and shape that are probably not meaningful to a receiving cell, the information content, or entropy of the signal depends on only the timing of action potentials, and because there is no external clock, only the interspike intervals, and not the absolute spike times, are significant. Estimating spike train entropy is a difficult task, particularly with small data sets, and many methods of entropy estimation have been proposed. Here we present two related model-based methods for estimating the entropy of neural signals and compare them to existing methods. One of the methods is fast and reasonably accurate, and it converges well with short spike time records; the other is impractically time-consuming but apparently very accurate, relying on generating artificial data that are a statistical match to the experimental data. Using the slow, accurate method to generate a best-estimate entropy value, we find that the faster estimator converges to this value more closely and with smaller data sets than many existing entropy estimators.

Noise Shaping in Neural Populations with Global Delayed Feedback

The interplay between intrinsic and network dynamics has been the focus of many investigations. Here we use a combination of theoretical and numerical approaches to study the effects of delayed global feedback on the information transmission properties of neural networks. Specifically, we compare networks of neurons that display intrinsic interspike interval correlations (nonrenewal) to networks that do not (renewal). We find that excitatory and inhibitory delays can tune information transmission by single neurons but not by the entire network. Most surprisingly, addition of a delay can change the dependence of the information on the coupling strength for renewal neurons and not for nonrenewal neurons. Our results show that intrinsic ISI correlations can have nontrivial interactions with network-induced phenomena.

Estimating the temporal interval entropy of neuronal discharge

To better understand the role of timing in the function of the nervous system, we have developed a methodology that allows the entropy of neuronal discharge activity to be estimated from a spike train record when it may be assumed that successive interspike intervals are temporally uncorrelated. The so-called interval entropy obtained by this methodology is based on an implicit enumeration of all possible spike trains that are statistically indistinguishable from a given spike train. The interval entropy is calculated from an analytic distribution whose parameters are obtained by maximum likelihood estimation from the interval probability distribution associated with a given spike train. We show that this approach reveals features of neuronal discharge not seen with two alternative methods of entropy estimation. The methodology allows for validation of the obtained data models by calculation of confidence intervals for the parameters of the analytic distribution and the testing of the significance of the fit between the observed and analytic interval distributions by means of Kolmogorov-Smirnov and Anderson-Darling statistics. The method is demonstrated by analysis of two different data sets: simulated spike trains evoked by either Poissonian or near-synchronous pulsed activation of a model cerebellar Purkinje neuron and spike trains obtained by extracellular recording from spontaneously discharging cultured rat hippocampal neurons.

Effects of collateral inhibition in a model of the immature rat cerebellar cortex: multineuron correlations

A model of the immature rat cerebellar cortex is used to simulate the effect of the inhibitory recurrent collateral axons of the Purkinje cells on the spike trains in the network. Inhibition induces an important overall change in the statistical characteristics of individual spike trains. It is also instrumental in producing a strong cooperativity between the different neurons. Moreover, a functional spatial anisotropy appears. A specific entropy index is used to analyze levels of information transfer between clustered and faraway neurons in the network. The formatting effect of recurrent collateral inhibition on spike trains and on network functional dynamics is studied by means of a model of the newborn rat cerebellar cortex. This immature structure has simpler morphological characteristics and fewer physiological parameters than the adult one. It is thus a good candidate for the comparison between experimental and theoretical data. The model network is made of 256 formal neurons (FN), arranged in a square lattice. Each neuron is coupled to its eight nearest neighbors by inhibitory links. All the parameters of the different elements of the model--in particular integration of inhibitory and excitatory inputs--are given anatomical and physiological values derived from biological data. Activities of single FNs and correlations between spatially distant ones are analyzed with classical statistical techniques as well as with a specific informational entropy method we introduce. Simulation results indicate that inhibition is instrumental in: (1) the transformation of the spike train characteristics. This includes a lengthening of the mean interspike interval as well as an overall change in the statistical distribution of intervals, with an emergence of long-lasting ones; (2) the functional structuration of the network. Inhibitory connections between nearest neighbors induce a strong cooperativity between FNs. Furthermore a clear spatial anisotropy occurs in the functioning of the network, with inhibitory effects extending beyond local connectivity in preferential directions. We propose an interpretation of this functional structuration in terms of the various routes followed by the inhibition, including relay effects. The parameters of the model (levels of activities, inhibition rules and connectivities) were varied in order to test the robustness of the above results. Finally, the results are compared with those obtained in an experimental situation.

How to use EEG/ERBP phenomena

Hitherto the study of cerebral slow waves has been mainly empirical phenomenology, in spite of 50 years of effort to discover their origins and possible functions. Since 1970 the increasing use of both analog and digital filters, combined with more sensitive averaging techniques, has led to better understanding of some possible functional significance for the electroencephalogram (EEG) and the endogenous slow waves of event-related brain potentials (ERBP). The developing concepts and mathematics of non-linear dynamicsand coupled oscillators, when considered in the light of cerebral circuit geometry and topology, of local neuronal circuitry, and of both observed and experienced biological behavior, now offer hope for future, more complete, understanding of the development of 'conscious and goal-directed' activity from robotic and reflexive stimulus-response paradigms. Immediate practical results of a biophysical approach to brain waves should be: Models of vertebrate brains as distributed systems of non-linearly coupled oscillators; and revision of traditional methodologies for recording and interpreting EEG/ERBP data.

Do neurons process information by relative intervals in spike trains?

We suggest the possibility that neurons process information in terms of the relative duration of clusters of adjacent and successive inter-action potential intervals ("bytes" of intervals). If this concept is plausible, as is supported by research from several laboratories which have specifically addressed this possibility, one should be able to see evidence for such patterning in the published illustrations from studies in which this concept was not considered. We present some of this evidence here, along with some illustrations from the original publications. Byte patterns are evident in these examples, even though they went unrecognized by authors and readers alike. It is true that interval patterns are not obvious in all published illustrations of spike trains, and we suggest that this can be explained by one or more of the following: (1) some neurons may operate with an interval-pattern code while others do not, (2) a given neuron may use an interval-pattern code only under certain conditions, and (3) even when such a code exists, it may be difficult to detect for identifiable technical reasons. Therefore, we believe that the relative-internal-pattern concept is a valid scientific hypothesis which merits specific testing of its validity and range of applicability.

Serial dependencies and Markov properties of neuronal interspike intervals from rat cerebellum

Using three different approaches, each with different theoretical assumptions, we showed that mammalian neuronal spike trains contain serial ordering. We demonstrated that: (1) when intervals are categorized according to whether their durations are short, medium, or long, sequential groupings of adjacent interval categories exhibit Markov dependencies, extending to at least the 4th order; (2) the observed incidence of specified patterns of these groups of adjacent interval categories differs from the independent case, based on Chi square goodness-of-fit tests, and by using similar procedures; (3) there is divergence from independence when adjacent interval patterns are described in terms of relative lengths of adjacent intervals. The statistical indicators of serial dependence were significantly greater when applied to the original data than when applied to the same data after shuffling. Each of these approaches leads to the notion that "information" is carried in clusters of adjacent intervals ("bytes" or "words") and moreover, we can identify which specific patterns of interspike intervals contribute most to the statistical significance (i.e., those clusters that are potential candidates for "information carriers"). In most of the ten neurons, the "memory" of the system appears to be at least 36--45 msecs.