The phonochrome: a coherent spectro-temporal representation of sound

The role of ECoG magnitude and phase in decoding position, velocity, and acceleration during continuous motor behavior

In neuronal population signals, including the electroencephalogram (EEG) and electrocorticogram (ECoG), the low-frequency component (LFC) is particularly informative about motor behavior and can be used for decoding movement parameters for brain-machine interface (BMI) applications. An idea previously expressed, but as of yet not quantitatively tested, is that it is the LFC phase that is the main source of decodable information. To test this issue, we analyzed human ECoG recorded during a game-like, one-dimensional, continuous motor task with a novel decoding method suitable for unfolding magnitude and phase explicitly into a complex-valued, time-frequency signal representation, enabling quantification of the decodable information within the temporal, spatial and frequency domains and allowing disambiguation of the phase contribution from that of the spectral magnitude. The decoding accuracy based only on phase information was substantially (at least 2 fold) and significantly higher than that based only on magnitudes for position, velocity and acceleration. The frequency profile of movement-related information in the ECoG data matched well with the frequency profile expected when assuming a close time-domain correlate of movement velocity in the ECoG, e.g., a (noisy) "copy" of hand velocity. No such match was observed with the frequency profiles expected when assuming a copy of either hand position or acceleration. There was also no indication of additional magnitude-based mechanisms encoding movement information in the LFC range. Thus, our study contributes to elucidating the nature of the informative LFC of motor cortical population activity and may hence contribute to improve decoding strategies and BMI performance.

Context dependence of spectro-temporal receptive fields with implications for neural coding

The spectro-temporal receptive field (STRF) is frequently used to characterize the linear frequency-time filter properties of the auditory system up to the neuron recorded from. STRFs are extremely stimulus dependent, reflecting the strong non-linearities in the auditory system. Changes in the STRF with stimulus type (tonal, noise-like, vocalizations), sound level and spectro-temporal sound density are reviewed here. Effects on STRF shape of task and attention are also briefly reviewed. Models to account for these changes, potential improvements to STRF analysis, and implications for neural coding are discussed.

Social experience influences the development of a central auditory area

Vocal communication develops under social influences that can enhance attention, an important factor in memory formation and perceptual tuning. In songbirds, social conditions can delay sensitive periods of development, overcome learning inhibitions and enable exceptional learning or induce selective learning. However, we do not know how social conditions influence auditory processing in the brain. In the present study, we raised young naive starlings under different social conditions but with the same auditory experience of adult songs, and we compared the effects of these different conditions on the development of the auditory cortex analogue. Several features appeared to be influenced by the social experience, among which the proportion of auditory neuronal sites and the neuronal selectivity. Both physical and social isolation from adult models altered the development of the auditory area in parallel to alterations in vocal development. To our knowledge, this is the first evidence that social deprivation has as much influence on neuronal responsiveness as sensory deprivation.

A rapid correlation method for the analysis of spectro-temporal receptive fields of auditory neurons

The evaluation of spectro-temporal receptive fields in auditory neurons makes use of a correlation technique that needs a high amount of calculation time. We present a method in which 2-ms time bins of post-stimulus time histograms, rather than every action potential, are the basis for correlation with the acoustic spectrum of the stimulus. In a t test the content of each bin is classified as excitatory, inhibitory or responseless. The bin width is adjusted to the temporal resolution of the units as determined in a gap detection analysis. The method presented here saves a substantial amount of analysis time and reveals adequately spectro-temporal receptive fields.

Wiener and Volterra analyses applied to the auditory system

The application of a particular branch of non-linear system analysis, the functional series expansion or integral method, to the auditory system is reviewed. Both the Volterra and Wiener approach are discussed and an extension of the Wiener method from its traditional white-noise stimulus approach to that of Poisson distributed clicks is presented. This type of analysis has been applied to compound and single-unit responses from the auditory nerve, cochlear nucleus, auditory midbrain and medial geniculate body. Most studies have estimated only first-order Wiener kernels but in recent years second-order Wiener and Volterra kernels have been estimated, particularly with reference to dynamic non-linearities. A particular form of second-order analysis, the Spectro Temporal Receptive Field, offers an alternative to first-order cross-correlation when phase-lock is absent. The correlation method has revealed that neural synchronization is less affected by intensity changes and damage to the hair cells than is neural firing rate. Although the presence of the static cochlear non-linearity could be demonstrated on the basis of the intensity dependence of the first-order Wiener kernel, the identification of the exact form of the nonlinearity of the peripheral auditory system on basis of higher-order Wiener kernels has so far been inconclusive. However, successes of the method can be found in the description of the dynamic non-linearities and non-linear neural interactions.

A spectrotemporal analysis of DCN single unit responses to wideband noise in guinea pig

Spectrotemporal receptive fields (STRFs) were estimated for chopper and pauser units recorded in guinea pig dorsal cochlear nucleus (DCN). Sixteen wideband, periodic noise stimuli, represented as time-frequency surfaces of energy density, were cross correlated in time with the unit's corresponding period histograms to determine if specific energy patterns tended to precede spike occurrence. The STRFs obtained were unique to the DCN, as compared to the ventral cochlear nucleus (VCN) [Clopton and Backoff, 1991, Hear. Res. 52, 329-344] in their degree of temporal and spectral complexity. Certain unit response types, classified from their peristimulus-time histograms (PSTHs) to tonebursts, were associated with distinctive patterns in the STRFs. All STRFs had at least one region of elevated energy density (peak region) closely preceding spike occurrence, which may reflect a short-pathway, primary excitatory input (or inputs) to the neuron. In addition, some units displayed low-energy regions (troughs) with greater temporal precedences on their STRFs, particularly when higher stimulus intensities were used. This analysis approach appears to have potential for investigating functional neural connectivity and predicting responses to novel complex stimuli, although specific implementations of the technique impose limitations on the interpretation of results.

Spectrotemporal receptive fields of neurons in cochlear nucleus of guinea pig

Spectrotemporal receptive fields (STRFs) [Hermes et al., Hear. Res. 5, 147-178, 1981] for neurons in the cochlear nuclei (CN) of guinea pig were estimated. Sixteen periodic segments of bandlimited, synthesized noise evoked replicable, distinctive period histograms for spike discharges. All driven units in the major divisions of the CN having their characteristic frequency (CF) within the noise bandlimits had unique STRFs for a given intensity of noise stimulation. The STRF maximum corresponded to the unit's CF, and details of the STRF patterns differed over CN divisions and response classes derived from tonebursts. The sizes of features in STRFs from this mammal appeared significantly smaller in their temporal and spectral extents than those reported in the torus semicircularis of an amphibian and were roughly comparable to the few units reported from cat ventral CN [Eggermont et al., Quart. Rev. Biophys. 16, 341-414, 1983]. STRFs, as they are presently obtained, provide useful insight into some aspects of afferent processing and perhaps connectivity, but their interpretation is specific to the level of stimulation and limited by the need to choose a specific energy distribution to represent the stimulus.

Spectro-temporal interpretation of activity patterns of auditory neurons

Sensory receptors transform an external sensory stimulus into an internal neural activity pattern. This mapping is studied through its inverse. An earlier paper showed that within the context of a neuron model composed of a linear filter followed by an exponential pulse generator and a Gaussian stimulus ensemble a unique "most plausible" first-order stimulus estimate can be constructed. This method, applicable only to neurons showing phase-lock, is extended to neurons without phase-lock. In this situation second-order spectro-temporal stimulus estimates are produced; examples are given from simulation. The method is applied to activity of neurons in the auditory system of the frog.

Sensitivity of neurons in the dorsal medullary nucleus of the grassfrog to spectral and temporal characteristics of sound

The responses of 58 dorsal medullary nucleus units to a set of spectrally and temporally structured stimuli were investigated. Responses to tonepips and noise indicated monomodal spectral sensitivities, with diverse response patterns. Phase-locking was strong for frequencies from 0.1 to 0.2 kHz, and in one unit extended up to 0.6 kHz. To clicks, amplitude modulated tonebursts and natural and artificial versions of the mating call various responses were found. Most low-frequency units fired tonically. They showed a non-selective or low-pass rate response to increasing modulation frequency, and a low-pass synchronization behavior to the envelope. A group of mid-frequency units fired phasically and exhibited a band-pass rate characteristic of amplitude modulated tonebursts. Frequently this was combined with a low-pass rate characteristic of click trains. These units hardly responded to the time-reversed mating call, but fired in a time-locked fashion to the pulses of the original mating call, up to a signal-to-noise ratio of 0 dB. This suggests that aspects of pulse envelope and interpulse interval are coded in the dorsal medullary nucleus.

Single-unit characteristics in the auditory midbrain of the immobilized grassfrog

The anuran auditory midbrain of the grassfrog (Rana temporaria L.) was studied by a combined spectro-temporal analysis of sound preceding neural events. From the spectro-temporal sensitivities (STS) estimates of best frequencies (BF) and latencies (LT) were derived. Several types of STSs were observed: monomodal excitatory STSs comprised about half of the cases. Bimodal excitatory STSs, i.e. STSs with two discrete excitation regions, were observed in about 25%. Trimodal and broadly tuned STSs comprised about 5%. The remaining 20% of the STSs were characterized by inhibitory phenomena such as pure inhibition, sideband inhibition and post-activation inhibition. The distribution of best frequencies matches the frequency spectrum of the animal's vocalizations. A relative absence of monomodal units was noted in the mid frequency range. The distribution of latencies was bimodal over the range 7-108 ms. For each unit 6 functional parameters were determined; besides BF and LT these were: form of the STS (i.e. monomodality versus multimodality), spontaneous activity, binaural interaction, and firing mode (i.e. sustained versus transient) upon continuous noises stimulation. In addition, two structural parameters were considered: location in the torus and action potential waveform. Large correlations appeared between LT and action potential waveform, and between BF and binaural interaction type. Tonotopy was not found. A comparison was made between results from this study with a previous study on lightly anesthetized grassfrogs, using the same stimulus paradigms (D.J. Hermes et al. (1981): Hearing Res. 5, 147-178; D.J. Hermes et al. (1982): Hearing Res. 6, 103-126). Spontaneous activity, inhibitory phenomena and complex STSs were common using immobilization, whereas these have hardly been observed using anesthesia. Furthermore, interdependencies between the neural characteristics are substantially weaker for the immobilized preparation.

Relation of binaural interaction and spectro-temporal characteristics in the auditory midbrain of the grassfrog

The relation between binaural interaction type and spectro-temporal characteristics was studied for single units in the auditory midbrain of the grassfrog. Tonal and continuous wideband noise ensembles have been used as stimuli. Spectro-temporal sensitivities were determined for ipsi-, contra- and bilateral stimulus presentation by a closed sound system. Binaural interaction was classified in monaural EO (one ear excitatory), binaural EE (both ears excitatory) and EI (one ear excitatory, the other inhibitory) and purely inhibitory categories. Binaural interaction appeared to be rather invariant to alterations in stimulus intensity and type. A very clear correlation was observed between best frequency and binaural interaction type: EE units are predominantly of high best frequency, whereas EI units are predominantly of low best frequency. The correlation with latency was less significant: EE units tended to have somewhat shorter latencies that EI units. EO units take an intermediate position. Comparisons of ipsi-, contra- and bilateral spectro-temporal sensitivities, revealed differences in best frequency, latency and temporal discharge pattern. In some units a complex interplay of excitatory and inhibitory monaural influences was demonstrated. A number of units was recorded, which were characterized by multiple activation or suppression areas. The majority of these units exhibited frequency-dependent binaural interaction types. In some units it was noticed that binaural interaction type can be dependent on state of adaptation. A comparison of binaural interaction types of neighbouring units provided only weak evidence for a binaural organization in the anuran auditory midbrain, since simultaneously recorded pairs shared the same binaural interaction type only slightly more than expected by mere chance (chi 2-test, P less than 0.10).

Reverse-correlation methods in auditory research

Single unit recordings have provided us with a basis for understanding the auditory system, especially about how it behaves under stimulation with simple sounds such as clicks and tones. The experimental as well as the theoretical approach to single unit studies has been dichotomous. One approach, the more familiar, gives a representation of nervous system activity in the form of peri-stimulus-time (PST) histograms, period histograms, iso-intensity rate curves and frequency tuning curves. This approach observes the neural output of units in the various nuclei in the auditory nervous system, and, faced with the random way in which the neurons respond to sound, proceeds by repeatedly presenting the same stimulus in order to obtain averaged results. These are the various histogram procedures (Gerstein & Kiang, 1960; Kiang et al. 1965).

Statistical and dimensional analysis of the neural representation of the acoustic biotope of the frog

The field of investigation is the neural representation of acoustic stimuli occurring in the natural environment of the frog. The point of departure is the description of a stimulus ensemble consisting of natural sounds: the acoustic biotope. A relation of statistical and dimensional structure of the acoustic biotope is indicated. The animal used in the neurophysiological experiments is the grass frog, Rana temporaria L.; microelectrode recordings are made in the auditory midbrain. A method is described to determine the existence of a relation between acoustic stimulus and neural events. The form of this relation has been investigated by first- and second-order stimulus-event correlation. While the first one does not give significant results, the second one leads to the spectrotemporal receptive field of the neuron for natural stimuli. Questions are formulated to estimate the value of this receptive field as a functional descriptor of the neuron. Finally, an outline is sketched for a synthetic construction of the bioacoustic space from neuroacoustic subspaces.

Spectro-temporal characteristics of single units in the auditory midbrain of the lightly anaesthetised grass frog (Rana temporaria L) investigated with noise stimuli

About 30% of the auditory units in the midbrain of the lightly anaesthetised grass frog respond in a sustained way to stationary pseudorandom noise. This response is described by the spectro-temporal receptive field (STRF), the regions in the spectro-temporal domain where the average second-order functional of those parts of the stimulus ensemble that precede the action potentials differ from the average second-order functional of the stimulus ensemble. By means of the STRF frequency selectivity, postactivation suppression and lateral suppression can quantitatively be studied under one and the same experimental condition. Auditory units that respond to stationary noise are localised in those parts of the torus where fibres enter from the olivary nucleus. They are characterised by relatively short latencies to tones and probably represent the first information-processing stage in the torus semicircularis.

A comparison of the spectro-temporal sensitivity of auditory neurons to tonal and natural stimuli

The spectro-temporal sensitivity of auditory neurons has been investigated experimentally by averaging the spectrograms of stimuli preceding the occurrence of action potentials or neural events ( the APES : Aertsen et al., 1980, 1981). The properties of the stimulus ensemble are contained in this measure of neural selectivity. The spectro-temporal receptive field (STRF) has been proposed as a theoretical concept which should give a stimulus-invariant representation of the second order characteristics of the neuron's system function (Aertsen and Johannesma, 1981). The present paper investigates the relation between the experimental and the theoretical description of the neuron's spectro-temporal sensitivity for sound. The aim is to derive a formally based stimulus-normalization procedure for the results of the experimental averaging procedure. Under particular assumptions, regarding both the neuron and the stimulus ensemble, an integral equation connecting the APES and the STRF is derived. This integral expression enables to calculate the APES from the STRF by taking into account the stimulus spectral composition and the characteristics of the spectrogram analysis. The inverse relation, i.e. starting from the experimental results and by application of a formal normalization procedure arriving at the theoretical STRF, is effectively hindered by the nature of the spectrogram analysis. An approximative "normalization" procedure, based on intuitive manipulation of the integral equation, has been applied to a number of single unit recordings from the grassfrog's auditory midbrain area to tonal and natural stimulus ensembles. The results indicate tha spectrogram analysis, while being a useful real-time tool in investigating the spectro-temporal transfer properties of auditory neurons, shows fundamental shortcomings for a theoretical treatment of the questions of interest.

The spectro-temporal receptive field. A functional characteristic of auditory neurons

The Spectro-Temporal Receptive Field (STRF) of an auditory neuron has been introduced experimentally on the base of the average spectro-temporal structure of the acoustic stimuli which precede the occurrence of action potentials (Aertsen et al., 1980, 1981). In the present paper the STRF is considered in the general framework of nonlinear system theory, especially in the form of the Volterra integral representation. The STRF is proposed to be formally identified with a linear functional of the second order Volterra kernel. The experimental determination of the STRF leads to a formulation in terms of the Wiener expansion where the kernels can be identified by evaluation of the system's input-output correlations. For a Gaussian stimulus ensemble and a nonlinear system with no even order contributions of order higher than two, it is shown that the second order cross correlation of stimulus and response, normalized with respect to the spectral contents of the stimulus ensemble, leads to the stimulus-invariant spectro-temporal receptive field. The investigation of stimulus-invariance of the STRF for more general nonlinear systems and for stimulus ensembles which can be generated by nonlinear transformations of Gaussian noise involve the evaluation of higher order stimulus-response correlation functions.