Perceived continuity and pitch perception

Dynamics of the Auditory Continuity Illusion

Illusions give intriguing insights into perceptual and neural dynamics. In the auditory continuity illusion, two brief tones separated by a silent gap may be heard as one continuous tone if a noise burst with appropriate characteristics fills the gap. This illusion probes the conditions under which listeners link related sounds across time and maintain perceptual continuity in the face of sudden changes in sound mixtures. Conceptual explanations of this illusion have been proposed, but its neural basis is still being investigated. In this work we provide a dynamical systems framework, grounded in principles of neural dynamics, to explain the continuity illusion. We construct an idealized firing rate model of a neural population and analyze the conditions under which firing rate responses persist during the interruption between the two tones. First, we show that sustained inputs and hysteresis dynamics (a mismatch between tone levels needed to activate and inactivate the population) can produce continuous responses. Second, we show that transient inputs and bistable dynamics (coexistence of two stable firing rate levels) can also produce continuous responses. Finally, we combine these input types together to obtain neural dynamics consistent with two requirements for the continuity illusion as articulated in a well-known theory of auditory scene analysis: responses persist through the noise-filled gap if noise provides sufficient evidence that the tone continues and if there is no evidence of discontinuities between the tones and noise. By grounding these notions in a quantitative model that incorporates elements of neural circuits (recurrent excitation, and mutual inhibition, specifically), we identify plausible mechanisms for the continuity illusion. Our findings can help guide future studies of neural correlates of this illusion and inform development of more biophysically-based models of the auditory continuity illusion.

Insights on the Neuromagnetic Representation of Temporal Asymmetry in Human Auditory Cortex

Communication sounds are typically asymmetric in time and human listeners are highly sensitive to this short-term temporal asymmetry. Nevertheless, causal neurophysiological correlates of auditory perceptual asymmetry remain largely elusive to our current analyses and models. Auditory modelling and animal electrophysiological recordings suggest that perceptual asymmetry results from the presence of multiple time scales of temporal integration, central to the auditory periphery. To test this hypothesis we recorded auditory evoked fields (AEF) elicited by asymmetric sounds in humans. We found a strong correlation between perceived tonal salience of ramped and damped sinusoids and the AEFs, as quantified by the amplitude of the N100m dynamics. The N100m amplitude increased with stimulus half-life time, showing a maximum difference between the ramped and damped stimulus for a modulation half-life time of 4 ms which is greatly reduced at 0.5 ms and 32 ms. This behaviour of the N100m closely parallels psychophysical data in a manner that: i) longer half-life times are associated with a stronger tonal percept, and ii) perceptual differences between damped and ramped are maximal at 4 ms half-life time. Interestingly, differences in evoked fields were significantly stronger in the right hemisphere, indicating some degree of hemispheric specialisation. Furthermore, the N100m magnitude was successfully explained by a pitch perception model using multiple scales of temporal integration of auditory nerve activity patterns. This striking correlation between AEFs, perception, and model predictions suggests that the physiological mechanisms involved in the processing of pitch evoked by temporal asymmetric sounds are reflected in the N100m.

Pitch and spectral resolution: A systematic comparison of bottom-up cues for top-down repair of degraded speech

The brain is capable of restoring missing parts of speech, a top-down repair mechanism that enhances speech understanding in noisy environments. This enhancement can be quantified using the phonemic restoration paradigm, i.e., the improvement in intelligibility when silent interruptions of interrupted speech are filled with noise. Benefit from top-down repair of speech differs between cochlear implant (CI) users and normal-hearing (NH) listeners. This difference could be due to poorer spectral resolution and/or weaker pitch cues inherent to CI transmitted speech. In CIs, those two degradations cannot be teased apart because spectral degradation leads to weaker pitch representation. A vocoding method was developed to evaluate independently the roles of pitch and spectral resolution for restoration in NH individuals. Sentences were resynthesized with different spectral resolutions and with either retaining the original pitch cues or discarding them all. The addition of pitch significantly improved restoration only at six-bands spectral resolution. However, overall intelligibility of interrupted speech was improved both with the addition of pitch and with the increase in spectral resolution. This improvement may be due to better discrimination of speech segments from the filler noise, better grouping of speech segments together, and/or better bottom-up cues available in the speech segments.

Diverse cortical codes for scene segmentation in primate auditory cortex

The temporal coherence of amplitude fluctuations is a critical cue for segmentation of complex auditory scenes. The auditory system must accurately demarcate the onsets and offsets of acoustic signals. We explored how and how well the timing of onsets and offsets of gated tones are encoded by auditory cortical neurons in awake rhesus macaques. Temporal features of this representation were isolated by presenting otherwise identical pure tones of differing durations. Cortical response patterns were diverse, including selective encoding of onset and offset transients, tonic firing, and sustained suppression. Spike train classification methods revealed that many neurons robustly encoded tone duration despite substantial diversity in the encoding process. Excellent discrimination performance was achieved by neurons whose responses were primarily phasic at tone offset and by those that responded robustly while the tone persisted. Although diverse cortical response patterns converged on effective duration discrimination, this diversity significantly constrained the utility of decoding models referenced to a spiking pattern averaged across all responses or averaged within the same response category. Using maximum likelihood-based decoding models, we demonstrated that the spike train recorded in a single trial could support direct estimation of stimulus onset and offset. Comparisons between different decoding models established the substantial contribution of bursts of activity at sound onset and offset to demarcating the temporal boundaries of gated tones. Our results indicate that relatively few neurons suffice to provide temporally precise estimates of such auditory "edges," particularly for models that assume and exploit the heterogeneity of neural responses in awake cortex.

Language experience enhances early cortical pitch-dependent responses

Pitch processing at cortical and subcortical stages of processing is shaped by language experience. We recently demonstrated that specific components of the cortical pitch response (CPR) index the more rapidly-changing portions of the high rising Tone 2 of Mandarin Chinese, in addition to marking pitch onset and sound offset. In this study, we examine how language experience (Mandarin vs. English) shapes the processing of different temporal attributes of pitch reflected in the CPR components using stimuli representative of within-category variants of Tone 2. Results showed that the magnitude of CPR components (Na-Pb and Pb-Nb) and the correlation between these two components and pitch acceleration were stronger for the Chinese listeners compared to English listeners for stimuli that fell within the range of Tone 2 citation forms. Discriminant function analysis revealed that the Na-Pb component was more than twice as important as Pb-Nb in grouping listeners by language affiliation. In addition, a stronger stimulus-dependent, rightward asymmetry was observed for the Chinese group at the temporal, but not frontal, electrode sites. This finding may reflect selective recruitment of experience-dependent, pitch-specific mechanisms in right auditory cortex to extract more complex, time-varying pitch patterns. Taken together, these findings suggest that long-term language experience shapes early sensory level processing of pitch in the auditory cortex, and that the sensitivity of the CPR may vary depending on the relative linguistic importance of specific temporal attributes of dynamic pitch.

Cortical pitch response components index stimulus onset/offset and dynamic features of pitch contours

Voice pitch is an important information-bearing component of language that is subject to experience dependent plasticity at both early cortical and subcortical stages of processing. We have already demonstrated that pitch onset component (Na) of the cortical pitch response (CPR) is sensitive to flat pitch and its salience … CPR responses from Chinese listeners were elicited by three citation forms varying in pitch acceleration and duration. Results showed that the pitch onset component (Na) was invariant to changes in acceleration. In contrast, Na–Pb and Pb–Nb showed a systematic decrease in the interpeak latency and decrease in amplitude with increase in pitch acceleration that followed the time course of pitch change across the three stimuli. A strong correlation with pitch acceleration was observed for these two components only – a putative index of pitch-relevant neural activity associated with the more rapidly-changing portions of the pitch contour. Pc–Nc marks unambiguously the stimulus offset … and their functional roles as related to sensory and cognitive properties of the stimulus. [Corrected]

The function of offset neurons in auditory information processing

Offset neurons which respond to the termination of the sound stimulation may play important roles in auditory temporal information processing, sound signal recognition, and complex distinction. Two additional possible mechanisms were reviewed: neural inhibition and the intrinsic conductance property of offset neuron membranes. The underlying offset response was postulated to be located in the superior paraolivary nucleus of mice. The biological significance of the offset neurons was discussed as well.

Hearing an illusory vowel in noise: suppression of auditory cortical activity

Human hearing is constructive. For example, when a voice is partially replaced by an extraneous sound (e.g., on the telephone due to a transmission problem), the auditory system may restore the missing portion so that the voice can be perceived as continuous (Miller and Licklider, 1950; for review, see Bregman, 1990; Warren, 1999). The neural mechanisms underlying this continuity illusion have been studied mostly with schematic stimuli (e.g., simple tones) and are still a matter of debate (for review, see Petkov and Sutter, 2011). The goal of the present study was to elucidate how these mechanisms operate under more natural conditions. Using psychophysics and electroencephalography (EEG), we assessed simultaneously the perceived continuity of a human vowel sound through interrupting noise and the concurrent neural activity. We found that vowel continuity illusions were accompanied by a suppression of the 4 Hz EEG power in auditory cortex (AC) that was evoked by the vowel interruption. This suppression was stronger than the suppression accompanying continuity illusions of a simple tone. Finally, continuity perception and 4 Hz power depended on the intactness of the sound that preceded the vowel (i.e., the auditory context). These findings show that a natural sound may be restored during noise due to the suppression of 4 Hz AC activity evoked early during the noise. This mechanism may attenuate sudden pitch changes, adapt the resistance of the auditory system to extraneous sounds across auditory scenes, and provide a useful model for assisted hearing devices.

Effect of speech degradation on top-down repair: phonemic restoration with simulations of cochlear implants and combined electric-acoustic stimulation

The brain, using expectations, linguistic knowledge, and context, can perceptually restore inaudible portions of speech. Such top-down repair is thought to enhance speech intelligibility in noisy environments. Hearing-impaired listeners with cochlear implants commonly complain about not understanding speech in noise. We hypothesized that the degradations in the bottom-up speech signals due to the implant signal processing may have a negative effect on the top-down repair mechanisms, which could partially be responsible for this complaint. To test the hypothesis, phonemic restoration of interrupted sentences was measured with young normal-hearing listeners using a noise-band vocoder simulation of implant processing. Decreasing the spectral resolution (by reducing the number of vocoder processing channels from 32 to 4) systematically degraded the speech stimuli. Supporting the hypothesis, the size of the restoration benefit varied as a function of spectral resolution. A significant benefit was observed only at the highest spectral resolution of 32 channels. With eight channels, which resembles the resolution available to most implant users, there was no significant restoration effect. Combined electric–acoustic hearing has been previously shown to provide better intelligibility of speech in adverse listening environments. In a second configuration, combined electric–acoustic hearing was simulated by adding low-pass-filtered acoustic speech to the vocoder processing. There was a slight improvement in phonemic restoration compared to the first configuration; the restoration benefit was observed at spectral resolutions of both 16 and 32 channels. However, the restoration was not observed at lower spectral resolutions (four or eight channels). Overall, the findings imply that the degradations in the bottom-up signals alone (such as occurs in cochlear implants) may reduce the top-down restoration of speech.

The sound of silence: ionic mechanisms encoding sound termination

Offset responses upon termination of a stimulus are crucial for perceptual grouping and gap detection. These gaps are key features of vocal communication, but an ionic mechanism capable of generating fast offsets from auditory stimuli has proven elusive. Offset firing arises in the brainstem superior paraolivary nucleus (SPN), which receives powerful inhibition during sound and converts this into precise action potential (AP) firing upon sound termination. Whole-cell patch recording in vitro showed that offset firing was triggered by IPSPs rather than EPSPs. We show that AP firing can emerge from inhibition through integration of large IPSPs, driven by an extremely negative chloride reversal potential (E(Cl)), combined with a large hyperpolarization-activated nonspecific cationic current (I(H)), with a secondary contribution from a T-type calcium conductance (I(TCa)). On activation by the IPSP, I(H) potently accelerates the membrane time constant, so when the sound ceases, a rapid repolarization triggers multiple offset APs that match onset timing accuracy.

Binaural sensitivity changes between cortical on and off responses

Neurons exhibiting on and off responses with different frequency tuning have previously been described in the primary auditory cortex (A1) of anesthetized and awake animals, but it is unknown whether other tuning properties, including sensitivity to binaural localization cues, also differ between on and off responses. We measured the sensitivity of A1 neurons in anesthetized ferrets to 1) interaural level differences (ILDs), using unmodulated broadband noise with varying ILDs and average binaural levels, and 2) interaural time delays (ITDs), using sinusoidally amplitude-modulated broadband noise with varying envelope ITDs. We also assessed fine-structure ITD sensitivity and frequency tuning, using pure-tone stimuli. Neurons most commonly responded to stimulus onset only, but purely off responses and on-off responses were also recorded. Of the units exhibiting significant binaural sensitivity nearly one-quarter showed binaural sensitivity in both on and off responses, but in almost all (∼97%) of these units the binaural tuning of the on responses differed significantly from that seen in the off responses. Moreover, averaged, normalized ILD and ITD tuning curves calculated from all units showing significant sensitivity to binaural cues indicated that on and off responses displayed different sensitivity patterns across the population. A principal component analysis of ITD response functions suggested a continuous cortical distribution of binaural sensitivity, rather than discrete response classes. Rather than reflecting a release from inhibition without any functional significance, we propose that binaural off responses may be important to cortical encoding of sound-source location.

Tracking vocal pitch through noise: neural correlates in nonprimary auditory cortex

In natural environments, a sound can be heard as stable despite the presence of other occasionally louder sounds. For example, when a portion in a voice is replaced by masking noise, the interrupted voice may still appear illusorily continuous. Previous research found that continuity illusions of simple interrupted sounds, such as tones, are accompanied by weaker activity in the primary auditory cortex (PAC) during the interruption than veridical discontinuity percepts of these sounds. Here, we studied whether continuity illusions of more natural and more complex sounds also emerge from this mechanism. We used psychophysics and functional magnetic resonance imaging in humans to measure simultaneously continuity ratings and blood oxygenation level-dependent activity to vowels that were partially replaced by masking noise. Consistent with previous results on tone continuity illusions, we found listeners' reports of more salient vowel continuity illusions associated with weaker activity in auditory cortex (compared with reports of veridical discontinuity percepts of physically identical stimuli). In contrast to the reduced activity to tone continuity illusions in PAC, this reduction was localized in the right anterolateral Heschl's gyrus, a region that corresponds more to the non-PAC. Our findings suggest that the ability to hear differently complex sounds as stable during other louder sounds may be attributable to a common suppressive mechanism that operates at different levels of sound representation in auditory cortex.

Perceived continuity and pitch shifts for complex tones with unresolved harmonics

Brief complex tone bursts with fundamental frequencies (F0s) of 100, 125, 166.7, and 250 Hz were bandpass filtered between the 22nd and 30th harmonics, to produce waveforms with five regularly occurring envelope peaks ("pitch pulses") that evoked pitches associated with their repetition period. Two such tone bursts were presented sequentially and separated by a silent interval of two periods (2/F0). When the relative phases of the two bursts were varied, such that the interpulse interval (IPI) between the last pulse of the first burst and the first pulse of the second burst was varied, the pitch of the whole sequence was little affected. This is consistent with previous results suggesting that the pitch integration window may be "reset" by a discontinuity. However, when the interval between the two bursts was filled with a noise with the same spectral envelope as the complex, variations in IPI had substantial effects on the pitch of the sequence. It is suggested that the presence of the noise causes the two tones bursts to appear continuous, hence, resetting does not occur, and the pitch mechanism is sensitive to the phase discontinuity across the silent interval.

Evolutionary conservation and neuronal mechanisms of auditory perceptual restoration

Auditory perceptual 'restoration' occurs when the auditory system restores an occluded or masked sound of interest. Behavioral work on auditory restoration in humans began over 50 years ago using it to model a noisy environmental scene with competing sounds. It has become clear that not only humans experience auditory restoration: restoration has been broadly conserved in many species. Behavioral studies in humans and animals provide a necessary foundation to link the insights being obtained from human EEG and fMRI to those from animal neurophysiology. The aggregate of data resulting from multiple approaches across species has begun to clarify the neuronal bases of auditory restoration. Different types of neural responses supporting restoration have been found, supportive of multiple mechanisms working within a species. Yet a general principle has emerged that responses correlated with restoration mimic the response that would have been given to the uninterrupted sound of interest. Using the same technology to study different species will help us to better harness animal models of 'auditory scene analysis' to clarify the conserved neural mechanisms shaping the perceptual organization of sound and to advance strategies to improve hearing in natural environmental settings.

Testing an auditory illusion in frogs: Perceptual restoration or sensory bias?

The human auditory system perceptually restores short deleted segments of speech and other sounds (e.g. tones) when the resulting silent gaps are filled by a potential masking noise. When this phenomenon, known as 'auditory induction', occurs, listeners experience the illusion of hearing an ongoing sound continuing through the interrupting noise even though the perceived sound is not physically present. Such illusions suggest that a key function of the auditory system is to allow listeners to perceive complete auditory objects with incomplete acoustic information, as may often be the case in multisource acoustic environments. At present, however, we know little about the possible functions of auditory induction in the sound-mediated behaviours of animals. The present study used two-choice phonotaxis experiments to test the hypothesis that female grey treefrogs, Hyla chrysoscelis, experience the illusory perceptual restoration of discrete pulses in the male advertisement call when pulses are deleted and replaced by a potential masking noise. While added noise restored some attractiveness to calls with missing pulses, there was little evidence to suggest that the frogs actually experienced the illusion of perceiving the missing pulses. Instead, the added noise appeared to function as an acoustic appendage that made some calls more attractive than others as a result of sensory biases, the expression of which depended on the temporal order and acoustic structure of the added appendages.

Nonoverlapping sets of synapses drive on responses and off responses in auditory cortex

Neurons in visual, somatosensory, and auditory cortex can respond to the termination as well as the onset of a sensory stimulus. In auditory cortex, these off responses may underlie the ability of the auditory system to use sound offsets as cues for perceptual grouping. Off responses have been widely proposed to arise from postinhibitory rebound, but this hypothesis has never been directly tested. We used in vivo whole-cell recordings to measure the synaptic inhibition evoked by sound onset. We find that inhibition is invariably transient, indicating that off responses are not caused by postinhibitory rebound in auditory cortical neurons. Instead, on and off responses appear to be driven by distinct sets of synapses, because they have distinct frequency tuning and different excitatory-inhibitory balance. Furthermore, an on-on sequence causes complete forward suppression, whereas an off-on sequence causes no suppression at all. We conclude that on and off responses are driven by largely nonoverlapping sets of synaptic inputs.

Understanding pitch perception as a hierarchical process with top-down modulation

Pitch is one of the most important features of natural sounds, underlying the perception of melody in music and prosody in speech. However, the temporal dynamics of pitch processing are still poorly understood. Previous studies suggest that the auditory system uses a wide range of time scales to integrate pitch-related information and that the effective integration time is both task- and stimulus-dependent. None of the existing models of pitch processing can account for such task- and stimulus-dependent variations in processing time scales. This study presents an idealized neurocomputational model, which provides a unified account of the multiple time scales observed in pitch perception. The model is evaluated using a range of perceptual studies, which have not previously been accounted for by a single model, and new results from a neurophysiological experiment. In contrast to other approaches, the current model contains a hierarchy of integration stages and uses feedback to adapt the effective time scales of processing at each stage in response to changes in the input stimulus. The model has features in common with a hierarchical generative process and suggests a key role for efferent connections from central to sub-cortical areas in controlling the temporal dynamics of pitch processing.

Illusory Vowels Resulting from Perceptual Continuity: A Functional Magnetic Resonance Imaging Study

We used functional magnetic resonance imaging to study the neural processing of vowels whose perception depends on the continuity illusion. Participants heard sequences of two-formant vowels under a number of listening conditions. In the “vowel conditions,” both formants were always present simultaneously and the stimuli were perceived as speech-like. Contrasted with a range of nonspeech sounds, these vowels elicited activity in the posterior middle temporal gyrus (MTG) and superior temporal sulcus (STS). When the two formants alternated in time, the “speech-likeness” of the sounds was reduced. It could be partially restored by filling the silent gaps in each formant with bands of noise (the “Illusion” condition) because the noise induced an illusion of continuity in each formant region, causing the two formants to be perceived as simultaneous. However, this manipulation was only effective at low formant-to-noise ratios (FNRs). When the FNR was increased, the illusion broke down (the “illusion-break” condition). Activation in vowel-sensitive regions of the MTG was greater in the illusion than in the illusion-break condition, consistent with the perception of Illusion stimuli as vowels. Activity in Heschl's gyri (HG), the approximate location of the primary auditory cortex, showed the opposite pattern, and may depend instead on the number of perceptual onsets in a sound. Our results demonstrate that speech-sensitive regions of the MTG are sensitive not to the physical characteristics of the stimulus but to the perception of the stimulus as speech, and also provide an anatomically distinct, objective physiological correlate of the continuity illusion in human listeners.

A cascade autocorrelation model of pitch perception

Autocorrelation algorithms, in combination with computational models of the auditory periphery, have been successfully used to predict the pitch of a wide range of complex stimuli. However, new stimuli are frequently offered as counterexamples to the viability of this approach. This study addresses the issue of whether in the light of these challenges the predictive power of autocorrelation can be preserved by changes to the peripheral model and the computational algorithm. An existing model is extended by the addition of a low-pass filter of the summary integration of the individual within-channel autocorrelations. Other recent developments are also incorporated, including nonlinear processing on the basilar membrane and the use of integration time constants that are proportional to the autocorrelation lags. The modified and extended model predicts with reasonable success the pitches of a range of stimuli that have proved problematic for earlier implementations of the autocorrelation principle. The evaluation stimuli include short tone sequences, click trains consisting of alternating interclick intervals, click trains consisting of mixtures of regular and irregular intervals, shuffled click trains, and transposed tones.

A model of perceptual segregation based on clustering the time series of the simulated auditory nerve firing probability

This paper introduces a model that accounts quantitatively for a phenomenon of perceptual segregation, the simultaneous perception of more than one pitch in a single complex sound. The method is based on a characterization of the time-varying spike probability generated by a model of cochlear responses to sounds. It demonstrates how the autocorrelation theories of pitch perception contain the necessary elements to define a specific measure in the phase space of the simulated auditory nerve probability of firing time series. This measure was motivated in the first instance by the correlation dimension of the attractor; however, it has been modified in several ways in order to increase the neurobiological plausibility. This quantity characterizes each of the cochlear frequency channels and gives rise to a channel clustering criterion. The model computes the clusters and the pitch estimates simultaneously using the same processing mechanisms of delay lines; therefore, it respects the biological constraints in a similar way to temporal theories of pitch. The model successfully explains a wide range of perceptual experiments.

The effect of a flashing visual stimulus on the auditory continuity illusion

The effect of a visual stimulus on the auditory continuity illusion was examined. Observers judged whether a tone that was repeatedly alternated with a band-pass noise was continuous or discontinuous. In most observers, a transient visual stimulus that was synchronized with the onset of the noise increased the limit of illusory continuity in terms of maximum noise duration and maximum tone level. The smaller the asynchrony between the noise onset and the visual stimulus onset, the larger the visual effect on this illusion. On the other hand, detection of a tone added to the noise was not enhanced by the visual stimulus. These results cannot be fully explained by the conventional theory that illusory continuity is created by the decomposition of peripheral excitation produced by the occluding sound.

Dynamic aspects of the continuity illusion: perception of level and of the depth, rate, and phase of modulation

The perception of modulation of a tone interrupted by a noise burst was investigated. The tone and its modulation were perceived as continuing through the noise. In experiment 1, subjects rated the similarity of an uninterrupted tone and a tone interrupted by noise, in terms of the perceived level and modulation depth of the sinusoidal carrier. The values of these parameters in the central portion of the uninterrupted tone were systematically varied. Both amplitude and frequency modulation (AM and FM) were used. The results indicated that the perceived level and modulation depth of the carrier did not change greatly during the noise burst. When the modulation rate differed before and after the noise burst, the modulation-rate transition was perceived to occur near the end of the noise burst for the FM stimuli. Hence, for these stimuli, the continuity illusion appears to be dominated by the portion of the tone before, rather than after, the interruption. Results for the AM stimuli showed a non-significant trend in the same direction. Experiment 2 used forced-choice tasks to evaluate the ability to detect a change in the ongoing phase of AM and FM following interruption by a noise burst. The results confirmed earlier findings for FM tones, and extended them to AM tones, showing that listeners lost track of the phase of the modulation, even though the modulation was perceived as continuous.

How the brain separates sounds

In everyday life we often listen to one sound, such as someone's voice, in a background of competing sounds. To do this, we must assign simultaneously occurring frequency components to the correct source, and organize sounds appropriately over time. The physical cues that we exploit to do so are well-established; more recent research has focussed on the underlying neural bases, where most progress has been made in the study of a form of sequential organization known as "auditory streaming". Listeners' sensitivity to streaming cues can be captured in the responses of neurons in the primary auditory cortex, and in EEG wave components with a short latency (< 200ms). However, streaming can be strongly affected by attention, suggesting that this early processing either receives input from non-auditory areas, or feeds into processes that do.

Across-frequency interference effects in fundamental frequency discrimination: questioning evidence for two pitch mechanisms

Carlyon and Shackleton [J. Acoust. Soc. Am. 95, 3541-3554 (1994)] presented an influential study supporting the existence of two pitch mechanisms, one for complex tones containing resolved and one for complex tones containing only unresolved components. The current experiments provide an alternative explanation for their finding, namely the existence of across-frequency interference in fundamental frequency (F0) discrimination. Sensitivity (d') was measured for F0 discrimination between two sequentially presented 400 ms complex (target) tones containing only unresolved components. In experiment 1, the target was filtered between 1375 and 15,000 Hz, had a nominal F0 of 88 Hz, and was presented either alone or with an additional complex tone ("interferer"). The interferer was filtered between 125-625 Hz, and its F0 varied between 88 and 114.4 Hz across blocks. Sensitivity was significantly reduced in the presence of the interferer, and this effect decreased as its F0 was moved progressively further from that of the target. Experiment 2 showed that increasing the level of a synchronously gated lowpass noise that spectrally overlapped with the interferer reduced this "pitch discrimination interference (PDI)". In experiment 3A, the target was filtered between 3900 and 5400 Hz and had an F0 of either 88 or 250 Hz. It was presented either alone or with an interferer, filtered between 1375 and 1875 Hz with an F0 corresponding to the nominal target F0. PDI was larger in the presence of the resolved (250 Hz F0) than in the presence of the unresolved (88 Hz F0) interferer, presumably because the pitch of the former was more salient than that of the latter. Experiments 4A and 4B showed that PDI was reduced but not eliminated when the interferer was gated on 200 ms before and off 200 ms after the target, and that some PDI was observed with a continuous interferer. The current findings provide an alternative interpretation of a study supposedly providing strong evidence for the existence of two pitch mechanisms.

The effects of real and illusory glides on pure-tone frequency discrimination

Experiment 1 measured pure-tone frequency difference limens (DLs) at 1 and 4 kHz. The stimuli had two steady-state portions, which differed in frequency for the target. These portions were separated by a middle section of varying length, which consisted of a silent gap, a frequency glide, or a noise burst (conditions: gap, glide, and noise, respectively). The noise burst created an illusion of the tone continuing through the gap. In the first condition, the stimuli had an overall duration of 500 ms. In the second condition, stimuli had a fixed 50-ms middle section, and the overall duration was varied. DLs were lower for the glide than for the gap condition, consistent with the idea that the auditory system contains a mechanism specific for the detection of dynamic changes. DLs were generally lower for the noise than for the gap condition, suggesting that this mechanism extracts information from an illusory glide. In a second experiment, pure-tone frequency direction-discrimination thresholds were measured using similar stimuli as for the first experiment. For this task, the type of the middle section hardly affected the thresholds, suggesting that the frequency-change detection mechanism does not facilitate the identification of the direction of frequency changes.

Neural sensitivity to periodicity in the inferior colliculus: evidence for the role of cochlear distortions

Responses of low characteristic-frequency (CF) neurons in the inferior colliculus were obtained to amplitude-modulated (AM) high-frequency tones in which the modulation rate was equal to the neuron's CF. Despite all spectral components lying outside the pure tone-evoked response areas, discharge rates were modulated by the AM signals. Introducing a low-frequency tone (CF - 1 Hz) to the same ear as the AM tones produced a 1-Hz beat in the neural response. Introducing a tone (CF - 1 Hz) to the opposite ear to the AM tone also produced a beat in the neural response, with the beat at the period of the interaural phase difference between the CF - 1 Hz tone in one ear, and the AM rate in the other ear. The monaural and interaural interactions of the AM signals with introduced pure tones suggest that AM tones generate combination tones, (inter-modulation distortion) on the basilar membrane. These interact with low-frequency tones presented to the same ear to produce monaural beats on the basilar membrane, modulating the responses of inferior colliculus (IC) neurons on the 1-Hz period of the monaural beats or interacting binaurally with neural input generated in response to stimulation of the opposite ear. The auditory midbrain appears to show a robust representation of cochlear distortions generated by amplitude-modulated sounds.

Auditory steady-state responses reveal amplitude modulation gap detection thresholds

Auditory evoked magnetic fields were recorded from the left hemisphere of healthy subjects using a 37-channel magnetometer while stimulating the right ear with 40-Hz amplitude modulated (AM) tone-bursts with 500-Hz carrier frequency in order to study the time-courses of amplitude and phase of auditory steady-state responses (ASSRs). The stimulus duration of 300 ms and the duration of the silent periods (3-300 ms) between succeeding stimuli were chosen to address the question whether the time-course of the ASSR can reflect both temporal integration and temporal resolution in the central auditory processing. Long lasting perturbations of the ASSR were found after gaps in the AM sound, even for gaps of short duration. These were interpreted as evidences for an auditory reset mechanism. Concomitant psycho-acoustical tests corroborated that gap durations perturbing the ASSR were in the same range as the threshold for AM gap detection. Magnetic source localizations estimated the ASSR sources in the primary auditory cortex, suggesting that the processing of temporal structures in the sound is performed at or below the cortical level.

The Neurophysiological Basis of the Auditory Continuity Illusion: A Mismatch Negativity Study

A sound turned off for a short moment can be perceived as continuous if the silent gap is filled with noise. The neural mechanisms underlying this “continuity illusion” were investigated using the mismatch negativity (MMN), an eventrelated potential reflecting the perception of a sudden change in an otherwise regular stimulus sequence. The MMN was recorded in four conditions using an oddball paradigm. The standards consisted of 500-Hz, 120-msec tone pips that were either physically continuous (Condition 1) or were interrupted by a 40-msec silent gap (Condition 2). The deviants consisted of the interrupted tone, but with the silent gap filled by a burst of bandpass-filtered noise. The noise either occupied the same frequency region as the tone and elicited the continuity illusion (Conditions 1a and 2a), or occupied a remote frequency region and did not elicit the illusion (Conditions 1b and 2b). We predicted that, if the continuity illusion is determined before MMN generation, then, other things being equal, the MMN should be larger in conditions where the deviants are perceived as continuous and the standards as interrupted or vice versa, than when both were perceived as continuous or both interrupted. Consistent with this prediction, we observed an interaction between standard type and noise frequency region, with the MMN being larger in Condition 1a than in Condition 1b, but smaller in Condition 2a than in Condition 2b. Because the subjects were instructed to ignore the tones and watch a silent movie during the recordings, the results indicate that the continuity illusion can occur outside the focus of attention. Furthermore, the latency of the MMN (less than approximately 200 msec postdeviance onset) places an upper limit on the stage of neural processing responsible for the illusion.

Virtual pitch integration for asynchronous harmonics

This experiment examined the generation of virtual pitch for harmonically related tones that do not overlap in time. The interval between successive tones was systematically varied in order to gauge the integration period for virtual pitch. A pitch discrimination task was employed, and both harmonic and nonharmonic tone series were tested. The results confirmed that a virtual pitch can be generated by a series of brief, harmonically related tones that are separated in time. Robust virtual pitch information can be derived for intervals between successive 40-ms tones of up to about 45 ms, consistent with a minimum estimate of integration period of about 210 ms. Beyond intertone intervals of 45 ms, performance becomes more variable and approaches an upper limit where discrimination of tone sequences can be undertaken on the basis of the individual frequency components. The individual differences observed in this experiment suggest that the ability to derive a salient virtual pitch varies across listeners.

Temporal pitch mechanisms in acoustic and electric hearing

Two experiments investigated pitch perception for stimuli where the place of excitation was held constant. Experiment 1 used pulse trains in which the interpulse interval alternated between 4 and 6 ms. In experiment 1a these "4-6" pulse trains were bandpass filtered between 3900 and 5300 Hz and presented acoustically against a noise background to normal listeners. The rate of an isochronous pulse train (in which all the interpulse intervals were equal) was adjusted so that its pitch matched that of the "4-6" stimulus. The pitch matches were distributed unimodally, had a mean of 5.7 ms, and never corresponded to either 4 or to 10 ms (the period of the stimulus). In experiment 1b the pulse trains were presented both acoustically to normal listeners and electrically to users of the LAURA cochlear implant, via a single channel of their device. A forced-choice procedure was used to measure psychometric functions, in which subjects judged whether the 4-6 stimulus was higher or lower in pitch than isochronous pulse trains having periods of 3, 4, 5, 6, or 7 ms. For both groups of listeners, the point of subjective equality corresponded to a period of 5.6 to 5.7 ms. Experiment 1c confirmed that these psychometric functions were monotonic over the range 4-12 ms. In experiment 2, normal listeners adjusted the rate of an isochronous filtered pulse train to match the pitch of mixtures of pulse trains having rates of F1 and F2 Hz, passed through the same bandpass filter (3900-5400 Hz). The ratio F2/F1 was 1.29 and F1 was either 70, 92, 109, or 124 Hz. Matches were always close to F2 Hz. It is concluded that the results of both experiments are inconsistent with models of pitch perception which rely on higher-order intervals. Together with those of other published data on purely temporal pitch perception, the data are consistent with a model in which only first-order interpulse intervals contribute to pitch, and in which, over the range 0-12 ms, longer intervals receive higher weights than short intervals.

Searching for the time constant of neural pitch extraction

Multichannel, auditory models have been repeatedly used to explain many aspects of human pitch perception. Among the most successful ones are models where pitch is estimated based on an analysis of periodicity in the simulated auditory-nerve firing. This periodicity analysis is typically implemented as a running autocorrelation, i.e., the autocorrelation is calculated within a temporal window which is shifted along the time axis. The window was suggested to have an exponential decay with time-constant estimates between 1.5 and 100 ms. The window length determines the minimal integration time of pitch extraction. The present experiments are designed to quantify the temporal window of pitch extraction using regular-interval noises (RINs). RINs were generated by concatenating equal-duration noise samples which produce a pitch corresponding to the reciprocal of the sample duration when the samples are identical (periodic noise). When the samples are independent, the stimulus is Gaussian noise and produces no pitch. Using RIN stimuli where periodic portions interchange with aperiodic portions, it is shown that the temporal window of pitch extraction cannot be modeled using a single time constant but that the size of the temporal window depends on the pitch itself.