Contextual effects in the identification of nonspeech auditory patterns

Cues to reduce modulation informational masking

The detectability of target amplitude modulation (AM) can be reduced by masker AM in the same carrier-frequency region. It can be reduced even further, however, if the masker-AM rate is uncertain [Conroy and Kidd, J. Acoust. Soc. Am. 149, 3665-3673 (2021)]. This study examined the effectiveness of contextual cues in reducing this latter, uncertainty-related effect (modulation informational masking). Observers were tasked with detecting fixed-rate target sinusoidal amplitude modulation (SAM) in the presence of masker SAM applied simultaneously to the same broadband-noise carrier. A single-interval, two-alternative forced-choice detection procedure was used to measure sensitivity for the target SAM; masker-AM-rate uncertainty was created by randomly selecting the AM rate of the masker SAM on each trial. Relative to an uncued condition, a pretrial cue to the masker SAM significantly improved sensitivity for the target SAM; a cue to the target SAM, however, did not. The delay between the cue-interval offset and trial-interval onset did not affect the size of the masker-cue benefit, suggesting that adaptation of the masker SAM was not responsible. A simple model of within-AM-channel masking captured important trends in the psychophysical data, suggesting that reduced masker-AM-rate uncertainty may have played a relatively minor role in the masker-cue benefit.

Evaluating peripheral versus central contributions to spectral context effects in speech perception

Perception of a sound is influenced by spectral properties of surrounding sounds. When frequencies are absent in a preceding acoustic context before being introduced in a subsequent target sound, detection of those frequencies is facilitated via an auditory enhancement effect (EE). When spectral composition differs across a preceding context and subsequent target sound, those differences are perceptually magnified and perception shifts via a spectral contrast effect (SCE). Each effect is thought to receive contributions from peripheral and central neural processing, but the relative contributions are unclear. The present experiments manipulated ear of presentation to elucidate the degrees to which peripheral and central processes contributed to each effect in speech perception. In Experiment 1, EE and SCE magnitudes in consonant categorization were substantially diminished through contralateral presentation of contexts and targets compared to ipsilateral or bilateral presentations. In Experiment 2, spectrally complementary contexts were presented dichotically followed by the target in only one ear. This arrangement was predicted to produce context effects peripherally and cancel them centrally, but the competing contralateral context minimally decreased effect magnitudes. Results confirm peripheral and central contributions to EEs and SCEs in speech perception, but both effects appear to be primarily due to peripheral processing.

Auditory enhancement under forward masking in normal-hearing and hearing-impaired listeners

A target within a spectrally notched masker can be enhanced by a preceding copy of the masker. Enhancement can also increase the effectiveness of the target as a forward masker. Enhancement has been reported in hearing-impaired listeners under simultaneous but not forward masking. However, previous studies of enhancement under forward masking did not fully assess the potential effect of differences in sensation level or spectral resolution between the normal-hearing and hearing-impaired listeners. This study measured enhancement via forward masking in hearing-impaired and age-matched normal-hearing listeners with different spectral notches in the masker, to account for potential differences in frequency selectivity, and with levels equated by adding a background masking noise to equate both sensation level and sound pressure level or by reducing the sound pressure level of the stimuli to equate sensation level. Hearing-impaired listeners showed no significant enhancement, regardless of spectral notch width. Normal-hearing listeners showed enhancement at high levels, but showed less enhancement when sensation levels were reduced to match those of the hearing-impaired group, either by reducing sound levels or by adding a masking noise. The results confirm a lack of forward-masked enhancement in hearing-impaired listeners but suggest this may be partly due to reduced sensation level.

Auditory enhancement and spectral contrast effects in speech perception

The auditory system is remarkably sensitive to changes in the acoustic environment. This is exemplified by two classic effects of preceding spectral context on perception. In auditory enhancement effects (EEs), the absence and subsequent insertion of a frequency component increases its salience. In spectral contrast effects (SCEs), spectral differences between earlier and later (target) sounds are perceptually magnified, biasing target sound categorization. These effects have been suggested to be related, but have largely been studied separately. Here, EEs and SCEs are demonstrated using the same speech materials. In Experiment 1, listeners categorized vowels (/ɪ/-/ɛ/) or consonants (/d/-/g/) following a sentence processed by a bandpass or bandstop filter (vowel tasks: 100-400 or 550-850 Hz; consonant tasks: 1700-2700 or 2700-3700 Hz). Bandpass filtering produced SCEs and bandstop filtering produced EEs, with effect magnitudes significantly correlated at the individual differences level. In Experiment 2, context sentences were processed by variable-depth notch filters in these frequency regions (-5 to -20 dB). EE magnitudes increased at larger notch depths, growing linearly in consonant categorization. This parallels previous research where SCEs increased linearly for larger spectral peaks in the context sentence. These results link EEs and SCEs, as both shape speech categorization in orderly ways.

Auditory sequential accumulation of spectral information

In many listening situations, information about the spectral content of a target sound may be distributed over time, and estimating the target spectrum requires efficient sequential processing. Listeners' ability to estimate the spectrum of a random-frequency, six-tone complex was investigated and the spectral content of the complex was revealed using a sequence of bursts. Whether each of the six tones was presented within each burst was determined at random according to a presentation probability. In separate conditions, the presentation probabilities (p) ranged from 0.2 to 1, the total number of bursts varied from 1 to 16, and the inter-burst interval was either 0 or 200 ms. To evaluate the information acquired by the listener, the burst sequence was followed, after a 500-ms silent interval, by the six-tone complex acting as an informational masker and the listener was required to detect a pure-tone target presented simultaneously with the masker. Greater performance in this task indicates more accurate estimation of the spectrum of the complex by the listener. Evidence for integration of information across bursts was observed, and the integration process did not significantly depend on inter-burst interval.

The effect of frequency cueing on the perceptual segregation of simultaneous tones: Bottom-up and top-down contributions

Listeners were presented with two simultaneous tones of different frequencies (more than one octave apart) and asked to identify the tone that was amplitude-modulated while a tonal precursor was presented to cue the frequency of the lower frequency tone. Performance thresholds were estimated based on the duration of the tone-pair. In Exp. I the duration of the precursor varied from 100 to 400 ms and the inter-stimulus interval (ISI) between the precursor and the tone-pair varied from 0 to 1 s. The presence of the precursor facilitated segregation. As the ISI increased, the facilitation effect of the precursor increased for the precursor durations of 100 and 200 ms, but not for the 400-ms precursor duration. When the precursor was presented to the contralateral ear relative to the tone-pair in Exp. II, no significant change to the precursor effect was observed. These observations contradict the predictions of the model based solely on bottom-up processing, suggesting the likely involvement of top-down processes.

Neural Correlates of Auditory Figure-Ground Segregation Based on Temporal Coherence

To make sense of natural acoustic environments, listeners must parse complex mixtures of sounds that vary in frequency, space, and time. Emerging work suggests that, in addition to the well-studied spectral cues for segregation, sensitivity to temporal coherence-the coincidence of sound elements in and across time-is also critical for the perceptual organization of acoustic scenes. Here, we examine pre-attentive, stimulus-driven neural processes underlying auditory figure-ground segregation using stimuli that capture the challenges of listening in complex scenes where segregation cannot be achieved based on spectral cues alone. Signals ("stochastic figure-ground": SFG) comprised a sequence of brief broadband chords containing random pure tone components that vary from 1 chord to another. Occasional tone repetitions across chords are perceived as "figures" popping out of a stochastic "ground." Magnetoencephalography (MEG) measurement in naïve, distracted, human subjects revealed robust evoked responses, commencing from about 150 ms after figure onset that reflect the emergence of the "figure" from the randomly varying "ground." Neural sources underlying this bottom-up driven figure-ground segregation were localized to planum temporale, and the intraparietal sulcus, demonstrating that this area, outside the "classic" auditory system, is also involved in the early stages of auditory scene analysis."

Informational Masking in Normal-Hearing and Hearing-Impaired Listeners Measured in a Nonspeech Pattern Identification Task

Individuals with sensorineural hearing loss (SNHL) often experience more difficulty with listening in multisource environments than do normal-hearing (NH) listeners. While the peripheral effects of sensorineural hearing loss certainly contribute to this difficulty, differences in central processing of auditory information may also contribute. To explore this issue, it is important to account for peripheral differences between NH and these hearing-impaired (HI) listeners so that central effects in multisource listening can be examined. In the present study, NH and HI listeners performed a tonal pattern identification task at two distant center frequencies (CFs), 850 and 3500 Hz. In an attempt to control for differences in the peripheral representations of the stimuli, the patterns were presented at the same sensation level (15 dB SL), and the frequency deviation of the tones comprising the patterns was adjusted to obtain equal quiet pattern identification performance across all listeners at both CFs. Tonal sequences were then presented at both CFs simultaneously (informational masking conditions), and listeners were asked either to selectively attend to a source (CF) or to divide attention between CFs and identify the pattern at a CF designated after each trial. There were large differences between groups in the frequency deviations necessary to perform the pattern identification task. After compensating for these differences, there were small differences between NH and HI listeners in the informational masking conditions. HI listeners showed slightly greater performance asymmetry between the low and high CFs than did NH listeners, possibly due to central differences in frequency weighting between groups.

Effects of a consistent target or masker voice on target speech intelligibility in two- and three-talker mixtures

When the spatial location or identity of a sound is held constant, it is not masked as effectively by competing sounds. This suggests that experience with a particular voice over time might facilitate perceptual organization in multitalker environments. The current study examines whether listeners benefit from experience with a voice only when it is the target, or also when it is a masker, using diotic presentation and a closed-set task (coordinate response measure). A reliable interaction was observed such that, in two-talker mixtures, consistency of masker or target voice over 3-7 trials significantly benefited target recognition performance, whereas in three-talker mixtures, target, but not masker, consistency was beneficial. Overall, this work suggests that voice consistency improves intelligibility, although somewhat differently when two talkers, compared to three talkers, are present, suggesting that consistent-voice information facilitates intelligibility in at least two different ways. Listeners can use a template-matching strategy to extract a known voice from a mixture when it is the target. However, consistent-voice information facilitates segregation only when two, but not three, talkers are present.

New perspectives on the measurement and time course of auditory enhancement

A target sound can become more audible and may "pop out" from a simultaneously presented masker if the masker is presented first by itself, as a precursor. This phenomenon, known as auditory enhancement, may reflect the general perceptual principle of contrast enhancement, which facilitates adaptation to ongoing acoustic conditions and the detection of new events. Little is known about the mechanisms underlying enhancement, and potential confounding factors have made the size of the effect and its time course a point of contention. Here we measured enhancement as a function of precursor duration and delay between precursor offset and target onset, using 2 single-interval pitch comparison tasks, which involve either same-different or up-down judgments, to avoid the potential confounds of earlier studies. Although these 2 tasks elicit different levels of performance and may reflect different underlying mechanisms, they produced similar amounts of enhancement. The effect decreased with decreasing precursor duration, but remained present for precursors as short as 62.5 ms, and decreased with increasing gap between the precursor and target, but remained measurable 1 s after the precursor. Additional conditions, examining the effect of precursor/masker similarity and the possible role of grouping and cueing, suggest multiple sources of auditory enhancement.

The salience of enhanced components within inharmonic complexes

A subjective listening task was used to measure the salience of enhanced components using typical intensity-enhancement stimuli, time-reversed versions of those stimuli, and stimuli which contained a frequency shift of the target component. Twenty-five listeners judged whether or not a pitch "stood out" within an inharmonic complex. For comparison, judgments also were made for stimuli with a single segment that consisted of a simultaneously masked target. The results indicate that the perceived salience of enhanced components is greater than might be predicted by the effective magnitude of those components, and that informational masking is likely involved.

Segregation of complex acoustic scenes based on temporal coherence

In contrast to the complex acoustic environments we encounter everyday, most studies of auditory segregation have used relatively simple signals. Here, we synthesized a new stimulus to examine the detection of coherent patterns (‘figures’) from overlapping ‘background’ signals. In a series of experiments, we demonstrate that human listeners are remarkably sensitive to the emergence of such figures and can tolerate a variety of spectral and temporal perturbations. This robust behavior is consistent with the existence of automatic auditory segregation mechanisms that are highly sensitive to correlations across frequency and time. The observed behavior cannot be explained purely on the basis of adaptation-based models used to explain the segregation of deterministic narrowband signals. We show that the present results are consistent with the predictions of a model of auditory perceptual organization based on temporal coherence. Our data thus support a role for temporal coherence as an organizational principle underlying auditory segregation.

DOI:http://dx.doi.org/10.7554/eLife.00699.001

Even when seated in the middle of a crowded restaurant, we are still able to distinguish the speech of the person sitting opposite us from the conversations of fellow diners and a host of other background noise. While we generally perform this task almost effortlessly, it is unclear how the brain solves what is in reality a complex information processing problem.

In the 1970s, researchers began to address this question using stimuli consisting of simple tones. When subjects are played a sequence of alternating high and low frequency tones, they perceive them as two independent streams of sound. Similar experiments in macaque monkeys reveal that each stream activates a different area of auditory cortex, suggesting that the brain may distinguish acoustic stimuli on the basis of their frequency.

However, the simple tones that are used in laboratory experiments bear little resemblance to the complex sounds we encounter in everyday life. These are often made up of multiple frequencies, and overlap—both in frequency and in time—with other sounds in the environment. Moreover, recent experiments have shown that if a subject hears two tones simultaneously, he or she perceives them as belonging to a single stream of sound even if they have different frequencies: models that assume that we distinguish stimuli from noise on the basis of frequency alone struggle to explain this observation.

Now, Teki, Chait, et al. have used more complex sounds, in which frequency components of the target stimuli overlap with those of background signals, to obtain new insights into how the brain solves this problem. Subjects were extremely good at discriminating these complex target stimuli from background noise, and computational modelling confirmed that they did so via integration of both frequency and temporal information. The work of Teki, Chait, et al. thus offers the first explanation for our ability to home in on speech and other pertinent sounds, even amidst a sea of background noise.

DOI:http://dx.doi.org/10.7554/eLife.00699.002

No Need for Templates in the Auditory Enhancement Effect

The audibility of a target tone in a multitone background masker is enhanced by the presentation of a precursor sound consisting of the masker alone. There is evidence that precursor-induced neural adaptation plays a role in this perceptual enhancement. However, the precursor may also be strategically used by listeners as a spectral template of the following masker to better segregate it from the target. In the present study, we tested this hypothesis by measuring the audibility of a target tone in a multitone masker after the presentation of precursors which, in some conditions, were made dissimilar to the masker by gating their components asynchronously. The precursor and the following sound were presented either to the same ear or to opposite ears. In either case, we found no significant difference in the amount of enhancement produced by synchronous and asynchronous precursors. In a second experiment, listeners had to judge whether a synchronous multitone complex contained exactly the same tones as a preceding precursor complex or had one tone less. In this experiment, listeners performed significantly better with synchronous than with asynchronous precursors, showing that asynchronous precursors were poorer perceptual templates of the synchronous multitone complexes. Overall, our findings indicate that precursor-induced auditory enhancement cannot be fully explained by the strategic use of the precursor as a template of the following masker. Our results are consistent with an explanation of enhancement based on selective neural adaptation taking place at a central locus of the auditory system.

The perceptual enhancement of tones by frequency shifts

In a chord of pure tones with a flat spectral profile, one tone can be perceptually enhanced relative to the other tones by the previous presentation of a slightly different chord. "Intensity enhancement" (IE) is obtained when the component tones of the two chords have the same frequencies, but in the first chord the target of enhancement is attenuated relative to the other tones. "Frequency enhancement" (FE) is obtained when both chords have a flat spectral profile, but the target of enhancement shifts in frequency from the first to the second chord. We report here an experiment in which IE and FE were measured using a task requiring the listener to indicate whether or not the second chord included a tone identical to a subsequent probe tone. The results showed that a global attenuation of the first chord relative to the second chord disrupted IE more than FE. This suggests that the mechanisms of IE and FE are not the same. In accordance with this suggestion, computations of the auditory excitation patterns produced by the chords indicate that the mechanism of IE is not sufficient to explain FE for small frequency shifts.

Enhancement of increments in spectral amplitude: further evidence for a mechanism based on central adaptation

The threshold for detecting a tone in a multitone masker is lower when the masker-plus-signal stimulus is preceded by a copy of the masker. One potential explanation of this "enhancement" phenomenon is that the -precursor stimulus acts as a "template" of the subsequent masker, thus helping listeners to segregate the signal from the masker. To assess this idea, we measured enhancement for precursors that were perceptually similar to the masker and for precursors that were made dissimilar to the masker by gating their components asynchronously. We found that the two types of precursor produced similar amounts of enhancement. This was true not only when the precursor and the subsequent test stimulus were presented to the same ear but also when they were presented to opposite ears. In a second experiment, we checked that the precursors with asynchronously gated components were perceptually poor templates of the subsequent maskers. Listeners now had to discriminate between test stimuli -containing the same components as the precursor and test stimuli containing all but one of the precursor components. We found that in this experimental situation, where enhancement could play no role, gating the precursor components asynchronously disrupted performance. Overall, our results are inconsistent with the hypothesis that precursors producing enhancement are beneficial because they are used as perceptual templates of the masker. Our results are instead consistent with an -explanation of enhancement based on selective neural adaptation taking place at a central locus of the auditory system.

Auditory enhancement of increments in spectral amplitude stems from more than one source

A component of a test sound consisting of simultaneous pure tones perceptually "pops out" if the test sound is preceded by a copy of itself with that component attenuated. Although this "enhancement" effect was initially thought to be purely monaural, it is also observable when the test sound and the precursor sound are presented contralaterally (i.e., to opposite ears). In experiment 1, we assessed the magnitude of ipsilateral and contralateral enhancement as a function of the time interval between the precursor and test sounds (10, 100, or 600 ms). The test sound, randomly transposed in frequency from trial to trial, was followed by a probe tone, either matched or mismatched in frequency to the test sound component which was the target of enhancement. Listeners' ability to discriminate matched probes from mismatched probes was taken as an index of enhancement magnitude. The results showed that enhancement decays more rapidly for ipsilateral than for contralateral precursors, suggesting that ipsilateral enhancement and contralateral enhancement stem from at least partly different sources. It could be hypothesized that, in experiment 1, contralateral precursors were effective only because they provided attentional cues about the target tone frequency. In experiment 2, this hypothesis was tested by presenting the probe tone before the precursor sound rather than after the test sound. Although the probe tone was then serving as a frequency cue, contralateral precursors were again found to produce enhancement. This indicates that contralateral enhancement cannot be explained by cuing alone and is a genuine sensory phenomenon.