Rhythmic masking release: effects of asynchrony, temporal overlap, harmonic relations, and source separation on cross-spectral grouping

Informational Masking in Aging and Brain-lesioned Individuals

Auditory stream segregation and informational masking were investigated in brain-lesioned individuals, age-matched controls with no neurological disease, and young college-age students. A psychophysical paradigm known as rhythmic masking release (RMR) was used to examine the ability of participants to identify a change in the rhythmic sequence of 20-ms Gaussian noise bursts presented through headphones and filtered through generalized head-related transfer functions to produce the percept of an externalized auditory image (i.e., a 3D virtual reality sound). The target rhythm was temporally interleaved with a masker sequence comprising similar noise bursts in a manner that resulted in a uniform sequence with no information remaining about the target rhythm when the target and masker were presented from the same location (an impossible task). Spatially separating the target and masker sequences allowed participants to determine if there was a change in the target rhythm midway during its presentation. RMR thresholds were defined as the minimum spatial separation between target and masker sequences that resulted in 70.7% correct-performance level in a single-interval 2-alternative forced-choice adaptive tracking procedure. The main findings were (1) significantly higher RMR thresholds for individuals with brain lesions (especially those with damage to parietal areas) and (2) a left-right spatial asymmetry in performance for lesion (but not control) participants. These findings contribute to a better understanding of spatiotemporal relations in informational masking and the neural bases of auditory scene analysis.

Temporal Processing in Audition: Insights from Music

Music is a curious example of a temporally patterned acoustic stimulus, and a compelling pan-cultural phenomenon. This review strives to bring some insights from decades of music psychology and sensorimotor synchronization (SMS) literature into the mainstream auditory domain, arguing that musical rhythm perception is shaped in important ways by temporal processing mechanisms in the brain. The feature that unites these disparate disciplines is an appreciation of the central importance of timing, sequencing, and anticipation. Perception of musical rhythms relies on an ability to form temporal predictions, a general feature of temporal processing that is equally relevant to auditory scene analysis, pattern detection, and speech perception. By bringing together findings from the music and auditory literature, we hope to inspire researchers to look beyond the conventions of their respective fields and consider the cross-disciplinary implications of studying auditory temporal sequence processing. We begin by highlighting music as an interesting sound stimulus that may provide clues to how temporal patterning in sound drives perception. Next, we review the SMS literature and discuss possible neural substrates for the perception of, and synchronization to, musical beat. We then move away from music to explore the perceptual effects of rhythmic timing in pattern detection, auditory scene analysis, and speech perception. Finally, we review the neurophysiology of general timing processes that may underlie aspects of the perception of rhythmic patterns. We conclude with a brief summary and outlook for future research.

Scanning for Digital Content

The widespread availability of digital media has changed the way that people consume information and has impacted the consumption of auditory information. Despite this recent popularity among sighted people, the use of auditory feedback to access digital information is not new for visually impaired users. However, its sequential nature undermines both blind and sighted people’s ability to efficiently find relevant information in the midst of several potentially useful items. We propose taking advantage of the Cocktail Party Effect , which states that people are able to focus on a single speech source among several conversations, but still identify relevant content in the background. Therefore, in contrast to one sequential speech channel, we hypothesize that people can leverage concurrent speech channels to quickly get the gist of digital information. In this article, we present an experiment with 46 (23 blind, 23 sighted) participants, which aims to understand people’s ability to search for relevant content listening to two, three, or four concurrent speech channels. Our results suggest that both blind and sighted people are able to process concurrent speech in scanning scenarios. In particular, the use of two concurrent sources may be used both to identify and understand the content of the relevant sentence. Moreover, three sources may be used for most people depending on the task intelligibility demands and user characteristics. Contrasting with related work, the use of different voices did not affect the perception of concurrent speech but was highly preferred by participants. To complement the analysis, we propose a set of scenarios that may benefit from the use of concurrent speech sources, for both blind and sighted people, toward a Design for All paradigm.

Assessing the effects of temporal coherence on auditory stream formation through comodulation masking release

Recent studies of auditory streaming have suggested that repeated synchronous onsets and offsets over time, referred to as "temporal coherence," provide a strong grouping cue between acoustic components, even when they are spectrally remote. This study uses a measure of auditory stream formation, based on comodulation masking release (CMR), to assess the conditions under which a loss of temporal coherence across frequency can lead to auditory stream segregation. The measure relies on the assumption that the CMR, produced by flanking bands remote from the masker and target frequency, only occurs if the masking and flanking bands form part of the same perceptual stream. The masking and flanking bands consisted of sequences of narrowband noise bursts, and the temporal coherence between the masking and flanking bursts was manipulated in two ways: (a) By introducing a fixed temporal offset between the flanking and masking bands that varied from zero to 60 ms and (b) by presenting the flanking and masking bursts at different temporal rates, so that the asynchronies varied from burst to burst. The results showed reduced CMR in all conditions where the flanking and masking bands were temporally incoherent, in line with expectations of the temporal coherence hypothesis.

Weighting of spatial and spectro-temporal cues for auditory scene analysis by human listeners

The auditory system creates a neuronal representation of the acoustic world based on spectral and temporal cues present at the listener's ears, including cues that potentially signal the locations of sounds. Discrimination of concurrent sounds from multiple sources is especially challenging. The current study is part of an effort to better understand the neuronal mechanisms governing this process, which has been termed "auditory scene analysis". In particular, we are interested in spatial release from masking by which spatial cues can segregate signals from other competing sounds, thereby overcoming the tendency of overlapping spectra and/or common temporal envelopes to fuse signals with maskers. We studied detection of pulsed tones in free-field conditions in the presence of concurrent multi-tone non-speech maskers. In "energetic" masking conditions, in which the frequencies of maskers fell within the ± 1/3-octave band containing the signal, spatial release from masking at low frequencies (~600 Hz) was found to be about 10 dB. In contrast, negligible spatial release from energetic masking was seen at high frequencies (~4000 Hz). We observed robust spatial release from masking in broadband "informational" masking conditions, in which listeners could confuse signal with masker even though there was no spectral overlap. Substantial spatial release was observed in conditions in which the onsets of the signal and all masker components were synchronized, and spatial release was even greater under asynchronous conditions. Spatial cues limited to high frequencies (>1500 Hz), which could have included interaural level differences and the better-ear effect, produced only limited improvement in signal detection. Substantially greater improvement was seen for low-frequency sounds, for which interaural time differences are the dominant spatial cue.

Temporal coherence versus harmonicity in auditory stream formation

This study sought to investigate the influence of temporal incoherence and inharmonicity on concurrent stream segregation, using performance-based measures. Subjects discriminated frequency shifts in a temporally regular sequence of target pure tones, embedded in a constant or randomly varying multi-tone background. Depending on the condition tested, the target tones were either temporally coherent or incoherent with, and either harmonically or inharmonically related to, the background tones. The results provide further evidence that temporal incoherence facilitates stream segregation and they suggest that deviations from harmonicity can cause similar facilitation effects, even when the targets and the maskers are temporally coherent.

Perceptual grouping over time within and across auditory and tactile modalities

In auditory scene analysis, population separation and temporal coherence have been proposed to explain how auditory features are grouped together and streamed over time. The present study investigated whether these two theories can be applied to tactile streaming and whether temporal coherence theory can be applied to crossmodal streaming. The results show that synchrony detection between two tones/taps at different frequencies/locations became difficult when one of the tones/taps was embedded in a perceptual stream. While the taps applied to the same location were streamed over time, the taps applied to different locations were not. This observation suggests that tactile stream formation can be explained by population-separation theory. On the other hand, temporally coherent auditory stimuli at different frequencies were streamed over time, but temporally coherent tactile stimuli applied to different locations were not. When there was within-modality streaming, temporally coherent auditory stimuli and tactile stimuli were not streamed over time, either. This observation suggests the limitation of temporal coherence theory when it is applied to perceptual grouping over time.

Neural mechanisms of rhythmic masking release in monkey primary auditory cortex: implications for models of auditory scene analysis

The ability to detect and track relevant acoustic signals embedded in a background of other sounds is crucial for hearing in complex acoustic environments. This ability is exemplified by a perceptual phenomenon known as "rhythmic masking release" (RMR). To demonstrate RMR, a sequence of tones forming a target rhythm is intermingled with physically identical "Distracter" sounds that perceptually mask the rhythm. The rhythm can be "released from masking" by adding "Flanker" tones in adjacent frequency channels that are synchronous with the Distracters. RMR represents a special case of auditory stream segregation, whereby the target rhythm is perceptually segregated from the background of Distracters when they are accompanied by the synchronous Flankers. The neural basis of RMR is unknown. Previous studies suggest the involvement of primary auditory cortex (A1) in the perceptual organization of sound patterns. Here, we recorded neural responses to RMR sequences in A1 of awake monkeys in order to identify neural correlates and potential mechanisms of RMR. We also tested whether two current models of stream segregation, when applied to these responses, could account for the perceptual organization of RMR sequences. Results suggest a key role for suppression of Distracter-evoked responses by the simultaneous Flankers in the perceptual restoration of the target rhythm in RMR. Furthermore, predictions of stream segregation models paralleled the psychoacoustics of RMR in humans. These findings reinforce the view that preattentive or "primitive" aspects of auditory scene analysis may be explained by relatively basic neural mechanisms at the cortical level.

Method-of-adjustment measures of informational masking between auditory streams

A method-of-adjustment procedure was used to measure thresholds for detecting a continuous sequence of brief 2-kHz tonal pulses in the presence of random-frequency masking sequences. Masker pulses consisted of either one or eight sinusoidal components and were either synchronous or asynchronous with the signal pulses. Effects of pulse rate and asynchronous gating were generally consistent with a reduction in informational masking due to segregation of the signal and masker streams. Despite use of continuous stimulus presentation to encourage stream segregation, masking was still obtained from most listeners in most conditions.

Effects of frequency disparities on trading of an ambiguous tone between two competing auditory objects

Listeners are relatively good at estimating the true content of each physical source in a sound mixture in most everyday situations. However, if there is a spectrotemporal element that logically could belong to more than one object, the correct way to group that element can be ambiguous. Many psychoacoustic experiments have implicitly assumed that when a sound mixture contains ambiguous sound elements, the ambiguous elements "trade" between competing sources, such that the elements contribute more to one object in conditions when they contribute less to others. However, few studies have directly tested whether such trading occurs. While some studies found trading, trading failed in some recent studies in which spatial cues were manipulated to alter the perceptual organization. The current study extended this work by exploring whether trading occurs for similar sound mixtures when frequency content, rather than spatial cues, was manipulated to alter grouping. Unlike when spatial cues were manipulated, results are roughly consistent with trading. Together, results suggest that the degree to which trading is obeyed depends on how stimuli are manipulated to affect object formation.

Neural substrate of concurrent sound perception: direct electrophysiological recordings from human auditory cortex

In everyday life, consciously or not, we are constantly disentangling the multiple auditory sources contributing to our acoustical environment. To better understand the neural mechanisms involved in concurrent sound processing, we manipulated sound onset asynchrony to induce the segregation or grouping of two concurrent sounds. Each sound consisted of amplitude-modulated tones at different carrier and modulation frequencies, allowing a cortical tagging of each sound. Electrophysiological recordings were carried out in epileptic patients with pharmacologically resistant partial epilepsy, implanted with depth electrodes in the temporal cortex. Patients were presented with the stimuli while they performed an auditory distracting task. We found that transient and steady-state evoked responses, and induced gamma oscillatory activities were enhanced in the case of onset synchrony. These effects were mainly located in the Heschl's gyrus for steady-state responses whereas they were found in the lateral superior temporal gyrus for evoked transient responses and induced gamma oscillations. They can be related to distinct neural mechanisms such as frequency selectivity and habituation. These results in the auditory cortex provide an anatomically refined description of the neurophysiological components which might be involved in the perception of concurrent sounds.

A sound element gets lost in perceptual competition

Our ability to understand auditory signals depends on properly separating the mixture of sound arriving from multiple sources. Sound elements tend to belong to only one object at a time, consistent with the principle of disjoint allocation, although there are instances of duplex perception or coallocation, in which two sound objects share one sound element. Here we report an effect of "nonallocation," in which a sound element "disappears" when two ongoing objects compete for its ownership. When a target tone is presented either as one of a sequence of tones or simultaneously with a harmonic vowel complex, it is heard as part of the corresponding object. However, depending on the spatial configuration of the scene, if the target, the tones, and the vowel are all presented together, the target may not be perceived in either the tones or the vowel, even though it is not perceived as a separate entity. This finding suggests an asymmetry in the strength of the perceptual evidence required to reject vs. to include an element within the auditory foreground, a result with important implications for how we process complex auditory scenes containing ambiguous information.

Contralateral influences of wideband inhibition on the effect of onset asynchrony as a cue for auditory grouping

Onset asynchrony is an important cue for segregating sound mixtures. A harmonic of a vowel that begins before the other components contributes less to vowel quality. This asynchrony effect can be partly reversed by accompanying the leading portion of the harmonic with an octave-higher captor tone. The original interpretation was that the captor and leading portion formed a perceptual group, but it has recently been shown that the captor effect depends on neither a common onset time nor harmonic relations with the leading portion. Instead, it has been proposed that the captor effect depends on wideband inhibition in the central auditory system. Physiological evidence suggests that such inhibition occurs both within and across ears. Experiment 1 compared the efficacy of a pure-tone captor presented in the same or opposite ear to the vowel and leading harmonic. Contralateral presentation was at least as effective as ipsilateral presentation. Experiment 2 used multicomponent captors in a more comprehensive evaluation of harmonic influences on captor efficacy. Three captors with different fundamental frequencies were used, one of which formed a consecutive harmonic series with the leading harmonic. All captors were equally effective, irrespective of the harmonic relationship. These findings support and refine the inhibitory account.