Forward masking in the amplitude-modulation domain for tone carriers: psychophysical results and physiological correlates

Evidence for proactive and retroactive temporal pattern analysis in simultaneous maskinga)

Amplitude modulation (AM) of a masker reduces its masking on a simultaneously presented unmodulated pure-tone target, which likely involves dip listening. This study tested the idea that dip-listening efficiency may depend on stimulus context, i.e., the match in AM peakedness (AMP) between the masker and a precursor or postcursor stimulus, assuming a form of temporal pattern analysis process. Masked thresholds were measured in normal-hearing listeners using Schroeder-phase harmonic complexes as maskers and precursors or postcursors. Experiment 1 showed threshold elevation (i.e., interference) when a flat cursor preceded or followed a peaked masker, suggesting proactive and retroactive temporal pattern analysis. Threshold decline (facilitation) was observed when the masker AMP was matched to the precursor, irrespective of stimulus AMP, suggesting only proactive processing. Subsequent experiments showed that both interference and facilitation (1) remained robust when a temporal gap was inserted between masker and cursor, (2) disappeared when an F0-difference was introduced between masker and precursor, and (3) decreased when the presentation level was reduced. These results suggest an important role of envelope regularity in dip listening, especially when masker and cursor are F0-matched and, therefore, form one perceptual stream. The reported effects seem to represent a time-domain variant of comodulation masking release.

Adaptation in auditory processing

Adaptation is an essential feature of auditory neurons, which reduces their responses to unchanging and recurring sounds and allows their response properties to be matched to the constantly changing statistics of sounds that reach the ears. As a consequence, processing in the auditory system highlights novel or unpredictable sounds and produces an efficient representation of the vast range of sounds that animals can perceive by continually adjusting the sensitivity and, to a lesser extent, the tuning properties of neurons to the most commonly encountered stimulus values. Together with attentional modulation, adaptation to sound statistics also helps to generate neural representations of sound that are tolerant to background noise and therefore plays a vital role in auditory scene analysis. In this review, we consider the diverse forms of adaptation that are found in the auditory system in terms of the processing levels at which they arise, the underlying neural mechanisms, and their impact on neural coding and perception. We also ask what the dynamics of adaptation, which can occur over multiple timescales, reveal about the statistical properties of the environment. Finally, we examine how adaptation to sound statistics is influenced by learning and experience and changes as a result of aging and hearing loss.

Cues to reduce modulation informational masking

The detectability of target amplitude modulation (AM) can be reduced by masker AM in the same carrier-frequency region. It can be reduced even further, however, if the masker-AM rate is uncertain [Conroy and Kidd, J. Acoust. Soc. Am. 149, 3665-3673 (2021)]. This study examined the effectiveness of contextual cues in reducing this latter, uncertainty-related effect (modulation informational masking). Observers were tasked with detecting fixed-rate target sinusoidal amplitude modulation (SAM) in the presence of masker SAM applied simultaneously to the same broadband-noise carrier. A single-interval, two-alternative forced-choice detection procedure was used to measure sensitivity for the target SAM; masker-AM-rate uncertainty was created by randomly selecting the AM rate of the masker SAM on each trial. Relative to an uncued condition, a pretrial cue to the masker SAM significantly improved sensitivity for the target SAM; a cue to the target SAM, however, did not. The delay between the cue-interval offset and trial-interval onset did not affect the size of the masker-cue benefit, suggesting that adaptation of the masker SAM was not responsible. A simple model of within-AM-channel masking captured important trends in the psychophysical data, suggesting that reduced masker-AM-rate uncertainty may have played a relatively minor role in the masker-cue benefit.

Amplitude-modulation forward masking for listeners with and without hearing loss

Amplitude-modulation (AM) forward masking was measured for listeners with normal hearing and sensorineural hearing loss at 4000 and 1000 Hz, using continuous and noncontinuous masker and signal carriers, respectively. A low-fluctuation noise (LFN) carrier was used for the "continuous carrier" conditions. An unmodulated low-fluctuation noise (U-LFN), an unmodulated Gaussian noise (U-GN), and an amplitude-modulation low-fluctuation noise (AM-LFN) were maskers for the "noncontinuous carrier" conditions. As predicted, U-GN yielded more masking than U-LFN and similar masking to AM-LFN, suggesting that U-GN resulted in AM forward masking. Contrary to predictions, no differences in masked thresholds were observed between listener groups.

Forward masking of spectrotemporal modulation detection

Recent work has suggested that there may be specialized mechanisms in the auditory system for coding spectrotemporal modulations (STMs), tuned to different combinations of spectral modulation frequency, temporal modulation frequency, and STM sweep direction. The current study sought evidence of such mechanisms using a psychophysical forward masking paradigm. The detectability of a target comprising upward sweeping STMs was measured following the presentation of modulated maskers applied to the same carrier. Four maskers were tested, which had either (1) the same spectral modulation frequency as the target but a flat temporal envelope, (2) the same temporal modulation frequency as the target but a flat spectral envelope, (3) the same spectral and temporal modulation frequencies as the target but the opposite sweep direction (downward sweeping STMs), or (4) the same spectral and temporal modulation frequencies as the target and the same sweep direction (upward sweeping STMs). Forward masking was greatest for the masker fully matched to the target (4), intermediate for the masker with the opposite sweep direction (3), and negligible for the other two (1, 2). These findings are consistent with the suggestion that the detectability of the target was mediated by an STM-specific coding mechanism with sweep-direction selectivity.

Informational masking in the modulation domain

Uncertainty regarding the frequency spectrum of a masker can have an adverse effect on the ability to focus selective attention on a target frequency channel, yielding informational masking (IM). This study sought to determine if uncertainty regarding the modulation spectrum of a masker can have an analogous adverse effect on the ability to focus selective attention on a target modulation channel, yielding IM in the modulation domain, or "modulation IM." A single-interval, two-alternative forced-choice (yes-no) procedure was used. The task was to detect 32-Hz target sinusoidal amplitude modulation (SAM) imposed on a broadband-noise carrier in the presence of masker SAM imposed on the same carrier. Six maskers, spanning the range from 8 to 128 Hz in half-octave steps, were tested, excluding those that fell within a two-octave protected zone surrounding the target. Psychometric functions (d'-vs-target modulation depth) were measured for each masker under two conditions: a fixed (low-uncertainty/low-IM) condition, in which the masker was the same on all trials within a block, and a random (high-uncertainty/high-IM) condition, in which it varied randomly from presentation-to-presentation. Thresholds and slopes extracted from the psychometric functions differed markedly between the conditions. These results are consistent with the idea that IM occurs in the modulation domain.

Second Language Experience Facilitates Sentence Recognition in Temporally-Modulated Noise for Non-native Listeners

Non-native listeners deal with adverse listening conditions in their daily life much harder than native listeners. However, previous work in our laboratories found that native Chinese listeners with native English exposure may improve the use of temporal fluctuations of noise for English vowel identification. The purpose of this study was to investigate whether Chinese listeners can generalize the use of temporal cues for the English sentence recognition in noise. Institute of Electrical and Electronics Engineers (IEEE) sentence recognition in quiet condition, stationary noise, and temporally-modulated noise were measured for native American English listeners (EN), native Chinese listeners in the United States (CNU), and native Chinese listeners in China (CNC). Results showed that in general, EN listeners outperformed the two groups of CN listeners in quiet and noise, while CNU listeners had better scores of sentence recognition than CNC listeners. Moreover, the native English exposure helped CNU listeners use high-level linguistic cues more effectively and take more advantage of temporal fluctuations of noise to process English sentence in severely degraded listening conditions [i.e., the signal-to-noise ratio (SNR) of −12 dB] than CNC listeners. These results suggest a significant effect of language experience on the auditory processing of both speech and noise.

Frequency selectivity in the modulation domain estimated using forward masking: Effects of masker modulation depth and masker-signal delay

The threshold for detecting amplitude modulation (AM) of a sinusoidal or noise carrier is elevated when the signal AM is preceded by masker AM applied to the same carrier. This effect, called AM forward masking, shows selectivity in the AM domain, consistent with the existence of a modulation filter bank (MFB). In this paper we explore the effect of two factors that can influence AM forward masking, using an 8-kHz sinusoidal carrier and a range of masker AM frequencies, f_m, both below and above the signal AM frequency, f_s, of 40 Hz. The first factor was the time delay, t_d, between the end of the masker AM and the start of the signal AM. The second was the AM depth, m, of the masker, which was either 1 or 0.25. The AM forward masking patterns in all conditions showed tuning in the AM domain; signal thresholds were highest when f_m was close to f_s. The amount of AM forward masking decreased with increasing t_d in a similar way for all f_m, so the shapes of the masking patterns did not change markedly with t_d. Remarkably, the amount of AM forward masking decreased by only about 3 dB (a non-significant effect) when the masker m was decreased from 1 to 0.25. This result appears to be inconsistent with an explanation of AM forward masking in terms of adaptation in a MFB or in terms of a sliding temporal integrator.

Forward masking of amplitude modulation across ears and its tuning in the modulation domain

Frequency selectivity in the amplitude modulation (AM) domain has been demonstrated using both simultaneous AM masking and forward AM masking. This has been explained using the concept of a modulation filter bank (MFB). Here, we assessed whether the MFB occurs before or after the point of binaural interaction in the auditory pathway by using forward masking in the AM domain in an ipsilateral condition (masker AM and signal AM applied to the left ear with an unmodulated carrier in the right ear) and a contralateral condition (masker AM applied to the right ear and signal AM applied to the left ear). The carrier frequency was 8 kHz, the signal AM frequency, f_s, was 40 or 80 Hz, and the masker AM frequency ranged from 0.25 to 4 times f_s. Contralateral forward AM masking did occur, but it was smaller than ipsilateral AM masking. Tuning in the AM domain was slightly sharper for ipsilateral than for contralateral masking, perhaps reflecting confusion of the signal and masker AM in the ipsilateral condition when their AM frequencies were the same. The results suggest that there might be an MFB both before and after the point in the auditory pathway where binaural interaction occurs.

Adaptation to Noise in Human Speech Recognition Depends on Noise-Level Statistics and Fast Dynamic-Range Compression

Human hearing adapts to background noise, as evidenced by the fact that listeners recognize more isolated words when words are presented later rather than earlier in noise. This adaptation likely occurs because the leading noise shifts ("adapts") the dynamic range of auditory neurons, which can improve the neural encoding of speech spectral and temporal cues. Because neural dynamic range adaptation depends on stimulus-level statistics, here we investigated the importance of "statistical" adaptation for improving speech recognition in noisy backgrounds. We compared the recognition of noised-masked words in the presence and in the absence of adapting noise precursors whose level was either constant or was changing every 50 ms according to different statistical distributions. Adaptation was measured for 28 listeners (9 men) and was quantified as the recognition improvement in the precursor relative to the no-precursor condition. Adaptation was largest for constant-level precursors and did not occur for highly fluctuating precursors, even when the two types of precursors had the same mean level and both activated the medial olivocochlear reflex. Instantaneous amplitude compression of the highly fluctuating precursor produced as much adaptation as the constant-level precursor did without compression. Together, results suggest that noise adaptation in speech recognition is probably mediated by neural dynamic range adaptation to the most frequent sound level. Further, they suggest that auditory peripheral compression per se, rather than the medial olivocochlear reflex, could facilitate noise adaptation by reducing the level fluctuations in the noise.SIGNIFICANCE STATEMENT Recognizing speech in noise is challenging but can be facilitated by noise adaptation. The neural mechanisms underlying this adaptation remain unclear. Here, we report some benefits of adaptation for word-in-noise recognition and show that (1) adaptation occurs for stationary but not for highly fluctuating precursors with equal mean level; (2) both stationary and highly fluctuating noises activate the medial olivocochlear reflex; and (3) adaptation occurs even for highly fluctuating precursors when the stimuli are passed through a fast amplitude compressor. These findings suggest that noise adaptation reflects neural dynamic range adaptation to the most frequent noise level and that auditory peripheral compression, rather than the medial olivocochlear reflex, could facilitate noise adaptation.

Masking of short tones in noise: Evidence for envelope-based, rather than energy-based detection

The "temporal effect" in simultaneous masking may be characterized by better probe detection thresholds for a short, tonal probe presented at the temporal center of a masker compared to at the onset of a masker. Energy-based models of masking have been used to interpret the temporal effect as evidence that the gain of the auditory system decreases during acoustic stimulation. This study shows that masking from temporal-envelope fluctuations of a precursor or from a temporal gap between stimuli violates the assumptions of energy-based models and complicates the interpretation of temporal effects in terms of a reduction in gain. Detection thresholds were measured for a 6-ms, 4000-Hz probe preceded by a narrowband precursor and presented 2-, 197-, or 392-ms after the onset of a narrowband masker. The delay between the precursor offset and masker onset ranged from -2 to 250 ms. Probe thresholds were elevated in the presence of precursors with fluctuating compared to flattened temporal envelopes and when a temporal gap was inserted between the precursor and masker. The results suggest that the interpretation and design of temporal-effect studies should consider the masking effects of temporal-envelope fluctuations. These findings are consistent with speech-perception experiments that show masking from temporal-envelope fluctuations.

Modulation detection interference in cochlear implant listeners under forward masking conditions

Little is known about cochlear implant (CI) users' ability to process amplitude modulation (AM) under conditions of forward masking (forward-modulation detection/discrimination interference, or F-MDI). In this study, F-MDI was investigated in adult CI listeners using direct electrical stimulation via research interface. The target was sinusoidally amplitude modulated at 50 Hz, and presented to a fixed electrode in the middle of the array. The forward masker was either amplitude modulated at the same rate (AM) or unmodulated and presented at the peak amplitude of its AM counterpart (steady-state peak, SSP). Results showed that the AM masker produced higher modulation thresholds in the target than the SSP masker. The difference (F-MDI) was estimated to be 4.6 dB on average, and did not change with masker-target delays up to 100 ms or with masker-target spatial electrode distances up to eight electrodes. Results with a coherent remote cue presented with the masker showed that confusion effects did not play a role in the observed F-MDI. Traditional recovery from forward masking using the same maskers and a 20-ms probe, measured in four of the subjects, confirmed the expected result: higher thresholds with the SSP masker than the AM masker. Collectively, the results indicate that significant F-MDI occurs in CI users.

Inherent envelope fluctuations in forward maskers: Effects of masker-probe delay for listeners with normal and impaired hearing

Forward-masked thresholds increase as the magnitude of inherent masker envelope fluctuations increase for both normal-hearing (NH) and hearing-impaired (HI) adults for a short masker-probe delay (25 ms). The slope of the recovery from forward masking is shallower for HI than for NH listeners due to reduced cochlear nonlinearities. However, effects of hearing loss on additional masking due to inherent envelope fluctuations across masker-probe delays remain unknown. The current study assessed effects of hearing loss on the slope and amount of recovery from forward maskers that varied in inherent envelope fluctuations. Forward-masked thresholds were measured at 2000 and 4000 Hz, for masker-probe delays of 25, 50, and 75 ms, for NH and HI adults. Four maskers at each center frequency varied in inherent envelope fluctuations: Gaussian noise (GN) or low-fluctuation noise (LFN), with 1 or 1/3 equivalent rectangular bandwidths (ERBs). Results suggested that slopes of recovery from forward masking were shallower for HI than for NH listeners regardless of masker fluctuations. Additional masking due to inherent envelope fluctuations was greater for HI than for NH listeners at longer masker-probe delays, suggesting that inherent envelope fluctuations are more disruptive for HI than for NH listeners for a longer time course.

Synaptic plasticity as a cortical coding scheme

Processing of auditory information requires constant adjustment due to alterations of the environment and changing conditions in the nervous system with age, health, and experience. Consequently, patterns of activity in cortical networks have complex dynamics over a wide range of timescales, from milliseconds to days and longer. In the primary auditory cortex (AI), multiple forms of adaptation and plasticity shape synaptic input and action potential output. However, the variance of neuronal responses has made it difficult to characterize AI receptive fields and to determine the function of AI in processing auditory information such as vocalizations. Here we describe recent studies on the temporal modulation of cortical responses and consider the relation of synaptic plasticity to neural coding.

Vowel identification in temporal-modulated noise for native and non-native listeners: Effect of language experience

A previous study found that English vowel identification in babble was significantly different between Chinese-native listeners in China and in the U.S. One possible explanation is that native English experiences might change Chinese-native listeners' ability to take advantage of the temporal modulation in noise for their English vowel perception. As a follow-up, the current study explored whether there was a difference between the two groups of Chinese listeners in using temporal gaps in noise for English vowel identification. Vowel identification in temporally modulated noise and a temporal modulation transfer function (TMTF) was measured for American-English-native listeners (EN), Chinese-native listeners in the U.S. (CNU), and Chinese-native listeners in China (CNC). The results revealed that TMTFs were similar across the three groups, indicating that psychophysical temporal processing was independent of listeners' language backgrounds. However, for vowel identification in noise, EN and CNU listeners showed significantly greater masking release from the temporal modulation of noise than CNC listeners at low signal-to-noise ratios (e.g., -12 dB). Altogether, native English experiences may change the use of temporal cues in noise for English vowel identification for Chinese-native listeners.

Specificity of the Human Frequency Following Response for Carrier and Modulation Frequency Assessed Using Adaptation

The frequency following response (FFR) is a scalp-recorded measure of phase-locked brainstem activity to stimulus-related periodicities. Three experiments investigated the specificity of the FFR for carrier and modulation frequency using adaptation. FFR waveforms evoked by alternating-polarity stimuli were averaged for each polarity and added, to enhance envelope, or subtracted, to enhance temporal fine structure information. The first experiment investigated peristimulus adaptation of the FFR for pure and complex tones as a function of stimulus frequency and fundamental frequency (F0). It showed more adaptation of the FFR in response to sounds with higher frequencies or F0s than to sounds with lower frequency or F0s. The second experiment investigated tuning to modulation rate in the FFR. The FFR to a complex tone with a modulation rate of 213 Hz was not reduced more by an adaptor that had the same modulation rate than by an adaptor with a different modulation rate (90 or 504 Hz), thus providing no evidence that the FFR originates mainly from neurons that respond selectively to the modulation rate of the stimulus. The third experiment investigated tuning to audio frequency in the FFR using pure tones. An adaptor that had the same frequency as the target (213 or 504 Hz) did not generally reduce the FFR to the target more than an adaptor that differed in frequency (by 1.24 octaves). Thus, there was no evidence that the FFR originated mainly from neurons tuned to the frequency of the target. Instead, the results are consistent with the suggestion that the FFR for low-frequency pure tones at medium to high levels mainly originates from neurons tuned to higher frequencies. Implications for the use and interpretation of the FFR are discussed.

Modulation-frequency-specific adaptation in awake auditory cortex

Amplitude modulations are fundamental features of natural signals, including human speech and nonhuman primate vocalizations. Because natural signals frequently occur in the context of other competing signals, we used a forward-masking paradigm to investigate how the modulation context of a prior signal affects cortical responses to subsequent modulated sounds. Psychophysical "modulation masking," in which the presentation of a modulated "masker" signal elevates the threshold for detecting the modulation of a subsequent stimulus, has been interpreted as evidence of a central modulation filterbank and modeled accordingly. Whether cortical modulation tuning is compatible with such models remains unknown. By recording responses to pairs of sinusoidally amplitude modulated (SAM) tones in the auditory cortex of awake squirrel monkeys, we show that the prior presentation of the SAM masker elicited persistent and tuned suppression of the firing rate to subsequent SAM signals. Population averages of these effects are compatible with adaptation in broadly tuned modulation channels. In contrast, modulation context had little effect on the synchrony of the cortical representation of the second SAM stimuli and the tuning of such effects did not match that observed for firing rate. Our results suggest that, although the temporal representation of modulated signals is more robust to changes in stimulus context than representations based on average firing rate, this representation is not fully exploited and psychophysical modulation masking more closely mirrors physiological rate suppression and that rate tuning for a given stimulus feature in a given neuron's signal pathway appears sufficient to engender context-sensitive cortical adaptation.

Hierarchical effects of task engagement on amplitude modulation encoding in auditory cortex

We recorded from middle lateral belt (ML) and primary (A1) auditory cortical neurons while animals discriminated amplitude-modulated (AM) sounds and also while they sat passively. Engagement in AM discrimination improved ML and A1 neurons' ability to discriminate AM with both firing rate and phase-locking; however, task engagement affected neural AM discrimination differently in the two fields. The results suggest that these two areas utilize different AM coding schemes: a "single mode" in A1 that relies on increased activity for AM relative to unmodulated sounds and a "dual-polar mode" in ML that uses both increases and decreases in neural activity to encode modulation. In the dual-polar ML code, nonsynchronized responses might play a special role. The results are consistent with findings in the primary and secondary somatosensory cortices during discrimination of vibrotactile modulation frequency, implicating a common scheme in the hierarchical processing of temporal information among different modalities. The time course of activity differences between behaving and passive conditions was also distinct in A1 and ML and may have implications for auditory attention. At modulation depths ≥ 16% (approximately behavioral threshold), A1 neurons' improvement in distinguishing AM from unmodulated noise is relatively constant or improves slightly with increasing modulation depth. In ML, improvement during engagement is most pronounced near threshold and disappears at highly suprathreshold depths. This ML effect is evident later in the stimulus, and mainly in nonsynchronized responses. This suggests that attention-related increases in activity are stronger or longer-lasting for more difficult stimuli in ML.

Forward masking of frequency modulation

Forward masking of sinusoidal frequency modulation (FM) was measured with three types of maskers: FM, amplitude modulation (AM), and a masker created by combining the magnitude spectrum of an FM tone with random component phases. For the signal FM rates used (5, 20, and 40 Hz), an FM masker raised detection thresholds in terms of frequency deviation by a factor of about 5 relative to without a masker. The AM masker produced a much smaller effect, suggesting that FM-to-AM conversion did not contribute substantially to the FM forward masking. The modulation depth of an FM masker had a nonmonotonic effect, with maximal masking observed at an intermediate value within the range of possible depths, while the random-phase FM masker produced less masking, arguing against a spectrally-based explanation for FM forward masking. Broad FM-rate selectivity for forward masking was observed for both 4-kHz and 500-Hz carriers. Thresholds measured as a function of the masker-signal delay showed slow recovery from FM forward masking, with residual masking for delays up to 500 ms. The FM forward-masking effect resembles that observed for AM [Wojtczak and Viemeister (2005). J. Acoust. Soc. Am. 188, 3198-3210] and may reflect modulation-rate selective neural adaptation to FM.

Stream segregation in the perception of sinusoidally amplitude-modulated tones

Amplitude modulation can serve as a cue for segregating streams of sounds from different sources. Here we evaluate stream segregation in humans using ABA- sequences of sinusoidally amplitude modulated (SAM) tones. A and B represent SAM tones with the same carrier frequency (1000, 4000 Hz) and modulation depth (30, 100%). The modulation frequency of the A signals (f_modA) was 30, 100 or 300 Hz, respectively. The modulation frequency of the B signals was up to four octaves higher (Δf_mod). Three different ABA- tone patterns varying in tone duration and stimulus onset asynchrony were presented to evaluate the effect of forward suppression. Subjects indicated their 1- or 2-stream percept on a touch screen at the end of each ABA- sequence (presentation time 5 or 15 s). Tone pattern, f_modA, Δf_mod, carrier frequency, modulation depth and presentation time significantly affected the percentage of a 2-stream percept. The human psychophysical results are compared to responses of avian forebrain neurons evoked by different ABA- SAM tone conditions [1] that were broadly overlapping those of the present study. The neurons also showed significant effects of tone pattern and Δf_mod that were comparable to effects observed in the present psychophysical study. Depending on the carrier frequency, modulation frequency, modulation depth and the width of the auditory filters, SAM tones may provide mainly temporal cues (sidebands fall within the range of the filter), spectral cues (sidebands fall outside the range of the filter) or possibly both. A computational model based on excitation pattern differences was used to predict the 50% threshold of 2-stream responses. In conditions for which the model predicts a considerably larger 50% threshold of 2-stream responses (i.e., larger Δf_mod at threshold) than was observed, it is unlikely that spectral cues can provide an explanation of stream segregation by SAM.