Modulation masking produced by second-order modulators

Frequency selectivity in the modulation domain estimated using forward masking: Effects of masker modulation depth and masker-signal delay

The threshold for detecting amplitude modulation (AM) of a sinusoidal or noise carrier is elevated when the signal AM is preceded by masker AM applied to the same carrier. This effect, called AM forward masking, shows selectivity in the AM domain, consistent with the existence of a modulation filter bank (MFB). In this paper we explore the effect of two factors that can influence AM forward masking, using an 8-kHz sinusoidal carrier and a range of masker AM frequencies, f_m, both below and above the signal AM frequency, f_s, of 40 Hz. The first factor was the time delay, t_d, between the end of the masker AM and the start of the signal AM. The second was the AM depth, m, of the masker, which was either 1 or 0.25. The AM forward masking patterns in all conditions showed tuning in the AM domain; signal thresholds were highest when f_m was close to f_s. The amount of AM forward masking decreased with increasing t_d in a similar way for all f_m, so the shapes of the masking patterns did not change markedly with t_d. Remarkably, the amount of AM forward masking decreased by only about 3 dB (a non-significant effect) when the masker m was decreased from 1 to 0.25. This result appears to be inconsistent with an explanation of AM forward masking in terms of adaptation in a MFB or in terms of a sliding temporal integrator.

Cascaded Amplitude Modulations in Sound Texture Perception

Sound textures, such as crackling fire or chirping crickets, represent a broad class of sounds defined by their homogeneous temporal structure. It has been suggested that the perception of texture is mediated by time-averaged summary statistics measured from early auditory representations. In this study, we investigated the perception of sound textures that contain rhythmic structure, specifically second-order amplitude modulations that arise from the interaction of different modulation rates, previously described as “beating” in the envelope-frequency domain. We developed an auditory texture model that utilizes a cascade of modulation filterbanks that capture the structure of simple rhythmic patterns. The model was examined in a series of psychophysical listening experiments using synthetic sound textures—stimuli generated using time-averaged statistics measured from real-world textures. In a texture identification task, our results indicated that second-order amplitude modulation sensitivity enhanced recognition. Next, we examined the contribution of the second-order modulation analysis in a preference task, where the proposed auditory texture model was preferred over a range of model deviants that lacked second-order modulation rate sensitivity. Lastly, the discriminability of textures that included second-order amplitude modulations appeared to be perceived using a time-averaging process. Overall, our results demonstrate that the inclusion of second-order modulation analysis generates improvements in the perceived quality of synthetic textures compared to the first-order modulation analysis considered in previous approaches.

Amplitude modulation detection with concurrent frequency modulation

Human speech consists of concomitant temporal modulations in amplitude and frequency that are crucial for speech perception. In this study, amplitude modulation (AM) detection thresholds were measured for 550 and 5000 Hz carriers with and without concurrent frequency modulation (FM), at AM rates crucial for speech perception. Results indicate that adding 40 Hz FM interferes with AM detection, more so for 5000 Hz carrier and for frequency deviations exceeding the critical bandwidth of the carrier frequency. These findings suggest that future cochlear implant processors, encoding speech fine-structures may consider limiting the FM to narrow bandwidth and to low frequencies.

Modulation masking and glimpsing of natural and vocoded speech during single-talker modulated noise: Effect of the modulation spectrum

Compared to notionally steady-state noise, modulated maskers provide a perceptual benefit for speech recognition, in part due to preserved speech information during the amplitude dips of the masker. However, overlap in the modulation spectrum between the target speech and the competing modulated masker may potentially result in modulation masking, and thereby offset the release from energetic masking. The current study investigated masking release provided by single-talker modulated noise. The overlap in the modulation spectra of the target speech and the modulated noise masker was varied through time compression or expansion of the competing masker. Younger normal hearing adults listened to sentences that were unprocessed or noise vocoded to primarily limit speech recognition to the preserved temporal envelope cues. For unprocessed speech, results demonstrated improved performance with masker modulation spectrum shifted up or down compared to the target modulation spectrum, except for the most extreme time expansion. For vocoded speech, significant masking release was observed with the slowest masker rate. Perceptual results combined with acoustic analyses of the preserved glimpses of the target speech suggest contributions of modulation masking and cognitive-linguistic processing as factors contributing to performance.

Modulation masking and glimpsing of natural and vocoded speech during single-talker modulated noise: Effect of the modulation spectrum

Compared to notionally steady-state noise, modulated maskers provide a perceptual benefit for speech recognition, in part due to preserved speech information during the amplitude dips of the masker. However, overlap in the modulation spectrum between the target speech and the competing modulated masker may potentially result in modulation masking, and thereby offset the release from energetic masking. The current study investigated masking release provided by single-talker modulated noise. The overlap in the modulation spectra of the target speech and the modulated noise masker was varied through time compression or expansion of the competing masker. Younger normal hearing adults listened to sentences that were unprocessed or noise vocoded to primarily limit speech recognition to the preserved temporal envelope cues. For unprocessed speech, results demonstrated improved performance with masker modulation spectrum shifted up or down compared to the target modulation spectrum, except for the most extreme time expansion. For vocoded speech, significant masking release was observed with the slowest masker rate. Perceptual results combined with acoustic analyses of the preserved glimpses of the target speech suggest contributions of modulation masking and cognitive-linguistic processing as factors contributing to performance.

On The (Un)importance of Working Memory in Speech-in-Noise Processing for Listeners with Normal Hearing Thresholds

With the advent of cognitive hearing science, increased attention has been given to individual differences in cognitive functioning and their explanatory power in accounting for inter-listener variability in the processing of speech in noise (SiN). The psychological construct that has received much interest in recent years is working memory. Empirical evidence indeed confirms the association between WM capacity (WMC) and SiN identification in older hearing-impaired listeners. However, some theoretical models propose that variations in WMC are an important predictor for variations in speech processing abilities in adverse perceptual conditions for all listeners, and this notion has become widely accepted within the field. To assess whether WMC also plays a role when listeners without hearing loss process speech in adverse listening conditions, we surveyed published and unpublished studies in which the Reading-Span test (a widely used measure of WMC) was administered in conjunction with a measure of SiN identification, using sentence material routinely used in audiological and hearing research. A meta-analysis revealed that, for young listeners with audiometrically normal hearing, individual variations in WMC are estimated to account for, on average, less than 2% of the variance in SiN identification scores. This result cautions against the (intuitively appealing) assumption that individual variations in WMC are predictive of SiN identification independently of the age and hearing status of the listener.

Age-group differences in speech identification despite matched audiometrically normal hearing: contributions from auditory temporal processing and cognition

Hearing loss with increasing age adversely affects the ability to understand speech, an effect that results partly from reduced audibility. The aims of this study were to establish whether aging reduces speech intelligibility for listeners with normal audiograms, and, if so, to assess the relative contributions of auditory temporal and cognitive processing. Twenty-one older normal-hearing (ONH; 60-79 years) participants with bilateral audiometric thresholds ≤ 20 dB HL at 0.125-6 kHz were matched to nine young (YNH; 18-27 years) participants in terms of mean audiograms, years of education, and performance IQ. Measures included: (1) identification of consonants in quiet and in noise that was unmodulated or modulated at 5 or 80 Hz; (2) identification of sentences in quiet and in co-located or spatially separated two-talker babble; (3) detection of modulation of the temporal envelope (TE) at frequencies 5-180 Hz; (4) monaural and binaural sensitivity to temporal fine structure (TFS); (5) various cognitive tests. Speech identification was worse for ONH than YNH participants in all types of background. This deficit was not reflected in self-ratings of hearing ability. Modulation masking release (the improvement in speech identification obtained by amplitude modulating a noise background) and spatial masking release (the benefit obtained from spatially separating masker and target speech) were not affected by age. Sensitivity to TE and TFS was lower for ONH than YNH participants, and was correlated positively with speech-in-noise (SiN) identification. Many cognitive abilities were lower for ONH than YNH participants, and generally were correlated positively with SiN identification scores. The best predictors of the intelligibility of SiN were composite measures of cognition and TFS sensitivity. These results suggest that declines in speech perception in older persons are partly caused by cognitive and perceptual changes separate from age-related changes in audiometric sensitivity.

The origin of binaural interaction in the modulation domain

The purpose of these experiments was to assess whether the detection of diotic 5 Hz "probe" modulation of a 4000 Hz sinusoidal carrier was influenced by binaural interaction of "masker" modulators presented separately to each ear and applied to the same carrier. A 50 Hz masker modulator was applied to one ear and the masker modulator applied to the other ear had a frequency of 55 or 27.5 Hz. The starting phase of the masker modulators was fixed, and the starting phase of the probe modulator was varied. For both pairs of masker modulators, the threshold for detecting the probe modulation varied slightly but significantly with probe starting phase. Further experiments measuring probe detectability as a function of probe modulation depth did not provide clear evidence to support the idea that the internal representations of the masker modulators interacted binaurally to produce a weak distortion component in the internal representation of the modulation at a 5 Hz frequency. Also, the obtained phase effects were not correctly predicted using a model based on short-term loudness fluctuations.

Estimation of the center frequency of the highest modulation filter

For high-frequency sinusoidal carriers, the threshold for detecting sinusoidal amplitude modulation increases when the signal modulation frequency increases above about 120 Hz. Using the concept of a modulation filter bank, this effect might be explained by (1) a decreasing sensitivity or greater internal noise for modulation filters with center frequencies above 120 Hz; and (2) a limited span of center frequencies of the modulation filters, the top filter being tuned to about 120 Hz. The second possibility was tested by measuring modulation masking in forward masking using an 8 kHz sinusoidal carrier. The signal modulation frequency was 80, 120, or 180 Hz and the masker modulation frequencies covered a range above and below each signal frequency. Four highly trained listeners were tested. For the 80-Hz signal, the signal threshold was usually maximal when the masker frequency equaled the signal frequency. For the 180-Hz signal, the signal threshold was maximal when the masker frequency was below the signal frequency. For the 120-Hz signal, two listeners showed the former pattern, and two showed the latter pattern. The results support the idea that the highest modulation filter has a center frequency in the range 100-120 Hz.

Comparison of level discrimination, increment detection, and comodulation masking release in the audio- and envelope-frequency domains

In general, the temporal structure of stimuli must be considered to account for certain observations made in detection and masking experiments in the audio-frequency domain. Two such phenomena are (1) a heightened sensitivity to amplitude increments with a temporal fringe compared to gated level discrimination performance and (2) lower tone-in-noise detection thresholds using a modulated masker compared to those using an unmodulated masker. In the current study, translations of these two experiments were carried out to test the hypothesis that analogous cues might be used in the envelope-frequency domain. Pure-tone carrier amplitude-modulation (AM) depth-discrimination thresholds were found to be similar using both traditional gated stimuli and using a temporally modulated fringe for a fixed standard depth (ms = 0.25) and a range of AM frequencies (4-64 Hz). In a second experiment, masked sinusoidal AM detection thresholds were compared in conditions with and without slow and regular fluctuations imposed on the instantaneous masker AM depth. Release from masking was obtained only for very slow masker fluctuations (less than 2 Hz). A physiologically motivated model that effectively acts as a first-order envelope change detector accounted for several, but not all, of the key aspects of the data.

Effect of modulation maskers on the detection of second-order amplitude modulation with and without notched noise

The mechanisms underlying the detection of second-order amplitude modulation (AM) were explored. The detectability of second-order AM (fixed depth for each subject) was measured for first- and second-order modulation rates of 16 and 2 Hz, respectively (slow-rate pair), and 50 and 10 Hz, respectively (fast-rate pair), with no masker, a low-band modulation masker (centered at 2 or 10 Hz), and a high-band modulation masker (centered at 16 or 50 Hz). This was done in the absence and presence of an audio-frequency notched noise centered at the carrier frequency of 4000 Hz. Both modulation maskers were "low-noise" noises, to prevent overmodulation. In the absence of notched noise, both modulation maskers impaired performance for the slow-rate pair, but only the low-band masker impaired performance for the fast-rate pair. When notched noise was present, the low-band masker had no significant effect for either rate pair and the high-band masker had an effect only for the slow-rate pair. These results suggest that second-order AM detection is mediated both by an envelope distortion component at the second-order rate and by slow fluctuations in the output of a modulation filter tuned to the first-order rate. When notched noise is present, the distortion component plays little role.

Perception of amplitude modulation by hearing-impaired listeners: the audibility of component modulation and detection of phase change in three-component modulators

Two experiments were conducted to assess whether hearing-impaired listeners have a reduced ability to process suprathreshold complex patterns of modulation applied to a 4-kHz sinusoidal carrier. Experiment 1 examined the ability to "hear out" the modulation frequency of the central component of a three-component modulator, using the method described by Sek and Moore [J. Acoust. Soc. Am. 113, 2801-2811 (2003)]. Scores were around 70-80% correct when the components in the three-component modulator were widely spaced and when the frequencies of the target and comparison different sufficiently, but decreased when the components in the modulator were closely spaced. Experiment 2 examined the ability to hear a change in the relative phase of the components in a three-component modulator with harmonically spaced components. The frequency of the central component, f, was either 50 or 100 Hz. Scores were about 70% correct when the component spacing was < or = 0.5fc, but decreased markedly for greater spacings. Performance was only slightly impaired by randomizing the overall modulation depth from one stimulus to the next. For both experiments, performance was only slightly worse than for normally hearing listeners, indicating that cochlear hearing loss does not markedly affect the ability to process suprathreshold complex patterns of modulation.

Perception of the envelope-beat frequency of inharmonic complex temporal envelopes

Listeners can hear slow sinusoidal variations in the depth of sinusoidally amplitude-modulated (SAM) stimuli. Here, the SAM stimulus of frequency f(m) acts as the carrier, and the slow variation in depth of frequency f'm (referred to as "second-order" amplitude modulation) corresponds to a beat in the temporal envelope. Recent studies have suggested that second-order amplitude modulation perception is based on a modulation-distortion component or the "venelope" (the Hilbert envelope of the ac-coupled Hilbert envelope), both occurring at the envelope-beat frequency f'm. This was tested by transposing to the modulation domain the matching paradigm used by Schouten et al. [J. Acoust. Soc. Am. 34, 1418-1424 (1962)]. Listeners estimated the envelope-beat frequency evoked by a 5-Hz, second-order SAM white noise with f(m) either an integer multiple of f'm or shifted in frequency to make the complex envelope inharmonic. The results indicate that the perception of the envelope-beat frequency was affected by these shifts when f(m) < or = 20 Hz. This suggests that, at least at low modulation frequencies, the perceived envelope beat is not determined by a modulation-distortion or venelope component, but rather relies on the time intervals between the main peaks of the first-order envelope.

Masking release for consonant features in temporally fluctuating background noise

Consonant identification was measured for normal-hearing listeners using Vowel-Consonant-Vowel stimuli that were either unprocessed or spectrally degraded to force listeners to use temporal-envelope cues. Stimuli were embedded in a steady state or fluctuating noise masker and presented at a fixed signal-to-noise ratio. Fluctuations in the maskers were obtained by applying sinusoidal modulation to: (i) the amplitude of the noise (1st-order SAM masker) or (ii) the modulation depth of a 1st-order SAM noise (2nd-order SAM masker). The frequencies of the amplitude variation fm and the depth variation f'm were systematically varied. Consistent with previous studies, identification scores obtained with unprocessed speech were highest in an 8-Hz, 1st-order SAM masker. Reception of voicing and manner also peaked around fm=8 Hz, while the reception of place of articulation was maximal at a higher frequency (fm=32 Hz). When 2nd-order SAM maskers were used, identification scores and received information for each consonant feature were found to be independent of f'm. They decreased progressively with increasing carrier modulation frequency fm, and ranged between those obtained with the steady state and the 1st-order SAM maskers. Finally, the results obtained with spectrally degraded speech were similar across all types of maskers, although an 8% improvement in the reception of voicing was observed for modulated maskers with fm < 64 Hz compared to the steady-state masker. These data provide additional evidence that listeners take advantage of temporal minima in fluctuating background noises, and suggest that: (i) minima of different durations are required for an optimal reception of the three consonant features and (ii) complex (i.e., 2nd-order) envelope fluctuations in background noise do not degrade speech identification by interfering with speech-envelope processing.

Forward masking of amplitude modulation: basic characteristics

In this study we demonstrate an effect for amplitude modulation (AM) that is analogous to forward making of audio frequencies, i.e., the modulation threshold for detection of AM (signal) is raised by preceding AM (masker). In the study we focused on the basic characteristics of the forward-masking effect. Functions representing recovery from AM forward masking measured with a 150- ms 40- Hz masker AM and a 50- ms signal AM of the same rate imposed on the same broadband-noise carrier, showed an exponential decay of forward masking with increasing delay from masker offset. Thresholds remained elevated by more than 2 dB over an interval of at least 150 ms following the masker. Masked-threshold patterns, measured with a fixed signal rate (20, 40, and 80 Hz) and a variable masker rate, showed tuning of the AM forward-masking effect. The tuning was approximately constant across signal modulation rates used and consistent with the idea of modulation-rate selective channels. Combining two equally effective forward maskers of different frequencies did not lead to an increase in forward masking relative to that produced by either component alone. Overall, the results are consistent with modulation-rate selective neural channels that adapt and recover from the adaptation relatively quickly.