Modulation masking produced by complex tone modulators

Age-related reduction of amplitude modulation frequency selectivity

The perception of amplitude modulations (AMs) has been characterized by a frequency-selective process in the temporal envelope domain and simulated in computational auditory processing and perception models using a modulation filterbank. Such AM frequency-selective processing has been argued to be critical for the perception of complex sounds, including speech. This study aimed at investigating the effects of age on behavioral AM frequency selectivity in young (n = 11, 22-29 years) versus older (n = 10, 57-77 years) listeners with normal hearing, using a simultaneous AM masking paradigm with a sinusoidal carrier (2.8 kHz), target modulation frequencies of 4, 16, 64, and 128 Hz, and narrowband-noise modulation maskers. A reduction of AM frequency selectivity by a factor of up to 2 was found in the older listeners. While the observed AM selectivity co-varied with the unmasked AM detection sensitivity, the age-related broadening of the masked threshold patterns remained stable even when AM sensitivity was similar across groups for an extended stimulus duration. The results from the present study might provide a valuable basis for further investigations exploring the effects of age and reduced AM frequency selectivity on complex sound perception as well as the interaction of age and hearing impairment on AM processing and perception.

Forward masking of amplitude modulation across ears and its tuning in the modulation domain

Frequency selectivity in the amplitude modulation (AM) domain has been demonstrated using both simultaneous AM masking and forward AM masking. This has been explained using the concept of a modulation filter bank (MFB). Here, we assessed whether the MFB occurs before or after the point of binaural interaction in the auditory pathway by using forward masking in the AM domain in an ipsilateral condition (masker AM and signal AM applied to the left ear with an unmodulated carrier in the right ear) and a contralateral condition (masker AM applied to the right ear and signal AM applied to the left ear). The carrier frequency was 8 kHz, the signal AM frequency, f_s, was 40 or 80 Hz, and the masker AM frequency ranged from 0.25 to 4 times f_s. Contralateral forward AM masking did occur, but it was smaller than ipsilateral AM masking. Tuning in the AM domain was slightly sharper for ipsilateral than for contralateral masking, perhaps reflecting confusion of the signal and masker AM in the ipsilateral condition when their AM frequencies were the same. The results suggest that there might be an MFB both before and after the point in the auditory pathway where binaural interaction occurs.

Cascaded Amplitude Modulations in Sound Texture Perception

Sound textures, such as crackling fire or chirping crickets, represent a broad class of sounds defined by their homogeneous temporal structure. It has been suggested that the perception of texture is mediated by time-averaged summary statistics measured from early auditory representations. In this study, we investigated the perception of sound textures that contain rhythmic structure, specifically second-order amplitude modulations that arise from the interaction of different modulation rates, previously described as “beating” in the envelope-frequency domain. We developed an auditory texture model that utilizes a cascade of modulation filterbanks that capture the structure of simple rhythmic patterns. The model was examined in a series of psychophysical listening experiments using synthetic sound textures—stimuli generated using time-averaged statistics measured from real-world textures. In a texture identification task, our results indicated that second-order amplitude modulation sensitivity enhanced recognition. Next, we examined the contribution of the second-order modulation analysis in a preference task, where the proposed auditory texture model was preferred over a range of model deviants that lacked second-order modulation rate sensitivity. Lastly, the discriminability of textures that included second-order amplitude modulations appeared to be perceived using a time-averaging process. Overall, our results demonstrate that the inclusion of second-order modulation analysis generates improvements in the perceived quality of synthetic textures compared to the first-order modulation analysis considered in previous approaches.

Comodulation masking release in an off-frequency masking paradigm

Detection threshold of a sinusoidal signal masked by a broadband masker is lower when on- and off-frequency masker components have a correlated envelope, compared to a condition in which these masker components have different envelopes. This effect is commonly referred to as comodulation masking release (CMR). The present study investigated if there is a CMR in the absence of a masker component at the signal frequency, i.e., in an off-frequency masking paradigm. Thresholds were measured for a 500-Hz signal in the presence of a broadband masker with a spectral notch at the signal frequency. Thresholds were significantly lower for a (co-)modulated than for an unmodulated masker for all notch widths up to 400 Hz. An additional experiment showed that the particularly large CMR for the no-notch condition was due to the way the modulated masker was generated. No CMR was measured when the notched-noise masker was replaced by a pair of narrowband noises. The addition of more remote masker bands resulted in a CMR of about 3-4 dB. The notched-noise data were predicted on the basis of a modulation-filterbank model. The predictions of the narrowband noise conditions indicated that all mechanisms underlying CMR might still not be fully understood.

Lateralization based on interaural differences in the second-order amplitude modulator

Second-order amplitude modulation is a relatively slow variation of the modulation depth of a first-order amplitude modulation with higher frequency. In contrast to first-order modulation, which appears as a physical component in the stimulus spectrum after half-wave rectification, second-order modulation is not necessarily demodulated by the auditory periphery. For binaural processing of second-order amplitude modulated stimuli it is unknown whether interaural time differences (ITDs) in the second-order modulation result in a lateralized percept. Thus, second-order modulation can serve as a tool to investigate whether demodulation of interaurally delayed components is a prerequisite for lateralization. In most of the psychoacoustic experiments presented here, a 25 Hz sinusoidally amplitude-modulated (SAM) 160 Hz tone was either transposed to 4 kHz by half-wave rectifying this SAM waveform before multiplication with a 4 kHz tone (TSAM), or by adding an offset before multiplication (SAMAM). The experiments revealed an inability to lateralize the SAMAM based on ITDs in the 25 Hz component, whereas subjects could lateralize the TSAM. Given that only the TSAM results in a demodulated 25 Hz component after peripheral auditory processing, this result supports the hypothesis that demodulation is a prerequisite for lateralization, which has consequences for temporal modulation processing in models of binaural interaction.

The origin of binaural interaction in the modulation domain

The purpose of these experiments was to assess whether the detection of diotic 5 Hz "probe" modulation of a 4000 Hz sinusoidal carrier was influenced by binaural interaction of "masker" modulators presented separately to each ear and applied to the same carrier. A 50 Hz masker modulator was applied to one ear and the masker modulator applied to the other ear had a frequency of 55 or 27.5 Hz. The starting phase of the masker modulators was fixed, and the starting phase of the probe modulator was varied. For both pairs of masker modulators, the threshold for detecting the probe modulation varied slightly but significantly with probe starting phase. Further experiments measuring probe detectability as a function of probe modulation depth did not provide clear evidence to support the idea that the internal representations of the masker modulators interacted binaurally to produce a weak distortion component in the internal representation of the modulation at a 5 Hz frequency. Also, the obtained phase effects were not correctly predicted using a model based on short-term loudness fluctuations.

Estimation of the center frequency of the highest modulation filter

For high-frequency sinusoidal carriers, the threshold for detecting sinusoidal amplitude modulation increases when the signal modulation frequency increases above about 120 Hz. Using the concept of a modulation filter bank, this effect might be explained by (1) a decreasing sensitivity or greater internal noise for modulation filters with center frequencies above 120 Hz; and (2) a limited span of center frequencies of the modulation filters, the top filter being tuned to about 120 Hz. The second possibility was tested by measuring modulation masking in forward masking using an 8 kHz sinusoidal carrier. The signal modulation frequency was 80, 120, or 180 Hz and the masker modulation frequencies covered a range above and below each signal frequency. Four highly trained listeners were tested. For the 80-Hz signal, the signal threshold was usually maximal when the masker frequency equaled the signal frequency. For the 180-Hz signal, the signal threshold was maximal when the masker frequency was below the signal frequency. For the 120-Hz signal, two listeners showed the former pattern, and two showed the latter pattern. The results support the idea that the highest modulation filter has a center frequency in the range 100-120 Hz.

Discrimination of starting phase with sinusoidal envelope modulation

Modulation-filterbank models discard phase information above very low rates of amplitude modulation (AM). The present work evaluated this restriction by measuring thresholds for discriminating the starting phase of sinusoidal modulators of wideband-noise carriers. Results showed a low-pass characteristic with some listeners unable to perform the task once the modulation rate was greater than 12.5 Hz. For others, however, thresholds were obtained with AM rates of up to one to two octaves higher. Intersubject variability may in part relate to the presence of multiple discrimination cues, with only some based on comparison of the ongoing pattern of envelope fluctuation.

Effect of modulation maskers on the detection of second-order amplitude modulation with and without notched noise

The mechanisms underlying the detection of second-order amplitude modulation (AM) were explored. The detectability of second-order AM (fixed depth for each subject) was measured for first- and second-order modulation rates of 16 and 2 Hz, respectively (slow-rate pair), and 50 and 10 Hz, respectively (fast-rate pair), with no masker, a low-band modulation masker (centered at 2 or 10 Hz), and a high-band modulation masker (centered at 16 or 50 Hz). This was done in the absence and presence of an audio-frequency notched noise centered at the carrier frequency of 4000 Hz. Both modulation maskers were "low-noise" noises, to prevent overmodulation. In the absence of notched noise, both modulation maskers impaired performance for the slow-rate pair, but only the low-band masker impaired performance for the fast-rate pair. When notched noise was present, the low-band masker had no significant effect for either rate pair and the high-band masker had an effect only for the slow-rate pair. These results suggest that second-order AM detection is mediated both by an envelope distortion component at the second-order rate and by slow fluctuations in the output of a modulation filter tuned to the first-order rate. When notched noise is present, the distortion component plays little role.

Perception of amplitude modulation by hearing-impaired listeners: the audibility of component modulation and detection of phase change in three-component modulators

Two experiments were conducted to assess whether hearing-impaired listeners have a reduced ability to process suprathreshold complex patterns of modulation applied to a 4-kHz sinusoidal carrier. Experiment 1 examined the ability to "hear out" the modulation frequency of the central component of a three-component modulator, using the method described by Sek and Moore [J. Acoust. Soc. Am. 113, 2801-2811 (2003)]. Scores were around 70-80% correct when the components in the three-component modulator were widely spaced and when the frequencies of the target and comparison different sufficiently, but decreased when the components in the modulator were closely spaced. Experiment 2 examined the ability to hear a change in the relative phase of the components in a three-component modulator with harmonically spaced components. The frequency of the central component, f, was either 50 or 100 Hz. Scores were about 70% correct when the component spacing was < or = 0.5fc, but decreased markedly for greater spacings. Performance was only slightly impaired by randomizing the overall modulation depth from one stimulus to the next. For both experiments, performance was only slightly worse than for normally hearing listeners, indicating that cochlear hearing loss does not markedly affect the ability to process suprathreshold complex patterns of modulation.

Perception of the envelope-beat frequency of inharmonic complex temporal envelopes

Listeners can hear slow sinusoidal variations in the depth of sinusoidally amplitude-modulated (SAM) stimuli. Here, the SAM stimulus of frequency f(m) acts as the carrier, and the slow variation in depth of frequency f'm (referred to as "second-order" amplitude modulation) corresponds to a beat in the temporal envelope. Recent studies have suggested that second-order amplitude modulation perception is based on a modulation-distortion component or the "venelope" (the Hilbert envelope of the ac-coupled Hilbert envelope), both occurring at the envelope-beat frequency f'm. This was tested by transposing to the modulation domain the matching paradigm used by Schouten et al. [J. Acoust. Soc. Am. 34, 1418-1424 (1962)]. Listeners estimated the envelope-beat frequency evoked by a 5-Hz, second-order SAM white noise with f(m) either an integer multiple of f'm or shifted in frequency to make the complex envelope inharmonic. The results indicate that the perception of the envelope-beat frequency was affected by these shifts when f(m) < or = 20 Hz. This suggests that, at least at low modulation frequencies, the perceived envelope beat is not determined by a modulation-distortion or venelope component, but rather relies on the time intervals between the main peaks of the first-order envelope.

Masking release for consonant features in temporally fluctuating background noise

Consonant identification was measured for normal-hearing listeners using Vowel-Consonant-Vowel stimuli that were either unprocessed or spectrally degraded to force listeners to use temporal-envelope cues. Stimuli were embedded in a steady state or fluctuating noise masker and presented at a fixed signal-to-noise ratio. Fluctuations in the maskers were obtained by applying sinusoidal modulation to: (i) the amplitude of the noise (1st-order SAM masker) or (ii) the modulation depth of a 1st-order SAM noise (2nd-order SAM masker). The frequencies of the amplitude variation fm and the depth variation f'm were systematically varied. Consistent with previous studies, identification scores obtained with unprocessed speech were highest in an 8-Hz, 1st-order SAM masker. Reception of voicing and manner also peaked around fm=8 Hz, while the reception of place of articulation was maximal at a higher frequency (fm=32 Hz). When 2nd-order SAM maskers were used, identification scores and received information for each consonant feature were found to be independent of f'm. They decreased progressively with increasing carrier modulation frequency fm, and ranged between those obtained with the steady state and the 1st-order SAM maskers. Finally, the results obtained with spectrally degraded speech were similar across all types of maskers, although an 8% improvement in the reception of voicing was observed for modulated maskers with fm < 64 Hz compared to the steady-state masker. These data provide additional evidence that listeners take advantage of temporal minima in fluctuating background noises, and suggest that: (i) minima of different durations are required for an optimal reception of the three consonant features and (ii) complex (i.e., 2nd-order) envelope fluctuations in background noise do not degrade speech identification by interfering with speech-envelope processing.

Modulation masking produced by second-order modulators

Recent studies suggest that an auditory nonlinearity converts second-order sinusoidal amplitude modulation (SAM) (i.e., modulation of SAM depth) into a first-order SAM component, which contributes to the perception of second-order SAM. However, conversion may also occur in other ways such as cochlear filtering. The present experiments explored the source of the first-order SAM component by investigating the ability to detect a 5-Hz, first-order SAM probe in the presence of a second-order SAM masker beating at the probe frequency. Detection performance was measured as a function of masker-carrier modulation frequency, phase relationship between the probe and masker modulator, and probe modulation depth. In experiment 1, the carrier was a 5-kHz sinusoid presented either alone or within a notched-noise masker in order to restrict off-frequency listening. In experiment 2, the carrier was a white noise. The data obtained in both carrier conditions are consistent with the existence of a modulation distortion component. However, the phase yielding poorest detection performance varied across experimental conditions between 0 degrees and 180 degrees, confirming that, in addition to nonlinear mechanisms, cochlear filtering and off-frequency listening play a role in second-order SAM perception. The estimated magnitude of the modulation distortion component ranges from 5%-12%.

Estimation of the level and phase of the simple distortion tone the modulation domain

These experiments were designed to test the idea that nonlinearities in the auditory system can introduce a distortion component into the internal representation of the envelope of a sound, and to estimate the phase of the hypothetical distortion component. In experiment 1, a two-alternative forced-choice (2AFC) task with feedback was used to measure psychometric functions for detecting 5-Hz probe modulation of a 4-kHz sinusoidal carrier in the presence of a masker modulator with components at 50 and 55 Hz (m=0.3 for each component). Performance was measured as a function of the relative phase, delta[symbol see text], of the probe relative to the "venelope" (envelope of the envelope) of the masker. Performance was poorest for delta[symbol see text]= 135 degrees. In experiment 2, delta[symbol see text] was fixed at 135 degrees, m was set to 0.48 for each masker component, and psychometric functions for detecting probe modulation were measured using a 2AFC task without feedback. For small probe modulation depths (m approximately 0.03), the detectability index, d', was consistently negative, consistent with the existence of a weak distortion product which can "cancel" the probe modulation. The distortion component for the conditions of the experiment was estimated to have a phase of about -25 degrees relative to the venelope.