Testing the concept of a modulation filter bank: the audibility of component modulation and detection of phase change in three-component modulators

Disentangling the effects of hearing loss and age on amplitude modulation frequency selectivity

The processing and perception of amplitude modulation (AM) in the auditory system reflect a frequency-selective process, often described as a modulation filterbank. Previous studies on perceptual AM masking reported similar results for older listeners with hearing impairment (HI listeners) and young listeners with normal hearing (NH listeners), suggesting no effects of age or hearing loss on AM frequency selectivity. However, recent evidence has shown that age, independently of hearing loss, adversely affects AM frequency selectivity. Hence, this study aimed to disentangle the effects of hearing loss and age. A simultaneous AM masking paradigm was employed, using a sinusoidal carrier at 2.8 kHz, narrowband noise modulation maskers, and target modulation frequencies of 4, 16, 64, and 128 Hz. The results obtained from young (n = 3, 24-30 years of age) and older (n = 10, 63-77 years of age) HI listeners were compared to previously obtained data from young and older NH listeners. Notably, the HI listeners generally exhibited lower (unmasked) AM detection thresholds and greater AM frequency selectivity than their NH counterparts in both age groups. Overall, the results suggest that age negatively affects AM frequency selectivity for both NH and HI listeners, whereas hearing loss improves AM detection and AM selectivity, likely due to the loss of peripheral compression.

Informational masking in the modulation domain

Uncertainty regarding the frequency spectrum of a masker can have an adverse effect on the ability to focus selective attention on a target frequency channel, yielding informational masking (IM). This study sought to determine if uncertainty regarding the modulation spectrum of a masker can have an analogous adverse effect on the ability to focus selective attention on a target modulation channel, yielding IM in the modulation domain, or "modulation IM." A single-interval, two-alternative forced-choice (yes-no) procedure was used. The task was to detect 32-Hz target sinusoidal amplitude modulation (SAM) imposed on a broadband-noise carrier in the presence of masker SAM imposed on the same carrier. Six maskers, spanning the range from 8 to 128 Hz in half-octave steps, were tested, excluding those that fell within a two-octave protected zone surrounding the target. Psychometric functions (d'-vs-target modulation depth) were measured for each masker under two conditions: a fixed (low-uncertainty/low-IM) condition, in which the masker was the same on all trials within a block, and a random (high-uncertainty/high-IM) condition, in which it varied randomly from presentation-to-presentation. Thresholds and slopes extracted from the psychometric functions differed markedly between the conditions. These results are consistent with the idea that IM occurs in the modulation domain.

Frequency selectivity in the modulation domain estimated using forward masking: Effects of masker modulation depth and masker-signal delay

The threshold for detecting amplitude modulation (AM) of a sinusoidal or noise carrier is elevated when the signal AM is preceded by masker AM applied to the same carrier. This effect, called AM forward masking, shows selectivity in the AM domain, consistent with the existence of a modulation filter bank (MFB). In this paper we explore the effect of two factors that can influence AM forward masking, using an 8-kHz sinusoidal carrier and a range of masker AM frequencies, f_m, both below and above the signal AM frequency, f_s, of 40 Hz. The first factor was the time delay, t_d, between the end of the masker AM and the start of the signal AM. The second was the AM depth, m, of the masker, which was either 1 or 0.25. The AM forward masking patterns in all conditions showed tuning in the AM domain; signal thresholds were highest when f_m was close to f_s. The amount of AM forward masking decreased with increasing t_d in a similar way for all f_m, so the shapes of the masking patterns did not change markedly with t_d. Remarkably, the amount of AM forward masking decreased by only about 3 dB (a non-significant effect) when the masker m was decreased from 1 to 0.25. This result appears to be inconsistent with an explanation of AM forward masking in terms of adaptation in a MFB or in terms of a sliding temporal integrator.

Forward masking of amplitude modulation across ears and its tuning in the modulation domain

Frequency selectivity in the amplitude modulation (AM) domain has been demonstrated using both simultaneous AM masking and forward AM masking. This has been explained using the concept of a modulation filter bank (MFB). Here, we assessed whether the MFB occurs before or after the point of binaural interaction in the auditory pathway by using forward masking in the AM domain in an ipsilateral condition (masker AM and signal AM applied to the left ear with an unmodulated carrier in the right ear) and a contralateral condition (masker AM applied to the right ear and signal AM applied to the left ear). The carrier frequency was 8 kHz, the signal AM frequency, f_s, was 40 or 80 Hz, and the masker AM frequency ranged from 0.25 to 4 times f_s. Contralateral forward AM masking did occur, but it was smaller than ipsilateral AM masking. Tuning in the AM domain was slightly sharper for ipsilateral than for contralateral masking, perhaps reflecting confusion of the signal and masker AM in the ipsilateral condition when their AM frequencies were the same. The results suggest that there might be an MFB both before and after the point in the auditory pathway where binaural interaction occurs.

The rhythm of attention: Perceptual modulation via rhythmic entrainment is lowpass and attention mediated

Modulation patterns are known to carry critical predictive cues to signal detection in complex acoustic environments. The current study investigated the persistence of masker modulation effects on postmodulation detection of probe signals. Hickok, Farahbod, and Saberi (Psychological Science, 26, 1006-1013, 2015) demonstrated that thresholds for a tone pulse in stationary noise follow a predictable periodic pattern when preceded by a 3-Hz amplitude modulated masker. They found entrainment of detection patterns to the modulation envelope lasting for approximately two cycles after termination of modulation. The current study extends these results to a wide range of modulation rates by mapping the temporal modulation transfer function for persistent modulatory effects. We found significant entrainment to modulation rates of 2 and 3 Hz, a weaker effect at 5 Hz, and no entrainment at higher rates (8 to 32 Hz). The effect seems critically dependent on attentional mechanisms, requiring temporal and level uncertainty of the probe signal. Our findings suggest that the persistence of modulatory effects on signal detection is lowpass in nature and attention based.

Comparison of perceptual properties of auditory streaming between spectral and amplitude modulation domains

The two-tone sequence (ABA_), which comprises two different sounds (A and B) and a silent gap, has been used to investigate how the auditory system organizes sequential sounds depending on various stimulus conditions or brain states. Auditory streaming can be evoked by differences not only in the tone frequency ("spectral cue": ΔF_TONE, TONE condition) but also in the amplitude modulation rate ("AM cue": ΔF_AM, AM condition). The aim of the present study was to explore the relationship between the perceptual properties of auditory streaming for the TONE and AM conditions. A sequence with a long duration (400 repetitions of ABA_) was used to examine the property of the bistability of streaming. The ratio of feature differences that evoked an equivalent probability of the segregated percept was close to the ratio of the Q-values of the auditory and modulation filters, consistent with a "channeling theory" of auditory streaming. On the other hand, for values of ΔF_AM and ΔF_TONE evoking equal probabilities of the segregated percept, the number of perceptual switches was larger for the TONE condition than for the AM condition, indicating that the mechanism(s) that determine the bistability of auditory streaming are different between or sensitive to the two domains. Nevertheless, the number of switches for individual listeners was positively correlated between the spectral and AM domains. The results suggest a possibility that the neural substrates for spectral and AM processes share a common switching mechanism but differ in location and/or in the properties of neural activity or the strength of internal noise at each level.

Dynamic Reweighting of Auditory Modulation Filters

Sound waveforms convey information largely via amplitude modulations (AM). A large body of experimental evidence has provided support for a modulation (bandpass) filterbank. Details of this model have varied over time partly reflecting different experimental conditions and diverse datasets from distinct task strategies, contributing uncertainty to the bandwidth measurements and leaving important issues unresolved. We adopt here a solely data-driven measurement approach in which we first demonstrate how different models can be subsumed within a common 'cascade' framework, and then proceed to characterize the cascade via system identification analysis using a single stimulus/task specification and hence stable task rules largely unconstrained by any model or parameters. Observers were required to detect a brief change in level superimposed onto random level changes that served as AM noise; the relationship between trial-by-trial noisy fluctuations and corresponding human responses enables targeted identification of distinct cascade elements. The resulting measurements exhibit a dynamic complex picture in which human perception of auditory modulations appears adaptive in nature, evolving from an initial lowpass to bandpass modes (with broad tuning, Q∼1) following repeated stimulus exposure.

Modulation masking within and across carriers for subjects with normal and impaired hearing

The detection of amplitude modulation (AM) of a carrier can be impaired by additional (masker) AM applied to the same carrier (within-carrier modulation masking, MM) or to a different carrier (across-carrier MM). These two types of MM were compared for young normal-hearing and older hearing-impaired subjects. The signal was 4- or 16-Hz sinusoidal AM of a 4000-Hz carrier. Masker AM with depth 0.4 was applied either to the same carrier or to a carrier at 3179 or 2518 Hz. The masker AM rate was 0.25, 0.5, 1, 2, or 4 times the signal rate. The signal AM depth was varied adaptively to determine the threshold. Both within-carrier and across-carrier MM patterns were similar for the two groups, suggesting that the hypothetical modulation filters are not affected by hearing loss or age. The signal AM detection thresholds were also similar for the two groups. Thresholds in the absence of masker AM were lower (better) for the older hearing-impaired than for the young normal-hearing subjects. Since the masked modulation thresholds were similar for the two groups, it seems unlikely that abnormal MM contributes to the difficulties experienced by older hearing-impaired people in understanding speech in background sounds.

Modulation-frequency-specific adaptation in awake auditory cortex

Amplitude modulations are fundamental features of natural signals, including human speech and nonhuman primate vocalizations. Because natural signals frequently occur in the context of other competing signals, we used a forward-masking paradigm to investigate how the modulation context of a prior signal affects cortical responses to subsequent modulated sounds. Psychophysical "modulation masking," in which the presentation of a modulated "masker" signal elevates the threshold for detecting the modulation of a subsequent stimulus, has been interpreted as evidence of a central modulation filterbank and modeled accordingly. Whether cortical modulation tuning is compatible with such models remains unknown. By recording responses to pairs of sinusoidally amplitude modulated (SAM) tones in the auditory cortex of awake squirrel monkeys, we show that the prior presentation of the SAM masker elicited persistent and tuned suppression of the firing rate to subsequent SAM signals. Population averages of these effects are compatible with adaptation in broadly tuned modulation channels. In contrast, modulation context had little effect on the synchrony of the cortical representation of the second SAM stimuli and the tuning of such effects did not match that observed for firing rate. Our results suggest that, although the temporal representation of modulated signals is more robust to changes in stimulus context than representations based on average firing rate, this representation is not fully exploited and psychophysical modulation masking more closely mirrors physiological rate suppression and that rate tuning for a given stimulus feature in a given neuron's signal pathway appears sufficient to engender context-sensitive cortical adaptation.

On the near non-existence of "pure" energetic masking release for speech

Stone et al. [(2012). J. Acoust. Soc. Am. 132, 317-326] showed that a masker constructed to produce a near-constant envelope at the output of each auditory filter reduced speech intelligibility less than maskers of the same mean level with fluctuating envelopes, produced by 100% sinusoidal amplitude modulation (SAM) at 8 Hz. Here, this effect was explored for a range of SAM rates from 1 to 81 Hz. Speech was filtered into 28 channels. A sinusoidal masker centered on each channel was added to the channel signal. The maskers were either unmodulated or had 100% SAM. In most conditions, even-numbered channels were presented to one ear and odd-numbered channels to the other. The signal-to-masker ratio was adapted to measure the Speech Reception Threshold (SRT) corresponding to 50% correct. The fluctuating masker benefit (FMB), the difference in SRT between the SAM and unmodulated masker, was negative for all SAM frequencies except 1 Hz. Due to the different slopes of the psychometric functions, when SRTs were inferred for more realistic performance levels, 74% or more, FMB was zero or negative for all SAM rates. It is concluded that a positive FMB, when it occurs, is a release from modulation and not energetic masking.

The origin of binaural interaction in the modulation domain

The purpose of these experiments was to assess whether the detection of diotic 5 Hz "probe" modulation of a 4000 Hz sinusoidal carrier was influenced by binaural interaction of "masker" modulators presented separately to each ear and applied to the same carrier. A 50 Hz masker modulator was applied to one ear and the masker modulator applied to the other ear had a frequency of 55 or 27.5 Hz. The starting phase of the masker modulators was fixed, and the starting phase of the probe modulator was varied. For both pairs of masker modulators, the threshold for detecting the probe modulation varied slightly but significantly with probe starting phase. Further experiments measuring probe detectability as a function of probe modulation depth did not provide clear evidence to support the idea that the internal representations of the masker modulators interacted binaurally to produce a weak distortion component in the internal representation of the modulation at a 5 Hz frequency. Also, the obtained phase effects were not correctly predicted using a model based on short-term loudness fluctuations.

Estimation of the center frequency of the highest modulation filter

For high-frequency sinusoidal carriers, the threshold for detecting sinusoidal amplitude modulation increases when the signal modulation frequency increases above about 120 Hz. Using the concept of a modulation filter bank, this effect might be explained by (1) a decreasing sensitivity or greater internal noise for modulation filters with center frequencies above 120 Hz; and (2) a limited span of center frequencies of the modulation filters, the top filter being tuned to about 120 Hz. The second possibility was tested by measuring modulation masking in forward masking using an 8 kHz sinusoidal carrier. The signal modulation frequency was 80, 120, or 180 Hz and the masker modulation frequencies covered a range above and below each signal frequency. Four highly trained listeners were tested. For the 80-Hz signal, the signal threshold was usually maximal when the masker frequency equaled the signal frequency. For the 180-Hz signal, the signal threshold was maximal when the masker frequency was below the signal frequency. For the 120-Hz signal, two listeners showed the former pattern, and two showed the latter pattern. The results support the idea that the highest modulation filter has a center frequency in the range 100-120 Hz.

Neural response properties of primary, rostral, and rostrotemporal core fields in the auditory cortex of marmoset monkeys

The core region of primate auditory cortex contains a primary and two primary-like fields (AI, primary auditory cortex; R, rostral field; RT, rostrotemporal field). Although it is reasonable to assume that multiple core fields provide an advantage for auditory processing over a single primary field, the differential roles these fields play and whether they form a functional pathway collectively such as for the processing of spectral or temporal information are unknown. In this report we compare the response properties of neurons in the three core fields to pure tones and sinusoidally amplitude modulated tones in awake marmoset monkeys (Callithrix jacchus). The main observations are as follows. (1) All three fields are responsive to spectrally narrowband sounds and are tonotopically organized. (2) Field AI responds more strongly to pure tones than fields R and RT. (3) Field RT neurons have lower best sound levels than those of neurons in fields AI and R. In addition, rate-level functions in field RT are more commonly nonmonotonic than in fields AI and R. (4) Neurons in fields RT and R have longer minimum latencies than those of field AI neurons. (5) Fields RT and R have poorer stimulus synchronization than that of field AI to amplitude-modulated tones. (6) Between the three core fields the more rostral regions (R and RT) have narrower firing-rate-based modulation transfer functions than that of AI. This effect was seen only for the nonsynchronized neurons. Synchronized neurons showed no such trend.

Investigation of perceptual constancy in the temporal-envelope domain

The ability to discriminate complex temporal envelope patterns submitted to temporal compression or expansion was assessed in normal-hearing listeners. An XAB, matching-to-sample-procedure was used. X, the reference stimulus, is obtained by applying the sum of two, inharmonically related, sinusoids to a broadband noise carrier. A and B are obtained by multiplying the frequency of each modulation component of X by the same time expansion/compression factor, alpha (alphain[0.35-2.83]). For each trial, A or B is a time-reversed rendering of X, and the listeners' task is to choose which of the two is matched by X. Overall, the results indicate that discrimination performance degrades for increasing amounts of time expansion/compression (i.e., when alpha departs from 1), regardless of the frequency spacing of modulation components and the peak-to-trough ratio of the complex envelopes. An auditory model based on envelope extraction followed by a memory-limited, template-matching process accounted for results obtained without time scaling of stimuli, but generally underestimated discrimination ability with either time expansion or compression, especially with the longer stimulus durations. This result is consistent with partial or incomplete perceptual normalization of envelope patterns.

Effect of modulation maskers on the detection of second-order amplitude modulation with and without notched noise

The mechanisms underlying the detection of second-order amplitude modulation (AM) were explored. The detectability of second-order AM (fixed depth for each subject) was measured for first- and second-order modulation rates of 16 and 2 Hz, respectively (slow-rate pair), and 50 and 10 Hz, respectively (fast-rate pair), with no masker, a low-band modulation masker (centered at 2 or 10 Hz), and a high-band modulation masker (centered at 16 or 50 Hz). This was done in the absence and presence of an audio-frequency notched noise centered at the carrier frequency of 4000 Hz. Both modulation maskers were "low-noise" noises, to prevent overmodulation. In the absence of notched noise, both modulation maskers impaired performance for the slow-rate pair, but only the low-band masker impaired performance for the fast-rate pair. When notched noise was present, the low-band masker had no significant effect for either rate pair and the high-band masker had an effect only for the slow-rate pair. These results suggest that second-order AM detection is mediated both by an envelope distortion component at the second-order rate and by slow fluctuations in the output of a modulation filter tuned to the first-order rate. When notched noise is present, the distortion component plays little role.

Perception of amplitude modulation by hearing-impaired listeners: the audibility of component modulation and detection of phase change in three-component modulators

Two experiments were conducted to assess whether hearing-impaired listeners have a reduced ability to process suprathreshold complex patterns of modulation applied to a 4-kHz sinusoidal carrier. Experiment 1 examined the ability to "hear out" the modulation frequency of the central component of a three-component modulator, using the method described by Sek and Moore [J. Acoust. Soc. Am. 113, 2801-2811 (2003)]. Scores were around 70-80% correct when the components in the three-component modulator were widely spaced and when the frequencies of the target and comparison different sufficiently, but decreased when the components in the modulator were closely spaced. Experiment 2 examined the ability to hear a change in the relative phase of the components in a three-component modulator with harmonically spaced components. The frequency of the central component, f, was either 50 or 100 Hz. Scores were about 70% correct when the component spacing was < or = 0.5fc, but decreased markedly for greater spacings. Performance was only slightly impaired by randomizing the overall modulation depth from one stimulus to the next. For both experiments, performance was only slightly worse than for normally hearing listeners, indicating that cochlear hearing loss does not markedly affect the ability to process suprathreshold complex patterns of modulation.

Perception of the envelope-beat frequency of inharmonic complex temporal envelopes

Listeners can hear slow sinusoidal variations in the depth of sinusoidally amplitude-modulated (SAM) stimuli. Here, the SAM stimulus of frequency f(m) acts as the carrier, and the slow variation in depth of frequency f'm (referred to as "second-order" amplitude modulation) corresponds to a beat in the temporal envelope. Recent studies have suggested that second-order amplitude modulation perception is based on a modulation-distortion component or the "venelope" (the Hilbert envelope of the ac-coupled Hilbert envelope), both occurring at the envelope-beat frequency f'm. This was tested by transposing to the modulation domain the matching paradigm used by Schouten et al. [J. Acoust. Soc. Am. 34, 1418-1424 (1962)]. Listeners estimated the envelope-beat frequency evoked by a 5-Hz, second-order SAM white noise with f(m) either an integer multiple of f'm or shifted in frequency to make the complex envelope inharmonic. The results indicate that the perception of the envelope-beat frequency was affected by these shifts when f(m) < or = 20 Hz. This suggests that, at least at low modulation frequencies, the perceived envelope beat is not determined by a modulation-distortion or venelope component, but rather relies on the time intervals between the main peaks of the first-order envelope.

Masking release for consonant features in temporally fluctuating background noise

Consonant identification was measured for normal-hearing listeners using Vowel-Consonant-Vowel stimuli that were either unprocessed or spectrally degraded to force listeners to use temporal-envelope cues. Stimuli were embedded in a steady state or fluctuating noise masker and presented at a fixed signal-to-noise ratio. Fluctuations in the maskers were obtained by applying sinusoidal modulation to: (i) the amplitude of the noise (1st-order SAM masker) or (ii) the modulation depth of a 1st-order SAM noise (2nd-order SAM masker). The frequencies of the amplitude variation fm and the depth variation f'm were systematically varied. Consistent with previous studies, identification scores obtained with unprocessed speech were highest in an 8-Hz, 1st-order SAM masker. Reception of voicing and manner also peaked around fm=8 Hz, while the reception of place of articulation was maximal at a higher frequency (fm=32 Hz). When 2nd-order SAM maskers were used, identification scores and received information for each consonant feature were found to be independent of f'm. They decreased progressively with increasing carrier modulation frequency fm, and ranged between those obtained with the steady state and the 1st-order SAM maskers. Finally, the results obtained with spectrally degraded speech were similar across all types of maskers, although an 8% improvement in the reception of voicing was observed for modulated maskers with fm < 64 Hz compared to the steady-state masker. These data provide additional evidence that listeners take advantage of temporal minima in fluctuating background noises, and suggest that: (i) minima of different durations are required for an optimal reception of the three consonant features and (ii) complex (i.e., 2nd-order) envelope fluctuations in background noise do not degrade speech identification by interfering with speech-envelope processing.

Auditory processing of real and illusory changes in frequency modulation (FM) phase

Auditory processing of frequency modulation (FM) was explored. In experiment 1, detection of a tau-radians modulator phase shift deteriorated as modulation rate increased from 2.5 to 20 Hz, for 1- and 6-kHz carriers. In experiment 2, listeners discriminated between two 1-kHz carriers, where, mid-way through, the 10-Hz frequency modulator had either a phase shift or increased in depth by deltaD% for half a modulator period. Discrimination was poorer for deltaD = 4% than for smaller or larger increases. These results are consistent with instantaneous frequency being smoothed by a time window with a total duration of about 110 ms. In experiment 3, the central 200-ms of a 1-s 1-kHz carrier modulated at 5 Hz was replaced by noise, or by a faster FM applied to a more intense 1-kHz carrier. Listeners heard the 5-Hz FM continue at the same depth throughout the stimulus. Experiments 4 and 5 showed that, after an FM tone had been interrupted by a 200-ms noise, listeners were insensitive to the phase at which the FM resumed. It is argued that the auditory system explicitly encodes the presence, and possibly the rate and depth, of FM in a way that does not preserve information on FM phase.

Estimation of the level and phase of the simple distortion tone the modulation domain

These experiments were designed to test the idea that nonlinearities in the auditory system can introduce a distortion component into the internal representation of the envelope of a sound, and to estimate the phase of the hypothetical distortion component. In experiment 1, a two-alternative forced-choice (2AFC) task with feedback was used to measure psychometric functions for detecting 5-Hz probe modulation of a 4-kHz sinusoidal carrier in the presence of a masker modulator with components at 50 and 55 Hz (m=0.3 for each component). Performance was measured as a function of the relative phase, delta[symbol see text], of the probe relative to the "venelope" (envelope of the envelope) of the masker. Performance was poorest for delta[symbol see text]= 135 degrees. In experiment 2, delta[symbol see text] was fixed at 135 degrees, m was set to 0.48 for each masker component, and psychometric functions for detecting probe modulation were measured using a 2AFC task without feedback. For small probe modulation depths (m approximately 0.03), the detectability index, d', was consistently negative, consistent with the existence of a weak distortion product which can "cancel" the probe modulation. The distortion component for the conditions of the experiment was estimated to have a phase of about -25 degrees relative to the venelope.