The role of cochlear place coding in the perception of frequency modulation

Individual Differences Elucidate the Perceptual Benefits Associated with Robust Temporal Fine-Structure Processing

The auditory system is unique among sensory systems in its ability to phase lock to and precisely follow very fast cycle-by-cycle fluctuations in the phase of sound-driven cochlear vibrations. Yet, the perceptual role of this temporal fine structure (TFS) code is debated. This fundamental gap is attributable to our inability to experimentally manipulate TFS cues without altering other perceptually relevant cues. Here, we circumnavigated this limitation by leveraging individual differences across 200 participants to systematically compare variations in TFS sensitivity to performance in a range of speech perception tasks. TFS sensitivity was assessed through detection of interaural time/phase differences, while speech perception was evaluated by word identification under noise interference. Results suggest that greater TFS sensitivity is not associated with greater masking release from fundamental-frequency or spatial cues, but appears to contribute to resilience against the effects of reverberation. We also found that greater TFS sensitivity is associated with faster response times, indicating reduced listening effort. These findings highlight the perceptual significance of TFS coding for everyday hearing.

The Rapid Decline in Interaural-Time-Difference Sensitivity for Pure Tones Can Be Explained by Peripheral Filtering

PURPOSE

The interaural time difference (ITD) is a primary horizontal-plane sound localization cue computed in the auditory brainstem. ITDs are accessible in the temporal fine structure of pure tones with a frequency of no higher than about 1400 Hz. How listeners' ITD sensitivity transitions from very best sensitivity near 700 Hz to impossible to detect within 1 octave currently lacks a fully compelling physiological explanation. Here, it was hypothesized that the rapid decline in ITD sensitivity is dictated not by a central neural limitation but by initial peripheral sound encoding, specifically, the low-frequency (apical) portion of the cochlear excitation pattern produced by a pure tone.

METHODS

ITD sensitivity was measured in 16 normal-hearing listeners as a joint function of frequency (900-1500 Hz) and level (10-50 dB sensation level).

RESULTS

Performance decreased with increasing frequency and decreasing sound level. The slope of performance decline was 90 dB/octave, consistent with the low-frequency slope of the cochlear excitation pattern.

CONCLUSION

Fine-structure ITD sensitivity near 1400 Hz may be conveyed primarily by "off-frequency" activation of neurons tuned to lower frequencies near 700 Hz. Physiologically, this could be realized by having neurons sensitive to fine-structure ITD up to only about 700 Hz. A more extreme model would have only a single narrow channel near 700 Hz that conveys fine-structure ITDs. Such a model is a major simplification and departure from the classic formulation of the binaural display, which consists of a matrix of neurons tuned to a wide range of relevant frequencies and ITDs.

Web-based psychoacoustics: Hearing screening, infrastructure, and validation

Anonymous web-based experiments are increasingly used in many domains of behavioral research. However, online studies of auditory perception, especially of psychoacoustic phenomena pertaining to low-level sensory processing, are challenging because of limited available control of the acoustics, and the inability to perform audiometry to confirm normal-hearing status of participants. Here, we outline our approach to mitigate these challenges and validate our procedures by comparing web-based measurements to lab-based data on a range of classic psychoacoustic tasks. Individual tasks were created using jsPsych, an open-source JavaScript front-end library. Dynamic sequences of psychoacoustic tasks were implemented using Django, an open-source library for web applications, and combined with consent pages, questionnaires, and debriefing pages. Subjects were recruited via Prolific, a subject recruitment platform for web-based studies. Guided by a meta-analysis of lab-based data, we developed and validated a screening procedure to select participants for (putative) normal-hearing status based on their responses in a suprathreshold task and a survey. Headphone use was standardized by supplementing procedures from prior literature with a binaural hearing task. Individuals meeting all criteria were re-invited to complete a range of classic psychoacoustic tasks. For the re-invited participants, absolute thresholds were in excellent agreement with lab-based data for fundamental frequency discrimination, gap detection, and sensitivity to interaural time delay and level difference. Furthermore, word identification scores, consonant confusion patterns, and co-modulation masking release effect also matched lab-based studies. Our results suggest that web-based psychoacoustics is a viable complement to lab-based research. Source code for our infrastructure is provided.

The rapid decline in interaural-time-difference sensitivity for pure tones can be explained by peripheral filtering

Purpose

The interaural time difference (ITD) is a primary horizontal-plane sound localization cue computed in the auditory brainstem. ITDs are accessible in the temporal fine structure of pure tones with a frequency of no higher than about 1400 Hz. Explaining how listeners' ITD sensitivity transitions from very best sensitivity near 700 Hz to impossible to detect within 1 octave currently lacks a fully compelling physiological explanation. Here, it was hypothesized that the rapid decline in ITD sensitivity is dictated not by a central neural limitation but by initial peripheral sound encoding, specifically, the low-frequency (apical) edge of the cochlear excitation pattern produced by a pure tone.

Methods

ITD sensitivity was measured in 16 normal-hearing listeners as a joint function of frequency (900-1500 Hz) and level (10-50 dB sensation level).

Results

Performance decreased with increasing frequency and decreasing sound level. The slope of performance decline was 90 dB/octave, consistent with the low-frequency slope of the cochlear excitation pattern.

Conclusion

Fine-structure ITD sensitivity near 1400 Hz may be conveyed primarily by "off-frequency" activation of neurons tuned to lower frequencies near 700 Hz. Physiologically, this could be realized by having neurons sensitive to fine-structure ITD up to only about 700 Hz. A more extreme model would have only a single narrow channel near 700 Hz that conveys fine-structure ITDs. Such a model is a major simplification and departure from the classic formulation of the binaural display, which consists of a matrix of neurons tuned to a wide range of relevant frequencies and ITDs.

Exploiting individual differences to assess the role of place and phase locking cues in auditory frequency discrimination at 2 kHz

The relative role of place and temporal mechanisms in auditory frequency discrimination was assessed for a centre frequency of 2 kHz. Four measures of frequency discrimination were obtained for 63 normal-hearing participants: detection of frequency modulation using modulation rates of 2 Hz (FM2) and 20 Hz (FM20); detection of a change in frequency across successive pure tones (difference limen for frequency, DLF); and detection of changes in the temporal fine structure of bandpass filtered complex tones centred at 2 kHz (TFS). Previous work has suggested that: FM2 depends on the use of both temporal and place cues; FM20 depends primarily on the use of place cues because the temporal mechanism cannot track rapid changes in frequency; DLF depends primarily on temporal cues; TFS depends exclusively on temporal cues. This led to the following predicted patterns of the correlations of scores across participants: DLF and TFS should be highly correlated; FM2 should be correlated with DLF and TFS; FM20 should not be correlated with DLF or TFS. The results were broadly consistent with these predictions and with the idea that frequency discrimination at 2 kHz depends partly or primarily on temporal cues except for frequency modulation detection at a high rate.

The role of carrier spectral composition in the perception of musical pitch

Temporal envelope fluctuations of natural sounds convey critical information to speech and music processing. In particular, musical pitch perception is assumed to be primarily underlined by temporal envelope encoding. While increasing evidence demonstrates the importance of carrier fine structure to complex pitch perception, how carrier spectral information affects musical pitch perception is less clear. Here, transposed tones designed to convey identical envelope information across different carriers were used to assess the effects of carrier spectral composition to pitch discrimination and musical-interval and melody identifications. Results showed that pitch discrimination thresholds became lower (better) with increasing carrier frequencies from 1k to 10k Hz, with performance comparable to that of pure sinusoids. Musical interval and melody defined by the periodicity of sine- or harmonic complex envelopes across carriers were identified with greater than 85% accuracy even on a 10k-Hz carrier. Moreover, enhanced interval and melody identification performance was observed with increasing carrier frequency up to 6k Hz. Findings suggest a perceptual enhancement of temporal envelope information with increasing carrier spectral region in musical pitch processing, at least for frequencies up to 6k Hz. For carriers in the extended high-frequency region (8-20k Hz), the use of temporal envelope information to music pitch processing may vary depending on task requirement. Collectively, these results implicate the fidelity of temporal envelope information to musical pitch perception is more pronounced than previously considered, with ecological implications.

Assessing mechanisms of frequency discrimination by comparison of different measures over a wide frequency range

It has been hypothesized that auditory detection of frequency modulation (FM) for low FM rates depends on the use of both temporal (phase locking) and place cues, depending on the carrier frequency, while detection of FM at high rates depends primarily on the use of place cues. To test this, FM detection for 2 and 20 Hz rates was measured over a wide frequency range, 1-10 kHz, including high frequencies for which temporal cues are assumed to be very weak. Performance was measured over the same frequency range for a task involving detection of changes in the temporal fine structure (TFS) of bandpass filtered complex tones, for which performance is assumed to depend primarily on the use of temporal cues. FM thresholds were better for the 2- than for the 20-Hz rate for center frequencies up to 4 kHz, while the reverse was true for higher center frequencies. For both FM rates, the thresholds, expressed as a proportion of the center frequency, were roughly constant for center frequencies from 6 to 10 Hz, consistent with the use of place cues. For the TFS task, thresholds worsened progressively with increasing frequency above 4 kHz, consistent with the weakening of temporal cues.

Sensitivity to Frequency Modulation is Limited Centrally

Modulations in both amplitude and frequency are prevalent in natural sounds and are critical in defining their properties. Humans are exquisitely sensitive to frequency modulation (FM) at the slow modulation rates and low carrier frequencies that are common in speech and music. This enhanced sensitivity to slow-rate and low-frequency FM has been widely believed to reflect precise, stimulus-driven phase locking to temporal fine structure in the auditory nerve. At faster modulation rates and/or higher carrier frequencies, FM is instead thought to be coded by coarser frequency-to-place mapping, where FM is converted to amplitude modulation (AM) via cochlear filtering. Here, we show that patterns of human FM perception that have classically been explained by limits in peripheral temporal coding are instead better accounted for by constraints in the central processing of fundamental frequency (F0) or pitch. We measured FM detection in male and female humans using harmonic complex tones with an F0 within the range of musical pitch but with resolved harmonic components that were all above the putative limits of temporal phase locking (>8 kHz). Listeners were more sensitive to slow than fast FM rates, even though all components were beyond the limits of phase locking. In contrast, AM sensitivity remained better at faster than slower rates, regardless of carrier frequency. These findings demonstrate that classic trends in human FM sensitivity, previously attributed to auditory nerve phase locking, may instead reflect the constraints of a unitary code that operates at a more central level of processing.SIGNIFICANCE STATEMENT Natural sounds involve dynamic frequency and amplitude fluctuations. Humans are particularly sensitive to frequency modulation (FM) at slow rates and low carrier frequencies, which are prevalent in speech and music. This sensitivity has been ascribed to encoding of stimulus temporal fine structure (TFS) via phase-locked auditory nerve activity. To test this long-standing theory, we measured FM sensitivity using complex tones with a low F0 but only high-frequency harmonics beyond the limits of phase locking. Dissociating the F0 from TFS showed that FM sensitivity is limited not by peripheral encoding of TFS but rather by central processing of F0, or pitch. The results suggest a unitary code for FM detection limited by more central constraints.

Questions and controversies surrounding the perception and neural coding of pitch

Pitch is a fundamental aspect of auditory perception that plays an important role in our ability to understand speech, appreciate music, and attend to one sound while ignoring others. The questions surrounding how pitch is represented in the auditory system, and how our percept relates to the underlying acoustic waveform, have been a topic of inquiry and debate for well over a century. New findings and technological innovations have led to challenges of some long-standing assumptions and have raised new questions. This article reviews some recent developments in the study of pitch coding and perception and focuses on the topic of how pitch information is extracted from peripheral representations based on frequency-to-place mapping (tonotopy), stimulus-driven auditory-nerve spike timing (phase locking), or a combination of both. Although a definitive resolution has proved elusive, the answers to these questions have potentially important implications for mitigating the effects of hearing loss via devices such as cochlear implants.

The Relative Contribution of Cochlear Synaptopathy and Reduced Inhibition to Age-Related Hearing Impairment for People With Normal Audiograms

Older people often show auditory temporal processing deficits and speech-in-noise intelligibility difficulties even when their audiogram is clinically normal. The causes of such problems remain unclear. Some studies have suggested that for people with normal audiograms, age-related hearing impairments may be due to a cognitive decline, while others have suggested that they may be caused by cochlear synaptopathy. Here, we explore an alternative hypothesis, namely that age-related hearing deficits are associated with decreased inhibition. For human adults (N = 30) selected to cover a reasonably wide age range (25-59 years), with normal audiograms and normal cognitive function, we measured speech reception thresholds in noise (SRTNs) for disyllabic words, gap detection thresholds (GDTs), and frequency modulation detection thresholds (FMDTs). We also measured the rate of growth (slope) of auditory brainstem response wave-I amplitude with increasing level as an indirect indicator of cochlear synaptopathy, and the interference inhibition score in the Stroop color and word test (SCWT) as a proxy for inhibition. As expected, performance in the auditory tasks worsened (SRTNs, GDTs, and FMDTs increased), and wave-I slope and SCWT inhibition scores decreased with ageing. Importantly, SRTNs, GDTs, and FMDTs were not related to wave-I slope but worsened with decreasing SCWT inhibition. Furthermore, after partialling out the effect of SCWT inhibition, age was no longer related to SRTNs or GDTs and became less strongly related to FMDTs. Altogether, results suggest that for people with normal audiograms, age-related deficits in auditory temporal processing and speech-in-noise intelligibility are mediated by decreased inhibition rather than cochlear synaptopathy.

Selecting auditory alerting stimuli for eagles on the basis of auditory evoked potentials

Development of wind energy facilities results in interactions between wildlife and wind turbines. Raptors, including bald and golden eagles, are among the species known to incur mortality from these interactions. Several alerting technologies have been proposed to mitigate this mortality by increasing eagle avoidance of wind energy facilities. However, there has been little attempt to match signals used as alerting stimuli with the sensory capabilities of target species like eagles. One potential approach to tuning signals is to use sensory physiology to determine what stimuli the target eagle species are sensitive to even in the presence of background noise, thereby allowing the development of a maximally stimulating signal. To this end, we measured auditory evoked potentials of bald and golden eagles to determine what types of sounds eagles can process well, especially in noisy conditions. We found that golden eagles are significantly worse than bald eagles at processing rapid frequency changes in sounds, but also that noise effects on hearing in both species are minimal in response to rapidly changing sounds. Our findings therefore suggest that sounds of intermediate complexity may be ideal both for targeting bald and golden eagle hearing and for ensuring high stimulation in noisy field conditions. These results suggest that the sensory physiology of target species is likely an important consideration when selecting auditory alerting sounds and may provide important insight into what sounds have a reasonable probability of success in field applications under variable conditions and background noise.

The hunt for hidden hearing loss in humans: From preclinical studies to effective interventions

Many individuals experience hearing problems that are hidden under a normal audiogram. This not only impacts on individual sufferers, but also on clinicians who can offer little in the way of support. Animal studies using invasive methodologies have developed solid evidence for a range of pathologies underlying this hidden hearing loss (HHL), including cochlear synaptopathy, auditory nerve demyelination, elevated central gain, and neural mal-adaptation. Despite progress in pre-clinical models, evidence supporting the existence of HHL in humans remains inconclusive, and clinicians lack any non-invasive biomarkers sensitive to HHL, as well as a standardized protocol to manage hearing problems in the absence of elevated hearing thresholds. Here, we review animal models of HHL as well as the ongoing research for tools with which to diagnose and manage hearing difficulties associated with HHL. We also discuss new research opportunities facilitated by recent methodological tools that may overcome a series of barriers that have hampered meaningful progress in diagnosing and treating of HHL.

The burst gap is a peripheral temporal code for pitch perception that is shared across audition and touch

When tactile afferents were manipulated to fire in periodic bursts of spikes, we discovered that the perceived pitch corresponded to the inter-burst interval (burst gap) in a spike train, rather than the spike rate or burst periodicity as previously thought. Given that tactile frequency mechanisms have many analogies to audition, and indications that temporal frequency channels are linked across the two modalities, we investigated whether there is burst gap temporal encoding in the auditory system. To link this putative neural code to perception, human subjects (n = 13, 6 females) assessed pitch elicited by trains of temporally-structured acoustic pulses in psychophysical experiments. Each pulse was designed to excite a fixed population of cochlear neurons, precluding place of excitation cues, and to elicit desired temporal spike trains in activated afferents. We tested periodicities up to 150 Hz using a variety of burst patterns and found striking deviations from periodicity-predicted pitch. Like the tactile system, the duration of the silent gap between successive bursts of neural activity best predicted perceived pitch, emphasising the role of peripheral temporal coding in shaping pitch. This suggests that temporal patterning of stimulus pulses in cochlear implant users might improve pitch perception.

Human discrimination and modeling of high-frequency complex tones shed light on the neural codes for pitch

Accurate pitch perception of harmonic complex tones is widely believed to rely on temporal fine structure information conveyed by the precise phase-locked responses of auditory-nerve fibers. However, accurate pitch perception remains possible even when spectrally resolved harmonics are presented at frequencies beyond the putative limits of neural phase locking, and it is unclear whether residual temporal information, or a coarser rate-place code, underlies this ability. We addressed this question by measuring human pitch discrimination at low and high frequencies for harmonic complex tones, presented either in isolation or in the presence of concurrent complex-tone maskers. We found that concurrent complex-tone maskers impaired performance at both low and high frequencies, although the impairment introduced by adding maskers at high frequencies relative to low frequencies differed between the tested masker types. We then combined simulated auditory-nerve responses to our stimuli with ideal-observer analysis to quantify the extent to which performance was limited by peripheral factors. We found that the worsening of both frequency discrimination and F0 discrimination at high frequencies could be well accounted for (in relative terms) by optimal decoding of all available information at the level of the auditory nerve. A Python package is provided to reproduce these results, and to simulate responses to acoustic stimuli from the three previously published models of the human auditory nerve used in our analyses.

Individualized Assays of Temporal Coding in the Ascending Human Auditory System

Neural phase-locking to temporal fluctuations is a fundamental and unique mechanism by which acoustic information is encoded by the auditory system. The perceptual role of this metabolically expensive mechanism, the neural phase-locking to temporal fine structure (TFS) in particular, is debated. Although hypothesized, it is unclear whether auditory perceptual deficits in certain clinical populations are attributable to deficits in TFS coding. Efforts to uncover the role of TFS have been impeded by the fact that there are no established assays for quantifying the fidelity of TFS coding at the individual level. While many candidates have been proposed, for an assay to be useful, it should not only intrinsically depend on TFS coding, but should also have the property that individual differences in the assay reflect TFS coding per se over and beyond other sources of variance. Here, we evaluate a range of behavioral and electroencephalogram (EEG)-based measures as candidate individualized measures of TFS sensitivity. Our comparisons of behavioral and EEG-based metrics suggest that extraneous variables dominate both behavioral scores and EEG amplitude metrics, rendering them ineffective. After adjusting behavioral scores using lapse rates, and extracting latency or percent-growth metrics from EEG, interaural timing sensitivity measures exhibit robust behavior-EEG correlations. Together with the fact that unambiguous theoretical links can be made relating binaural measures and phase-locking to TFS, our results suggest that these "adjusted" binaural assays may be well suited for quantifying individual TFS processing.

Double-pass consistency for amplitude- and frequency-modulation detection in normal-hearing listeners

Amplitude modulation (AM) and frequency modulation (FM) provide crucial auditory information. If FM is encoded as AM, it should be possible to give a unified account of AM and FM perception both in terms of response consistency and performance. These two aspects of behavior were estimated for normal-hearing participants using a constant-stimuli, forced-choice detection task repeated twice with the same stimuli (double pass). Sinusoidal AM or FM with rates of 2 or 20 Hz were applied to a 500-Hz pure-tone carrier and presented at detection threshold. All stimuli were masked by a modulation noise. Percent agreement of responses across passes and percent-correct detection for the two passes were used to estimate consistency and performance, respectively. These data were simulated using a model implementing peripheral processes, a central modulation filterbank, an additive internal noise, and a template-matching device. Different levels of internal noise were required to reproduce AM and FM data, but a single level could account for the 2- and 20-Hz AM data. As for FM, two levels of internal noise were needed to account for detection at slow and fast rates. Finally, the level of internal noise yielding best predictions increased with the level of the modulation-noise masker. Overall, these results suggest that different sources of internal variability are involved for AM and FM detection at low audio frequencies.

Temporal integration of monaural and dichotic frequency modulation

Frequency modulation (FM) detection at low modulation frequencies is commonly used as an index of temporal fine-structure processing. The present study evaluated the rate of improvement in monaural and dichotic FM across a range of test parameters. In experiment I, dichotic and monaural FM detection was measured as a function of duration and modulator starting phase. Dichotic FM thresholds were lower than monaural FM thresholds and the modulator starting phase had no effect on detection. Experiment II measured monaural FM detection for signals that differed in modulation rate and duration such that the improvement with duration in seconds (carrier) or cycles (modulator) was compared. Monaural FM detection improved monotonically with the number of modulation cycles, suggesting that the modulator is extracted prior to detection. Experiment III measured dichotic FM detection for shorter signal durations to test the hypothesis that dichotic FM relies primarily on the signal onset. The rate of improvement decreased as duration increased, which is consistent with the use of primarily onset cues for the detection of dichotic FM. These results establish that improvement with duration occurs as a function of the modulation cycles at a rate consistent with the independent-samples model for monaural FM, but later cycles contribute less to detection in dichotic FM.

Harmonic Cancellation-A Fundamental of Auditory Scene Analysis

This paper reviews the hypothesis of harmonic cancellation according to which an interfering sound is suppressed or canceled on the basis of its harmonicity (or periodicity in the time domain) for the purpose of Auditory Scene Analysis. It defines the concept, discusses theoretical arguments in its favor, and reviews experimental results that support it, or not. If correct, the hypothesis may draw on time-domain processing of temporally accurate neural representations within the brainstem, as required also by the classic equalization-cancellation model of binaural unmasking. The hypothesis predicts that a target sound corrupted by interference will be easier to hear if the interference is harmonic than inharmonic, all else being equal. This prediction is borne out in a number of behavioral studies, but not all. The paper reviews those results, with the aim to understand the inconsistencies and come up with a reliable conclusion for, or against, the hypothesis of harmonic cancellation within the auditory system.

Effects of noise precursors on the detection of amplitude and frequency modulation for tones in noise

Recent studies on amplitude modulation (AM) detection for tones in noise reported that AM-detection thresholds improve when the AM stimulus is preceded by a noise precursor. The physiological mechanisms underlying this AM unmasking are unknown. One possibility is that adaptation to the level of the noise precursor facilitates AM encoding by causing a shift in neural rate-level functions to optimize level encoding around the precursor level. The aims of this study were to investigate whether such a dynamic-range adaptation is a plausible mechanism for the AM unmasking and whether frequency modulation (FM), thought to be encoded via AM, also exhibits the unmasking effect. Detection thresholds for AM and FM of tones in noise were measured with and without a fixed-level precursor. Listeners showing the unmasking effect were then tested with the precursor level roved over a wide range to modulate the effect of adaptation to the precursor level on the detection of the subsequent AM. It was found that FM detection benefits from a precursor and the magnitude of FM unmasking correlates with that of AM unmasking. Moreover, consistent with dynamic-range adaptation, the unmasking magnitude weakens as the level difference between the precursor and simultaneous masker of the tone increases.