Frequency-Following Responses to Complex Tones at Different Frequencies Reflect Different Source Configurations

Different binaural processing of the envelope component and the temporal fine structure component of a narrowband noise in rat inferior colliculus

Complex broadband sounds are decomposed by peripheral auditory filters into a series of relatively narrowband signals, each with a slowly varying envelope (ENV) and a rapidly fluctuating temporal fine structure (TFS). ENV and TFS information at the bilateral ears contribute differentially to auditory perception. However, whether the difference could attribute to mechanisms of binaural integration remains an open question. As a potential neural correlate, subsets of neurons in the central nucleus of the inferior colliculus (ICC) are known to integrate binaural information with binaural inhibition or binaural summation. Therefore, we recorded the frequency-following responses (FFRs) to the ENV and TFS components of narrowband noises in the ICC of anesthetized rats and examined changes in FFR amplitude and stimulus-response coherence under various sound-delivery settings. We showed that binaural FFR_ENV was predominantly elicited by contralateral inputs and inhibited by ipsilateral inputs, exhibiting a "binaural-inhibition" like property. On the other hand, binaural FFR_TFS received a balanced contribution from both sides, echoing the "binaural-summation" mechanism. What is more, binaural FFR_ENV was significantly correlated with contralateral-evoked but not ipsilateral-evoked FFR_ENV, while binaural FFR_TFS correlated with both contralateral- and ipsilateral-evoked FFR_TFS. Overall, these results suggest distinct binaural processing of ENV and TFS information at the midbrain level.

Context-dependent Plasticity and Strength of Subcortical Encoding of Musical Sounds Independently Underlie Pitch Discrimination for Music Melodies

Subcortical auditory nuclei contribute to pitch perception, but how subcortical sound encoding is related to pitch processing for music perception remains unclear. Conventionally, enhanced subcortical sound encoding is considered underlying superior pitch discrimination. However, associations between superior auditory perception and the context-dependent plasticity of subcortical sound encoding are also documented. Here, we explored the subcortical neural correlates to music pitch perception by analyzing frequency-following responses (FFRs) to musical sounds presented in a predictable context and a random context. We found that the FFR inter-trial phase-locking (ITPL) was negatively correlated with behavioral performances of discrimination of pitches in music melodies. It was also negatively correlated with the plasticity indices measuring the variability of FFRs to physically identical sounds between the two contexts. The plasticity indices were consistently positively correlated with pitch discrimination performances, suggesting the subcortical context-dependent plasticity underlying music pitch perception. Moreover, the raw FFR spectral strength was not significantly correlated with pitch discrimination performances. However, it was positively correlated with behavioral performances when the FFR ITPL was controlled by partial correlations, suggesting that the strength of subcortical sound encoding underlies music pitch perception. When the spectral strength was controlled by partial correlations, the negative ITPL-behavioral correlations were maintained. Furthermore, the FFR ITPL, the plasticity indices, and the FFR spectral strength were more correlated with pitch than with rhythm discrimination performances. These findings suggest that the context-dependent plasticity and the strength of subcortical encoding of musical sounds are independently and perhaps specifically associated with pitch perception for music melodies.

Neural encoding of voice pitch and formant structure at birth as revealed by frequency-following responses

Detailed neural encoding of voice pitch and formant structure plays a crucial role in speech perception, and is of key importance for an appropriate acquisition of the phonetic repertoire in infants since birth. However, the extent to what newborns are capable of extracting pitch and formant structure information from the temporal envelope and the temporal fine structure of speech sounds, respectively, remains unclear. Here, we recorded the frequency-following response (FFR) elicited by a novel two-vowel, rising-pitch-ending stimulus to simultaneously characterize voice pitch and formant structure encoding accuracy in a sample of neonates and adults. Data revealed that newborns tracked changes in voice pitch reliably and no differently than adults, but exhibited weaker signatures of formant structure encoding, particularly at higher formant frequency ranges. Thus, our results indicate a well-developed encoding of voice pitch at birth, while formant structure representation is maturing in a frequency-dependent manner. Furthermore, we demonstrate the feasibility to assess voice pitch and formant structure encoding within clinical evaluation times in a hospital setting, and suggest the possibility to use this novel stimulus as a tool for longitudinal developmental studies of the auditory system.

Subcortical rather than cortical sources of the frequency-following response (FFR) relate to speech-in-noise perception in normal-hearing listeners

Scalp-recorded frequency-following responses (FFRs) reflect a mixture of phase-locked activity across the auditory pathway. FFRs have been widely used as a neural barometer of complex listening skills, especially speech-in noise (SIN) perception. Applying individually optimized source reconstruction to speech-FFRs recorded via EEG (FFR_EEG), we assessed the relative contributions of subcortical [auditory nerve (AN), brainstem/midbrain (BS)] and cortical [bilateral primary auditory cortex, PAC] source generators with the aim of identifying which source(s) drive the brain-behavior relation between FFRs and SIN listening skills. We found FFR strength declined precipitously from AN to PAC, consistent with diminishing phase-locking along the ascending auditory neuroaxis. FFRs to the speech fundamental (F0) were robust to noise across sources, but were largest in subcortical sources (BS > AN > PAC). PAC FFRs were only weakly observed above the noise floor and only at the low pitch of speech (F0≈100 Hz). Brain-behavior regressions revealed (i) AN and BS FFRs were sufficient to describe listeners' QuickSIN scores and (ii) contrary to neuromagnetic (MEG) FFRs, neither left nor right PAC FFR_EEG related to SIN performance. Our findings suggest subcortical sources not only dominate the electrical FFR but also the link between speech-FFRs and SIN processing in normal-hearing adults as observed in previous EEG studies.

Subcortical neural generators of the envelope-following response in sleeping children: A transfer function analysis

Multiple auditory structures, from cochlea to cortex, phase-lock to the envelope of complex stimuli. The relative contributions of these structures to the human surface-recorded envelope-following response (EFR) are still uncertain. Identification of the active contributor(s) is complicated by the fact that even the simplest two-tone (f₁&f₂) stimulus, targeting its (f₂-f₁) envelope, evokes additional linear (f₁&f₂) and non-linear (2f₁-f₂) phase-locked components as well as a transient auditory brainstem response (ABR). Here, we took advantage of the generalized primary tone phase variation method to isolate each predictable component in the time domain, allowing direct measurements of onset latency, duration and phase discontinuity values from which the involved generators were inferred. Targeting several envelope frequencies (0.22-1 kHz), we derived the EFR transfer functions along a vertical vertex-to-neck and a horizontal earlobe-to-earlobe recording channels, yielding respectively EFR-V and EFR-H waveforms. Subjects (N= 30) were sleeping children with normal electrophysiological thresholds and normal oto-acoustic emissions. Both EFR-H and EFR-V phase-locking values (PLV) transfer functions had a low-pass profile, EFR-V showing a lower cut-off frequency than EFR-H. We also computed the frequency-latency relationships of both EFRs onset latencies. EFR-H data fitted a power-law function incorporating a frequency-dependent traveling wave delay and a fixed one amounting to 1.2 ms. The fitted function nicely fell within five published estimations of the latency-frequency function of the ABR wave-I, thus pointing to a cochlear nerve origin. The absence of phase discontinuity and overall response durations that were equal to that of the stimulus indicated no contribution from a later generator. The recording of an entirely similar EFR-H response in a patient who had severe brainstem encephalitis with a normal, isolated, ABR wave-I but complete absence of later waves, further substantiated a cochlear nerve origin. Modeling of the EFR-V latency-frequency functions indicated a fixed transport time of 2 ms with respect to EFR-H onset, suggesting a cochlear nucleus (CN) origin, here also, without indication for multiple generators. Other features of the EFR-V response pointing to the CN were, at least for the EFR frequency below the cut-off values of the transfer functions, higher PLVs coupled with increased harmonic distortion. Such a behavior has been described in the so-called highly-synchronized neurons of the ventral cochlear nucleus (VCN). The present study compellingly demonstrated the advantage of isolating the EFR in the temporal domain so as to extract detailed spectro-temporal parameters that, combined with orthogonal recording channels, shed new light on the involved neural generators.

Speech frequency-following response in human auditory cortex is more than a simple tracking

The human auditory cortex is recently found to contribute to the frequency following response (FFR) and the cortical component has been shown to be more relevant to speech perception. However, it is not clear how cortical FFR may contribute to the processing of speech fundamental frequency (F0) and the dynamic pitch. Using intracranial EEG recordings, we observed a significant FFR at the fundamental frequency (F0) for both speech and speech-like harmonic complex stimuli in the human auditory cortex, even in the missing fundamental condition. Both the spectral amplitude and phase coherence of the cortical FFR showed a significant harmonic preference, and attenuated from the primary auditory cortex to the surrounding associative auditory cortex. The phase coherence of the speech FFR was found significantly higher than that of the harmonic complex stimuli, especially in the left hemisphere, showing a high timing fidelity of the cortical FFR in tracking dynamic F0 in speech. Spectrally, the frequency band of the cortical FFR was largely overlapped with the range of the human vocal pitch. Taken together, our study parsed the intrinsic properties of the cortical FFR and reveals a preference for speech-like sounds, supporting its potential role in processing speech intonation and lexical tones.

What Can Computational Models Learn From Human Selective Attention? A Review From an Audiovisual Unimodal and Crossmodal Perspective

Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments and computational modeling. Although an increasing number of findings on crossmodal selective attention have shed light on humans' behavioral patterns and neural underpinnings, a much better understanding is still necessary to yield the same benefit for intelligent computational agents. This article reviews studies of selective attention in unimodal visual and auditory and crossmodal audiovisual setups from the multidisciplinary perspectives of psychology and cognitive neuroscience, and evaluates different ways to simulate analogous mechanisms in computational models and robotics. We discuss the gaps between these fields in this interdisciplinary review and provide insights about how to use psychological findings and theories in artificial intelligence from different perspectives.

Human Frequency Following Response Correlates of Spatial Release From Masking

Purpose Speech recognition in complex listening environments is enhanced by the extent of spatial separation between the speech source and background competing sources, an effect known as spatial release from masking (SRM). The aim of this study was to investigate whether the phase-locked neural activity in the central auditory pathways, reflected in the frequency following response (FFR), exhibits SRM. Method Eighteen normal-hearing adults (8 men and 10 women, ranging in age from 20 to 42 years) with no known neurological disorders participated in this study. FFRs were recorded from the participants in response to a target vowel /u/ presented with spatially colocated and separated competing talkers at 3 ranges of signal-to-noise ratios (SNRs), with median SNRs of -5.4, 0.5, and 6.8 dB and for different attentional conditions (attention and no attention). Results Amplitude of the FFR at the fundamental frequency was significantly larger in the spatially separated condition as compared to the colocated condition for only the lowest (< -2.4 dB SNR) of the 3 SNR ranges tested. A significant effect of attention was found when subjects were actively focusing on the target stimuli. No significant interaction effects were found between spatial separation and attention. Conclusions The enhanced representation of the target stimulus in the separated condition suggests that the temporal pattern of phase-locked brainstem neural activity generating the FFR may contain information relevant to the binaural processes underlying SRM but only in challenging listening environments. Attention may modulate FFR fundamental frequency amplitude but does not seem to modulate spatial processing at the level of generating the FFR. Supplemental Material https://doi.org/10.23641/asha.9992597.

Evolving perspectives on the sources of the frequency-following response

The auditory frequency-following response (FFR) is a non-invasive index of the fidelity of sound encoding in the brain, and is used to study the integrity, plasticity, and behavioral relevance of the neural encoding of sound. In this Perspective, we review recent evidence suggesting that, in humans, the FFR arises from multiple cortical and subcortical sources, not just subcortically as previously believed, and we illustrate how the FFR to complex sounds can enhance the wider field of auditory neuroscience. Far from being of use only to study basic auditory processes, the FFR is an uncommonly multifaceted response yielding a wealth of information, with much yet to be tapped.

Auditory cortical generators of the Frequency Following Response are modulated by intermodal attention

The efferent auditory system suggests that brainstem auditory regions could also be sensitive to top-down processes. In electrophysiology, the Frequency Following Response (FFR) to speech stimuli has been used extensively to study brainstem areas. Despite seemingly straight-forward in addressing the issue of attentional modulations of brainstem regions by means of the FFR, the existing results are inconsistent. Moreover, the notion that the FFR exclusively represents subcortical generators has been challenged. We aimed to gain a more differentiated perspective on how the generators of the FFR are modulated by either attending to the visual or auditory input while neural activity was recorded using magnetoencephalography (MEG). In a first step our results confirm the strong contribution of also cortical regions to the FFR. Interestingly, of all regions exhibiting a measurable FFR response, only the right primary auditory cortex was significantly affected by intermodal attention. By showing a clear cortical contribution to the attentional FFR effect, our work significantly extends previous reports that focus on surface level recordings only. It underlines the importance of making a greater effort to disentangle the different contributing sources of the FFR and serves as a clear precaution of simplistically interpreting the FFR as brainstem response.