An integrative model of subcortical auditory plasticity

Auditory Pitch Perception in Autism Spectrum Disorder: A Systematic Review and Meta-Analysis

PURPOSE

Pitch plays an important role in auditory perception of music and language. This study provides a systematic review with meta-analysis to investigate whether individuals with autism spectrum disorder (ASD) have enhanced pitch processing ability and to identify the potential factors associated with processing differences between ASD and neurotypicals.

METHOD

We conducted a systematic search through six major electronic databases focusing on the studies that used nonspeech stimuli to provide a qualitative and quantitative assessment across existing studies on pitch perception in autism. We identified potential participant- and methodology-related moderators and conducted metaregression analyses using mixed-effects models.

RESULTS

On the basis of 22 studies with a total of 464 participants with ASD, we obtained a small-to-medium positive effect size (g = 0.26) in support of enhanced pitch perception in ASD. Moreover, the mean age and nonverbal IQ of participants were found to significantly moderate the between-studies heterogeneity.

CONCLUSIONS

Our study provides the first meta-analysis on auditory pitch perception in ASD and demonstrates the existence of different developmental trajectories between autistic individuals and neurotypicals. In addition to age, nonverbal ability is found to be a significant contributor to the lower level/local processing bias in ASD. We highlight the need for further investigation of pitch perception in ASD under challenging listening conditions. Future neurophysiological and brain imaging studies with a longitudinal design are also needed to better understand the underlying neural mechanisms of atypical pitch processing in ASD and to help guide auditory-based interventions for improving language and social functioning.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21614271.

Neonatal Frequency-Following Responses: A Methodological Framework for Clinical Applications

The frequency-following response (FFR) to periodic complex sounds is a noninvasive scalp-recorded auditory evoked potential that reflects synchronous phase-locked neural activity to the spectrotemporal components of the acoustic signal along the ascending auditory hierarchy. The FFR has gained recent interest in the fields of audiology and auditory cognitive neuroscience, as it has great potential to answer both basic and applied questions about processes involved in sound encoding, language development, and communication. Specifically, it has become a promising tool in neonates, as its study may allow both early identification of future language disorders and the opportunity to leverage brain plasticity during the first 2 years of life, as well as enable early interventions to prevent and/or ameliorate sound and language encoding disorders. Throughout the present review, we summarize the state of the art of the neonatal FFR and, based on our own extensive experience, present methodological approaches to record it in a clinical environment. Overall, the present review is the first one that comprehensively focuses on the neonatal FFRs applications, thus supporting the feasibility to record the FFR during the first days of life and the predictive potential of the neonatal FFR on detecting short- and long-term language abilities and disruptions.

Pattern of frustration formation in the functional brain network

The brain is a frustrated system that contains conflictual link arrangements named frustration. The frustration as a source of disorder prevents the system from settling into low-energy states and provides flexibility for brain network organization. In this research, we tried to identify the pattern of frustration formation in the brain at the levels of region, connection, canonical network, and hemisphere. We found that frustration formation has no uniform pattern. Some subcortical elements have an active role in frustration formation, despite low contributions from many cortical elements. Frustrating connections are mostly between-network connections, and triadic frustrations are mainly formed between three regions from three distinct canonical networks. We did not find any significant differences between brain hemispheres or any robust differences between the frustration formation patterns of various life-span stages. Our results may be interesting for those who study the organization of brain links and promising for those who want to manipulate brain networks.

A Linear Superposition Model of Envelope and Frequency Following Responses May Help Identify Generators Based on Latency

Envelope and frequency-following responses (FFR_ENV and FFR_TFS) are scalp-recorded electrophysiological potentials that closely follow the periodicity of complex sounds such as speech. These signals have been established as important biomarkers in speech and learning disorders. However, despite important advances, it has remained challenging to map altered FFR_ENV and FFR_TFS to altered processing in specific brain regions. Here we explore the utility of a deconvolution approach based on the assumption that FFR_ENV and FFR_TFS reflect the linear superposition of responses that are triggered by the glottal pulse in each cycle of the fundamental frequency (F0 responses). We tested the deconvolution method by applying it to FFR_ENV and FFR_TFS of rhesus monkeys to human speech and click trains with time-varying pitch patterns. Our analyses show that F0_ENV responses could be measured with high signal-to-noise ratio and featured several spectro-temporally and topographically distinct components that likely reflect the activation of brainstem (<5 ms; 200-1000 Hz), midbrain (5-15 ms; 100-250 Hz), and cortex (15-35 ms; ~90 Hz). In contrast, F0_TFS responses contained only one spectro-temporal component that likely reflected activity in the midbrain. In summary, our results support the notion that the latency of F0 components map meaningfully onto successive processing stages. This opens the possibility that pathologically altered FFR_ENV or FFR_TFS may be linked to altered F0_ENV or F0_TFS and from there to specific processing stages and ultimately spatially targeted interventions.

Auditory Behavior in Adult-Blinded Mice

Cross-modal plasticity occurs when the function of remaining senses is enhanced following deprivation or loss of a sensory modality. Auditory neural responses are enhanced in the auditory cortex, including increased sensitivity and frequency selectivity, following short-term visual deprivation in adult mice (Petrus et al. Neuron 81:664-673, 2014). Whether or not these visual deprivation-induced neural changes translate into improved auditory perception and performance remains unclear. As an initial investigation of the effects of adult visual deprivation on auditory behaviors, CBA/CaJ mice underwent binocular enucleation at 3-4 weeks old and were tested on a battery of learned behavioral tasks, acoustic startle response (ASR), and prepulse inhibition (PPI) tests beginning at least 2 weeks after the enucleation procedure. Auditory brain stem responses (ABRs) were also measured to screen for potential effects of visual deprivation on non-behavioral hearing function. Control and enucleated mice showed similar tone detection sensitivity and frequency discrimination in a conditioned lick suppression test. Both groups showed normal reactivity to sound as measured by ASR in a quiet background. However, when startle-eliciting stimuli were presented in noise, enucleated mice showed decreased ASR amplitude relative to controls. Control and enucleated mice displayed no significant differences in ASR habituation, PPI tests, or ABR thresholds, or wave morphology. Our findings suggest that while adult-onset visual deprivation induces cross-modal plasticity at the synaptic and circuit levels, it does not substantially influence simple auditory behavioral performance.

Altered processing of communication signals in the subcortical auditory sensory pathway in autism

Autism spectrum disorder (ASD) is characterised by social communication difficulties. These difficulties have been mainly explained by cognitive, motivational, and emotional alterations in ASD. The communication difficulties could, however, also be associated with altered sensory processing of communication signals. Here, we assessed the functional integrity of auditory sensory pathway nuclei in ASD in three independent functional magnetic resonance imaging experiments. We focused on two aspects of auditory communication that are impaired in ASD: voice identity perception, and recognising speech‐in‐noise. We found reduced processing in adults with ASD as compared to typically developed control groups (pairwise matched on sex, age, and full‐scale IQ) in the central midbrain structure of the auditory pathway (inferior colliculus [IC]). The right IC responded less in the ASD as compared to the control group for voice identity, in contrast to speech recognition. The right IC also responded less in the ASD as compared to the control group when passively listening to vocal in contrast to non‐vocal sounds. Within the control group, the left and right IC responded more when recognising speech‐in‐noise as compared to when recognising speech without additional noise. In the ASD group, this was only the case in the left, but not the right IC. The results show that communication signal processing in ASD is associated with reduced subcortical sensory functioning in the midbrain. The results highlight the importance of considering sensory processing alterations in explaining communication difficulties, which are at the core of ASD.

Modulation of the Primary Auditory Thalamus When Recognizing Speech with Background Noise

Recognizing speech in background noise is a strenuous daily activity, yet most humans can master it. An explanation of how the human brain deals with such sensory uncertainty during speech recognition is to-date missing. Previous work has shown that recognition of speech without background noise involves modulation of the auditory thalamus (medial geniculate body; MGB): there are higher responses in left MGB for speech recognition tasks that require tracking of fast-varying stimulus properties in contrast to relatively constant stimulus properties (e.g., speaker identity tasks) despite the same stimulus input. Here, we tested the hypotheses that (1) this task-dependent modulation for speech recognition increases in parallel with the sensory uncertainty in the speech signal, i.e., the amount of background noise; and that (2) this increase is present in the ventral MGB, which corresponds to the primary sensory part of the auditory thalamus. In accordance with our hypothesis, we show, by using ultra-high-resolution functional magnetic resonance imaging (fMRI) in male and female human participants, that the task-dependent modulation of the left ventral MGB (vMGB) for speech is particularly strong when recognizing speech in noisy listening conditions in contrast to situations where the speech signal is clear. The results imply that speech in noise recognition is supported by modifications at the level of the subcortical sensory pathway providing driving input to the auditory cortex.SIGNIFICANCE STATEMENT Speech recognition in noisy environments is a challenging everyday task. One reason why humans can master this task is the recruitment of additional cognitive resources as reflected in recruitment of non-language cerebral cortex areas. Here, we show that also modulation in the primary sensory pathway is specifically involved in speech in noise recognition. We found that the left primary sensory thalamus (ventral medial geniculate body; vMGB) is more involved when recognizing speech signals as opposed to a control task (speaker identity recognition) when heard in background noise versus when the noise was absent. This finding implies that the brain optimizes sensory processing in subcortical sensory pathway structures in a task-specific manner to deal with speech recognition in noisy environments.

Context-dependent Plasticity and Strength of Subcortical Encoding of Musical Sounds Independently Underlie Pitch Discrimination for Music Melodies

Subcortical auditory nuclei contribute to pitch perception, but how subcortical sound encoding is related to pitch processing for music perception remains unclear. Conventionally, enhanced subcortical sound encoding is considered underlying superior pitch discrimination. However, associations between superior auditory perception and the context-dependent plasticity of subcortical sound encoding are also documented. Here, we explored the subcortical neural correlates to music pitch perception by analyzing frequency-following responses (FFRs) to musical sounds presented in a predictable context and a random context. We found that the FFR inter-trial phase-locking (ITPL) was negatively correlated with behavioral performances of discrimination of pitches in music melodies. It was also negatively correlated with the plasticity indices measuring the variability of FFRs to physically identical sounds between the two contexts. The plasticity indices were consistently positively correlated with pitch discrimination performances, suggesting the subcortical context-dependent plasticity underlying music pitch perception. Moreover, the raw FFR spectral strength was not significantly correlated with pitch discrimination performances. However, it was positively correlated with behavioral performances when the FFR ITPL was controlled by partial correlations, suggesting that the strength of subcortical sound encoding underlies music pitch perception. When the spectral strength was controlled by partial correlations, the negative ITPL-behavioral correlations were maintained. Furthermore, the FFR ITPL, the plasticity indices, and the FFR spectral strength were more correlated with pitch than with rhythm discrimination performances. These findings suggest that the context-dependent plasticity and the strength of subcortical encoding of musical sounds are independently and perhaps specifically associated with pitch perception for music melodies.

Sensory cortical and subcortical auditory neurophysiological changes predict cue-specific extinction behavior enabled by the pharmacological inhibition of an epigenetic regulator during memory formation

Extinction learning and memory have been broadly investigated at both behavioral and neural levels, but sensory system contributions to extinction processes have been less explored. Using a sound-reward extinction paradigm in male rats, we reveal both cortical and subcortical forms of plasticity associated with the cue-specificity of behavioral extinction memory. In the auditory cortex, frequency tuning narrowed by up to two-thirds of an octave around the remembered extinguished sound cue. Subcortical signals revealed in the auditory brainstem response (ABR) in the same animals developed smaller amplitudes of some (but not all) ABR peaks evoked by the extinguished sound frequency. Interestingly, treatment with an inhibitor of histone deacetylase 3 (HDAC3-i) facilitated both auditory cortical tuning bandwidth changes and changes in subcortical peak amplitude evoked only by the extinguished sound frequency. These neurophysiological changes were correlated to each other, and to the highly precise extinction behavior enabled by HDAC3-i (compared to vehicle controls). Thus, we show for the first time that HDAC3 regulates the specificity of sensory features consolidated in extinction memory. Further, the sensory cortical changes in tuning bandwidth recapitulate known effects of blocking HDAC3 to enhance cue specificity in other behavioral tasks. Therefore, the findings demonstrate how some forms of sensory neuroplasticity may encode specific sensory features of learning experiences in order to enable cue-specific behaviors.

Neural generators of the frequency-following response elicited to stimuli of low and high frequency: A magnetoencephalographic (MEG) study

The frequency-following response (FFR) to periodic complex sounds has gained recent interest in auditory cognitive neuroscience as it captures with great fidelity the tracking accuracy of the periodic sound features in the ascending auditory system. Seminal studies suggested the FFR as a correlate of subcortical sound encoding, yet recent studies aiming to locate its sources challenged this assumption, demonstrating that FFR receives some contribution from the auditory cortex. Based on frequency-specific phase-locking capabilities along the auditory hierarchy, we hypothesized that FFRs to higher frequencies would receive less cortical contribution than those to lower frequencies, hence supporting a major subcortical involvement for these high frequency sounds. Here, we used a magnetoencephalographic (MEG) approach to trace the neural sources of the FFR elicited in healthy adults (N = 19) to low (89 Hz) and high (333 Hz) frequency sounds. FFRs elicited to the high and low frequency sounds were clearly observable on MEG and comparable to those obtained in simultaneous electroencephalographic recordings. Distributed source modeling analyses revealed midbrain, thalamic, and cortical contributions to FFR, arranged in frequency-specific configurations. Our results showed that the main contribution to the high-frequency sound FFR originated in the inferior colliculus and the medial geniculate body of the thalamus, with no significant cortical contribution. In contrast, the low-frequency sound FFR had a major contribution located in the auditory cortices, and also received contributions originating in the midbrain and thalamic structures. These findings support the multiple generator hypothesis of the FFR and are relevant for our understanding of the neural encoding of sounds along the auditory hierarchy, suggesting a hierarchical organization of periodicity encoding.

Acoustically Controlled Binaural Auditory Training with Vocal Duets: Assessment and Effectiveness

OBJECTIVES

We aimed to evaluate the effectiveness of a binaural auditory training program with vocal duets by comparing skills through outcomes from behavioral and electrophysiological assessment instruments at three moments: before the intervention, moment one (M1); immediately after training, moment two (M2); and 3 months after, moment three (M3).

METHODS

This interventional, longitudinal, prospective, and uncontrolled study was approved by our Research Ethics Committee. Binaural auditory training with vocal duets (ATVD) was applied in 10 adults with normal audiometric thresholds and auditory processing disorders. ATVD used four different vocals of a public domain song sung in a cappella as stimuli. Participants were asked to register any perceived difference in frequency for each syllable of the song during 30-minute sessions twice a week. The number of sessions required ranged from 12 (6 hours) to 20 (10 hours).

RESULTS

Regarding behavioral tests, the dichotic consonant-vowel test showed significant evidence of an improved advantage in the left ear (LE) in the non-forced condition and a significant reduction in the number of errors at M2 and M3 in the forced left condition. The speech-in-noise test and frequency pattern test showed a significant reduction in impaired results at M2 and M3. Electrophysiological results showed a significant increase in the LE amplitude in the P3 long-latency auditory evoked potentials test, as well as a decrease in the auditory brainstem response test (III-V and I-V inter-peak latencies in the right ear and wave I and I-III inter-peak latencies in LE).

CONCLUSION

The effectiveness of ATVD was evidenced, and the results were maintained after 3 months.

Lifelong Tone Language Experience does not Eliminate Deficits in Neural Encoding of Pitch in Autism Spectrum Disorder

Atypical pitch processing is a feature of Autism Spectrum Disorder (ASD), which affects non-tone language speakers' communication. Lifelong auditory experience has been demonstrated to modify genetically-predisposed risks for pitch processing. We examined individuals with ASD to test the hypothesis that lifelong auditory experience in tone language may eliminate impaired pitch processing in ASD. We examined children's and adults' Frequency-following Response (FFR), a neurophysiological component indexing early neural sensory encoding of pitch. Univariate and machine-learning-based analytics suggest less robust pitch encoding and diminished pitch distinctions in the FFR from individuals with ASD. Contrary to our hypothesis, results point to a linguistic pitch encoding impairment associated with ASD that may not be eliminated even by lifelong sensory experience.

Meter enhances the subcortical processing of speech sounds at a strong beat

The temporal structure of sound such as in music and speech increases the efficiency of auditory processing by providing listeners with a predictable context. Musical meter is a good example of a sound structure that is temporally organized in a hierarchical manner, with recent studies showing that meter optimizes neural processing, particularly for sounds located at a higher metrical position or strong beat. Whereas enhanced cortical auditory processing at times of high metric strength has been studied, there is to date no direct evidence showing metrical modulation of subcortical processing. In this work, we examined the effect of meter on the subcortical encoding of sounds by measuring human auditory frequency-following responses to speech presented at four different metrical positions. Results show that neural encoding of the fundamental frequency of the vowel was enhanced at the strong beat, and also that the neural consistency of the vowel was the highest at the strong beat. When comparing musicians to non-musicians, musicians were found, at the strong beat, to selectively enhance the behaviorally relevant component of the speech sound, namely the formant frequency of the transient part. Our findings indicate that the meter of sound influences subcortical processing, and this metrical modulation differs depending on musical expertise.

Neural Representation of Articulable and Inarticulable Novel Sound Contrasts: The Role of the Dorsal Stream

The extent that articulatory information embedded in incoming speech contributes to the formation of new perceptual categories for speech sounds has been a matter of discourse for decades. It has been theorized that the acquisition of new speech sound categories requires a network of sensory and speech motor cortical areas (the "dorsal stream") to successfully integrate auditory and articulatory information. However, it is possible that these brain regions are not sensitive specifically to articulatory information, but instead are sensitive to the abstract phonological categories being learned. We tested this hypothesis by training participants over the course of several days on an articulable non-native speech contrast and acoustically matched inarticulable nonspeech analogues. After reaching comparable levels of proficiency with the two sets of stimuli, activation was measured in fMRI as participants passively listened to both sound types. Decoding of category membership for the articulable speech contrast alone revealed a series of left and right hemisphere regions outside of the dorsal stream that have previously been implicated in the emergence of non-native speech sound categories, while no regions could successfully decode the inarticulable nonspeech contrast. Although activation patterns in the left inferior frontal gyrus, the middle temporal gyrus, and the supplementary motor area provided better information for decoding articulable (speech) sounds compared to the inarticulable (sine wave) sounds, the finding that dorsal stream regions do not emerge as good decoders of the articulable contrast alone suggests that other factors, including the strength and structure of the emerging speech categories are more likely drivers of dorsal stream activation for novel sound learning.

Precise memory for pure tones is predicted by measures of learning-induced sensory system neurophysiological plasticity at cortical and subcortical levels

Despite identical learning experiences, individuals differ in the memory formed of those experiences. Molecular mechanisms that control the neurophysiological bases of long-term memory formation might control how precisely the memory formed reflects the actually perceived experience. Memory formed with sensory specificity determines its utility for selectively cueing subsequent behavior, even in novel situations. Here, a rodent model of auditory learning capitalized on individual differences in learning-induced auditory neuroplasticity to identify and characterize neural substrates for sound-specific (vs. general) memory of the training signal's acoustic frequency. Animals that behaviorally revealed a naturally induced signal-"specific" memory exhibited long-lasting signal-specific neurophysiological plasticity in auditory cortical and subcortical evoked responses. Animals with "general" memories did not exhibit learning-induced changes in these same measures. Manipulating a histone deacetylase during memory consolidation biased animals to have more signal-specific memory. Individual differences validated this brain-behavior relationship in both natural and manipulated memory formation, such that the degree of change in sensory cortical and subcortical neurophysiological responses could be used to predict the behavioral precision of memory.

Effects of cTBS on the Frequency-Following Response and Other Auditory Evoked Potentials

The frequency-following response (FFR) is an auditory evoked potential (AEP) that follows the periodic characteristics of a sound. Despite being a widely studied biosignal in auditory neuroscience, the neural underpinnings of the FFR are still unclear. Traditionally, FFR was associated with subcortical activity, but recent evidence suggested cortical contributions which may be dependent on the stimulus frequency. We combined electroencephalography (EEG) with an inhibitory transcranial magnetic stimulation protocol, the continuous theta burst stimulation (cTBS), to disentangle the cortical contribution to the FFR elicited to stimuli of high and low frequency. We recorded FFR to the syllable /ba/ at two fundamental frequencies (Low: 113 Hz; High: 317 Hz) in healthy participants. FFR, cortical potentials, and auditory brainstem response (ABR) were recorded before and after real and sham cTBS in the right primary auditory cortex. Results showed that cTBS did not produce a significant change in the FFR recorded, in any of the frequencies. No effect was observed in the ABR and cortical potentials, despite the latter known contributions from the auditory cortex. Possible reasons behind the negative results include compensatory mechanisms from the non-targeted areas, intraindividual variability of the cTBS effectiveness, and the particular location of our target area, the primary auditory cortex.

Subcortical auditory neural synchronization is deficient in pre-reading children who develop dyslexia

Auditory processing of temporal information in speech is sustained by synchronized firing of neurons along the entire auditory pathway. In school-aged children and adults with dyslexia, neural synchronization deficits have been found at cortical levels of the auditory system, however, these deficits do not appear to be present in pre-reading children. An alternative role for subcortical synchronization in reading development and dyslexia has been suggested, but remains debated. By means of a longitudinal study, we assessed cognitive reading-related skills and subcortical auditory steady-state responses (80 Hz ASSRs) in a group of children before formal reading instruction (pre-reading), after 1 year of formal reading instruction (beginning reading), and after 3 years of formal reading instruction (more advanced reading). Children were retrospectively classified into three groups based on family risk and literacy achievement: typically developing children without a family risk for dyslexia, typically developing children with a family risk for dyslexia, and children who developed dyslexia. Our results reveal that children who developed dyslexia demonstrate decreased 80 Hz ASSRs at the pre-reading stage. This effect is no longer present after the onset of reading instruction, due to an atypical developmental increase in 80 Hz ASSRs between the pre-reading and the beginning reading stage. A forward stepwise logistic regression analysis showed that literacy achievement was predictable with an accuracy of 90.4% based on a model including three significant predictors, that is, family risk for dyslexia (R = .31), phonological awareness (R = .23), and 80 Hz ASSRs (R = .26). Given that (1) abnormalities in subcortical ASSRs preceded reading acquisition in children who developed dyslexia and (2) subcortical ASSRs contributed to the prediction of literacy achievement, subcortical auditory synchronization deficits may constitute a pre-reading risk factor in the emergence of dyslexia.

Effect of Auditory Predictability on the Human Peripheral Auditory System

Auditory perception is facilitated by prior knowledge about the statistics of the acoustic environment. Predictions about upcoming auditory stimuli are processed at various stages along the human auditory pathway, including the cortex and midbrain. Whether such auditory predictions are processed also at hierarchically lower stages-in the peripheral auditory system-is unclear. To address this question, we assessed outer hair cell (OHC) activity in response to isochronous tone sequences and varied the predictability and behavioral relevance of the individual tones (by manipulating tone-to-tone probabilities and the human participants' task, respectively). We found that predictability alters the amplitude of distortion-product otoacoustic emissions (DPOAEs, a measure of OHC activity) in a manner that depends on the behavioral relevance of the tones. Simultaneously recorded cortical responses showed a significant effect of both predictability and behavioral relevance of the tones, indicating that their experimental manipulations were effective in central auditory processing stages. Our results provide evidence for a top-down effect on the processing of auditory predictability in the human peripheral auditory system, in line with previous studies showing peripheral effects of auditory attention.

Auditory cortical generators of the Frequency Following Response are modulated by intermodal attention

The efferent auditory system suggests that brainstem auditory regions could also be sensitive to top-down processes. In electrophysiology, the Frequency Following Response (FFR) to speech stimuli has been used extensively to study brainstem areas. Despite seemingly straight-forward in addressing the issue of attentional modulations of brainstem regions by means of the FFR, the existing results are inconsistent. Moreover, the notion that the FFR exclusively represents subcortical generators has been challenged. We aimed to gain a more differentiated perspective on how the generators of the FFR are modulated by either attending to the visual or auditory input while neural activity was recorded using magnetoencephalography (MEG). In a first step our results confirm the strong contribution of also cortical regions to the FFR. Interestingly, of all regions exhibiting a measurable FFR response, only the right primary auditory cortex was significantly affected by intermodal attention. By showing a clear cortical contribution to the attentional FFR effect, our work significantly extends previous reports that focus on surface level recordings only. It underlines the importance of making a greater effort to disentangle the different contributing sources of the FFR and serves as a clear precaution of simplistically interpreting the FFR as brainstem response.

Biometric identification of listener identity from frequency following responses to speech

OBJECTIVE

We investigate the biometric specificity of the frequency following response (FFR), an EEG marker of early auditory processing that reflects phase-locked activity from neural ensembles in the auditory cortex and subcortex (Chandrasekaran and Kraus 2010, Bidelman, 2015a, 2018, Coffey et al 2017b). Our objective is two-fold: demonstrate that the FFR contains information beyond stimulus properties and broad group-level markers, and to assess the practical viability of the FFR as a biometric across different sounds, auditory experiences, and recording days.

APPROACH

We trained the hidden Markov model (HMM) to decode listener identity from FFR spectro-temporal patterns across multiple frequency bands. Our dataset included FFRs from twenty native speakers of English or Mandarin Chinese (10 per group) listening to Mandarin Chinese tones across three EEG sessions separated by days. We decoded subject identity within the same auditory context (same tone and session) and across different stimuli and recording sessions.

MAIN RESULTS

The HMM decoded listeners for averaging sizes as small as one single FFR. However, model performance improved for larger averaging sizes (e.g. 25 FFRs), similarity in auditory context (same tone and day), and lack of familiarity with the sounds (i.e. native English relative to native Chinese listeners). Our results also revealed important biometric contributions from frequency bands in the cortical and subcortical EEG.

SIGNIFICANCE

Our study provides the first deep and systematic biometric characterization of the FFR and provides the basis for biometric identification systems incorporating this neural signal.

Tracking Training-Related Plasticity by Combining fMRI and DTI: The Right Hemisphere Ventral Stream Mediates Musical Syntax Processing

As a functional homolog for left-hemispheric syntax processing in language, neuroimaging studies evidenced involvement of right prefrontal regions in musical syntax processing, of which underlying white matter connectivity remains unexplored so far. In the current experiment, we investigated the underlying pathway architecture in subjects with 3 levels of musical expertise. Employing diffusion tensor imaging tractography, departing from seeds from our previous functional magnetic resonance imaging study on music syntax processing in the same participants, we identified a pathway in the right ventral stream that connects the middle temporal lobe with the inferior frontal cortex via the extreme capsule, and corresponds to the left hemisphere ventral stream, classically attributed to syntax processing in language comprehension. Additional morphometric consistency analyses allowed dissociating tract core from more dispersed fiber portions. Musical expertise related to higher tract consistency of the right ventral stream pathway. Specifically, tract consistency in this pathway predicted the sensitivity for musical syntax violations. We conclude that enduring musical practice sculpts ventral stream architecture. Our results suggest that training-related pathway plasticity facilitates the right hemisphere ventral stream information transfer, supporting an improved sound-to-meaning mapping in music.

Interactive effects of linguistic abstraction and stimulus statistics in the online modulation of neural speech encoding

Speech processing is highly modulated by context. Prior studies examining frequency-following responses (FFRs), an electrophysiological 'neurophonic' potential that faithfully reflects phase-locked activity from neural ensembles within the auditory network, have demonstrated that stimulus context modulates the integrity of speech encoding. The extent to which context-dependent encoding reflects general auditory properties or interactivities between statistical and higher-level linguistic processes remains unexplored. Our study examined whether speech encoding, as reflected by FFRs, is modulated by abstract phonological relationships between a stimulus and surrounding contexts. FFRs were elicited to a Mandarin rising-tone syllable (/ji-TR/, 'second') randomly presented with other syllables in three contexts from 17 native listeners. In a contrastive context, /ji-TR/ occurred with meaning-contrastive high-level-tone syllables (/ji-H/, 'one'). In an allotone context, TR occurred with dipping-tone syllables /ji-D/, a non-meaning-contrastive variant of /ji-TR/. In a repetitive context, the same /ji-TR/ occurred with other speech tokens of /ji-TR/. Consistent with prior work, neural tracking of /ji-TR/ pitch contour was more faithful in the repetitive condition wherein /ji-TR/ occurred more predictably (p = 1) than in the contrastive condition (p = 0.34). Crucially, in the allotone context, neural tracking of /ji-TR/ was more accurate relative to the contrastive context, despite both having an identical transitional probability (p = 0.34). Mechanistically, the non-meaning-contrastive relationship may have augmented the probability to /ji-TR/ occurrence in the allotone context. Results indicate online interactions between bottom-up and top-down mechanisms, which facilitate speech perception. Such interactivities may predictively fine-tune incoming speech encoding using linguistic and statistical information from prior context.

Taking Attention Away from the Auditory Modality: Context-dependent Effects on Early Sensory Encoding of Speech

Increasing visual perceptual load can reduce pre-attentive auditory cortical activity to sounds, a reflection of the limited and shared attentional resources for sensory processing across modalities. Here, we demonstrate that modulating visual perceptual load can impact the early sensory encoding of speech sounds, and that the impact of visual load is highly dependent on the predictability of the incoming speech stream. Participants (n = 20, 9 females) performed a visual search task of high (target similar to distractors) and low (target dissimilar to distractors) perceptual load, while early auditory electrophysiological responses were recorded to native speech sounds. Speech sounds were presented either in a 'repetitive context', or a less predictable 'variable context'. Independent of auditory stimulus context, pre-attentive auditory cortical activity was reduced during high visual load, relative to low visual load. We applied a data-driven machine learning approach to decode speech sounds from the early auditory electrophysiological responses. Decoding performance was found to be poorer under conditions of high (relative to low) visual load, when the incoming acoustic stream was predictable. When the auditory stimulus context was less predictable, decoding performance was substantially greater for the high (relative to low) visual load conditions. Our results provide support for shared attentional resources between visual and auditory modalities that substantially influence the early sensory encoding of speech signals in a context-dependent manner.

Hidden Markov modeling of frequency-following responses to Mandarin lexical tones

BACKGROUND

The frequency-following response (FFR) is a scalp-recorded electrophysiological potential reflecting phase-locked activity from neural ensembles in the auditory system. The FFR is often used to assess the robustness of subcortical pitch processing. Due to low signal-to-noise ratio at the single-trial level, FFRs are typically averaged across thousands of stimulus repetitions. Prior work using this approach has shown that subcortical encoding of linguistically-relevant pitch patterns is modulated by long-term language experience.

NEW METHOD

We examine the extent to which a machine learning approach using hidden Markov modeling (HMM) can be utilized to decode Mandarin tone-categories from scalp-record electrophysiolgical activity. We then assess the extent to which the HMM can capture biologically-relevant effects (language experience-driven plasticity). To this end, we recorded FFRs to four Mandarin tones from 14 adult native speakers of Chinese and 14 of native English. We trained a HMM to decode tone categories from the FFRs with varying size of averages.

RESULTS AND COMPARISONS WITH EXISTING METHODS

Tone categories were decoded with above-chance accuracies using HMM. The HMM derived metric (decoding accuracy) revealed a robust effect of language experience, such that FFRs from native Chinese speakers yielded greater accuracies than native English speakers. Critically, the language experience-driven plasticity was captured with average sizes significantly smaller than those used in the extant literature.

CONCLUSIONS

Our results demonstrate the feasibility of HMM in assessing the robustness of neural pitch. Machine-learning approaches can complement extant analytical methods that capture auditory function and could reduce the number of trials needed to capture biological phenomena.

Dysfunction of Rapid Neural Adaptation in Dyslexia

Identification of specific neurophysiological dysfunctions resulting in selective reading difficulty (dyslexia) has remained elusive. In addition to impaired reading development, individuals with dyslexia frequently exhibit behavioral deficits in perceptual adaptation. Here, we assessed neurophysiological adaptation to stimulus repetition in adults and children with dyslexia for a wide variety of stimuli, spoken words, written words, visual objects, and faces. For every stimulus type, individuals with dyslexia exhibited significantly diminished neural adaptation compared to controls in stimulus-specific cortical areas. Better reading skills in adults and children with dyslexia were associated with greater repetition-induced neural adaptation. These results highlight a dysfunction of rapid neural adaptation as a core neurophysiological difference in dyslexia that may underlie impaired reading development. Reduced neurophysiological adaptation may relate to prior reports of reduced behavioral adaptation in dyslexia and may reveal a difference in brain functions that ultimately results in a specific reading impairment.

Differential sensitivity to changes in pitch acceleration in the auditory brainstem and cortex

The cortical pitch-specific response (CPR) is differentially sensitive to pitch contours varying in rate of acceleration-time-variant Mandarin Tone2 (T2) versus constant, linear rising ramp (Linear)-as a function of language experience (Krishnan, Gandour, & Suresh, 2014). CPR and brainstem frequency following response (FFR) data were recorded concurrently from native Mandarin listeners using the same stimuli. Results showed that T2 elicited larger responses than Linear at both cortical and brainstem levels (CPR: Na-Pb, Pb-Nb; FFR). However, Pb-Nb exhibited a larger difference in magnitude between T2 and Linear than either Na-Pb or FFR. This finding highlights differential weighting of brain responses elicited by a specific temporal attribute of pitch. Consistent with the notion of a distributed, integrated hierarchical pitch processing network, temporal attributes of pitch are differentially weighted by subcortical and cortical level processing.

Stability and plasticity in neural encoding of linguistically relevant pitch patterns

While lifelong language experience modulates subcortical encoding of pitch patterns, there is emerging evidence that short-term training introduced in adulthood also shapes subcortical pitch encoding. Here we use a cross-language design to examine the stability of language experience-dependent subcortical plasticity over multiple days. We then examine the extent to which behavioral relevance induced by sound-to-category training leads to plastic changes in subcortical pitch encoding in adulthood relative to adolescence, a period of ongoing maturation of subcortical and cortical auditory processing. Frequency-following responses (FFRs), which reflect phase-locked activity from subcortical neural ensembles, were elicited while participants passively listened to pitch patterns reflective of Mandarin tones. In experiment 1, FFRs were recorded across three consecutive days from native Chinese-speaking (n = 10) and English-speaking (n = 10) adults. In experiment 2, FFRs were recorded from native English-speaking adolescents (n = 20) and adults (n = 15) before, during, and immediately after a session of sound-to-category training, as well as a day after training ceased. Experiment 1 demonstrated the stability of language experience-dependent subcortical plasticity in pitch encoding across multiple days of passive exposure to linguistic pitch patterns. In contrast, experiment 2 revealed an enhancement in subcortical pitch encoding that emerged a day after the sound-to-category training, with some developmental differences observed. Taken together, these findings suggest that behavioral relevance is a critical component for the observation of plasticity in the subcortical encoding of pitch.NEW & NOTEWORTHY We examine the timescale of experience-dependent auditory plasticity to linguistically relevant pitch patterns. We find extreme stability in lifelong experience-dependent plasticity. We further demonstrate that subcortical function in adolescents and adults is modulated by a single session of sound-to-category training. Our results suggest that behavioral relevance is a necessary ingredient for neural changes in pitch encoding to be observed throughout human development. These findings contribute to the neurophysiological understanding of long- and short-term experience-dependent modulation of pitch.

Neural Correlates of Indicators of Sound Change in Cantonese: Evidence from Cortical and Subcortical Processes

Across time, languages undergo changes in phonetic, syntactic, and semantic dimensions. Social, cognitive, and cultural factors contribute to sound change, a phenomenon in which the phonetics of a language undergo changes over time. Individuals who misperceive and produce speech in a slightly divergent manner (called innovators) contribute to variability in the society, eventually leading to sound change. However, the cause of variability in these individuals is still unknown. In this study, we examined whether such misperceptions are represented in neural processes of the auditory system. We investigated behavioral, subcortical (via FFR), and cortical (via P300) manifestations of sound change processing in Cantonese, a Chinese language in which several lexical tones are merging. Across the merging categories, we observed a similar gradation of speech perception abilities in both behavior and the brain (subcortical and cortical processes). Further, we also found that behavioral evidence of tone merging correlated with subjects' encoding at the subcortical and cortical levels. These findings indicate that tone-merger categories, that are indicators of sound change in Cantonese, are represented neurophysiologically with high fidelity. Using our results, we speculate that innovators encode speech in a slightly deviant neurophysiological manner, and thus produce speech divergently that eventually spreads across the community and contributes to sound change.

Dysfunction of Rapid Neural Adaptation in Dyslexia

Identification of specific neurophysiological dysfunctions resulting in selective reading difficulty (dyslexia) has remained elusive. In addition to impaired reading development, individuals with dyslexia frequently exhibit behavioral deficits in perceptual adaptation. Here, we assessed neurophysiological adaptation to stimulus repetition in adults and children with dyslexia for a wide variety of stimuli, spoken words, written words, visual objects, and faces. For every stimulus type, individuals with dyslexia exhibited significantly diminished neural adaptation compared to controls in stimulus-specific cortical areas. Better reading skills in adults and children with dyslexia were associated with greater repetition-induced neural adaptation. These results highlight a dysfunction of rapid neural adaptation as a core neurophysiological difference in dyslexia that may underlie impaired reading development. Reduced neurophysiological adaptation may relate to prior reports of reduced behavioral adaptation in dyslexia and may reveal a difference in brain functions that ultimately results in a specific reading impairment.

Timing predictability enhances regularity encoding in the human subcortical auditory pathway

The encoding of temporal regularities is a critical property of the auditory system, as short-term neural representations of environmental statistics serve to auditory object formation and detection of potentially relevant novel stimuli. A putative neural mechanism underlying regularity encoding is repetition suppression, the reduction of neural activity to repeated stimulation. Although repetitive stimulation per se has shown to reduce auditory neural activity in animal cortical and subcortical levels and in the human cerebral cortex, other factors such as timing may influence the encoding of statistical regularities. This study was set out to investigate whether temporal predictability in the ongoing auditory input modulates repetition suppression in subcortical stages of the auditory processing hierarchy. Human auditory frequency–following responses (FFR) were recorded to a repeating consonant–vowel stimuli (/wa/) delivered in temporally predictable and unpredictable conditions. FFR amplitude was attenuated by repetition independently of temporal predictability, yet we observed an accentuated suppression when the incoming stimulation was temporally predictable. These findings support the view that regularity encoding spans across the auditory hierarchy and point to temporal predictability as a modulatory factor of regularity encoding in early stages of the auditory pathway.

Context-dependent plasticity in the subcortical encoding of linguistic pitch patterns

We examined the mechanics of online experience-dependent auditory plasticity by assessing the influence of prior context on the frequency-following responses (FFRs), which reflect phase-locked responses from neural ensembles within the subcortical auditory system. FFRs were elicited to a Cantonese falling lexical pitch pattern from 24 native speakers of Cantonese in a variable context, wherein the falling pitch pattern randomly occurred in the context of two other linguistic pitch patterns; in a patterned context, wherein, the falling pitch pattern was presented in a predictable sequence along with two other pitch patterns, and in a repetitive context, wherein the falling pitch pattern was presented with 100% probability. We found that neural tracking of the stimulus pitch contour was most faithful and accurate when listening context was patterned and least faithful when the listening context was variable. The patterned context elicited more robust pitch tracking relative to the repetitive context, suggesting that context-dependent plasticity is most robust when the context is predictable but not repetitive. Our study demonstrates a robust influence of prior listening context that works to enhance online neural encoding of linguistic pitch patterns. We interpret these results as indicative of an interplay between contextual processes that are responsive to predictability as well as novelty in the presentation context.

NEW & NOTEWORTHY

Human auditory perception in dynamic listening environments requires fine-tuning of sensory signal based on behaviorally relevant regularities in listening context, i.e., online experience-dependent plasticity. Our finding suggests what partly underlie online experience-dependent plasticity are interplaying contextual processes in the subcortical auditory system that are responsive to predictability as well as novelty in listening context. These findings add to the literature that looks to establish the neurophysiological bases of auditory system plasticity, a central issue in auditory neuroscience.

Sensorimotor Grounding of Musical Embodiment and the Role of Prediction: A Review

In a previous article, we reviewed empirical evidence demonstrating action-based effects on music perception to substantiate the musical embodiment thesis (Maes et al., 2014). Evidence was largely based on studies demonstrating that music perception automatically engages motor processes, or that body states/movements influence music perception. Here, we argue that more rigorous evidence is needed before any decisive conclusion in favor of a “radical” musical embodiment thesis can be posited. In the current article, we provide a focused review of recent research to collect further evidence for the “radical” embodiment thesis that music perception is a dynamic process firmly rooted in the natural disposition of sounds and the human auditory and motor system. Though, we emphasize that, on top of these natural dispositions, long-term processes operate, rooted in repeated sensorimotor experiences and leading to learning, prediction, and error minimization. This approach sheds new light on the development of musical repertoires, and may refine our understanding of action-based effects on music perception as discussed in our previous article (Maes et al., 2014). Additionally, we discuss two of our recent empirical studies demonstrating that music performance relies on similar principles of sensorimotor dynamics and predictive processing.

Language-experience plasticity in neural representation of changes in pitch salience

Neural representation of pitch-relevant information at the brainstem and cortical levels of processing is influenced by language experience. A well-known attribute of pitch is its salience. Brainstem frequency following responses and cortical pitch specific responses, recorded concurrently, were elicited by a pitch salience continuum spanning weak to strong pitch of a dynamic, iterated rippled noise pitch contour-homolog of a Mandarin tone. Our aims were to assess how language experience (Chinese, English) affects i) enhancement of neural activity associated with pitch salience at brainstem and cortical levels, ii) the presence of asymmetry in cortical pitch representation, and iii) patterns of relative changes in magnitude along the pitch salience continuum. Peak latency (Fz: Na, Pb, and Nb) was shorter in the Chinese than the English group across the continuum. Peak-to-peak amplitude (Fz: Na-Pb, Pb-Nb) of the Chinese group grew larger with increasing pitch salience, but an experience-dependent advantage was limited to the Na-Pb component. At temporal sites (T7/T8), the larger amplitude of the Chinese group across the continuum was both limited to the Na-Pb component and the right temporal site. At the brainstem level, F0 magnitude gets larger as you increase pitch salience, and it too reveals Chinese superiority. A direct comparison of cortical and brainstem responses for the Chinese group reveals different patterns of relative changes in magnitude along the pitch salience continuum. Such differences may point to a transformation in pitch processing at the cortical level presumably mediated by local sensory and/or extrasensory influence overlaid on the brainstem output.

Sound localization in a changing world

In natural environments, neural systems must be continuously updated to reflect changes in sensory inputs and behavioral goals. Recent studies of sound localization have shown that adaptation and learning involve multiple mechanisms that operate at different timescales and stages of processing, with other sensory and motor-related inputs playing a key role. We are only just beginning to understand, however, how these processes interact with one another to produce adaptive changes at the level of neuronal populations and behavior. Because there is no explicit map of auditory space in the cortex, studies of sound localization may also provide much broader insight into the plasticity of complex neural representations that are not topographically organized.

Neuronal adaptation, novelty detection and regularity encoding in audition

The ability to detect unexpected stimuli in the acoustic environment and determine their behavioral relevance to plan an appropriate reaction is critical for survival. This perspective article brings together several viewpoints and discusses current advances in understanding the mechanisms the auditory system implements to extract relevant information from incoming inputs and to identify unexpected events. This extraordinary sensitivity relies on the capacity to codify acoustic regularities, and is based on encoding properties that are present as early as the auditory midbrain. We review state-of-the-art studies on the processing of stimulus changes using non-invasive methods to record the summed electrical potentials in humans, and those that examine single-neuron responses in animal models. Human data will be based on mismatch negativity (MMN) and enhanced middle latency responses (MLR). Animal data will be based on the activity of single neurons at the cortical and subcortical levels, relating selective responses to novel stimuli to the MMN and to stimulus-specific neural adaptation (SSA). Theoretical models of the neural mechanisms that could create SSA and novelty responses will also be discussed.