The neuroanatomical and functional organization of speech perception

Phase-dependent word perception emerges from region-specific sensitivity to the statistics of language

Neural oscillations reflect fluctuations in excitability, which biases the percept of ambiguous sensory input. Why this bias occurs is still not fully understood. We hypothesized that neural populations representing likely events are more sensitive, and thereby become active on earlier oscillatory phases, when the ensemble itself is less excitable. Perception of ambiguous input presented during less-excitable phases should therefore be biased toward frequent or predictable stimuli that have lower activation thresholds. Here, we show such a frequency bias in spoken word recognition using psychophysics, magnetoencephalography (MEG), and computational modelling. With MEG, we found a double dissociation, where the phase of oscillations in the superior temporal gyrus and medial temporal gyrus biased word-identification behavior based on phoneme and lexical frequencies, respectively. This finding was reproduced in a computational model. These results demonstrate that oscillations provide a temporal ordering of neural activity based on the sensitivity of separable neural populations.

Perceiving and misperceiving speech: lexical and sublexical processing in the superior temporal lobes

Listeners can use prior knowledge to predict the content of noisy speech signals, enhancing perception. However, this process can also elicit misperceptions. For the first time, we employed a prime-probe paradigm and transcranial magnetic stimulation to investigate causal roles for the left and right posterior superior temporal gyri (pSTG) in the perception and misperception of degraded speech. Listeners were presented with spectrotemporally degraded probe sentences preceded by a clear prime. To produce misperceptions, we created partially mismatched pseudo-sentence probes via homophonic nonword transformations (e.g. The little girl was excited to lose her first tooth-Tha fittle girmn wam expited du roos har derst cooth). Compared to a control site (vertex), inhibitory stimulation of the left pSTG selectively disrupted priming of real but not pseudo-sentences. Conversely, inhibitory stimulation of the right pSTG enhanced priming of misperceptions with pseudo-sentences, but did not influence perception of real sentences. These results indicate qualitatively different causal roles for the left and right pSTG in perceiving degraded speech, supporting bilateral models that propose engagement of the right pSTG in sublexical processing.

On the interplay between speech perception and production: insights from research and theories

The study of spoken communication has long been entrenched in a debate surrounding the interdependence of speech production and perception. This mini review summarizes findings from prior studies to elucidate the reciprocal relationships between speech production and perception. We also discuss key theoretical perspectives relevant to speech perception-production loop, including hyper-articulation and hypo-articulation (H&H) theory, speech motor theory, direct realism theory, articulatory phonology, the Directions into Velocities of Articulators (DIVA) and Gradient Order DIVA (GODIVA) models, and predictive coding. Building on prior findings, we propose a revised auditory-motor integration model of speech and provide insights for future research in speech perception and production, focusing on the effects of impaired peripheral auditory systems.

Audiotactile Stimulation Can Improve Syllable Discrimination through Multisensory Integration in the Theta Frequency Band

Syllables are an essential building block of speech. We recently showed that tactile stimuli linked to the perceptual centers of syllables in continuous speech can improve speech comprehension. The rate of syllables lies in the theta frequency range, between 4 and 8 Hz, and the behavioral effect appears linked to multisensory integration in this frequency band. Because this neural activity may be oscillatory, we hypothesized that a behavioral effect may also occur not only while but also after this activity has been evoked or entrained through vibrotactile pulses. Here, we show that audiotactile integration regarding the perception of single syllables, both on the neural and on the behavioral level, is consistent with this hypothesis. We first stimulated participants with a series of vibrotactile pulses and then presented them with a syllable in background noise. We show that, at a delay of 200 msec after the last vibrotactile pulse, audiotactile integration still occurred in the theta band and syllable discrimination was enhanced. Moreover, the dependence of both the neural multisensory integration as well as of the behavioral discrimination on the delay of the audio signal with respect to the last tactile pulse was consistent with a damped oscillation. In addition, the multisensory gain is correlated with the syllable discrimination score. Our results therefore evidence the role of the theta band in audiotactile integration and provide evidence that these effects may involve oscillatory activity that still persists after the tactile stimulation.

Auditory phonological identification impairment in primary progressive aphasia

OBJECTIVE

To examine the audiological characteristics and neuroanatomical regions associated with auditory phonological identification impairment in primary progressive aphasia (PPA).

METHODS

Twenty-seven patients with PPA [13 non-fluent/agrammatic variant PPA (nfvPPA), three logopenic variant PPA (lvPPA), seven semantic variant PPA (svPPA), and four mixed type PPA] were included in the study. Neuropsychological, language, audiological, and neuroradiological examinations were also performed. Auditory function examinations consisted of a pure-tone threshold test, a phonological identification task, and temporal auditory acuity tests, such as click counting or fusion. As an evaluation value of phonological identification ability, we calculated the discrepancy scores, which were the smaller discrepancy (left or right ear) in phonological identification ability scores between measured and expected values from the pure-tone threshold. In the neuroradiological examination, we evaluated the regional cerebral blood flow using 123I-iodoamphetamine single-photon emission computed tomography.

RESULTS

Eight of the 27 patients were allocated to the impaired phonological identification group, and four were considered to have significant impairment on further analysis. Two of these patients, one with lvPPA and one with mixed type of lvPPA and nfvPPA, showed apparent phonological identification deficits that could be observed in daily life. The discrepancy scores were not significantly related to the results of neuropsychological, language, or any other auditory examinations, except for the click counting score in the left ear. Voxel-based correlation analyses revealed that regional cerebral blood flow in the bilateral superior temporal gyrus and bilateral primary auditory cortex was significantly and positively correlated with phonological identification ability.

CONCLUSIONS

Our results suggest that progressive dysfunction of the bilateral superior temporal gyrus and bilateral primary auditory cortex due to neurodegenerative diseases leads to phonological identification impairment in PPA syndrome.

Distinct neurophysiology during nonword repetition in logopenic and non-fluent variants of primary progressive aphasia

Overlapping clinical presentations in primary progressive aphasia (PPA) variants present challenges for diagnosis and understanding pathophysiology, particularly in the early stages of the disease when behavioral (speech) symptoms are not clearly evident. Divergent atrophy patterns (temporoparietal degeneration in logopenic variant lvPPA, frontal degeneration in nonfluent variant nfvPPA) can partially account for differential speech production errors in the two groups in the later stages of the disease. While the existing dogma states that neurodegeneration is the root cause of compromised behavior and cortical activity in PPA, the extent to which neurophysiological signatures of speech dysfunction manifest independent of their divergent atrophy patterns remain unknown. We test the hypothesis that nonword deficits in lvPPA and nfvPPA arise from distinct patterns of neural oscillations that are unrelated to atrophy. We use a novel structure-function imaging approach integrating magnetoencephalographic imaging of neural oscillations during a non-word repetition task with voxel-based morphometry-derived measures of gray matter volume to isolate neural oscillation abnormalities independent of atrophy. We find reduced beta band neural activity in left temporal regions associated with the late stages of auditory encoding unique to patients with lvPPA and reduced high-gamma neural activity over left frontal regions associated with the early stages of motor preparation in patients with nfvPPA. Neither of these patterns of reduced cortical oscillations was explained by cortical atrophy in our statistical model. These findings highlight the importance of structure-function imaging in revealing neurophysiological sequelae in early stages of dementia when neither structural atrophy nor behavioral deficits are clinically distinct.

A multidimensional characterization of the neurocognitive architecture underlying age-related temporal speech processing

Healthy aging is often associated with speech comprehension difficulties in everyday life situations despite a pure-tone hearing threshold in the normative range. Drawing on this background, we used a multidimensional approach to assess the functional and structural neural correlates underlying age-related temporal speech processing while controlling for pure-tone hearing acuity. Accordingly, we combined structural magnetic resonance imaging and electroencephalography, and collected behavioral data while younger and older adults completed a phonetic categorization and discrimination task with consonant-vowel syllables varying along a voice-onset time continuum. The behavioral results confirmed age-related temporal speech processing singularities which were reflected in a shift of the boundary of the psychometric categorization function, with older adults perceiving more syllable characterized by a short voice-onset time as /ta/ compared to younger adults. Furthermore, despite the absence of any between-group differences in phonetic discrimination abilities, older adults demonstrated longer N100/P200 latencies as well as increased P200 amplitudes while processing the consonant-vowel syllables varying in voice-onset time. Finally, older adults also exhibited a divergent anatomical gray matter infrastructure in bilateral auditory-related and frontal brain regions, as manifested in reduced cortical thickness and surface area. Notably, in the younger adults but not in the older adult cohort, cortical surface area in these two gross anatomical clusters correlated with the categorization of consonant-vowel syllables characterized by a short voice-onset time, suggesting the existence of a critical gray matter threshold that is crucial for consistent mapping of phonetic categories varying along the temporal dimension. Taken together, our results highlight the multifaceted dimensions of age-related temporal speech processing characteristics, and pave the way toward a better understanding of the relationships between hearing, speech and the brain in older age.

Language disorders in patients with striatal lesions: Deciphering the role of the striatum in language performance

The classical neural model of language refers to a cortical network involving frontal, parietal and temporal regions. However, patients with subcortical lesions of the striatum have language difficulties. We investigated whether the striatum is directly involved in language or whether its role in decision-making has an indirect effect on language performance, by testing carriers of Huntington's disease (HD) mutations and controls. HD is a genetic neurodegenerative disease primarily affecting the striatum and causing language disorders. We asked carriers of the HD mutation in the premanifest (before clinical diagnosis) and early disease stages, and controls to perform two discrimination tasks, one involving linguistic and the other non-linguistic stimuli. We used the hierarchical drift diffusion model (HDDM) to analyze the participants' responses and to assess the decision and non-decision parameters separately. We hypothesized that any language deficits related to decision-making impairments would be reflected in the decision parameters of linguistic and non-linguistic tasks. We also assessed the relative contributions of both HDDM decision and non-decision parameters to the participants' behavioral data (response time and discriminability). Finally, we investigated whether the decision and non-decision parameters of the HDDM were correlated with brain atrophy. The HDDM analysis showed that patients with early HD have impaired decision parameters relative to controls, regardless of the task. In both tasks, decision parameters better explained the variance of response time and discriminability performance than non-decision parameters. In the linguistic task, decision parameters were positively correlated with gray matter volume in the ventral striatum and putamen, whereas non-decision parameters were not. Language impairment in patients with striatal atrophy is better explained by a deficit of decision-making than by a deficit of core linguistic processing. These results suggest that the striatum is involved in language through the modulation of decision-making, presumably by regulating the process of choice between linguistic alternatives.

Distinct neuroanatomical and neuropsychological features of Down syndrome compared to related neurodevelopmental disorders: a systematic review

Objectives

We critically review research findings on the unique changes in brain structure and cognitive function characteristic of Down syndrome (DS) and summarize the similarities and differences with other neurodevelopmental disorders such as Williams syndrome, 22q11.2 deletion syndrome, and fragile X syndrome.

Methods

We conducted a meta-analysis and systematic literature review of 84 studies identified by searching PubMed, Google Scholar, and Web of Science from 1977 to October 2022. This review focuses on the following issues: (1) specific neuroanatomic and histopathological features of DS as revealed by autopsy and modern neuroimaging modalities, (2) language and memory deficits in DS, (3) the relationships between these neuroanatomical and neuropsychological features, and (4) neuroanatomic and neuropsychological differences between DS and related neurodevelopmental syndromes.

Results

Numerous post-mortem and morphometric neuroimaging investigations of individuals with DS have reported complex changes in regional brain volumes, most notably in the hippocampal formation, temporal lobe, frontal lobe, parietal lobe, and cerebellum. Moreover, neuropsychological assessments have revealed deficits in language development, emotional regulation, and memory that reflect these structural changes and are more severe than expected from general cognitive dysfunction. Individuals with DS also show relative preservation of multiple cognitive, linguistic, and social domains compared to normally developed controls and individuals with other neurodevelopmental disorders. However, all these neurodevelopment disorders exhibit substantial heterogeneity among individuals.

Conclusion

People with Down syndrome demonstrate unique neurodevelopmental abnormalities but cannot be regarded as a homogenous group. A comprehensive evaluation of individual intellectual skills is essential for all individuals with neurodevelopment disorders to develop personalized care programs.

An fNIRS investigation of novel expressed emotion stimulations in schizophrenia

Living in high expressed emotion (EE) environments tends to increase the relapse rate in schizophrenia (SZ). At present, the neural substrates responsible for high EE in SZ remain poorly understood. Functional near-infrared spectroscopy (fNIRS) may be of great use to quantitatively assess cortical hemodynamics and elucidate the pathophysiology of psychiatric disorders. In this study, we designed novel low- (positivity and warmth) and high-EE (criticism, negative emotion, and hostility) stimulations, in the form of audio, to investigate cortical hemodynamics. We used fNIRS to measure hemodynamic signals while participants listened to the recorded audio. Healthy controls (HCs, [Formula: see text]) showed increased hemodynamic activation in the major language centers across EE stimulations, with stronger activation in Wernicke's area during the processing of negative emotional language. Compared to HCs, people with SZ ([Formula: see text]) exhibited smaller hemodynamic activation in the major language centers across EE stimulations. In addition, people with SZ showed weaker or insignificant hemodynamic deactivation in the medial prefrontal cortex. Notably, hemodynamic activation in SZ was found to be negatively correlated with the negative syndrome scale score at high EE. Our findings suggest that the neural mechanisms in SZ are altered and disrupted, especially during negative emotional language processing. This supports the feasibility of using the designed EE stimulations to assess people who are vulnerable to high-EE environments, such as SZ. Furthermore, our findings provide preliminary evidence for future research on functional neuroimaging biomarkers for people with psychiatric disorders.

Predictive language comprehension in Parkinson's disease

Verb and action knowledge deficits are reported in persons with Parkinson's disease (PD), even in the absence of dementia or mild cognitive impairment. However, the impact of these deficits on combinatorial semantic processing is less well understood. Following on previous verb and action knowledge findings, we tested the hypothesis that PD impairs the ability to integrate event-based thematic fit information during online sentence processing. Specifically, we anticipated persons with PD with age-typical cognitive abilities would perform more poorly than healthy controls during a visual world paradigm task requiring participants to predict a target object constrained by the thematic fit of the agent-verb combination. Twenty-four PD and 24 healthy age-matched participants completed comprehensive neuropsychological assessments. We recorded participants' eye movements as they heard predictive sentences (The fisherman rocks the boat) alongside target, agent-related, verb-related, and unrelated images. We tested effects of group (PD/control) on gaze using growth curve models. There were no significant differences between PD and control participants, suggesting that PD participants successfully and rapidly use combinatory thematic fit information to predict upcoming language. Baseline sentences with no predictive information (e.g., Look at the drum) confirmed that groups showed equivalent sentence processing and eye movement patterns. Additionally, we conducted an exploratory analysis contrasting PD and controls' performance on low-motion-content versus high-motion-content verbs. This analysis revealed fewer predictive fixations in high-motion sentences only for healthy older adults. PD participants may adapt to their disease by relying on spared, non-action-simulation-based language processing mechanisms, although this conclusion is speculative, as the analyses of high- vs. low-motion items was highly limited by the study design. These findings provide novel evidence that individuals with PD match healthy adults in their ability to use verb meaning to predict upcoming nouns despite previous findings of verb semantic impairment in PD across a variety of tasks.

Long-term changes in cortical representation through perceptual learning of spectrally degraded speech

Listeners can adapt to acoustically degraded speech with perceptual training. The learning processes for long periods underlies the rehabilitation of patients with hearing aids or cochlear implants. Perceptual learning of acoustically degraded speech has been associated with the frontotemporal cortices. However, neural processes during and after long-term perceptual learning remain unclear. Here we conducted perceptual training of noise-vocoded speech sounds (NVSS), which is spectrally degraded signals, and measured the cortical activity for seven days and the follow up testing (approximately 1 year later) to investigate changes in neural activation patterns using functional magnetic resonance imaging. We demonstrated that young adult participants (n = 5) improved their performance across seven experimental days, and the gains were maintained after 10 months or more. Representational similarity analysis showed that the neural activation patterns of NVSS relative to clear speech in the left posterior superior temporal sulcus (pSTS) were significantly different across seven training days, accompanying neural changes in frontal cortices. In addition, the distinct activation patterns to NVSS in the frontotemporal cortices were also observed 10-13 months after the training. We, therefore, propose that perceptual training can induce plastic changes and long-term effects on neural representations of the trained degraded speech in the frontotemporal cortices. These behavioral improvements and neural changes induced by the perceptual learning of degraded speech will provide insights into cortical mechanisms underlying adaptive processes in difficult listening situations and long-term rehabilitation of auditory disorders.

Unraveling Spatial-Spectral Dynamics of Speech Categorization Speed Using Convolutional Neural Networks

The process of categorizing sounds into distinct phonetic categories is known as categorical perception (CP). Response times (RTs) provide a measure of perceptual difficulty during labeling decisions (i.e., categorization). The RT is quasi-stochastic in nature due to individuality and variations in perceptual tasks. To identify the source of RT variation in CP, we have built models to decode the brain regions and frequency bands driving fast, medium and slow response decision speeds. In particular, we implemented a parameter optimized convolutional neural network (CNN) to classify listeners' behavioral RTs from their neural EEG data. We adopted visual interpretation of model response using Guided-GradCAM to identify spatial-spectral correlates of RT. Our framework includes (but is not limited to): (i) a data augmentation technique designed to reduce noise and control the overall variance of EEG dataset; (ii) bandpower topomaps to learn the spatial-spectral representation using CNN; (iii) large-scale Bayesian hyper-parameter optimization to find best performing CNN model; (iv) ANOVA and posthoc analysis on Guided-GradCAM activation values to measure the effect of neural regions and frequency bands on behavioral responses. Using this framework, we observe that α-β (10-20 Hz) activity over left frontal, right prefrontal/frontal, and right cerebellar regions are correlated with RT variation. Our results indicate that attention, template matching, temporal prediction of acoustics, motor control, and decision uncertainty are the most probable factors in RT variation.

Auditory Pitch Perception in Autism Spectrum Disorder: A Systematic Review and Meta-Analysis

PURPOSE

Pitch plays an important role in auditory perception of music and language. This study provides a systematic review with meta-analysis to investigate whether individuals with autism spectrum disorder (ASD) have enhanced pitch processing ability and to identify the potential factors associated with processing differences between ASD and neurotypicals.

METHOD

We conducted a systematic search through six major electronic databases focusing on the studies that used nonspeech stimuli to provide a qualitative and quantitative assessment across existing studies on pitch perception in autism. We identified potential participant- and methodology-related moderators and conducted metaregression analyses using mixed-effects models.

RESULTS

On the basis of 22 studies with a total of 464 participants with ASD, we obtained a small-to-medium positive effect size (g = 0.26) in support of enhanced pitch perception in ASD. Moreover, the mean age and nonverbal IQ of participants were found to significantly moderate the between-studies heterogeneity.

CONCLUSIONS

Our study provides the first meta-analysis on auditory pitch perception in ASD and demonstrates the existence of different developmental trajectories between autistic individuals and neurotypicals. In addition to age, nonverbal ability is found to be a significant contributor to the lower level/local processing bias in ASD. We highlight the need for further investigation of pitch perception in ASD under challenging listening conditions. Future neurophysiological and brain imaging studies with a longitudinal design are also needed to better understand the underlying neural mechanisms of atypical pitch processing in ASD and to help guide auditory-based interventions for improving language and social functioning.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21614271.

Electroen cephalography correlates of word and non-word listening in children with specific language impairment: An observational study20F0

Auditory processing in children diagnosed with speech and language impairment (SLI) is atypical and characterized by reduced brain activation compared to typically developing (TD) children. In typical speech and language development processes, frontal, temporal, and posterior regions are engaged during single-word listening, while for non-word listening, it is highly unlikely that perceiving or speaking them is not followed by frequent neurones' activation enough to form stable network connections. This study aimed to investigate the electrophysiological cortical activity of alpha rhythm while listening words and non-words in children with SLI compared to TD children. The participants were 50 children with SLI, aged 4 to 6, and 50 age-related TD children. Groups were divided into 2 subgroups: first subgroup - children aged 4.0 to 5.0 years old (E = 25, C = 25) and second subgroup - children aged 5.0 to 6.0 years old (E = 25, C = 25). The younger children's group did not show statistically significant differences in alpha spectral power in word or non-word listening. In contrast, in the older age group for word and non-word listening, differences were present in the prefrontal, temporal, and parieto-occipital regions bilaterally. Children with SLI showed a certain lack of alpha desynchronization in word and non-word listening compared with TD children. Non-word perception arouses more brain regions because of the unknown presence of the word stimuli. The lack of adequate alpha desynchronization is consistent with established difficulties in lexical and phonological processing at the behavioral level in children with SLI.

Specific disruption of the ventral anterior temporo-frontal network reveals key implications for language comprehension and cognition

Recent investigations have raised the question of the role of the anterior lateral temporal cortex in language processing (ventral language network). Here we present the language and overall cognitive performance of a rare male patient with chronic middle cerebral artery cerebrovascular accident with a well-documented lesion restricted to the anterior temporal cortex and its connections via the extreme capsule with the pars triangularis of the inferior frontal gyrus (i.e. Broca’s region). The performance of this unique patient is compared with that of two chronic middle cerebral artery cerebrovascular accident male patients with damage to the classic dorsal posterior temporo-parietal language system. Diffusion tensor imaging is used to reconstruct the relevant white matter tracts of the three patients, which are also compared with those of 10 healthy individuals. The patient with the anterior temporo-frontal lesion presents with flawless and fluent speech, but selective impairment in accessing lexico-semantic information, in sharp contrast to the impairments in speech, sentence comprehension and repetition observed after lesions to the classic dorsal language system. The present results underline the contribution of the ventral language stream in lexico-semantic processing and higher cognitive functions, such as active selective controlled retrieval.

Neuropsychological profiling and clinical DTI of three stroke patients highlight the importance of the ventral language system in normal language comprehension and cognition.

Just give it time: Differential effects of disruption and delay on perceptual learning

Speech perception and production are critical skills when acquiring a new language. However, the nature of the relationship between these two processes is unclear, particularly for non-native speech sound contrasts. Although it has been assumed that perception and production are supportive, recent evidence has demonstrated that, under some circumstances, production can disrupt perceptual learning. Specifically, producing the to-be-learned contrast on each trial can disrupt perceptual learning of that contrast. Here, we treat speech perception and speech production as separate tasks. From this perspective, perceptual learning studies that include a production component on each trial create a task switch. We report two experiments that test how task switching can disrupt perceptual learning. One experiment demonstrates that the disruption caused by switching to production is sensitive to time delays: Increasing the delay between perception and production on a trial can reduce and even eliminate disruption of perceptual learning. The second experiment shows that if a task other than producing the to-be-learned contrast is imposed, the task-switching component of disruption is not influenced by a delay. These experiments provide a new understanding of the relationship between speech perception and speech production, and clarify conditions under which the two cooperate or compete.

Enhancement of speech-in-noise comprehension through vibrotactile stimulation at the syllabic rate

Syllables are important building blocks of speech. They occur at a rate between 4 and 8 Hz, corresponding to the theta frequency range of neural activity in the cerebral cortex. When listening to speech, the theta activity becomes aligned to the syllabic rhythm, presumably aiding in parsing a speech signal into distinct syllables. However, this neural activity cannot only be influenced by sound, but also by somatosensory information. Here, we show that the presentation of vibrotactile signals at the syllabic rate can enhance the comprehension of speech in background noise. We further provide evidence that this multisensory enhancement of speech comprehension reflects the multisensory integration of auditory and tactile information in the auditory cortex.

Speech unfolds over distinct temporal scales, in particular, those related to the rhythm of phonemes, syllables, and words. When a person listens to continuous speech, the syllabic rhythm is tracked by neural activity in the theta frequency range. The tracking plays a functional role in speech processing: Influencing the theta activity through transcranial current stimulation, for instance, can impact speech perception. The theta-band activity in the auditory cortex can also be modulated through the somatosensory system, but the effect on speech processing has remained unclear. Here, we show that vibrotactile feedback presented at the rate of syllables can modulate and, in fact, enhance the comprehension of a speech signal in background noise. The enhancement occurs when vibrotactile pulses occur at the perceptual center of the syllables, whereas a temporal delay between the vibrotactile signals and the speech stream can lead to a lower level of speech comprehension. We further investigate the neural mechanisms underlying the audiotactile integration through electroencephalographic (EEG) recordings. We find that the audiotactile stimulation modulates the neural response to the speech rhythm, as well as the neural response to the vibrotactile pulses. The modulations of these neural activities reflect the behavioral effects on speech comprehension. Moreover, we demonstrate that speech comprehension can be predicted by particular aspects of the neural responses. Our results evidence a role of vibrotactile information for speech processing and may have applications in future auditory prosthesis.

Multi-modal imaging of the auditory-larynx motor network for voicing perception

Voicing is one of the most important characteristics of phonetic speech sounds. Despite its importance, voicing perception mechanisms remain largely unknown. To explore auditory-motor networks associated with voicing perception, we firstly examined the brain regions that showed common activities for voicing production and perception using functional magnetic resonance imaging. Results indicated that the auditory and speech motor areas were activated with the operculum parietale 4 (OP4) during both voicing production and perception. Secondly, we used a magnetoencephalography and examined the dynamical functional connectivity of the auditory-motor networks during a perceptual categorization task of /da/-/ta/ continuum stimuli varying in voice onset time (VOT) from 0 to 40 ms in 10 ms steps. Significant functional connectivities from the auditory cortical regions to the larynx motor area via OP4 were observed only when perceiving the stimulus with VOT 30 ms. In addition, regional activity analysis showed that the neural representation of VOT in the auditory cortical regions was mostly correlated with categorical perception of voicing but did not reflect the perception of stimulus with VOT 30 ms. We suggest that the larynx motor area, which is considered to play a crucial role in voicing production, contributes to categorical perception of voicing by complementing the temporal processing in the auditory cortical regions.

The role of disrupted functional connectivity in aphasia

Language is one of the most complex and specialized higher cognitive processes. Brain damage to the distributed, primarily left-lateralized language network can result in aphasia, a neurologic disorder characterized by receptive and/or expressive deficits in spoken and/or written language. Most often, aphasia is the consequence of stroke-termed poststroke aphasia (PSA)-yet, aphasia can also manifest due to neurodegenerative disease, specifically, a disorder called primary progressive aphasia (PPA). In recent years, functional connectivity neuroimaging studies have provided emerging evidence supporting theories regarding the relationships between language impairments, structural brain damage, and functional network properties in these two disorders. This chapter reviews the current evidence for the "network phenotype of stroke injury" hypothesis (Siegel et al., 2016) as it pertains to PSA and the "network degeneration hypothesis" (Seeley et al., 2009) as it pertains to PPA. Methodologic considerations for functional connectivity studies, limitations of the current functional connectivity literature in aphasia, and future directions are also discussed.

Cross-modal integration and plasticity in the superior temporal cortex

In congenitally deaf people, temporal regions typically believed to be primarily auditory enhance their response to nonauditory information. The neural mechanisms and functional principles underlying this phenomenon, as well as its impact on auditory recovery after sensory restoration, yet remain debated. In this chapter, we demonstrate that the cross-modal recruitment of temporal regions by visual inputs in congenitally deaf people follows organizational principles known to be present in the hearing brain. We propose that the functional and structural mechanisms allowing optimal convergence of multisensory information in the temporal cortex of hearing people also provide the neural scaffolding for feeding visual or tactile information into the deafened temporal areas. Innate in their nature, such anatomo-functional links between the auditory and other sensory systems would represent the common substrate of both early multisensory integration and expression of selective cross-modal plasticity in the superior temporal cortex.

Hierarchical cortical networks of "voice patches" for processing voices in human brain

Humans have an extraordinary ability to recognize and differentiate voices. It is yet unclear whether voices are uniquely processed in the human brain. To explore the underlying neural mechanisms of voice processing, we recorded electrocorticographic signals from intracranial electrodes in epilepsy patients while they listened to six different categories of voice and nonvoice sounds. Subregions in the temporal lobe exhibited preferences for distinct voice stimuli, which were defined as "voice patches." Latency analyses suggested a dual hierarchical organization of the voice patches. We also found that voice patches were functionally connected under both task-engaged and resting states. Furthermore, the left motor areas were coactivated and correlated with the temporal voice patches during the sound-listening task. Taken together, this work reveals hierarchical cortical networks in the human brain for processing human voices.

Distinct neuron populations for simple and compound calls in the primary auditory cortex of awake marmosets

Marmosets are highly social non-human primates that live in families. They exhibit rich vocalization, but the neural basis underlying this complex vocal communication is largely unknown. Here we report the existence of specific neuron populations in marmoset A1 that respond selectively to distinct simple or compound calls made by conspecific marmosets. These neurons were spatially dispersed within A1 but distinct from those responsive to pure tones. Call-selective responses were markedly diminished when individual domains of the call were deleted or the domain sequence was altered, indicating the importance of the global rather than local spectral-temporal properties of the sound. Compound call-selective responses also disappeared when the sequence of the two simple-call components was reversed or their interval was extended beyond 1 s. Light anesthesia largely abolished call-selective responses. Our findings demonstrate extensive inhibitory and facilitatory interactions among call-evoked responses, and provide the basis for further study of circuit mechanisms underlying vocal communication in awake non-human primates.

Decoding Imagined Speech Based on Deep Metric Learning for Intuitive BCI Communication

Imagined speech is a highly promising paradigm due to its intuitive application and multiclass scalability in the field of brain-computer interfaces. However, optimal feature extraction and classifiers have not yet been established. Furthermore, retraining still requires a large number of trials when new classes are added. The aim of this study is (i) to increase the classification performance for imagined speech and (ii) to apply a new class using a pretrained classifier with a small number of trials. We propose a novel framework based on deep metric learning that learns the distance by comparing the similarity between samples. We also applied the instantaneous frequency and spectral entropy used for speech signals to electroencephalography signals during imagined speech. The method was evaluated on two public datasets (6-class Coretto DB and 5-class BCI Competition DB). We achieved a 6-class accuracy of 45.00 ± 3.13% and a 5-class accuracy of 48.10 ± 3.68% using the proposed method, which significantly outperformed state-of-the-art methods. Additionally, we verified that the new class could be detected through incremental learning with a small number of trials. As a result, the average accuracy is 44.50 ± 0.26% for Coretto DB and 47.12 ± 0.27% for BCI Competition DB, which shows similar accuracy to baseline accuracy without incremental learning. Our results have shown that the accuracy can be greatly improved even with a small number of trials by selecting appropriate features from imagined speech. The proposed framework could be directly used to help construct an extensible intuitive communication system based on brain-computer interfaces.

Reading direct speech quotes increases theta phase-locking: Evidence for cortical tracking of inner speech?

Growing evidence shows that theta-band (4-7 Hz) activity in the auditory cortex phase-locks to rhythms of overt speech. Does theta activity also encode the rhythmic dynamics of inner speech? Previous research established that silent reading of direct speech quotes (e.g., Mary said: "This dress is lovely!") elicits more vivid inner speech than indirect speech quotes (e.g., Mary said that the dress was lovely). As we cannot directly track the phase alignment between theta activity and inner speech over time, we used EEG to measure the brain's phase-locked responses to the onset of speech quote reading. We found that direct (vs. indirect) quote reading was associated with increased theta phase synchrony over trials at 250-500 ms post-reading onset, with sources of the evoked activity estimated in the speech processing network. An eye-tracking control experiment confirmed that increased theta phase synchrony in direct quote reading was not driven by eye movement patterns, and more likely reflects synchronous phase resetting at the onset of inner speech. These findings suggest a functional role of theta phase modulation in reading-induced inner speech.

The Same Ultra-Rapid Parallel Brain Dynamics Underpin the Production and Perception of Speech

The temporal dynamics by which linguistic information becomes available is one of the key properties to understand how language is organized in the brain. An unresolved debate between different brain language models is whether words, the building blocks of language, are activated in a sequential or parallel manner. In this study, we approached this issue from a novel perspective by directly comparing the time course of word component activation in speech production versus perception. In an overt object naming task and a passive listening task, we analyzed with mixed linear models at the single-trial level the event-related brain potentials elicited by the same lexico-semantic and phonological word knowledge in the two language modalities. Results revealed that both word components manifested simultaneously as early as 75 ms after stimulus onset in production and perception; differences between the language modalities only became apparent after 300 ms of processing. The data provide evidence for ultra-rapid parallel dynamics of language processing and are interpreted within a neural assembly framework where words recruit the same integrated cell assemblies across production and perception. These word assemblies ignite early on in parallel and only later on reverberate in a behavior-specific manner.

Neurophysiological Indices of Audiovisual Speech Processing Reveal a Hierarchy of Multisensory Integration Effects

Seeing a speaker's face benefits speech comprehension, especially in challenging listening conditions. This perceptual benefit is thought to stem from the neural integration of visual and auditory speech at multiple stages of processing, whereby movement of a speaker's face provides temporal cues to auditory cortex, and articulatory information from the speaker's mouth can aid recognizing specific linguistic units (e.g., phonemes, syllables). However, it remains unclear how the integration of these cues varies as a function of listening conditions. Here, we sought to provide insight on these questions by examining EEG responses in humans (males and females) to natural audiovisual (AV), audio, and visual speech in quiet and in noise. We represented our speech stimuli in terms of their spectrograms and their phonetic features and then quantified the strength of the encoding of those features in the EEG using canonical correlation analysis (CCA). The encoding of both spectrotemporal and phonetic features was shown to be more robust in AV speech responses than what would have been expected from the summation of the audio and visual speech responses, suggesting that multisensory integration occurs at both spectrotemporal and phonetic stages of speech processing. We also found evidence to suggest that the integration effects may change with listening conditions; however, this was an exploratory analysis and future work will be required to examine this effect using a within-subject design. These findings demonstrate that integration of audio and visual speech occurs at multiple stages along the speech processing hierarchy.SIGNIFICANCE STATEMENT During conversation, visual cues impact our perception of speech. Integration of auditory and visual speech is thought to occur at multiple stages of speech processing and vary flexibly depending on the listening conditions. Here, we examine audiovisual (AV) integration at two stages of speech processing using the speech spectrogram and a phonetic representation, and test how AV integration adapts to degraded listening conditions. We find significant integration at both of these stages regardless of listening conditions. These findings reveal neural indices of multisensory interactions at different stages of processing and provide support for the multistage integration framework.

Speech-evoked brain activity is more robust to competing speech when it is spoken by someone familiar

When speech is masked by competing sound, people are better at understanding what is said if the talker is familiar compared to unfamiliar. The benefit is robust, but how does processing of familiar voices facilitate intelligibility? We combined high-resolution fMRI with representational similarity analysis to quantify the difference in distributed activity between clear and masked speech. We demonstrate that brain representations of spoken sentences are less affected by a competing sentence when they are spoken by a friend or partner than by someone unfamiliar-effectively, showing a cortical signal-to-noise ratio (SNR) enhancement for familiar voices. This effect correlated with the familiar-voice intelligibility benefit. We functionally parcellated auditory cortex, and found that the most prominent familiar-voice advantage was manifest along the posterior superior and middle temporal gyri. Overall, our results demonstrate that experience-driven improvements in intelligibility are associated with enhanced multivariate pattern activity in posterior temporal cortex.

Directed functional connectivity of the hippocampus in patients with presbycusis

Presbycusis, associated with a diminished quality of life characterized by bilateral sensorineural hearing loss at high frequencies, has become an increasingly critical public health problem. This study aimed to identify directed functional connectivity (FC) of the hippocampus in patients with presbycusis and to explore the causes if the directed functional connections of the hippocampus were disrupted. Presbycusis patients (n = 32) and age-, sex-, and education-matched healthy controls (n = 40) were included in this study. The seed regions of bilateral hippocampus were selected to identify directed FC in patients with presbycusis using Granger causality analysis (GCA) approach. Correlation analyses were conducted to detect the associations of disrupted directed FC of hippocampus with clinical measures of presbycusis. Compared to healthy controls, decreased directed FC between inferior parietal lobule, insula, right supplementary motor area, middle temporal gyrus and hippocampus were detected in presbycusis patients. Furthermore, a negative correlation between TMB score and the decline of directed FC from left inferior parietal lobule to left hippocampus (r = -0.423, p = 0.025) and from right inferior parietal lobule to right hippocampus (r = -0.516, p = 0.005) were also observed. The decreased directed functional connections of the hippocampus were detected in patients with presbycusis, which was associated with specific cognitive performance. This study mainly emphasizes the crucial role of hippocampus in presbycusis and will enhance our understanding of the neuropathological mechanisms of presbycusis.

Cortical Regions Activated by Spectrally Degraded Speech in Adults With Single Sided Deafness or Bilateral Normal Hearing

Those with profound sensorineural hearing loss from single sided deafness (SSD) generally experience greater cognitive effort and fatigue in adverse sound environments. We studied cases with right ear, SSD compared to normal hearing (NH) individuals. SSD cases were significantly less correct in naming last words in spectrally degraded 8- and 16-band vocoded sentences, despite high semantic predictability. Group differences were not significant for less intelligible 4-band sentences, irrespective of predictability. SSD also had diminished BOLD percent signal changes to these same sentences in left hemisphere (LH) cortical regions of early auditory, association auditory, inferior frontal, premotor, inferior parietal, dorsolateral prefrontal, posterior cingulate, temporal-parietal-occipital junction, and posterior opercular. Cortical regions with lower amplitude responses in SSD than NH were mostly components of a LH language network, previously noted as concerned with speech recognition. Recorded BOLD signal magnitudes were averages from all vertices within predefined parcels from these cortex regions. Parcels from different regions in SSD showed significantly larger signal magnitudes to sentences of greater intelligibility (e.g., 8- or 16- vs. 4-band) in all except early auditory and posterior cingulate cortex. Significantly lower response magnitudes occurred in SSD than NH in regions prior studies found responsible for phonetics and phonology of speech, cognitive extraction of meaning, controlled retrieval of word meaning, and semantics. The findings suggested reduced activation of a LH fronto-temporo-parietal network in SSD contributed to difficulty processing speech for word meaning and sentence semantics. Effortful listening experienced by SSD might reflect diminished activation to degraded speech in the affected LH language network parcels. SSD showed no compensatory activity in matched right hemisphere parcels.

Evaluating cortical responses to speech in children: A functional near-infrared spectroscopy (fNIRS) study

Functional neuroimaging of speech processing has both research and clinical potential. This work is facilitating an ever-increasing understanding of the complex neural mechanisms involved in the processing of speech. Neural correlates of speech understanding also have potential clinical value, especially for infants and children, in whom behavioural assessments can be unreliable. Such measures would not only benefit normally hearing children experiencing speech and language delay, but also hearing impaired children with and without hearing devices. In the current study, we examined cortical correlates of speech intelligibility in normally hearing paediatric listeners. Cortical responses were measured using functional near-infrared spectroscopy (fNIRS), a non-invasive neuroimaging technique that is fully compatible with hearing devices, including cochlear implants. In nineteen normally hearing children (aged 6 - 13 years) we measured activity in temporal and frontal cortex bilaterally whilst participants listened to both clear- and noise-vocoded sentences targeting four levels of speech intelligibility. Cortical activation in superior temporal and inferior frontal cortex was generally stronger in the left hemisphere than in the right. Activation in left superior temporal cortex grew monotonically with increasing speech intelligibility. In the same region, we identified a trend towards greater activation on correctly vs. incorrectly perceived trials, suggesting a possible sensitivity to speech intelligibility per se, beyond sensitivity to changing acoustic properties across stimulation conditions. Outside superior temporal cortex, we identified other regions in which fNIRS responses varied with speech intelligibility. For example, channels overlying posterior middle temporal regions in the right hemisphere exhibited relative deactivation during sentence processing (compared to a silent baseline condition), with the amplitude of that deactivation being greater in more difficult listening conditions. This finding may represent sensitivity to components of the default mode network in lateral temporal regions, and hence effortful listening in normally hearing paediatric listeners. Our results indicate that fNIRS has the potential to provide an objective marker of speech intelligibility in normally hearing children. Should these results be found to apply to individuals experiencing language delay or to those listening through a hearing device, such as a cochlear implant, fNIRS may form the basis of a clinically useful measure of speech understanding.

Pre- and post-target cortical processes predict speech-in-noise performance

Understanding speech in noise (SiN) is a complex task that recruits multiple cortical subsystems. There is a variance in individuals' ability to understand SiN that cannot be explained by simple hearing profiles, which suggests that central factors may underlie the variance in SiN ability. Here, we elucidated a few cortical functions involved during a SiN task and their contributions to individual variance using both within- and across-subject approaches. Through our within-subject analysis of source-localized electroencephalography, we investigated how acoustic signal-to-noise ratio (SNR) alters cortical evoked responses to a target word across the speech recognition areas, finding stronger responses in left supramarginal gyrus (SMG, BA40 the dorsal lexicon area) with quieter noise. Through an individual differences approach, we found that listeners show different neural sensitivity to the background noise and target speech, reflected in the amplitude ratio of earlier auditory-cortical responses to speech and noise, named as an internal SNR. Listeners with better internal SNR showed better SiN performance. Further, we found that the post-speech time SMG activity explains a further amount of variance in SiN performance that is not accounted for by internal SNR. This result demonstrates that at least two cortical processes contribute to SiN performance independently: pre-target time processing to attenuate neural representation of background noise and post-target time processing to extract information from speech sounds.

Total GABA level in human auditory cortex is associated with speech-in-noise understanding in older age

Speech-in-noise (SIN) understanding often becomes difficult for older adults because of impaired hearing and aging-related changes in central auditory processing. Central auditory processing depends on a fine balance between excitatory and inhibitory neural mechanisms, which may be upset in older age by a change in the level of the inhibitory neurotransmitter gamma-aminobutyric acid (GABA). In this study, we used MEGA-PRESS magnetic resonance spectroscopy (MRS) to estimate GABA levels in both the left and right auditory cortices of young and older adults. We found that total auditory GABA levels were lower in older compared to young adults. To understand the relationship between GABA and hearing function, we correlated GABA levels with hearing loss and SIN performance. In older adults, the GABA level in the right auditory cortex was correlated with age and SIN performance. The relationship between chronological age and SIN loss was partially mediated by the GABA level in the right auditory cortex. These findings support the hypothesis that inhibitory mechanisms in the auditory system are reduced in aging, and this reduction relates to functional impairments.

Linguistic alignment and theory of mind impairments in schizophrenia patients' dialogic interactions

BACKGROUND

Impairments of contextual processing and theory of mind (ToM) have both been offered as accounts of the deviant language characterising formal thought disorder (FTD) in schizophrenia. This study investigated these processes in patients' dialogue. We predicted that FTD patients would show a decrement in linguistic alignment, associated with impaired ToM in dialogue.

METHODS

Speech samples were elicited via participation in an interactive computer-based task and a semi-structured interview to assess contextual processing abilities and ToM skills in dialogue, respectively, and from an interactive card-sorting task to measure syntactic alignment. Degree of alignment in dialogue and the syntactic task, and evidence of ToM in (i) dialogue and (ii) a traditional ToM task were compared across schizophrenia patients with FTD (n = 21), non-FTD patients (n = 22) and healthy controls (n = 21).

RESULTS

FTD patients showed less alignment than the other two groups in dialogue, and than healthy controls on the syntactic task. FTD patients showed poorer performance on the ToM task than the other two groups, but only compared to the healthy controls in dialogue. The FTD group's degree of alignment in dialogue was correlated with ToM performance in dialogue but not with the traditional ToM task or with syntactic alignment.

CONCLUSIONS

In dialogue, FTD patients demonstrate an impairment in employing available contextual information to facilitate their own subsequent production, which is associated with a ToM deficit. These findings indicate that a contextual processing deficit impacts on exploiting representations via the production system impoverishing the ability to make predictions about upcoming utterances in dialogue.

White matter-Maximien Parchappe and the integration of articulate language

The Imperial Academy of Medicine of Paris met in the spring of 1865 to discuss the localization of speech. One of the participants was Maximien Parchappe (1800-1866), an alienist whose research interests lay in the cerebral cortex. This article addresses Maximien Parchappe's concept that the cognitive elements of language-such as the translation of thoughts into words, the will to express them, and the means to do so-reside within the cortical gray matter, and that they are integrated through white-matter fibers. In so doing, Parchappe anticipated Carl Wernicke's linking of the posterior aspects of the dominant frontal and temporal lobes in verbal expression, and Jules Dejerine's linking of the angular gyrus and Wernicke's area in the understanding of written language. Functional imaging has revived interest in language as a network of neuronal aggregates and has given new relevance to Parchappe's concept of the functional organization of language.

Combined Brain-Perfusion SPECT and EEG Measurements Suggest Distinct Strategies for Speech Comprehension in CI Users With Higher and Lower Performance

Cochlear implantation constitutes a successful therapy of inner ear deafness, with the majority of patients showing good outcomes. There is, however, still some unexplained variability in outcomes with a number of cochlear-implant (CI) users, showing major limitations in speech comprehension. The current study used a multimodal diagnostic approach combining single-photon emission computed tomography (SPECT) and electroencephalography (EEG) to examine the mechanisms underlying speech processing in postlingually deafened CI users (N = 21). In one session, the participants performed a speech discrimination task, during which a 96-channel EEG was recorded and the perfusions marker 99mTc-HMPAO was injected intravenously. The SPECT scan was acquired 1.5 h after injection to measure the cortical activity during the speech task. The second session included a SPECT scan after injection without stimulation at rest. Analysis of EEG and SPECT data showed N400 and P600 event-related potentials (ERPs) particularly evoked by semantic violations in the sentences, and enhanced perfusion in a temporo-frontal network during task compared to rest, involving the auditory cortex bilaterally and Broca's area. Moreover, higher performance in testing for word recognition and verbal intelligence strongly correlated to the activation in this network during the speech task. However, comparing CI users with lower and higher speech intelligibility [median split with cutoff + 7.6 dB signal-to-noise ratio (SNR) in the Göttinger sentence test] revealed for CI users with higher performance additional activations of parietal and occipital regions and for those with lower performance stronger activation of superior frontal areas. Furthermore, SPECT activity was tightly coupled with EEG and cognitive abilities, as indicated by correlations between (1) cortical activation and the amplitudes in EEG, N400 (temporal and occipital areas)/P600 (parietal and occipital areas) and (2) between cortical activation in left-sided temporal and bilateral occipital/parietal areas and working memory capacity. These results suggest the recruitment of a temporo-frontal network in CI users during speech processing and a close connection between ERP effects and cortical activation in CI users. The observed differences in speech-evoked cortical activation patterns for CI users with higher and lower speech intelligibility suggest distinct processing strategies during speech rehabilitation with CI.

The time-course of speech perception revealed by temporally-sensitive neural measures

Recent advances in cognitive neuroscience have provided a detailed picture of the early time-course of speech perception. In this review, we highlight this work, placing it within the broader context of research on the neurobiology of speech processing, and discuss how these data point us toward new models of speech perception and spoken language comprehension. We focus, in particular, on temporally-sensitive measures that allow us to directly measure early perceptual processes. Overall, the data provide support for two key principles: (a) speech perception is based on gradient representations of speech sounds and (b) speech perception is interactive and receives input from higher-level linguistic context at the earliest stages of cortical processing. Implications for models of speech processing and the neurobiology of language more broadly are discussed. This article is categorized under: Psychology > Language Psychology > Perception and Psychophysics Neuroscience > Cognition.

Distinct auditory and visual tool regions with multisensory response properties in human parietal cortex

Left parietal cortex has been associated with the human-specific ability of sophisticated tool use. Yet, it is unclear how tool information is represented across senses. Here, we compared auditory and visual tool-specific activations within healthy human subjects to probe the relation of tool-specific networks, uni- and multisensory response properties, and functional and structural connectivity using functional and diffusion-weighted MRI. In each subject, we identified an auditory tool network with regions in left anterior inferior parietal cortex (aud-aIPL), bilateral posterior lateral sulcus, and left inferior precentral sulcus, and a visual tool network with regions in left aIPL (vis-aIPL) and bilateral inferior temporal gyrus. Aud-aIPL was largely separate and anterior/inferior from vis-aIPL, with varying degrees of overlap across subjects. Both regions displayed a strong preference for tools versus other stimuli presented within the same modality. Despite their modality preference, aud-aIPL and vis-aIPL and a region in left inferior precentral sulcus displayed multisensory response properties, as revealed in multivariate analyses. Thus, two largely separate tool networks are engaged by the visual and auditory modalities with nodes in parietal and prefrontal cortex potentially integrating information across senses. The diversification of tool processing in human parietal cortex underpins its critical role in complex object processing.

Neural Correlates of Phonetic Adaptation as Induced by Lexical and Audiovisual Context

When speech perception is difficult, one way listeners adjust is by reconfiguring phoneme category boundaries, drawing on contextual information. Both lexical knowledge and lipreading cues are used in this way, but it remains unknown whether these two differing forms of perceptual learning are similar at a neural level. This study compared phoneme boundary adjustments driven by lexical or audiovisual cues, using ultra-high-field 7-T fMRI. During imaging, participants heard exposure stimuli and test stimuli. Exposure stimuli for lexical retuning were audio recordings of words, and those for audiovisual recalibration were audio-video recordings of lip movements during utterances of pseudowords. Test stimuli were ambiguous phonetic strings presented without context, and listeners reported what phoneme they heard. Reports reflected phoneme biases in preceding exposure blocks (e.g., more reported /p/ after /p/-biased exposure). Analysis of corresponding brain responses indicated that both forms of cue use were associated with a network of activity across the temporal cortex, plus parietal, insula, and motor areas. Audiovisual recalibration also elicited significant occipital cortex activity despite the lack of visual stimuli. Activity levels in several ROIs also covaried with strength of audiovisual recalibration, with greater activity accompanying larger recalibration shifts. Similar activation patterns appeared for lexical retuning, but here, no significant ROIs were identified. Audiovisual and lexical forms of perceptual learning thus induce largely similar brain response patterns. However, audiovisual recalibration involves additional visual cortex contributions, suggesting that previously acquired visual information (on lip movements) is retrieved and deployed to disambiguate auditory perception.

Better Phonological Short-Term Memory Is Linked to Improved Cortical Memory Representations for Word Forms and Better Word Learning

Language learning relies on both short-term and long-term memory. Phonological short-term memory (pSTM) is thought to play an important role in the learning of novel word forms. However, language learners may differ in their ability to maintain word representations in pSTM during interfering auditory input. We used magnetoencephalography (MEG) to investigate how pSTM capacity in better and poorer pSTM groups is linked to language learning and the maintenance of pseudowords in pSTM. In particular, MEG was recorded while participants maintained pseudowords in pSTM by covert speech rehearsal, and while these brain representations were probed by presenting auditory pseudowords with first or third syllables matching or mismatching the rehearsed item. A control condition included identical stimuli but no rehearsal. Differences in response strength between matching and mismatching syllables were interpreted as the phonological mapping negativity (PMN). While PMN for the first syllable was found in both groups, it was observed for the third syllable only in the group with better pSTM. This suggests that individuals with better pSTM maintained representations of trisyllabic pseudowords more accurately during interference than individuals with poorer pSTM. Importantly, the group with better pSTM learned words faster in a paired-associate word learning task, linking the PMN findings to language learning.

Constrained Structure of Ancient Chinese Poetry Facilitates Speech Content Grouping

Ancient Chinese poetry is constituted by structured language that deviates from ordinary language usage [1, 2]; its poetic genres impose unique combinatory constraints on linguistic elements [3]. How does the constrained poetic structure facilitate speech segmentation when common linguistic [4-8] and statistical cues [5, 9] are unreliable to listeners in poems? We generated artificial Jueju, which arguably has the most constrained structure in ancient Chinese poetry, and presented each poem twice as an isochronous sequence of syllables to native Mandarin speakers while conducting magnetoencephalography (MEG) recording. We found that listeners deployed their prior knowledge of Jueju to build the line structure and to establish the conceptual flow of Jueju. Unprecedentedly, we found a phase precession phenomenon indicating predictive processes of speech segmentation-the neural phase advanced faster after listeners acquired knowledge of incoming speech. The statistical co-occurrence of monosyllabic words in Jueju negatively correlated with speech segmentation, which provides an alternative perspective on how statistical cues facilitate speech segmentation. Our findings suggest that constrained poetic structures serve as a temporal map for listeners to group speech contents and to predict incoming speech signals. Listeners can parse speech streams by using not only grammatical and statistical cues but also their prior knowledge of the form of language. VIDEO ABSTRACT.

Understanding rostral-caudal auditory cortex contributions to auditory perception

There are functional and anatomical distinctions between the neural systems involved in the recognition of sounds in the environment and those involved in the sensorimotor guidance of sound production and the spatial processing of sound. Evidence for the separation of these processes has historically come from disparate literatures on the perception and production of speech, music and other sounds. More recent evidence indicates that there are computational distinctions between the rostral and caudal primate auditory cortex that may underlie functional differences in auditory processing. These functional differences may originate from differences in the response times and temporal profiles of neurons in the rostral and caudal auditory cortex, suggesting that computational accounts of primate auditory pathways should focus on the implications of these temporal response differences.

Neuro-current response functions: A unified approach to MEG source analysis under the continuous stimuli paradigm

Characterizing the neural dynamics underlying sensory processing is one of the central areas of investigation in systems and cognitive neuroscience. Neuroimaging techniques such as magnetoencephalography (MEG) and Electroencephalography (EEG) have provided significant insights into the neural processing of continuous stimuli, such as speech, thanks to their high temporal resolution. Existing work in the context of auditory processing suggests that certain features of speech, such as the acoustic envelope, can be used as reliable linear predictors of the neural response manifested in M/EEG. The corresponding linear filters are referred to as temporal response functions (TRFs). While the functional roles of specific components of the TRF are well-studied and linked to behavioral attributes such as attention, the cortical origins of the underlying neural processes are not as well understood. In this work, we address this issue by estimating a linear filter representation of cortical sources directly from neuroimaging data in the context of continuous speech processing. To this end, we introduce Neuro-Current Response Functions (NCRFs), a set of linear filters, spatially distributed throughout the cortex, that predict the cortical currents giving rise to the observed ongoing MEG (or EEG) data in response to continuous speech. NCRF estimation is cast within a Bayesian framework, which allows unification of the TRF and source estimation problems, and also facilitates the incorporation of prior information on the structural properties of the NCRFs. To generalize this analysis to M/EEG recordings which lack individual structural magnetic resonance (MR) scans, NCRFs are extended to free-orientation dipoles and a novel regularizing scheme is put forward to lessen reliance on fine-tuned coordinate co-registration. We present a fast estimation algorithm, which we refer to as the Champ-Lasso algorithm, by leveraging recent advances in optimization, and demonstrate its utility through application to simulated and experimentally recorded MEG data under auditory experiments. Our simulation studies reveal significant improvements over existing methods that typically operate in a two-stage fashion, in terms of spatial resolution, response function reconstruction, and recovering dipole orientations. The analysis of experimentally-recorded MEG data without MR scans corroborates existing findings, but also delineates the distinct cortical distribution of the underlying neural processes at high spatiotemporal resolution. In summary, we provide a principled modeling and estimation paradigm for MEG source analysis tailored to extracting the cortical origin of electrophysiological responses to continuous stimuli.

Happy you, happy me: expressive changes on a stranger's voice recruit faster implicit processes than self-produced expressions

In social interactions, people have to pay attention both to the ‘what’ and ‘who’. In particular, expressive changes heard on speech signals have to be integrated with speaker identity, differentiating e.g. self- and other-produced signals. While previous research has shown that self-related visual information processing is facilitated compared to non-self stimuli, evidence in the auditory modality remains mixed. Here, we compared electroencephalography (EEG) responses to expressive changes in sequence of self- or other-produced speech sounds using a mismatch negativity (MMN) passive oddball paradigm. Critically, to control for speaker differences, we used programmable acoustic transformations to create voice deviants that differed from standards in exactly the same manner, making EEG responses to such deviations comparable between sequences. Our results indicate that expressive changes on a stranger’s voice are highly prioritized in auditory processing compared to identical changes on the self-voice. Other-voice deviants generate earlier MMN onset responses and involve stronger cortical activations in a left motor and somatosensory network suggestive of an increased recruitment of resources for less internally predictable, and therefore perhaps more socially relevant, signals.

Shared premotor activity in spoken and written communication

The aim of the present study was to uncover a possible common neural organizing principle in spoken and written communication, through the coupling of perceptual and motor representations. In order to identify possible shared neural substrates for processing the basic units of spoken and written language, a sparse sampling fMRI acquisition protocol was performed on the same subjects in two experimental sessions with similar sets of letters being read and written and of phonemes being heard and orally produced. We found evidence of common premotor regions activated in spoken and written language, both in perception and in production. The location of those brain regions was confined to the left lateral and medial frontal cortices, at locations corresponding to the premotor cortex, inferior frontal cortex and supplementary motor area. Interestingly, the speaking and writing tasks also appeared to be controlled by largely overlapping networks, possibly indicating some domain general cognitive processing. Finally, the spatial distribution of individual activation peaks further showed more dorsal and more left-lateralized premotor activations in written than in spoken language.

Task-General and Acoustic-Invariant Neural Representation of Speech Categories in the Human Brain

A significant neural challenge in speech perception includes extracting discrete phonetic categories from continuous and multidimensional signals despite varying task demands and surface-acoustic variability. While neural representations of speech categories have been previously identified in frontal and posterior temporal-parietal regions, the task dependency and dimensional specificity of these neural representations are still unclear. Here, we asked native Mandarin participants to listen to speech syllables carrying 4 distinct lexical tone categories across passive listening, repetition, and categorization tasks while they underwent functional magnetic resonance imaging (fMRI). We used searchlight classification and representational similarity analysis (RSA) to identify the dimensional structure underlying neural representation across tasks and surface-acoustic properties. Searchlight classification analyses revealed significant "cross-task" lexical tone decoding within the bilateral superior temporal gyrus (STG) and left inferior parietal lobule (LIPL). RSA revealed that the LIPL and LSTG, in contrast to the RSTG, relate to 2 critical dimensions (pitch height, pitch direction) underlying tone perception. Outside this core representational network, we found greater activation in the inferior frontal and parietal regions for stimuli that are more perceptually similar during tone categorization. Our findings reveal the specific characteristics of fronto-tempo-parietal regions that support speech representation and categorization processing.

Neural Networks Supporting Phoneme Monitoring Are Modulated by Phonology but Not Lexicality or Iconicity: Evidence From British and Swedish Sign Language

Sign languages are natural languages in the visual domain. Because they lack a written form, they provide a sharper tool than spoken languages for investigating lexicality effects which may be confounded by orthographic processing. In a previous study, we showed that the neural networks supporting phoneme monitoring in deaf British Sign Language (BSL) users are modulated by phonology but not lexicality or iconicity. In the present study, we investigated whether this pattern generalizes to deaf Swedish Sign Language (SSL) users. British and SSLs have a largely overlapping phoneme inventory but are mutually unintelligible because lexical overlap is small. This is important because it means that even when signs lexicalized in BSL are unintelligible to users of SSL they are usually still phonologically acceptable. During fMRI scanning, deaf users of the two different sign languages monitored signs that were lexicalized in either one or both of those languages for phonologically contrastive elements. Neural activation patterns relating to different linguistic levels of processing were similar across SLs; in particular, we found no effect of lexicality, supporting the notion that apparent lexicality effects on sublexical processing of speech may be driven by orthographic strategies. As expected, we found an effect of phonology but not iconicity. Further, there was a difference in neural activation between the two groups in a motion-processing region of the left occipital cortex, possibly driven by cultural differences, such as education. Importantly, this difference was not modulated by the linguistic characteristics of the material, underscoring the robustness of the neural activation patterns relating to different linguistic levels of processing.

Electrophysiological evidence of phonemotopic representations of vowels in the primary and secondary auditory cortex

How the brain encodes the speech acoustic signal into phonological representations is a fundamental question for the neurobiology of language. Determining whether this process is characterized by tonotopic maps in primary or secondary auditory areas, with bilateral or leftward activity, remains a long-standing challenge. Magnetoencephalographic studies failed to show hierarchical and asymmetric hints for speech processing. We employed high-density electroencephalography to map the Salento Italian vowel system onto cortical sources using the N1 auditory evoked component. We found evidence that the N1 is characterized by hierarchical and asymmetrical indexes in primary and secondary auditory areas structuring vowel representations. Importantly, the N1 was characterized by early and late phases. The early N1 peaked at 125-135 msec and was localized in the primary auditory cortex; the late N1 peaked at 145-155 msec and was localized in the left superior temporal gyrus. We showed that early in the primary auditory cortex, the cortical spatial arrangements-along the lateral-medial and anterior-posterior gradients-are broadly warped by phonemotopic patterns according to the distinctive feature principle. These phonemotopic patterns are carefully refined in the superior temporal gyrus along the inferior-superior and anterior-posterior gradients. The dynamical and hierarchical interface between primary and secondary auditory areas and the interaction effects between Height and Place features generate the categorical representation of the Salento Italian vowels.