Language prediction mechanisms in human auditory cortex

Stimulation of caudal inferior and middle frontal gyri disrupts planning during spoken interaction

Turn-taking is a central feature of conversation across languages and cultures.1,2,3,4 This key social behavior requires numerous sensorimotor and cognitive operations1,5,6 that can be organized into three general phases: comprehension of a partner's turn, preparation of a speaker's own turn, and execution of that turn. Using intracranial electrocorticography, we recently demonstrated that neural activity related to these phases is functionally distinct during turn-taking.7 In particular, networks active during the perceptual and articulatory stages of turn-taking consisted of structures known to be important for speech-related sensory and motor processing,8,9,10,11,12,13,14,15,16,17 while putative planning dynamics were most regularly observed in the caudal inferior frontal gyrus (cIFG) and the middle frontal gyrus (cMFG). To test if these structures are necessary for planning during spoken interaction, we used direct electrical stimulation (DES) to transiently perturb cortical function in neurosurgical patient-volunteers performing a question-answer task.7,18,19 We found that stimulating the cIFG and cMFG led to various response errors9,13,20,21 but not gross articulatory deficits, which instead resulted from DES of structures involved in motor control8,13,20,22 (e.g., the precentral gyrus). Furthermore, perturbation of the cIFG and cMFG delayed inter-speaker timing-consistent with slowed planning-while faster responses could result from stimulation of sites located in other areas. Taken together, our findings suggest that the cIFG and cMFG contain critical preparatory circuits that are relevant for interactive language use.

Planum temporale asymmetry in newborn monkeys predicts the future development of gestural communication's handedness

The planum temporale (PT), a key language area, is specialized in the left hemisphere in prelinguistic infants and considered as a marker of the pre-wired language-ready brain. However, studies have reported a similar structural PT left-asymmetry not only in various adult non-human primates, but also in newborn baboons. Its shared functional links with language are not fully understood. Here we demonstrate using previously obtained MRI data that early detection of PT left-asymmetry among 27 newborn baboons (Papio anubis, age range of 4 days to 2 months) predicts the future development of right-hand preference for communicative gestures but not for non-communicative actions. Specifically, only newborns with a larger left-than-right PT were more likely to develop a right-handed communication once juvenile, a contralateral brain-gesture link which is maintained in a group of 70 mature baboons. This finding suggests that early PT asymmetry may be a common inherited prewiring of the primate brain for the ontogeny of ancient lateralised properties shared between monkey gesture and human language.

Processing of auditory feedback in perisylvian and insular cortex

When we speak, we not only make movements with our mouth, lips, and tongue, but we also hear the sound of our own voice. Thus, speech production in the brain involves not only controlling the movements we make, but also auditory and sensory feedback. Auditory responses are typically suppressed during speech production compared to perception, but how this manifests across space and time is unclear. Here we recorded intracranial EEG in seventeen pediatric, adolescent, and adult patients with medication-resistant epilepsy who performed a reading/listening task to investigate how other auditory responses are modulated during speech production. We identified onset and sustained responses to speech in bilateral auditory cortex, with a selective suppression of onset responses during speech production. Onset responses provide a temporal landmark during speech perception that is redundant with forward prediction during speech production. Phonological feature tuning in these "onset suppression" electrodes remained stable between perception and production. Notably, the posterior insula responded at sentence onset for both perception and production, suggesting a role in multisensory integration during feedback control.

Timing and location of speech errors induced by direct cortical stimulation

Cortical regions supporting speech production are commonly established using neuroimaging techniques in both research and clinical settings. However, for neurosurgical purposes, structural function is routinely mapped peri-operatively using direct electrocortical stimulation. While this method is the gold standard for identification of eloquent cortical regions to preserve in neurosurgical patients, there is lack of specificity of the actual underlying cognitive processes being interrupted. To address this, we propose mapping the temporal dynamics of speech arrest across peri-sylvian cortices by quantifying the latency between stimulation and speech deficits. In doing so, we are able to substantiate hypotheses about distinct region-specific functional roles (e.g. planning versus motor execution). In this retrospective observational study, we analysed 20 patients (12 female; age range 14-43) with refractory epilepsy who underwent continuous extra-operative intracranial EEG monitoring of an automatic speech task during clinical bedside language mapping. Latency to speech arrest was calculated as time from stimulation onset to speech arrest onset, controlling for individual speech rate. Most instances of motor-based arrest (87.5% of 96 instances) were in sensorimotor cortex with mid-range latencies to speech arrest with a distributional peak at 0.47 s. Speech arrest occurred in numerous regions, with relatively short latencies in supramarginal gyrus (0.46 s), superior temporal gyrus (0.51 s) and middle temporal gyrus (0.54 s), followed by relatively long latencies in sensorimotor cortex (0.72 s) and especially long latencies in inferior frontal gyrus (0.95 s). Non-parametric testing for speech arrest revealed that region predicted latency; latencies in supramarginal gyrus and in superior temporal gyrus were shorter than in sensorimotor cortex and in inferior frontal gyrus. Sensorimotor cortex is primarily responsible for motor-based arrest. Latencies to speech arrest in supramarginal gyrus and superior temporal gyrus (and to a lesser extent middle temporal gyrus) align with latencies to motor-based arrest in sensorimotor cortex. This pattern of relatively quick cessation of speech suggests that stimulating these regions interferes with the outgoing motor execution. In contrast, the latencies to speech arrest in inferior frontal gyrus and in ventral regions of sensorimotor cortex were significantly longer than those in temporoparietal regions. Longer latencies in the more frontal areas (including inferior frontal gyrus and ventral areas of precentral gyrus and postcentral gyrus) suggest that stimulating these areas interrupts a higher-level speech production process involved in planning. These results implicate the ventral specialization of sensorimotor cortex (including both precentral and postcentral gyri) for speech planning above and beyond motor execution.

Attention to audiovisual speech shapes neural processing through feedback-feedforward loops between different nodes of the speech network

Selective attention-related top-down modulation plays a significant role in separating relevant speech from irrelevant background speech when vocal attributes separating concurrent speakers are small and continuously evolving. Electrophysiological studies have shown that such top-down modulation enhances neural tracking of attended speech. Yet, the specific cortical regions involved remain unclear due to the limited spatial resolution of most electrophysiological techniques. To overcome such limitations, we collected both electroencephalography (EEG) (high temporal resolution) and functional magnetic resonance imaging (fMRI) (high spatial resolution), while human participants selectively attended to speakers in audiovisual scenes containing overlapping cocktail party speech. To utilise the advantages of the respective techniques, we analysed neural tracking of speech using the EEG data and performed representational dissimilarity-based EEG-fMRI fusion. We observed that attention enhanced neural tracking and modulated EEG correlates throughout the latencies studied. Further, attention-related enhancement of neural tracking fluctuated in predictable temporal profiles. We discuss how such temporal dynamics could arise from a combination of interactions between attention and prediction as well as plastic properties of the auditory cortex. EEG-fMRI fusion revealed attention-related iterative feedforward-feedback loops between hierarchically organised nodes of the ventral auditory object related processing stream. Our findings support models where attention facilitates dynamic neural changes in the auditory cortex, ultimately aiding discrimination of relevant sounds from irrelevant ones while conserving neural resources.

Neural response properties predict perceived contents and locations elicited by intracranial electrical stimulation of human auditory cortex

Intracranial electrical stimulation (iES) of auditory cortex can elicit sound experiences with a variety of perceived contents (hallucination or illusion) and locations (contralateral or bilateral side), independent of actual acoustic inputs. However, the neural mechanisms underlying this elicitation heterogeneity remain undiscovered. Here, we collected subjective reports following iES at 3062 intracranial sites in 28 patients (both sexes) and identified 113 auditory cortical sites with iES-elicited sound experiences. We then decomposed the sound-induced intracranial electroencephalogram (iEEG) signals recorded from all 113 sites into time-frequency features. We found that the iES-elicited perceived contents can be predicted by the early high-γ features extracted from sound-induced iEEG. In contrast, the perceived locations elicited by stimulating hallucination sites and illusion sites are determined by the late high-γ and long-lasting α features, respectively. Our study unveils the crucial neural signatures of iES-elicited sound experiences in human and presents a new strategy to hearing restoration for individuals suffering from deafness.

Timing and location of speech errors induced by direct cortical stimulation

Cortical regions supporting speech production are commonly established using neuroimaging techniques in both research and clinical settings. However, for neurosurgical purposes, structural function is routinely mapped peri-operatively using direct electrocortical stimulation. While this method is the gold standard for identification of eloquent cortical regions to preserve in neurosurgical patients, there is lack of specificity of the actual underlying cognitive processes being interrupted. To address this, we propose mapping the temporal dynamics of speech arrest across peri-sylvian cortices by quantifying the latency between stimulation and speech deficits. In doing so, we are able to substantiate hypotheses about distinct region-specific functional roles (e.g., planning versus motor execution). In this retrospective observational study, we analyzed 20 patients (12 female; age range 14-43) with refractory epilepsy who underwent continuous extra-operative intracranial EEG monitoring of an automatic speech task during clinical bedside language mapping. Latency to speech arrest was calculated as time from stimulation onset to speech arrest onset, controlling for individual speech rate. Most instances of motor-based arrest (87.5% of 96 instances) were in sensorimotor cortex with mid-range latencies to speech arrest with a distributional peak at 0.47 seconds. Speech arrest occurred in numerous regions, with relatively short latencies in supramarginal gyrus (0.46 seconds), superior temporal gyrus (0.51 seconds), and middle temporal gyrus (0.54 seconds), followed by relatively long latencies in sensorimotor cortex (0.72 seconds) and especially long latencies in inferior frontal gyrus (0.95 seconds). Nonparametric testing for speech arrest revealed that region predicted latency; latencies in supramarginal gyrus and in superior temporal gyrus were shorter than in sensorimotor cortex and in inferior frontal gyrus. Sensorimotor cortex is primarily responsible for motor-based arrest. Latencies to speech arrest in supramarginal gyrus and superior temporal gyrus (and to a lesser extent middle temporal gyrus) align with latencies to motor-based arrest in sensorimotor cortex. This pattern of relatively quick cessation of speech suggests that stimulating these regions interferes with the outgoing motor execution. In contrast, the latencies to speech arrest in inferior frontal gyrus and in ventral regions of sensorimotor cortex were significantly longer than those in temporoparietal regions. Longer latencies in the more frontal areas (including inferior frontal gyrus and ventral areas of precentral gyrus and postcentral gyrus) suggest that stimulating these areas interrupts a higher-level speech production process involved in planning. These results implicate the ventral specialization of sensorimotor cortex (including both precentral and postcentral gyri) for speech planning above and beyond motor execution.

Auditory Cortex Asymmetry Associations with Individual Differences in Language and Cognition

A longstanding cerebral lateralization hypothesis predicts that disrupted development of typical leftward structural asymmetry of auditory cortex explains why children have problems learning to read. Small sample sizes and small effects, potential sex-specific effects, and associations that are limited to specific dimensions of language are thought to have contributed inconsistent results. The large ABCD study dataset (baseline visit: N = 11,859) was used to test the hypothesis of significant associations between surface area asymmetry of auditory cortex and receptive vocabulary performance across boys and girls, as well as an oral word reading effect that was specific to boys. The results provide modest support (Cohen's d effect sizes ≤ 0.10) for the cerebral lateralization hypothesis.

A frontal cortical network is critical for language planning during spoken interaction

Many brain areas exhibit activity correlated with language planning, but the impact of these dynamics on spoken interaction remains unclear. Here we use direct electrical stimulation to transiently perturb cortical function in neurosurgical patient-volunteers performing a question-answer task. Stimulating structures involved in speech motor function evoked diverse articulatory deficits, while perturbations of caudal inferior and middle frontal gyri - which exhibit preparatory activity during conversational turn-taking - led to response errors. Perturbation of the same planning-related frontal regions slowed inter-speaker timing, while faster responses could result from stimulation of sites located in other areas. Taken together, these findings further indicate that caudal inferior and middle frontal gyri constitute a critical planning network essential for interactive language use.

Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions

Models that predict brain responses to stimuli provide one measure of understanding of a sensory system and have many potential applications in science and engineering. Deep artificial neural networks have emerged as the leading such predictive models of the visual system but are less explored in audition. Prior work provided examples of audio-trained neural networks that produced good predictions of auditory cortical fMRI responses and exhibited correspondence between model stages and brain regions, but left it unclear whether these results generalize to other neural network models and, thus, how to further improve models in this domain. We evaluated model-brain correspondence for publicly available audio neural network models along with in-house models trained on 4 different tasks. Most tested models outpredicted standard spectromporal filter-bank models of auditory cortex and exhibited systematic model-brain correspondence: Middle stages best predicted primary auditory cortex, while deep stages best predicted non-primary cortex. However, some state-of-the-art models produced substantially worse brain predictions. Models trained to recognize speech in background noise produced better brain predictions than models trained to recognize speech in quiet, potentially because hearing in noise imposes constraints on biological auditory representations. The training task influenced the prediction quality for specific cortical tuning properties, with best overall predictions resulting from models trained on multiple tasks. The results generally support the promise of deep neural networks as models of audition, though they also indicate that current models do not explain auditory cortical responses in their entirety.

Genomic Structural Equation Modeling Reveals Latent Phenotypes in the Human Cortex with Distinct Genetic Architecture

Genetic contributions to human cortical structure manifest pervasive pleiotropy. This pleiotropy may be harnessed to identify unique genetically-informed parcellations of the cortex that are neurobiologically distinct from functional, cytoarchitectural, or other cortical parcellation schemes. We investigated genetic pleiotropy by applying genomic structural equation modeling (SEM) to map the genetic architecture of cortical surface area (SA) and cortical thickness (CT) for the 34 brain regions recently reported in the ENIGMA cortical GWAS. Genomic SEM uses the empirical genetic covariance estimated from GWAS summary statistics with LD score regression (LDSC) to discover factors underlying genetic covariance, which we are denoting genetically informed brain networks (GIBNs). Genomic SEM can fit a multivariate GWAS from summary statistics for each of the GIBNs, which can subsequently be used for LD score regression (LDSC). We found the best-fitting model of cortical SA identified 6 GIBNs and CT identified 4 GIBNs. The multivariate GWASs of these GIBNs identified 74 genome-wide significant (GWS) loci (p<5×10-8), including many previously implicated in neuroimaging phenotypes, behavioral traits, and psychiatric conditions. LDSC of GIBN GWASs found that SA-derived GIBNs had a positive genetic correlation with bipolar disorder (BPD), and cannabis use disorder, indicating genetic predisposition to a larger SA in the specific GIBN is associated with greater genetic risk of these disorders. A negative genetic correlation was observed with attention deficit hyperactivity disorder (ADHD), major depressive disorder (MDD), and insomnia, indicating genetic predisposition to a larger SA in the specific GIBN is associated with lower genetic risk of these disorders. CT GIBNs displayed a negative genetic correlation with alcohol dependence. Jointly modeling the genetic architecture of complex traits and investigating multivariate genetic links across phenotypes offers a new vantage point for mapping the cortex into genetically informed networks.

Functional geometry of auditory cortical resting state networks derived from intracranial electrophysiology

Understanding central auditory processing critically depends on defining underlying auditory cortical networks and their relationship to the rest of the brain. We addressed these questions using resting state functional connectivity derived from human intracranial electroencephalography. Mapping recording sites into a low-dimensional space where proximity represents functional similarity revealed a hierarchical organization. At a fine scale, a group of auditory cortical regions excluded several higher-order auditory areas and segregated maximally from the prefrontal cortex. On mesoscale, the proximity of limbic structures to the auditory cortex suggested a limbic stream that parallels the classically described ventral and dorsal auditory processing streams. Identities of global hubs in anterior temporal and cingulate cortex depended on frequency band, consistent with diverse roles in semantic and cognitive processing. On a macroscale, observed hemispheric asymmetries were not specific for speech and language networks. This approach can be applied to multivariate brain data with respect to development, behavior, and disorders.

Intraoperative cortical localization of music and language reveals signatures of structural complexity in posterior temporal cortex

Language and music involve the productive combination of basic units into structures. It remains unclear whether brain regions sensitive to linguistic and musical structure are co-localized. We report an intraoperative awake craniotomy in which a left-hemispheric language-dominant professional musician underwent cortical stimulation mapping (CSM) and electrocorticography of music and language perception and production during repetition tasks. Musical sequences were melodic or amelodic, and differed in algorithmic compressibility (Lempel-Ziv complexity). Auditory recordings of sentences differed in syntactic complexity (single vs. multiple phrasal embeddings). CSM of posterior superior temporal gyrus (pSTG) disrupted music perception and production, along with speech production. pSTG and posterior middle temporal gyrus (pMTG) activated for language and music (broadband gamma; 70-150 Hz). pMTG activity was modulated by musical complexity, while pSTG activity was modulated by syntactic complexity. This points to shared resources for music and language comprehension, but distinct neural signatures for the processing of domain-specific structural features.

Cortical speech tracking is related to individual prediction tendencies

Listening can be conceptualized as a process of active inference, in which the brain forms internal models to integrate auditory information in a complex interaction of bottom-up and top-down processes. We propose that individuals vary in their "prediction tendency" and that this variation contributes to experiential differences in everyday listening situations and shapes the cortical processing of acoustic input such as speech. Here, we presented tone sequences of varying entropy level, to independently quantify auditory prediction tendency (as the tendency to anticipate low-level acoustic features) for each individual. This measure was then used to predict cortical speech tracking in a multi speaker listening task, where participants listened to audiobooks narrated by a target speaker in isolation or interfered by 1 or 2 distractors. Furthermore, semantic violations were introduced into the story, to also examine effects of word surprisal during speech processing. Our results show that cortical speech tracking is related to prediction tendency. In addition, we find interactions between prediction tendency and background noise as well as word surprisal in disparate brain regions. Our findings suggest that individual prediction tendencies are generalizable across different listening situations and may serve as a valuable element to explain interindividual differences in natural listening situations.

Independency of Coding for Affective Similarities and for Word Co-occurrences in Temporal Perisylvian Neocortex

Word valence is one of the principal dimensions in the organization of word meaning. Co-occurrence-based similarities calculated by predictive natural language processing models are relatively poor at representing affective content, but very powerful in their own way. Here, we determined how these two canonical but distinct ways of representing word meaning relate to each other in the human brain both functionally and neuroanatomically. We re-analysed an fMRI study of word valence. A co-occurrence-based model was used and the correlation with the similarity of brain activity patterns was compared to that of affective similarities. The correlation between affective and co-occurrence-based similarities was low (r = 0.065), confirming that affect was captured poorly by co-occurrence modelling. In a whole-brain representational similarity analysis, word embedding similarities correlated significantly with the similarity between activity patterns in a region confined to the superior temporal sulcus to the left, and to a lesser degree to the right. Affective word similarities correlated with the similarity in activity patterns in this same region, confirming previous findings. The affective similarity effect extended more widely beyond the superior temporal cortex than the effect of co-occurrence-based similarities did. The effect of co-occurrence-based similarities remained unaltered after partialling out the effect of affective similarities (and vice versa). To conclude, different aspects of word meaning, derived from affective judgements or from word co-occurrences, are represented in superior temporal language cortex in a neuroanatomically overlapping but functionally independent manner.

Dynamic cortical and tractography atlases of proactive and reactive alpha and high-gamma activities

Alpha waves-posterior dominant rhythms at 8-12 Hz reactive to eye opening and closure-are among the most fundamental EEG findings in clinical practice and research since Hans Berger first documented them in the early 20th century. Yet, the exact network dynamics of alpha waves in regard to eye movements remains unknown. High-gamma activity at 70-110 Hz is also reactive to eye movements and a summary measure of local cortical activation supporting sensorimotor or cognitive function. We aimed to build the first-ever brain atlases directly visualizing the network dynamics of eye movement-related alpha and high-gamma modulations, at cortical and white matter levels. We studied 28 patients (age: 5-20 years) who underwent intracranial EEG and electro-oculography recordings. We measured alpha and high-gamma modulations at 2167 electrode sites outside the seizure onset zone, interictal spike-generating areas and MRI-visible structural lesions. Dynamic tractography animated white matter streamlines modulated significantly and simultaneously beyond chance, on a millisecond scale. Before eye-closure onset, significant alpha augmentation occurred at the occipital and frontal cortices. After eye-closure onset, alpha-based functional connectivity was strengthened, while high gamma-based connectivity was weakened extensively in both intra-hemispheric and inter-hemispheric pathways involving the central visual areas. The inferior fronto-occipital fasciculus supported the strengthened alpha co-augmentation-based functional connectivity between occipital and frontal lobe regions, whereas the posterior corpus callosum supported the inter-hemispheric functional connectivity between the occipital lobes. After eye-opening offset, significant high-gamma augmentation and alpha attenuation occurred at occipital, fusiform and inferior parietal cortices. High gamma co-augmentation-based functional connectivity was strengthened, whereas alpha-based connectivity was weakened in the posterior inter-hemispheric and intra-hemispheric white matter pathways involving central and peripheral visual areas. Our results do not support the notion that eye closure-related alpha augmentation uniformly reflects feedforward or feedback rhythms propagating from lower to higher order visual cortex, or vice versa. Rather, proactive and reactive alpha waves involve extensive, distinct white matter networks that include the frontal lobe cortices, along with low- and high-order visual areas. High-gamma co-attenuation coupled to alpha co-augmentation in shared brain circuitry after eye closure supports the notion of an idling role for alpha waves during eye closure. These normative dynamic tractography atlases may improve understanding of the significance of EEG alpha waves in assessing the functional integrity of brain networks in clinical practice; they also may help elucidate the effects of eye movements on task-related brain network measures observed in cognitive neuroscience research.

Evidence of a predictive coding hierarchy in the human brain listening to speech

Considerable progress has recently been made in natural language processing: deep learning algorithms are increasingly able to generate, summarize, translate and classify texts. Yet, these language models still fail to match the language abilities of humans. Predictive coding theory offers a tentative explanation to this discrepancy: while language models are optimized to predict nearby words, the human brain would continuously predict a hierarchy of representations that spans multiple timescales. To test this hypothesis, we analysed the functional magnetic resonance imaging brain signals of 304 participants listening to short stories. First, we confirmed that the activations of modern language models linearly map onto the brain responses to speech. Second, we showed that enhancing these algorithms with predictions that span multiple timescales improves this brain mapping. Finally, we showed that these predictions are organized hierarchically: frontoparietal cortices predict higher-level, longer-range and more contextual representations than temporal cortices. Overall, these results strengthen the role of hierarchical predictive coding in language processing and illustrate how the synergy between neuroscience and artificial intelligence can unravel the computational bases of human cognition.

Two stages of speech envelope tracking in human auditory cortex modulated by speech intelligibility

The envelope is essential for speech perception. Recent studies have shown that cortical activity can track the acoustic envelope. However, whether the tracking strength reflects the extent of speech intelligibility processing remains controversial. Here, using stereo-electroencephalogram technology, we directly recorded the activity in human auditory cortex while subjects listened to either natural or noise-vocoded speech. These 2 stimuli have approximately identical envelopes, but the noise-vocoded speech does not have speech intelligibility. According to the tracking lags, we revealed 2 stages of envelope tracking: an early high-γ (60-140 Hz) power stage that preferred the noise-vocoded speech and a late θ (4-8 Hz) phase stage that preferred the natural speech. Furthermore, the decoding performance of high-γ power was better in primary auditory cortex than in nonprimary auditory cortex, consistent with its short tracking delay, while θ phase showed better decoding performance in right auditory cortex. In addition, high-γ responses with sustained temporal profiles in nonprimary auditory cortex were dominant in both envelope tracking and decoding. In sum, we suggested a functional dissociation between high-γ power and θ phase: the former reflects fast and automatic processing of brief acoustic features, while the latter correlates to slow build-up processing facilitated by speech intelligibility.

Gender effects on the controllability of hemispheric white matter networks

Male and female adults exhibited significant group differences in brain white matter (WM) asymmetry and WM network controllability. However, gender differences in controllability of hemispheric WM networks between males and females remain to be determined. Based on 1 principal atlas and 1 replication atlas, this work characterized the average controllability (AC) and modal controllability (MC) of hemispheric WM network based on 1 principal dataset and 2 replication datasets. All results showed that males had higher AC of left hemispheric networks than females. And significant hemispheric asymmetry was revealed in regional AC and MC. Furthermore, significant gender differences in the AC asymmetry were mainly found in regions lie in the frontoparietal network, and the MC asymmetry was found in regions involving auditory and emotion process. Finally, we found significant associations between regional controllability and cognitive features. Taken together, this work could provide a novel perspective for understanding gender differences in hemispheric WM asymmetry and cognitive function between males and females.

A Redundant Cortical Code for Speech Envelope

Animal communication sounds exhibit complex temporal structure because of the amplitude fluctuations that comprise the sound envelope. In human speech, envelope modulations drive synchronized activity in auditory cortex (AC), which correlates strongly with comprehension (Giraud and Poeppel, 2012; Peelle and Davis, 2012; Haegens and Zion Golumbic, 2018). Studies of envelope coding in single neurons, performed in nonhuman animals, have focused on periodic amplitude modulation (AM) stimuli and use response metrics that are not easy to juxtapose with data from humans. In this study, we sought to bridge these fields. Specifically, we looked directly at the temporal relationship between stimulus envelope and spiking, and we assessed whether the apparent diversity across neurons' AM responses contributes to the population representation of speech-like sound envelopes. We gathered responses from single neurons to vocoded speech stimuli and compared them to sinusoidal AM responses in auditory cortex (AC) of alert, freely moving Mongolian gerbils of both sexes. While AC neurons displayed heterogeneous tuning to AM rate, their temporal dynamics were stereotyped. Preferred response phases accumulated near the onsets of sinusoidal AM periods for slower rates (<8 Hz), and an over-representation of amplitude edges was apparent in population responses to both sinusoidal AM and vocoded speech envelopes. Crucially, this encoding bias imparted a decoding benefit: a classifier could discriminate vocoded speech stimuli using summed population activity, while higher frequency modulations required a more sophisticated decoder that tracked spiking responses from individual cells. Together, our results imply that the envelope structure relevant to parsing an acoustic stream could be read-out from a distributed, redundant population code.SIGNIFICANCE STATEMENT Animal communication sounds have rich temporal structure and are often produced in extended sequences, including the syllabic structure of human speech. Although the auditory cortex (AC) is known to play a crucial role in representing speech syllables, the contribution of individual neurons remains uncertain. Here, we characterized the representations of both simple, amplitude-modulated sounds and complex, speech-like stimuli within a broad population of cortical neurons, and we found an overrepresentation of amplitude edges. Thus, a phasic, redundant code in auditory cortex can provide a mechanistic explanation for segmenting acoustic streams like human speech.

A critical analysis of Lin et al.'s (2021) failure to observe forward entrainment in pitch discrimination

Forward entrainment refers to that part of the entrainment process that outlasts the entraining stimulus. Several studies have demonstrated psychophysical forward entrainment in a pitch-discrimination task. In a recent paper, Lin et al. (2021) challenged these findings by demonstrating that a sequence of 4 entraining pure tones does not affect the ability to determine whether a frequency modulated pulse, presented after termination of the entraining sequence, has swept up or down in frequency. They concluded that rhythmic sequences do not facilitate pitch discrimination. Here, we describe several methodological and stimulus design flaws in Lin et al.'s study that may explain their failure to observe forward entrainment in pitch discrimination.

Echolocation-related reversal of information flow in a cortical vocalization network

The mammalian frontal and auditory cortices are important for vocal behavior. Here, using local-field potential recordings, we demonstrate that the timing and spatial patterns of oscillations in the fronto-auditory network of vocalizing bats (Carollia perspicillata) predict the purpose of vocalization: echolocation or communication. Transfer entropy analyses revealed predominant top-down (frontal-to-auditory cortex) information flow during spontaneous activity and pre-vocal periods. The dynamics of information flow depend on the behavioral role of the vocalization and on the timing relative to vocal onset. We observed the emergence of predominant bottom-up (auditory-to-frontal) information transfer during the post-vocal period specific to echolocation pulse emission, leading to self-directed acoustic feedback. Electrical stimulation of frontal areas selectively enhanced responses to sounds in auditory cortex. These results reveal unique changes in information flow across sensory and frontal cortices, potentially driven by the purpose of the vocalization in a highly vocal mammalian model.

Gamma Activation and Alpha Suppression within Human Auditory Cortex during a Speech Classification Task

The dynamics of information flow within the auditory cortical hierarchy associated with speech processing and the emergence of hemispheric specialization remain incompletely understood. To study these questions with high spatiotemporal resolution, intracranial recordings in 29 human neurosurgical patients of both sexes were obtained while subjects performed a semantic classification task. Neural activity was recorded from posteromedial portion of Heschl's gyrus (HGPM) and anterolateral portion of Heschl's gyrus (HGAL), planum temporale (PT), planum polare, insula, and superior temporal gyrus (STG). Responses to monosyllabic words exhibited early gamma power increases and a later suppression of alpha power, envisioned to represent feedforward activity and decreased feedback signaling, respectively. Gamma activation and alpha suppression had distinct magnitude and latency profiles. HGPM and PT had the strongest gamma responses with shortest onset latencies, indicating that they are the earliest auditory cortical processing stages. The origin of attenuated top-down influences in auditory cortex, as indexed by alpha suppression, was in STG and HGAL. Gamma responses and alpha suppression were typically larger to nontarget words than tones. Alpha suppression was uniformly greater to target versus nontarget stimuli. Hemispheric bias for words versus tones and for target versus nontarget words, when present, was left lateralized. Better task performance was associated with increased gamma activity in the left PT and greater alpha suppression in HGPM and HGAL bilaterally. The prominence of alpha suppression during semantic classification and its accessibility for noninvasive electrophysiologic studies suggests that this measure is a promising index of auditory cortical speech processing.SIGNIFICANCE STATEMENT Understanding the dynamics of cortical speech processing requires the use of active tasks. This is the first comprehensive intracranial electroencephalography study to examine cortical activity within the superior temporal plane, lateral superior temporal gyrus, and the insula during a semantic classification task. Distinct gamma activation and alpha suppression profiles clarify the functional organization of feedforward and feedback processing within the auditory cortical hierarchy. Asymmetries in cortical speech processing emerge at early processing stages. Relationships between cortical activity and task performance are interpreted in the context of current models of speech processing. Results lay the groundwork for iEEG studies using connectivity measures of the bidirectional information flow within the auditory processing hierarchy.

Event-Related Phase Synchronization Propagates Rapidly across Human Ventral Visual Cortex

Visual inputs to early visual cortex integrate with semantic, linguistic and memory inputs in higher visual cortex, in a manner that is rapid and accurate, and enables complex computations such as face recognition and word reading. This implies the existence of fundamental organizational principles that enable such efficiency. To elaborate on this, we performed intracranial recordings in 82 individuals while they performed tasks of varying visual and cognitive complexity. We discovered that visual inputs induce highly organized posterior-to-anterior propagating patterns of phase modulation across the ventral occipitotemporal cortex. At individual electrodes there was a stereotyped temporal pattern of phase progression following both stimulus onset and offset, consistent across trials and tasks. The phase of low frequency activity in anterior regions was predicted by the prior phase in posterior cortical regions. This spatiotemporal propagation of phase likely serves as a feed-forward organizational influence enabling the integration of information across the ventral visual stream. This phase modulation manifests as the early components of the event related potential; one of the most commonly used measures in human electrophysiology. These findings illuminate fundamental organizational principles of the higher order visual system that enable the rapid recognition and characterization of a variety of inputs.

Intraoperative localization and preservation of reading in ventral occipitotemporal cortex

OBJECTIVE

Resective surgery in language-dominant ventral occipitotemporal cortex (vOTC) carries the risk of causing impairment to reading. Because it is not on the lateral surface, it is not easily accessible for intraoperative mapping, and extensive stimulation mapping can be time-consuming. Here the authors assess the feasibility of using task-based electrocorticography (ECoG) recordings intraoperatively to help guide stimulation mapping of reading in vOTC.

METHODS

In 11 patients undergoing extraoperative, intracranial seizure mapping, the authors recorded induced broadband gamma activation (70-150 Hz) during a visual category localizer. In 2 additional patients, whose pathologies necessitated resections in language-dominant vOTC, task-based functional mapping was performed intraoperatively using subdural ECoG alongside direct cortical stimulation.

RESULTS

Word-responsive cortex localized using ECoG showed a high sensitivity (72%) to stimulation-induced reading deficits, and the confluence of ECoG and stimulation-positive sites appears to demarcate the visual word form area. Intraoperative task-based ECoG mapping was possible in < 3 minutes, providing a high signal quality, and initial intraoperative data analysis took < 3 minutes, allowing for rapid assessment of broad areas of cortex. Cortical areas critical for reading were mapped and successfully preserved, while also enabling pathological tissue to be completely removed.

CONCLUSIONS

Eloquent cortex in ventral visual cortex can be rapidly mapped intraoperatively using ECoG. This method acts to guide high-probability targets for stimulation with limited patient participation and can be used to avoid iatrogenic dyslexia following surgery.

Temporally and functionally distinct large-scale brain network dynamics supporting task switching

Objective:

Our daily activities require frequent switches among competing responses at the millisecond time scale. We determined the spatiotemporal characteristics and functional significance of rapid, large-scale brain network dynamics during task switching.

Methods:

This cross-sectional study investigated patients with drug-resistant focal epilepsy who played a Lumosity cognitive flexibility training game during intracranial electroencephalography (iEEG) recording. According to a given task rule, unpredictably switching across trials, participants had to swipe the screen in the direction the stimulus was pointing or moving. Using this data, we described the spatiotemporal characteristics of iEEG high-gamma augmentation occurring more intensely during switch than repeat trials, unattributable to the effect of task rule (pointing or moving), within-stimulus congruence (the direction of stimulus pointing and moving was same or different in a given trial), or accuracy of an immediately preceding response. Diffusion-weighted imaging (DWI) tractography determined whether distant cortical regions showing enhanced activation during task switch trials were directly connected by white matter tracts. Trial-by-trial iEEG analysis deduced whether the intensity of task switch-related high-gamma augmentation was altered through practice and whether high-gamma amplitude predicted the accuracy of an upcoming response among switch trials.

Results:

The average number of completed trials during five-minute gameplay was 221.4 per patient (range: 171–285). Task switch trials increased the response times, whereas later trials reduced them. Analysis of iEEG signals sampled from 860 brain sites effectively elucidated the distinct spatiotemporal characteristics of task switch, task rule, and post-error-specific high-gamma modulations. Post-cue, task switch-related high-gamma augmentation was initiated in the right calcarine cortex after 260 ms, right precuneus after 330 ms, right entorhinal after 420 ms, and bilateral anterior middle-frontal gyri after 450 ms. DWI tractography successfully showed the presence of direct white matter tracts connecting the right visual areas to the precuneus and anterior middle-frontal regions but not between the right precuneus and anterior middle-frontal regions. Task-related high-gamma amplitudes in later trials were reduced in the calcarine, entorhinal and anterior middle-frontal regions, but increased in the precuneus. Functionally, enhanced post-cue precuneus high-gamma augmentation improved the accuracy of subsequent responses among switch trials.

Conclusions:

Our multimodal analysis uncovered two temporally and functionally distinct network dynamics supporting task switching. High-gamma augmentation in the visual-precuneus pathway may reflect the neural process facilitating an attentional shift to a given updated task rule. High-gamma activity in the visual-dorsolateral prefrontal pathway, rapidly reduced through practice, may reflect the cost of executing appropriate stimulus-response translation.

The Listening Zone of Human Electrocorticographic Field Potential Recordings

Intracranial electroencephalographic (icEEG) recordings provide invaluable insights into neural dynamics in humans because of their unmatched spatiotemporal resolution. Yet, such recordings reflect the combined activity of multiple underlying generators, confounding the ability to resolve spatially distinct neural sources. To empirically quantify the listening zone of icEEG recordings, we computed correlations between signals as a function of distance (full width at half maximum; FWHM) between 8752 recording sites in 71 patients (33 female) implanted with either subdural electrodes (SDEs), stereo-encephalography electrodes (sEEG), or high-density sEEG electrodes. As expected, for both SDEs and sEEGs, higher frequency signals exhibited a sharper fall off relative to lower frequency signals. For broadband high γ (BHG) activity, the mean FWHM of SDEs (6.6 ± 2.5 mm) and sEEGs in gray matter (7.14 ± 1.7 mm) was not significantly different; however, FWHM for low frequencies recorded by sEEGs was 2.45 mm smaller than SDEs. White matter sEEGs showed much lower power for frequencies 17-200 Hz (q < 0.01) and a much broader decay (11.3 ± 3.2 mm) than gray matter electrodes (7.14 ± 1.7 mm). The use of a bipolar referencing scheme significantly lowered FWHM for sEEGs, relative to a white matter reference or a common average reference (CAR). These results outline the influence of array design, spectral bands, and referencing schema on local field potential recordings and source localization in icEEG recordings in humans. The metrics we derive have immediate relevance to the analysis and interpretation of both cognitive and epileptic data.

Parallel processing in speech perception with local and global representations of linguistic context

Speech processing is highly incremental. It is widely accepted that human listeners continuously use the linguistic context to anticipate upcoming concepts, words, and phonemes. However, previous evidence supports two seemingly contradictory models of how a predictive context is integrated with the bottom-up sensory input: Classic psycholinguistic paradigms suggest a two-stage process, in which acoustic input initially leads to local, context-independent representations, which are then quickly integrated with contextual constraints. This contrasts with the view that the brain constructs a single coherent, unified interpretation of the input, which fully integrates available information across representational hierarchies, and thus uses contextual constraints to modulate even the earliest sensory representations. To distinguish these hypotheses, we tested magnetoencephalography responses to continuous narrative speech for signatures of local and unified predictive models. Results provide evidence that listeners employ both types of models in parallel. Two local context models uniquely predict some part of early neural responses, one based on sublexical phoneme sequences, and one based on the phonemes in the current word alone; at the same time, even early responses to phonemes also reflect a unified model that incorporates sentence-level constraints to predict upcoming phonemes. Neural source localization places the anatomical origins of the different predictive models in nonidentical parts of the superior temporal lobes bilaterally, with the right hemisphere showing a relative preference for more local models. These results suggest that speech processing recruits both local and unified predictive models in parallel, reconciling previous disparate findings. Parallel models might make the perceptual system more robust, facilitate processing of unexpected inputs, and serve a function in language acquisition.

Ethical commitments, principles, and practices guiding intracranial neuroscientific research in humans

Leveraging firsthand experience, BRAIN-funded investigators conducting intracranial human neuroscience research propose two fundamental ethical commitments: (1) maintaining the integrity of clinical care and (2) ensuring voluntariness. Principles, practices, and uncertainties related to these commitments are offered for future investigation.

Speech-related auditory salience detection in the posterior superior temporal region

Processing auditory human speech requires both detection (early and transient) and analysis (sustained). We analyzed high gamma (70-110 Hz) activity of intracranial electroencephalography waveforms acquired during an auditory task that paired forward speech, reverse speech, and signal correlated noise. We identified widespread superior temporal sites with sustained activity responding only to forward and reverse speech regardless of paired order. More localized superior temporal auditory onset sites responded to all stimulus types when presented first in a pair and responded in recurrent fashion to the second paired stimulus in select conditions even in the absence of interstimulus silence; a novel finding. Auditory onset activity to a second paired sound recurred according to relative salience, with evidence of partial suppression during linguistic processing. We propose that temporal lobe auditory onset sites facilitate a salience detector function with hysteresis of 200 ms and are influenced by cortico-cortical feedback loops involving linguistic processing and articulation.

Mapping cortico-subcortical sensitivity to 4 Hz amplitude modulation depth in human auditory system with functional MRI

Temporal modulations in the envelope of acoustic waveforms at rates around 4 Hz constitute a strong acoustic cue in speech and other natural sounds. It is often assumed that the ascending auditory pathway is increasingly sensitive to slow amplitude modulation (AM), but sensitivity to AM is typically considered separately for individual stages of the auditory system. Here, we used blood oxygen level dependent (BOLD) fMRI in twenty human subjects (10 male) to measure sensitivity of regional neural activity in the auditory system to 4 Hz temporal modulations. Participants were exposed to AM noise stimuli varying parametrically in modulation depth to characterize modulation-depth effects on BOLD responses. A Bayesian hierarchical modeling approach was used to model potentially nonlinear relations between AM depth and group-level BOLD responses in auditory regions of interest (ROIs). Sound stimulation activated the auditory brainstem and cortex structures in single subjects. BOLD responses to noise exposure in core and belt auditory cortices scaled positively with modulation depth. This finding was corroborated by whole-brain cluster-level inference. Sensitivity to AM depth variations was particularly pronounced in the Heschl's gyrus but also found in higher-order auditory cortical regions. None of the sound-responsive subcortical auditory structures showed a BOLD response profile that reflected the parametric variation in AM depth. The results are compatible with the notion that early auditory cortical regions play a key role in processing low-rate modulation content of sounds in the human auditory system.

Learning nonnative speech sounds changes local encoding in the adult human cortex

Adults can learn to identify nonnative speech sounds with training, albeit with substantial variability in learning behavior. Increases in behavioral accuracy are associated with increased separability for sound representations in cortical speech areas. However, it remains unclear whether individual auditory neural populations all show the same types of changes with learning, or whether there are heterogeneous encoding patterns. Here, we used high-resolution direct neural recordings to examine local population response patterns, while native English listeners learned to recognize unfamiliar vocal pitch patterns in Mandarin Chinese tones. We found a distributed set of neural populations in bilateral superior temporal gyrus and ventrolateral frontal cortex, where the encoding of Mandarin tones changed throughout training as a function of trial-by-trial accuracy ("learning effect"), including both increases and decreases in the separability of tones. These populations were distinct from populations that showed changes as a function of exposure to the stimuli regardless of trial-by-trial accuracy. These learning effects were driven in part by more variable neural responses to repeated presentations of acoustically identical stimuli. Finally, learning effects could be predicted from speech-evoked activity even before training, suggesting that intrinsic properties of these populations make them amenable to behavior-related changes. Together, these results demonstrate that nonnative speech sound learning involves a wide array of changes in neural representations across a distributed set of brain regions.

Functional Topography of Auditory Areas Derived From the Combination of Electrophysiological Recordings and Cortical Electrical Stimulation

The posterior part of the superior temporal gyrus (STG) has long been known to be a crucial hub for auditory and language processing, at the crossroad of the functionally defined ventral and dorsal pathways. Anatomical studies have shown that this "auditory cortex" is composed of several cytoarchitectonic areas whose limits do not consistently match macro-anatomic landmarks like gyral and sulcal borders. The only method to record and accurately distinguish neuronal activity from the different auditory sub-fields of primary auditory cortex, located in the tip of Heschl and deeply buried in the Sylvian fissure, is to use stereotaxically implanted depth electrodes (Stereo-EEG) for pre-surgical evaluation of patients with epilepsy. In this prospective, we focused on how anatomo-functional delineation in Heschl's gyrus (HG), Planum Temporale (PT), the posterior part of the STG anterior to HG, the posterior superior temporal sulcus (STS), and the region at the parietal-temporal boundary commonly labeled "SPT" can be achieved using data from electrical cortical stimulation combined with electrophysiological recordings during listening to pure tones and syllables. We show the differences in functional roles between the primary and non-primary auditory areas, in the left and the right hemispheres. We discuss how these findings help understanding the auditory semiology of certain epileptic seizures and, more generally, the neural substrate of hemispheric specialization for language.

The impact of phase entrainment on auditory detection is highly variable: Revisiting a key finding

Ample evidence shows that the human brain carefully tracks acoustic temporal regularities in the input, perhaps by entraining cortical neural oscillations to the rate of the stimulation. To what extent the entrained oscillatory activity influences processing of upcoming auditory events remains debated. Here, we revisit a critical finding from Hickok et al. (2015) that demonstrated a clear impact of auditory entrainment on subsequent auditory detection. Participants were asked to detect tones embedded in stationary noise, following a noise that was amplitude modulated at 3 Hz. Tonal targets occurred at various phases relative to the preceding noise modulation. The original study (N = 5) showed that the detectability of the tones (presented at near-threshold intensity) fluctuated cyclically at the same rate as the preceding noise modulation. We conducted an exact replication of the original paradigm (N = 23) and a conceptual replication using a shorter experimental procedure (N = 24). Neither experiment revealed significant entrainment effects at the group level. A restricted analysis on the subset of participants (36%) who did show the entrainment effect revealed no consistent phase alignment between detection facilitation and the preceding rhythmic modulation. Interestingly, both experiments showed group-wide presence of a non-cyclic behavioural pattern, wherein participants' detection of the tonal targets was lower at early and late time points of the target period. The two experiments highlight both the sensitivity of the task to elicit oscillatory entrainment and the striking individual variability in performance.

Language Proficiency Entails Tuning Cortical Activity to Second Language Speech

Cortical tracking of linguistic structures in speech, such as phrases (<3 Hz, delta band) and syllables (3-8 Hz, theta band), is known to be crucial for speech comprehension. However, it has not been established whether this effect is related to language proficiency. Here, we investigate how auditory cortical activity in second language (L2) learners tracked L2 speech. Using magnetoencephalography, we recorded brain activity from participants listening to Spanish and Basque. Participants were Spanish native (L1) language speakers studying Basque (L2) at the same language center at three different levels: beginner (Grade 1), intermediate (Grade 2), and advanced (Grade 3). We found that 1) both delta and theta tracking to L2 speech in the auditory cortex were related to L2 learning proficiency and that 2) top-down modulations of activity in the left auditory regions during L2 speech listening-by the left inferior frontal and motor regions in delta band and by the left middle temporal regions in theta band-were also related to L2 proficiency. Altogether, these results indicate that the ability to learn an L2 is related to successful cortical tracking of L2 speech and its modulation by neuronal oscillations in higher-order cortical regions.