Predictive entrainment of natural speech through two fronto-motor top-down channels

The impact of speech rhythm and rate on comprehension in aphasia

BACKGROUND

Speech comprehension impairment in post-stroke aphasia is influenced by speech acoustics. This study investigated the impact of speech rhythm (syllabic isochrony) and rate on comprehension in people with aphasia (PWA). Rhythmical speech was hypothesised to support comprehension in PWA by reducing temporal variation, leading to enhanced speech tracking and more appropriate sampling of the speech stream. Speech rate was hypothesised to influence comprehension through auditory and linguistic processing time.

METHODS

One group of PWA (n = 19) and two groups of control participants (n = 10 and n = 18) performed a sentence-verification. Sentences were presented in two rhythm conditions (natural vs isochronous) and two rate conditions (typical, 3.6 Hz vs slow, 2.6 Hz) in a 2 × 2 factorial design. PWA and one group of controls performed the experiment with clear speech. The second group of controls performed the experiment with perceptually degraded speech.

RESULTS

D-prime analyses measured capacity to detect incongruent endings. Linear mixed effects models investigated the impact of group, rhythm, rate and clarity on d-prime scores. Control participants were negatively affected by isochronous rhythm in comparison to natural rhythm, likely due to alteration in linguistic cues. This negative impact remained or was exacerbated in control participants presented with degraded speech. In comparison, PWA were less affected by isochronous rhythm, despite producing d-prime scores matched to the degraded speech control group. Speech rate affected all groups, but only in interactions with rhythm, indicating that slow-rate isochronous speech was more comprehendible than typical-rate isochronous speech.

CONCLUSIONS

The comprehension network in PWA interacts differently with speech rhythm. Rhythmical speech may support acoustic speech tracking by enhancing predictability and ameliorate the detrimental impact of atypical rhythm on linguistic cues. Alternatively, reduced temporal prediction in aphasia may limit the impact of deviation from natural temporal structure. Reduction of speech rate below the typical range may not benefit comprehension in PWA.

Neural encoding of melodic expectations in music across EEG frequency bands

The human brain tracks regularities in the environment and extrapolates these to predict future events. Prior work on music cognition suggests that low-frequency (1-8 Hz) brain activity encodes melodic predictions beyond the stimulus acoustics. Building on this work, we aimed to disentangle the frequency-specific neural dynamics linked to melodic prediction uncertainty (modelled as entropy) and prediction error (modelled as surprisal) for temporal (note onset) and content (note pitch) information. By using multivariate temporal response function (TRF) models, we re-analysed the electroencephalogram (EEG) from 20 subjects (10 musicians) who listened to Western tonal music. Our results show that melodic expectation metrics improve the EEG reconstruction accuracy in all frequency bands below the gamma range (< 30 Hz). Crucially, we found that entropy contributed more strongly to the reconstruction accuracy enhancement compared to surprisal in all frequency bands. Additionally, we found that the encoding of temporal, but not content, information metrics was not limited to low frequencies, rather it extended to higher frequencies (> 8 Hz). An analysis of the TRF weights revealed that the temporal predictability of a note (entropy of note onset) may be encoded in the delta- (1-4 Hz) and beta-band (12-30 Hz) brain activity prior to the stimulus, suggesting that these frequency bands associate with temporal predictions. Strikingly, we also revealed that melodic expectations selectively enhanced EEG reconstruction accuracy in the beta band for musicians, and in the alpha band (8-12 Hz) for non-musicians, suggesting that musical expertise influences the neural dynamics underlying predictive processing in music cognition.

When direction matters: Neural correlates of interlimb coordination of rhythm and beat

In a previous experiment, we found evidence for a bodily hierarchy governing interlimb coordination of rhythm and beat, using five effectors: 1) Left foot, 2) Right foot, 3) Left hand, 4) Right hand and 5) Voice. The hierarchy implies that, during simultaneous rhythm and beat performance and using combinations of two of these effectors, executing the task by performing the rhythm with an effector that has a higher number than the beat effector is significantly easier than vice versa. To investigate the neural underpinnings of this proposed bodily hierarchy, we here scanned 46 professional musicians using fMRI as they performed a rhythmic pattern with one effector while keeping the beat with another. The conditions combined the voice and the right hand (V + RH), the right hand and the left hand (RH + LH), and the left hand and the right foot (LH + RF). Each effector combination was performed with and against the bodily hierarchy. Going against the bodily hierarchy increased tapping errors significantly and also increased activity in key brain areas functionally associated with top-down sensorimotor control and bottom-up feedback processing, such as the cerebellum and SMA. Conversely, going with the bodily hierarchy engaged areas functionally associated with the default mode network and regions involved in emotion processing. Theories of general brain function that hold prediction as a key principle, propose that action and perception are governed by the brain's attempt to minimise prediction error at different levels in the brain. Following this viewpoint, our results indicate that going against the hierarchy induces stronger prediction errors, while going with the hierarchy allows for a higher degree of automatization. Our results also support the notion of a bodily hierarchy in motor control that prioritizes certain conductive and supportive tapping roles in specific effector combinations.

Specificity of Motor Contributions to Auditory Statistical Learning

Statistical learning is the ability to extract patterned information from continuous sensory signals. Recent evidence suggests that auditory-motor mechanisms play an important role in auditory statistical learning from speech signals. The question remains whether auditory-motor mechanisms support such learning generally or in a domain-specific manner. In Experiment 1, we tested the specificity of motor processes contributing to learning patterns from speech sequences. Participants either whispered or clapped their hands while listening to structured speech. In Experiment 2, we focused on auditory specificity, testing whether whispering equally affects learning patterns from speech and non-speech sequences. Finally, in Experiment 3, we examined whether learning patterns from speech and non-speech sequences are correlated. Whispering had a stronger effect than clapping on learning patterns from speech sequences in Experiment 1. Moreover, whispering impaired statistical learning more strongly from speech than non-speech sequences in Experiment 2. Interestingly, while participants in the non-speech tasks spontaneously synchronized their motor movements with the auditory stream more than participants in the speech tasks, the effect of the motor movements on learning was stronger in the speech domain. Finally, no correlation between speech and non-speech learning was observed. Overall, our findings support the idea that learning statistical patterns from speech versus non-speech relies on segregated mechanisms, and that the speech motor system contributes to auditory statistical learning in a highly specific manner.

Frontal-Auditory Cortical Interactions and Sensory Prediction During Vocal Production in Marmoset Monkeys

The control of speech and vocal production involves the calculation of error between the intended vocal output and the resulting auditory feedback. Consistent with this model, recent evidence has demonstrated that the auditory cortex is suppressed immediately before and during vocal production, yet is still sensitive to differences between vocal output and altered auditory feedback. This suppression has been suggested to be the result of top-down signals containing information about the intended vocal output, potentially originating from motor or other frontal cortical areas. However, whether such frontal areas are the source of suppressive and predictive signaling to the auditory cortex during vocalization is unknown. Here, we simultaneously recorded neural activity from both the auditory and frontal cortices of marmoset monkeys while they produced self-initiated vocalizations. We found increases in neural activity in both brain areas preceding the onset of vocal production, notably changes in both multi-unit activity and local field potential theta-band power. Connectivity analysis using Granger causality demonstrated that frontal cortex sends directed signaling to the auditory cortex during this pre-vocal period. Importantly, this pre-vocal activity predicted both vocalization-induced suppression of the auditory cortex as well as the acoustics of subsequent vocalizations. These results suggest that frontal cortical areas communicate with the auditory cortex preceding vocal production, with frontal-auditory signals that may reflect the transmission of sensory prediction information. This interaction between frontal and auditory cortices may contribute to mechanisms that calculate errors between intended and actual vocal outputs during vocal communication.

Altered Brain Connectivity and Network Topological Organization in a Non-ordinary State of Consciousness Induced by Hypnosis

Hypnosis has been shown to be of clinical utility; however, its underlying neural mechanisms remain unclear. This study aims to investigate altered brain dynamics during the non-ordinary state of consciousness induced by hypnosis. We studied high-density EEG in 9 healthy participants during eyes-closed wakefulness and during hypnosis, induced by a muscle relaxation and eyes fixation procedure. Using hypotheses based on internal and external awareness brain networks, we assessed region-wise brain connectivity between six ROIs (right and left frontal, right and left parietal, upper and lower midline regions) at the scalp level and compared across conditions. Data-driven, graph-theory analyses were also carried out to characterize brain network topology in terms of brain network segregation and integration. During hypnosis, we observed (1) increased delta connectivity between left and right frontal, as well as between right frontal and parietal regions; (2) decreased connectivity for alpha (between right frontal and parietal and between upper and lower midline regions) and beta-2 bands (between upper midline and right frontal, frontal and parietal, also between upper and lower midline regions); and (3) increased network segregation (short-range connections) in delta and alpha bands, and increased integration (long-range connections) in beta-2 band. This higher network integration and segregation was measured bilaterally in frontal and right parietal electrodes, which were identified as central hub regions during hypnosis. This modified connectivity and increased network integration-segregation properties suggest a modification of the internal and external awareness brain networks that may reflect efficient cognitive-processing and lower incidences of mind-wandering during hypnosis.

Predictive coordination of breathing during intra-personal speaking and listening

It has long been known that human breathing is altered during listening and speaking compared to rest: during speaking, inhalation depth is adjusted to the air volume required for the upcoming utterance. During listening, inhalation is temporally aligned to inhalation of the speaker. While evidence for the former is relatively strong, it is virtually absent for the latter. We address both phenomena using recordings of speech envelope and respiration in 30 participants during 14 min of speaking and listening to one's own speech. First, we show that inhalation depth is positively correlated with the total power of the speech envelope in the following utterance. Second, we provide evidence that inhalation during listening to one's own speech is significantly more likely at time points of inhalation during speaking. These findings are compatible with models that postulate alignment of internal forward models of interlocutors with the aim to facilitate communication.

Exploring cognitive individuality and the underlying creativity in statistical learning and phase entrainment

Statistical learning starts at an early age and is intimately linked to brain development and the emergence of individuality. Through such a long period of statistical learning, the brain updates and constructs statistical models, with the model's individuality changing based on the type and degree of stimulation received. However, the detailed mechanisms underlying this process are unknown. This paper argues three main points of statistical learning, including 1) cognitive individuality based on "reliability" of prediction, 2) the construction of information "hierarchy" through chunking, and 3) the acquisition of "1-3Hz rhythm" that is essential for early language and music learning. We developed a Hierarchical Bayesian Statistical Learning (HBSL) model that takes into account both reliability and hierarchy, mimicking the statistical learning processes of the brain. Using this model, we conducted a simulation experiment to visualize the temporal dynamics of perception and production processes through statistical learning. By modulating the sensitivity to sound stimuli, we simulated three cognitive models with different reliability on bottom-up sensory stimuli relative to top-down prior prediction: hypo-sensitive, normal-sensitive, and hyper-sensitive models. We suggested that statistical learning plays a crucial role in the acquisition of 1-3 Hz rhythm. Moreover, a hyper-sensitive model quickly learned the sensory statistics but became fixated on their internal model, making it difficult to generate new information, whereas a hypo-sensitive model has lower learning efficiency but may be more likely to generate new information. Various individual characteristics may not necessarily confer an overall advantage over others, as there may be a trade-off between learning efficiency and the ease of generating new information. This study has the potential to shed light on the heterogeneous nature of statistical learning, as well as the paradoxical phenomenon in which individuals with certain cognitive traits that impede specific types of perceptual abilities exhibit superior performance in creative contexts.

Flexible structure learning under uncertainty

Experience is known to facilitate our ability to interpret sequences of events and make predictions about the future by extracting temporal regularities in our environments. Here, we ask whether uncertainty in dynamic environments affects our ability to learn predictive structures. We exposed participants to sequences of symbols determined by first-order Markov models and asked them to indicate which symbol they expected to follow each sequence. We introduced uncertainty in this prediction task by manipulating the: (a) probability of symbol co-occurrence, (b) stimulus presentation rate. Further, we manipulated feedback, as it is known to play a key role in resolving uncertainty. Our results demonstrate that increasing the similarity in the probabilities of symbol co-occurrence impaired performance on the prediction task. In contrast, increasing uncertainty in stimulus presentation rate by introducing temporal jitter resulted in participants adopting a strategy closer to probability maximization than matching and improving in the prediction tasks. Next, we show that feedback plays a key role in learning predictive statistics. Trial-by-trial feedback yielded stronger improvement than block feedback or no feedback; that is, participants adopted a strategy closer to probability maximization and showed stronger improvement when trained with trial-by-trial feedback. Further, correlating individual strategy with learning performance showed better performance in structure learning for observers who adopted a strategy closer to maximization. Our results indicate that executive cognitive functions (i.e., selective attention) may account for this individual variability in strategy and structure learning ability. Taken together, our results provide evidence for flexible structure learning; individuals adapt their decision strategy closer to probability maximization, reducing uncertainty in temporal sequences and improving their ability to learn predictive statistics in variable environments.

Spatiotemporal dynamics characterise spectral connectivity profiles of continuous speaking and listening

Speech production and perception are fundamental processes of human cognition that both rely on intricate processing mechanisms that are still poorly understood. Here, we study these processes by using magnetoencephalography (MEG) to comprehensively map connectivity of regional brain activity within the brain and to the speech envelope during continuous speaking and listening. Our results reveal not only a partly shared neural substrate for both processes but also a dissociation in space, delay, and frequency. Neural activity in motor and frontal areas is coupled to succeeding speech in delta band (1 to 3 Hz), whereas coupling in the theta range follows speech in temporal areas during speaking. Neural connectivity results showed a separation of bottom-up and top-down signalling in distinct frequency bands during speaking. Here, we show that frequency-specific connectivity channels for bottom-up and top-down signalling support continuous speaking and listening. These findings further shed light on the complex interplay between different brain regions involved in speech production and perception.

Naturalistic Spoken Language Comprehension Is Supported by Alpha and Beta Oscillations

Brain oscillations are prevalent in all species and are involved in numerous perceptual operations. α oscillations are thought to facilitate processing through the inhibition of task-irrelevant networks, while β oscillations are linked to the putative reactivation of content representations. Can the proposed functional role of α and β oscillations be generalized from low-level operations to higher-level cognitive processes? Here we address this question focusing on naturalistic spoken language comprehension. Twenty-two (18 female) Dutch native speakers listened to stories in Dutch and French while MEG was recorded. We used dependency parsing to identify three dependency states at each word: the number of (1) newly opened dependencies, (2) dependencies that remained open, and (3) resolved dependencies. We then constructed forward models to predict α and β power from the dependency features. Results showed that dependency features predict α and β power in language-related regions beyond low-level linguistic features. Left temporal, fundamental language regions are involved in language comprehension in α, while frontal and parietal, higher-order language regions, and motor regions are involved in β. Critically, α- and β-band dynamics seem to subserve language comprehension tapping into syntactic structure building and semantic composition by providing low-level mechanistic operations for inhibition and reactivation processes. Because of the temporal similarity of the α-β responses, their potential functional dissociation remains to be elucidated. Overall, this study sheds light on the role of α and β oscillations during naturalistic spoken language comprehension, providing evidence for the generalizability of these dynamics from perceptual to complex linguistic processes.SIGNIFICANCE STATEMENT It remains unclear whether the proposed functional role of α and β oscillations in perceptual and motor function is generalizable to higher-level cognitive processes, such as spoken language comprehension. We found that syntactic features predict α and β power in language-related regions beyond low-level linguistic features when listening to naturalistic speech in a known language. We offer experimental findings that integrate a neuroscientific framework on the role of brain oscillations as "building blocks" with spoken language comprehension. This supports the view of a domain-general role of oscillations across the hierarchy of cognitive functions, from low-level sensory operations to abstract linguistic processes.

Functional connectivity between parietal and temporal lobes mediates internal forward models during speech production

Internal forward models hypothesize functional links between motor and sensory systems for predicting the consequences of actions. Recently, the cascaded theory proposes that somatosensory estimation in the inferior parietal lobe (IPL) can be a relay computational structure, converting motor signals into predictions of auditory consequences in a serial processing manner during speech production. The study used fMRI with functional connectivity (FC) analyses to investigate the proposed cascaded processes using three speech tasks: overt articulation (OA), silent articulation (SA) and imagined articulation (IA). The FC results showed that connectivity between aIPL and STG was increased in OA compared with SA, suggesting that the relationship between somatosensory and auditory estimations can be modulated by speech tasks. Moreover, stronger connectivity between IFGoper and pIPL, and between pIPL and STG were observed in SA and IA compared with OA. These results are consistent with a cascaded process in the internal forward models.

Dynamics of Functional Networks for Syllable and Word-Level Processing

Speech comprehension requires the ability to temporally segment the acoustic input for higher-level linguistic analysis. Oscillation-based approaches suggest that low-frequency auditory cortex oscillations track syllable-sized acoustic information and therefore emphasize the relevance of syllabic-level acoustic processing for speech segmentation. How syllabic processing interacts with higher levels of speech processing, beyond segmentation, including the anatomical and neurophysiological characteristics of the networks involved, is debated. In two MEG experiments, we investigate lexical and sublexical word-level processing and the interactions with (acoustic) syllable processing using a frequency-tagging paradigm. Participants listened to disyllabic words presented at a rate of 4 syllables/s. Lexical content (native language), sublexical syllable-to-syllable transitions (foreign language), or mere syllabic information (pseudo-words) were presented. Two conjectures were evaluated: (i) syllable-to-syllable transitions contribute to word-level processing; and (ii) processing of words activates brain areas that interact with acoustic syllable processing. We show that syllable-to-syllable transition information compared to mere syllable information, activated a bilateral superior, middle temporal and inferior frontal network. Lexical content resulted, additionally, in increased neural activity. Evidence for an interaction of word- and acoustic syllable-level processing was inconclusive. Decreases in syllable tracking (cerebroacoustic coherence) in auditory cortex and increases in cross-frequency coupling between right superior and middle temporal and frontal areas were found when lexical content was present compared to all other conditions; however, not when conditions were compared separately. The data provide experimental insight into how subtle and sensitive syllable-to-syllable transition information for word-level processing is.

Late dominance of the right hemisphere during narrative comprehension

PET and fMRI studies suggest that auditory narrative comprehension is supported by a bilateral multilobar cortical network. The superior temporal resolution of magnetoencephalography (MEG) makes it an attractive tool to investigate the dynamics of how different neuroanatomic substrates engage during narrative comprehension. Using beta-band power changes as a marker of cortical engagement, we studied MEG responses during an auditory story comprehension task in 31 healthy adults. The protocol consisted of two runs, each interleaving 7 blocks of the story comprehension task with 15 blocks of an auditorily presented math task as a control for phonological processing, working memory, and attention processes. Sources at the cortical surface were estimated with a frequency-resolved beamformer. Beta-band power was estimated in the frequency range of 16-24 Hz over 1-sec epochs starting from 400 msec after stimulus onset until the end of a story or math problem presentation. These power estimates were compared to 1-second epochs of data before the stimulus block onset. The task-related cortical engagement was inferred from beta-band power decrements. Group-level source activations were statistically compared using non-parametric permutation testing. A story-math contrast of beta-band power changes showed greater bilateral cortical engagement within the fusiform gyrus, inferior and middle temporal gyri, parahippocampal gyrus, and left inferior frontal gyrus (IFG) during story comprehension. A math-story contrast of beta power decrements showed greater bilateral but left-lateralized engagement of the middle frontal gyrus and superior parietal lobule. The evolution of cortical engagement during five temporal windows across the presentation of stories showed significant involvement during the first interval of the narrative of bilateral opercular and insular regions as well as the ventral and lateral temporal cortex, extending more posteriorly on the left and medially on the right. Over time, there continued to be sustained right anterior ventral temporal engagement, with increasing involvement of the right anterior parahippocampal gyrus, STG, MTG, posterior superior temporal sulcus, inferior parietal lobule, frontal operculum, and insula, while left hemisphere engagement decreased. Our findings are consistent with prior imaging studies of narrative comprehension, but in addition, they demonstrate increasing right-lateralized engagement over the course of narratives, suggesting an important role for these right-hemispheric regions in semantic integration as well as social and pragmatic inference processing.

Dynamics of nonlinguistic statistical learning: From neural entrainment to the emergence of explicit knowledge

Humans are highly attuned to patterns in the environment. This ability to detect environmental patterns, referred to as statistical learning, plays a key role in many diverse aspects of cognition. However, the spatiotemporal neural mechanisms underlying implicit statistical learning, and how these mechanisms may relate or give rise to explicit learning, remain poorly understood. In the present study, we investigated these different aspects of statistical learning by using an auditory nonlinguistic statistical learning paradigm combined with magnetoencephalography. Twenty-four healthy volunteers were exposed to structured and random tone sequences, and statistical learning was quantified by neural entrainment. Already early during exposure, participants showed strong entrainment to the embedded tone patterns. A significant increase in entrainment over exposure was detected only in the structured condition, reflecting the trajectory of learning. While source reconstruction revealed a wide range of brain areas involved in this process, entrainment in areas around the left pre-central gyrus as well as right temporo-frontal areas significantly predicted behavioral performance. Sensor level results confirmed this relationship between neural entrainment and subsequent explicit knowledge. These results give insights into the dynamic relation between neural entrainment and explicit learning of triplet structures, suggesting that these two aspects are systematically related yet dissociable. Neural entrainment reflects robust, implicit learning of underlying patterns, whereas the emergence of explicit knowledge, likely built on the implicit encoding of structure, varies across individuals and may depend on factors such as sufficient exposure time and attention.

Expertise Modulates Neural Stimulus-Tracking

How does the brain anticipate information in language? When people perceive speech, low-frequency (<10 Hz) activity in the brain synchronizes with bursts of sound and visual motion. This phenomenon, called cortical stimulus-tracking, is thought to be one way that the brain predicts the timing of upcoming words, phrases, and syllables. In this study, we test whether stimulus-tracking depends on domain-general expertise or on language-specific prediction mechanisms. We go on to examine how the effects of expertise differ between frontal and sensory cortex. We recorded electroencephalography (EEG) from human participants who were experts in either sign language or ballet, and we compared stimulus-tracking between groups while participants watched videos of sign language or ballet. We measured stimulus-tracking by computing coherence between EEG recordings and visual motion in the videos. Results showed that stimulus-tracking depends on domain-general expertise, and not on language-specific prediction mechanisms. At frontal channels, fluent signers showed stronger coherence to sign language than to dance, whereas expert dancers showed stronger coherence to dance than to sign language. At occipital channels, however, the two groups of participants did not show different patterns of coherence. These results are difficult to explain by entrainment of endogenous oscillations, because neither sign language nor dance show any periodicity at the frequencies of significant expertise-dependent stimulus-tracking. These results suggest that the brain may rely on domain-general predictive mechanisms to optimize perception of temporally-predictable stimuli such as speech, sign language, and dance.

Statistical Properties of Musical Creativity: Roles of Hierarchy and Uncertainty in Statistical Learning

Creativity is part of human nature and is commonly understood as a phenomenon whereby something original and worthwhile is formed. Owing to this ability, humans can produce innovative information that often facilitates growth in our society. Creativity also contributes to esthetic and artistic productions, such as music and art. However, the mechanism by which creativity emerges in the brain remains debatable. Recently, a growing body of evidence has suggested that statistical learning contributes to creativity. Statistical learning is an innate and implicit function of the human brain and is considered essential for brain development. Through statistical learning, humans can produce and comprehend structured information, such as music. It is thought that creativity is linked to acquired knowledge, but so-called "eureka" moments often occur unexpectedly under subconscious conditions, without the intention to use the acquired knowledge. Given that a creative moment is intrinsically implicit, we postulate that some types of creativity can be linked to implicit statistical knowledge in the brain. This article reviews neural and computational studies on how creativity emerges within the framework of statistical learning in the brain (i.e., statistical creativity). Here, we propose a hierarchical model of statistical learning: statistically chunking into a unit (hereafter and shallow statistical learning) and combining several units (hereafter and deep statistical learning). We suggest that deep statistical learning contributes dominantly to statistical creativity in music. Furthermore, the temporal dynamics of perceptual uncertainty can be another potential causal factor in statistical creativity. Considering that statistical learning is fundamental to brain development, we also discuss how typical versus atypical brain development modulates hierarchical statistical learning and statistical creativity. We believe that this review will shed light on the key roles of statistical learning in musical creativity and facilitate further investigation of how creativity emerges in the brain.

Language Proficiency Entails Tuning Cortical Activity to Second Language Speech

Cortical tracking of linguistic structures in speech, such as phrases (<3 Hz, delta band) and syllables (3-8 Hz, theta band), is known to be crucial for speech comprehension. However, it has not been established whether this effect is related to language proficiency. Here, we investigate how auditory cortical activity in second language (L2) learners tracked L2 speech. Using magnetoencephalography, we recorded brain activity from participants listening to Spanish and Basque. Participants were Spanish native (L1) language speakers studying Basque (L2) at the same language center at three different levels: beginner (Grade 1), intermediate (Grade 2), and advanced (Grade 3). We found that 1) both delta and theta tracking to L2 speech in the auditory cortex were related to L2 learning proficiency and that 2) top-down modulations of activity in the left auditory regions during L2 speech listening-by the left inferior frontal and motor regions in delta band and by the left middle temporal regions in theta band-were also related to L2 proficiency. Altogether, these results indicate that the ability to learn an L2 is related to successful cortical tracking of L2 speech and its modulation by neuronal oscillations in higher-order cortical regions.

Speaking rhythmically can shape hearing

Evidence suggests that temporal predictions arising from the motor system can enhance auditory perception. However, in speech perception, we lack evidence of perception being modulated by production. Here we show a behavioural protocol that captures the existence of such auditory-motor interactions. Participants performed a syllable discrimination task immediately after producing periodic syllable sequences. Two speech rates were explored: a 'natural' (individually preferred) and a fixed 'non-natural' (2 Hz) rate. Using a decoding approach, we show that perceptual performance is modulated by the stimulus phase determined by a participant's own motor rhythm. Remarkably, for 'natural' and 'non-natural' rates, this finding is restricted to a subgroup of the population with quantifiable auditory-motor coupling. The observed pattern is compatible with a neural model assuming a bidirectional interaction of auditory and speech motor cortices. Crucially, the model matches the experimental results only if it incorporates individual differences in the strength of the auditory-motor connection.

Electrophysiological Dynamics of Visual Speech Processing and the Role of Orofacial Effectors for Cross-Modal Predictions

The human brain generates predictions about future events. During face-to-face conversations, visemic information is used to predict upcoming auditory input. Recent studies suggest that the speech motor system plays a role in these cross-modal predictions, however, usually only audio-visual paradigms are employed. Here we tested whether speech sounds can be predicted on the basis of visemic information only, and to what extent interfering with orofacial articulatory effectors can affect these predictions. We registered EEG and employed N400 as an index of such predictions. Our results show that N400's amplitude was strongly modulated by visemic salience, coherent with cross-modal speech predictions. Additionally, N400 ceased to be evoked when syllables' visemes were presented backwards, suggesting that predictions occur only when the observed viseme matched an existing articuleme in the observer's speech motor system (i.e., the articulatory neural sequence required to produce a particular phoneme/viseme). Importantly, we found that interfering with the motor articulatory system strongly disrupted cross-modal predictions. We also observed a late P1000 that was evoked only for syllable-related visual stimuli, but whose amplitude was not modulated by interfering with the motor system. The present study provides further evidence of the importance of the speech production system for speech sounds predictions based on visemic information at the pre-lexical level. The implications of these results are discussed in the context of a hypothesized trimodal repertoire for speech, in which speech perception is conceived as a highly interactive process that involves not only your ears but also your eyes, lips and tongue.

Synchronisation of Neural Oscillations and Cross-modal Influences

At any given moment, we receive multiple signals from our different senses. Prior research has shown that signals in one sensory modality can influence neural activity and behavioural performance associated with another sensory modality. Recent human and nonhuman primate studies suggest that such cross-modal influences in sensory cortices are mediated by the synchronisation of ongoing neural oscillations. In this review, we consider two mechanisms proposed to facilitate cross-modal influences on sensory processing, namely cross-modal phase resetting and neural entrainment. We consider how top-down processes may further influence cross-modal processing in a flexible manner, and we highlight fruitful directions for further research.

The Lateralization of Speech-Brain Coupling Is Differentially Modulated by Intrinsic Auditory and Top-Down Mechanisms

The lateralization of neuronal processing underpinning hearing, speech, language, and music is widely studied, vigorously debated, and still not understood in a satisfactory manner. One set of hypotheses focuses on the temporal structure of perceptual experience and links auditory cortex asymmetries to underlying differences in neural populations with differential temporal sensitivity (e.g., ideas advanced by Zatorre et al. (2002) and Poeppel (2003). The Asymmetric Sampling in Time theory (AST) (Poeppel, 2003), builds on cytoarchitectonic differences between auditory cortices and predicts that modulation frequencies within the range of, roughly, the syllable rate, are more accurately tracked by the right hemisphere. To date, this conjecture is reasonably well supported, since - while there is some heterogeneity in the reported findings - the predicted asymmetrical entrainment has been observed in various experimental protocols. Here, we show that under specific processing demands, the rightward dominance disappears. We propose an enriched and modified version of the asymmetric sampling hypothesis in the context of speech. Recent work (Rimmele et al., 2018b) proposes two different mechanisms to underlie the auditory tracking of the speech envelope: one derived from the intrinsic oscillatory properties of auditory regions; the other induced by top-down signals coming from other non-auditory regions of the brain. We propose that under non-speech listening conditions, the intrinsic auditory mechanism dominates and thus, in line with AST, entrainment is rightward lateralized, as is widely observed. However, (i) depending on individual brain structural/functional differences, and/or (ii) in the context of specific speech listening conditions, the relative weight of the top-down mechanism can increase. In this scenario, the typically observed auditory sampling asymmetry (and its rightward dominance) diminishes or vanishes.