The effects of stress and statistical cues on continuous speech segmentation: An event-related brain potential study

Auditory associative word learning in adults: The effects of musical experience and stimulus ordering

Evidence for sequential associative word learning in the auditory domain has been identified in infants, while adults have shown difficulties. To better understand which factors may facilitate adult auditory associative word learning, we assessed the role of auditory expertise as a learner-related property and stimulus order as a stimulus-related manipulation in the association of auditory objects and novel labels. We tested in the first experiment auditorily-trained musicians versus athletes (high-level control group) and in the second experiment stimulus ordering, contrasting object-label versus label-object presentation. Learning was evaluated from Event-Related Potentials (ERPs) during training and subsequent testing phases using a cluster-based permutation approach, as well as accuracy-judgement responses during test. Results revealed for musicians a late positive component in the ERP during testing, but neither an N400 (400-800 ms) nor behavioral effects were found at test, while athletes did not show any effect of learning. Moreover, the object-label-ordering group only exhibited emerging association effects during training, while the label-object-ordering group showed a trend-level late ERP effect (800-1200 ms) during test as well as above chance accuracy-judgement scores. Thus, our results suggest the learner-related property of auditory expertise and stimulus-related manipulation of stimulus ordering modulate auditory associative word learning in adults.

Chunk boundaries disrupt dependency processing in an AG: Reconciling incremental processing and discrete sampling

Language is rooted in our ability to compose: We link words together, fusing their meanings. Links are not limited to neighboring words but often span intervening words. The ability to process these non-adjacent dependencies (NADs) conflicts with the brain's sampling of speech: We consume speech in chunks that are limited in time, containing only a limited number of words. It is unknown how we link words together that belong to separate chunks. Here, we report that we cannot-at least not so well. In our electroencephalography (EEG) study, 37 human listeners learned chunks and dependencies from an artificial grammar (AG) composed of syllables. Multi-syllable chunks to be learned were equal-sized, allowing us to employ a frequency-tagging approach. On top of chunks, syllable streams contained NADs that were either confined to a single chunk or crossed a chunk boundary. Frequency analyses of the EEG revealed a spectral peak at the chunk rate, showing that participants learned the chunks. NADs that cross boundaries were associated with smaller electrophysiological responses than within-chunk NADs. This shows that NADs are processed readily when they are confined to the same chunk, but not as well when crossing a chunk boundary. Our findings help to reconcile the classical notion that language is processed incrementally with recent evidence for discrete perceptual sampling of speech. This has implications for language acquisition and processing as well as for the general view of syntax in human language.

Hebbian learning can explain rhythmic neural entrainment to statistical regularities

In many domains, learners extract recurring units from continuous sequences. For example, in unknown languages, fluent speech is perceived as a continuous signal. Learners need to extract the underlying words from this continuous signal and then memorize them. One prominent candidate mechanism is statistical learning, whereby learners track how predictive syllables (or other items) are of one another. Syllables within the same word predict each other better than syllables straddling word boundaries. But does statistical learning lead to memories of the underlying words-or just to pairwise associations among syllables? Electrophysiological results provide the strongest evidence for the memory view. Electrophysiological responses can be time-locked to statistical word boundaries (e.g., N400s) and show rhythmic activity with a periodicity of word durations. Here, I reproduce such results with a simple Hebbian network. When exposed to statistically structured syllable sequences (and when the underlying words are not excessively long), the network activation is rhythmic with the periodicity of a word duration and activation maxima on word-final syllables. This is because word-final syllables receive more excitation from earlier syllables with which they are associated than less predictable syllables that occur earlier in words. The network is also sensitive to information whose electrophysiological correlates were used to support the encoding of ordinal positions within words. Hebbian learning can thus explain rhythmic neural activity in statistical learning tasks without any memory representations of words. Learners might thus need to rely on cues beyond statistical associations to learn the words of their native language. RESEARCH HIGHLIGHTS: Statistical learning may be utilized to identify recurring units in continuous sequences (e.g., words in fluent speech) but may not generate explicit memory for words. Exposure to statistically structured sequences leads to rhythmic activity with a period of the duration of the underlying units (e.g., words). I show that a memory-less Hebbian network model can reproduce this rhythmic neural activity as well as putative encodings of ordinal positions observed in earlier research. Direct tests are needed to establish whether statistical learning leads to declarative memories for words.

Rhythmically Modulating Neural Entrainment during Exposure to Regularities Influences Statistical Learning

The ability to discover regularities in the environment, such as syllable patterns in speech, is known as statistical learning. Previous studies have shown that statistical learning is accompanied by neural entrainment, in which neural activity temporally aligns with repeating patterns over time. However, it is unclear whether these rhythmic neural dynamics play a functional role in statistical learning or whether they largely reflect the downstream consequences of learning, such as the enhanced perception of learned words in speech. To better understand this issue, we manipulated participants' neural entrainment during statistical learning using continuous rhythmic visual stimulation. Participants were exposed to a speech stream of repeating nonsense words while viewing either (1) a visual stimulus with a "congruent" rhythm that aligned with the word structure, (2) a visual stimulus with an incongruent rhythm, or (3) a static visual stimulus. Statistical learning was subsequently measured using both an explicit and implicit test. Participants in the congruent condition showed a significant increase in neural entrainment over auditory regions at the relevant word frequency, over and above effects of passive volume conduction, indicating that visual stimulation successfully altered neural entrainment within relevant neural substrates. Critically, during the subsequent implicit test, participants in the congruent condition showed an enhanced ability to predict upcoming syllables and stronger neural phase synchronization to component words, suggesting that they had gained greater sensitivity to the statistical structure of the speech stream relative to the incongruent and static groups. This learning benefit could not be attributed to strategic processes, as participants were largely unaware of the contingencies between the visual stimulation and embedded words. These results indicate that manipulating neural entrainment during exposure to regularities influences statistical learning outcomes, suggesting that neural entrainment may functionally contribute to statistical learning. Our findings encourage future studies using non-invasive brain stimulation methods to further understand the role of entrainment in statistical learning.

Neural correlates of statistical learning in developmental dyslexia: An electroencephalography study

The human brain extracts statistical regularities from the surrounding environment in a process called statistical learning. Behavioural evidence suggests that developmental dyslexia affects statistical learning. However, surprisingly few studies have assessed how developmental dyslexia affects the neural processing underlying this type of learning. We used electroencephalography to explore the neural correlates of an important aspect of statistical learning - sensitivity to transitional probabilities - in individuals with developmental dyslexia. Adults diagnosed with developmental dyslexia (n = 17) and controls (n = 19) were exposed to a continuous stream of sound triplets. Every so often, a triplet ending had a low transitional probability given the triplet's first two sounds ("statistical deviants"). Furthermore, every so often a triplet ending was presented from a deviant location ("acoustic deviants"). We examined mismatch negativity elicited by statistical deviants (sMMN), and MMN elicited by location deviants (i.e., acoustic changes). Acoustic deviants elicited a MMN which was larger in the control group than in the developmental dyslexia group. Statistical deviants elicited a small, yet significant, sMMN in the control group, but not in the developmental dyslexia group. However, the difference between the groups was not significant. Our findings indicate that the neural mechanisms underlying pre-attentive acoustic change detection and implicit statistical auditory learning are both affected in developmental dyslexia.

Order of statistical learning depends on perceptive uncertainty

Statistical learning (SL) is an innate mechanism by which the brain automatically encodes the n-th order transition probability (TP) of a sequence and grasps the uncertainty of the TP distribution. Through SL, the brain predicts a subsequent event (e _n+1 ) based on the preceding events (e _n ) that have a length of "n". It is now known that uncertainty modulates prediction in top-down processing by the human predictive brain. However, the manner in which the human brain modulates the order of SL strategies based on the degree of uncertainty remains an open question. The present study examined how uncertainty modulates the neural effects of SL and whether differences in uncertainty alter the order of SL strategies. It used auditory sequences in which the uncertainty of sequential information is manipulated based on the conditional entropy. Three sequences with different TP ratios of 90:10, 80:20, and 67:33 were prepared as low-, intermediate, and high-uncertainty sequences, respectively (conditional entropy: 0.47, 0.72, and 0.92 bit, respectively). Neural responses were recorded when the participants listened to the three sequences. The results showed that stimuli with lower TPs elicited a stronger neural response than those with higher TPs, as demonstrated by a number of previous studies. Furthermore, we found that participants adopted higher-order SL strategies in the high uncertainty sequence. These results may indicate that the human brain has an ability to flexibly alter the order based on the uncertainty. This uncertainty may be an important factor that determines the order of SL strategies. Particularly, considering that a higher-order SL strategy mathematically allows the reduction of uncertainty in information, we assumed that the brain may take higher-order SL strategies when encountering high uncertain information in order to reduce the uncertainty. The present study may shed new light on understanding individual differences in SL performance across different uncertain situations.

Explicit Instructions Do Not Enhance Auditory Statistical Learning in Children With Developmental Language Disorder: Evidence From Event-Related Potentials

A current issue in psycholinguistic research is whether the language difficulties exhibited by children with developmental language disorder [DLD, previously labeled specific language impairment (SLI)] are due to deficits in their abilities to pick up patterns in the sensory environment, an ability known as statistical learning (SL), and the extent to which explicit learning mechanisms can be used to compensate for those deficits. Studies designed to test the compensatory role of explicit learning mechanisms in children with DLD are, however, scarce, and the few conducted so far have led to inconsistent results. This work aimed to provide new insights into the role that explicit learning mechanisms might play on implicit learning deficits in children with DLD by resorting to a new approach. This approach involved not only the collection of event-related potentials (ERPs), while preschool children with DLD [relative to typical language developmental (TLD) controls] were exposed to a continuous auditory stream made of the repetition of three-syllable nonsense words but, importantly, the collection of ERPs when the same children performed analogous versions of the same auditory SL task first under incidental (implicit) and afterward under intentional (explicit) conditions. In each of these tasks, the level of predictability of the three-syllable nonsense words embedded in the speech streams was also manipulated (high vs. low) to mimic natural languages closely. At the end of both tasks' exposure phase, children performed a two-alternative forced-choice (2-AFC) task from which behavioral evidence of SL was obtained. Results from the 2-AFC tasks failed to show reliable signs of SL in both groups of children. The ERPs data showed, however, significant modulations in the N100 and N400 components, taken as neural signatures of word segmentation in the brain, even though a detailed analysis of the neural responses revealed that only children from the TLD group seem to have taken advantage of the previous knowledge to enhance SL functioning. These results suggest that children with DLD showed deficits both in implicit and explicit learning mechanisms, casting doubts on the efficiency of the interventions relying on explicit instructions to help children with DLD to overcome their language difficulties.

A Special Role of Syllables, But Not Vowels or Consonants, for Nonadjacent Dependency Learning

Successful language processing entails tracking (morpho)syntactic relationships between distant units of speech, so-called nonadjacent dependencies (NADs). Many cues to such dependency relations have been identified, yet the linguistic elements encoding them have received little attention. In the present investigation, we tested whether and how these elements, here syllables, consonants, and vowels, affect behavioral learning success as well as learning-related changes in neural activity in relation to item-specific NAD learning. In a set of two EEG studies with adults, we compared learning under conditions where either all segment types (Experiment 1) or only one segment type (Experiment 2) was informative. The collected behavioral and ERP data indicate that, when all three segment types are available, participants mainly rely on the syllable for NAD learning. With only one segment type available for learning, adults also perform most successfully with syllable-based dependencies. Although we find no evidence for successful learning across vowels in Experiment 2, dependencies between consonants seem to be identified at least passively at the phonetic-feature level. Together, these results suggest that successful item-specific NAD learning may depend on the availability of syllabic information. Furthermore, they highlight consonants' distinctive power to support lexical processes. Although syllables show a clear facilitatory function for NAD learning, the underlying mechanisms of this advantage require further research.

Learning Words While Listening to Syllables: Electrophysiological Correlates of Statistical Learning in Children and Adults

From an early age, exposure to a spoken language has allowed us to implicitly capture the structure underlying the succession of speech sounds in that language and to segment it into meaningful units (words). Statistical learning (SL), the ability to pick up patterns in the sensory environment without intention or reinforcement, is thus assumed to play a central role in the acquisition of the rule-governed aspects of language, including the discovery of word boundaries in the continuous acoustic stream. Although extensive evidence has been gathered from artificial languages experiments showing that children and adults are able to track the regularities embedded in the auditory input, as the probability of one syllable to follow another syllable in the speech stream, the developmental trajectory of this ability remains controversial. In this work, we have collected Event-Related Potentials (ERPs) while 5-year-old children and young adults (university students) were exposed to a speech stream made of the repetition of eight three-syllable nonsense words presenting different levels of predictability (high vs. low) to mimic closely what occurs in natural languages and to get new insights into the changes that the mechanisms underlying auditory statistical learning (aSL) might undergo through the development. The participants performed the aSL task first under implicit and, subsequently, under explicit conditions to further analyze if children take advantage of previous knowledge of the to-be-learned regularities to enhance SL, as observed with the adult participants. These findings would also contribute to extend our knowledge of the mechanisms available to assist SL at each developmental stage. Although behavioral signs of learning, even under explicit conditions, were only observed for the adult participants, ERP data showed evidence of online segmentation in the brain in both groups, as indexed by modulations in the N100 and N400 components. A detailed analysis of the neural data suggests, however, that adults and children rely on different mechanisms to assist the extraction of word-like units from the continuous speech stream, hence supporting the view that SL with auditory linguistic materials changes through development.

Neural processes underlying statistical learning for speech segmentation in dogs

To learn words, humans extract statistical regularities from speech. Multiple species use statistical learning also to process speech, but the neural underpinnings of speech segmentation in non-humans remain largely unknown. Here, we investigated computational and neural markers of speech segmentation in dogs, a phylogenetically distant mammal that efficiently navigates humans' social and linguistic environment. Using electroencephalography (EEG), we compared event-related responses (ERPs) for artificial words previously presented in a continuous speech stream with different distributional statistics. Results revealed an early effect (220-470 ms) of transitional probability and a late component (590-790 ms) modulated by both word frequency and transitional probability. Using fMRI, we searched for brain regions sensitive to statistical regularities in speech. Structured speech elicited lower activity in the basal ganglia, a region involved in sequence learning, and repetition enhancement in the auditory cortex. Speech segmentation in dogs, similar to that of humans, involves complex computations, engaging both domain-general and modality-specific brain areas. VIDEO ABSTRACT.

Dynamics of nonlinguistic statistical learning: From neural entrainment to the emergence of explicit knowledge

Humans are highly attuned to patterns in the environment. This ability to detect environmental patterns, referred to as statistical learning, plays a key role in many diverse aspects of cognition. However, the spatiotemporal neural mechanisms underlying implicit statistical learning, and how these mechanisms may relate or give rise to explicit learning, remain poorly understood. In the present study, we investigated these different aspects of statistical learning by using an auditory nonlinguistic statistical learning paradigm combined with magnetoencephalography. Twenty-four healthy volunteers were exposed to structured and random tone sequences, and statistical learning was quantified by neural entrainment. Already early during exposure, participants showed strong entrainment to the embedded tone patterns. A significant increase in entrainment over exposure was detected only in the structured condition, reflecting the trajectory of learning. While source reconstruction revealed a wide range of brain areas involved in this process, entrainment in areas around the left pre-central gyrus as well as right temporo-frontal areas significantly predicted behavioral performance. Sensor level results confirmed this relationship between neural entrainment and subsequent explicit knowledge. These results give insights into the dynamic relation between neural entrainment and explicit learning of triplet structures, suggesting that these two aspects are systematically related yet dissociable. Neural entrainment reflects robust, implicit learning of underlying patterns, whereas the emergence of explicit knowledge, likely built on the implicit encoding of structure, varies across individuals and may depend on factors such as sufficient exposure time and attention.

Perceptual uncertainty modulates auditory statistical learning: A magnetoencephalography study

Statistical learning allows comprehension of structured information, such as that in language and music. The brain computes a sequence's transition probability and predicts future states to minimise sensory reaction and derive entropy (uncertainty) from sequential information. Neurophysiological studies have revealed that early event-related neural responses (P1 and N1) reflect statistical learning - when the brain encodes transition probability in stimulus sequences, it predicts an upcoming stimulus with a high transition probability and suppresses the early event-related responses to a stimulus with a high transition probability. This amplitude difference between high and low transition probabilities reflects statistical learning effects. However, how a sequence's transition probability ratio affects neural responses contributing to statistical learning effects remains unknown. This study investigated how transition-probability ratios or conditional entropy (uncertainty) in auditory sequences modulate the early event-related neuromagnetic responses of P1m and N1m. Sequence uncertainties were manipulated using three different transition-probability ratios: 90:10%, 80:20%, and 67:33% (conditional entropy: 0.47, 0.72, and 0.92 bits, respectively). Neuromagnetic responses were recorded when participants listened to sequential sounds with these three transition probabilities. Amplitude differences between lower and higher probabilities were larger in sequences with transition-probability ratios of 90:10% and smaller in sequences with those of 67:33%, compared to sequences with those of 80:20%. This suggests that the transition-probability ratio finely tunes P1m and N1m. Our study also showed larger amplitude differences between frequent- and rare-transition stimuli in P1m than in N1m. This indicates that information about transition-probability differences may be calculated in earlier cognitive processes.

Statistical learning and prosodic bootstrapping differentially affect neural synchronization during speech segmentation

Neural oscillations constitute an intrinsic property of functional brain organization that facilitates the tracking of linguistic units at multiple time scales through brain-to-stimulus alignment. This ubiquitous neural principle has been shown to facilitate speech segmentation and word learning based on statistical regularities. However, there is no common agreement yet on whether speech segmentation is mediated by a transition of neural synchronization from syllable to word rate, or whether the two time scales are concurrently tracked. Furthermore, it is currently unknown whether syllable transition probability contributes to speech segmentation when lexical stress cues can be directly used to extract word forms. Using Inter-Trial Coherence (ITC) analyses in combinations with Event-Related Potentials (ERPs), we showed that speech segmentation based on both statistical regularities and lexical stress cues was accompanied by concurrent neural synchronization to syllables and words. In particular, ITC at the word rate was generally higher in structured compared to random sequences, and this effect was particularly pronounced in the flat condition. Furthermore, ITC at the syllable rate dynamically increased across the blocks of the flat condition, whereas a similar modulation was not observed in the stressed condition. Notably, in the flat condition ITC at both time scales correlated with each other, and changes in neural synchronization were accompanied by a rapid reconfiguration of the P200 and N400 components with a close relationship between ITC and ERPs. These results highlight distinct computational principles governing neural synchronization to pertinent linguistic units while segmenting speech under different listening conditions.

Maternal stress predicts neural responses during auditory statistical learning in 26-month-old children: An event-related potential study

Exposure to high levels of early life stress have been associated with long-term difficulties in learning, behavior, and health, with particular impact evident in the language domain. While some have proposed that the increased stress of living in a low-income household mediates observed associations between socioeconomic status (SES) and child outcomes, considerable individual differences have been observed. The extent to which specific variables associated with socioeconomic status - in particular exposure to stressful life events - influence the neurocognitive mechanisms underlying language acquisition are not well understood. Auditory statistical learning, or the ability to segment a continuous auditory stream based on its statistical properties, develops during early infancy and is one mechanism thought to underlie language learning. The present study used an event-related potential (ERP) paradigm to test whether maternal stress, adjusting for socioeconomic variables (e.g., family income, maternal education) was associated with neurocognitive processes underlying statistical learning in a sample of 26-month-old children (n = 23) from predominantly low- to middle-income backgrounds. Event-related potentials were recorded while children listened to a continuous stream of tri-tone "words" in which tone elements varied in transitional probability. "Tone-words" were presented in random order, such that Tone 1 always predicted Tones 2 and 3 (transitional probability for Tone 3 = 1.0), but Tone 1 appeared randomly. A larger P2 amplitude was observed in response to Tone 3 compared to Tone 1, demonstrating that children implicitly tracked differences in transitional probabilities during passive listening. Maternal reports of stress at 26 months, adjusting for SES, were negatively associated with difference in P2 amplitude between Tones 1 and 3. These findings suggest that maternal stress, within a low-SES context, is associated with the manner in which children process statistical properties of auditory input.

Cortical Tracking of Speech: Toward Collaboration between the Fields of Signal and Sentence Processing

In recent years, a growing number of studies have used cortical tracking methods to investigate auditory language processing. Although most studies that employ cortical tracking stem from the field of auditory signal processing, this approach should also be of interest to psycholinguistics-particularly the subfield of sentence processing-given its potential to provide insight into dynamic language comprehension processes. However, there has been limited collaboration between these fields, which we suggest is partly because of differences in theoretical background and methodological constraints, some mutually exclusive. In this paper, we first review the theories and methodological constraints that have historically been prioritized in each field and provide concrete examples of how some of these constraints may be reconciled. We then elaborate on how further collaboration between the two fields could be mutually beneficial. Specifically, we argue that the use of cortical tracking methods may help resolve long-standing debates in the field of sentence processing that commonly used behavioral and neural measures (e.g., ERPs) have failed to adjudicate. Similarly, signal processing researchers who use cortical tracking may be able to reduce noise in the neural data and broaden the impact of their results by controlling for linguistic features of their stimuli and by using simple comprehension tasks. Overall, we argue that a balance between the methodological constraints of the two fields will lead to an overall improved understanding of language processing as well as greater clarity on what mechanisms cortical tracking of speech reflects. Increased collaboration will help resolve debates in both fields and will lead to new and exciting avenues for research.

Predictive Processing during a Naturalistic Statistical Learning Task in ASD

Children's sensitivity to regularities within the linguistic stream, such as the likelihood that syllables co-occur, is foundational to speech segmentation and language acquisition. Yet, little is known about the neurocognitive mechanisms underlying speech segmentation in typical development and in neurodevelopmental disorders that impact language acquisition such as autism spectrum disorder (ASD). Here, we investigate the neural signals of statistical learning in 15 human participants (children ages 8-12) with a clinical diagnosis of ASD and 14 age-matched and gender-matched typically developing peers. We tracked the evoked neural responses to syllable sequences in a naturalistic statistical learning corpus using magnetoencephalography (MEG) in the left primary auditory cortex, posterior superior temporal gyrus (pSTG), and inferior frontal gyrus (IFG), across three repetitions of the passage. In typically developing children, we observed a neural index of learning in all three regions of interest (ROIs), measured by the change in evoked response amplitude as a function of syllable surprisal across passage repetitions. As surprisal increased, the amplitude of the neural response increased; this sensitivity emerged after repeated exposure to the corpus. Children with ASD did not show this pattern of learning in all three regions. We discuss two possible hypotheses related to children's sensitivity to bottom-up sensory deficits and difficulty with top-down incremental processing.

In the absence of visual input: Electrophysiological evidence of infants' mapping of labels onto auditory objects

Despite the prominence of non-visual semantic features for some words (e.g., siren or thunder), little is known about when and how the meanings of those words that refer to auditory objects can be acquired in early infancy. With associative learning being an important mechanism of word learning, we ask the question whether associations between sounds and words lead to similar learning effects as associations between visual objects and words. In an event-related potential (ERP) study, 10- to 12-month-old infants were presented with pairs of environmental sounds and pseudowords in either a consistent (where sound-word mapping can occur) or inconsistent manner. Subsequently, the infants were presented with sound-pseudoword combinations either matching or violating the consistent pairs from the training phase. In the training phase, we observed word-form familiarity effects and pairing consistency effects for ERPs time-locked to the onset of the word. The test phase revealed N400-like effects for violated pairs as compared to matching pairs. These results indicate that associative word learning is also possible for auditory objects before infants' first birthday. The specific temporal occurrence of the N400-like effect and topological distribution of the ERPs suggests that the object's modality has an impact on how novel words are processed.

Beta-Band Activity Is a Signature of Statistical Learning

Through statistical learning (SL), cognitive systems may discover the underlying regularities in the environment. Testing human adults (n = 35, 21 females), we document, in the context of a classical visual SL task, divergent rhythmic EEG activity in the interstimulus delay periods within patterns versus between patterns (i.e., pattern transitions). Our findings reveal increased oscillatory activity in the beta band (∼20 Hz) at triplet transitions that indexes learning: it emerges with increased pattern repetitions; and importantly, it is highly correlated with behavioral learning outcomes. These findings hold the promise of converging on an online measure of learning regularities and provide important theoretical insights regarding the mechanisms of SL and prediction.SIGNIFICANCE STATEMENT Statistical learning has become a major theoretical construct in cognitive science, providing the primary means by which organisms learn about regularities in the environment. As such, it is a critical building block for basic and higher-order cognitive functions. Here we identify, for the first time, a spectral neural index in the time window before stimulus presentation, which evolves with increased pattern exposure, and is predictive of learning performance. The manifestation of learning that is revealed, not in stimulus processing but in the blank interval between stimuli, makes a direct link between the fields of statistical learning on the one hand and either prediction or consolidation on the other hand, suggesting a possible mechanistic account of visual statistical learning.

Not All Words Are Equally Acquired: Transitional Probabilities and Instructions Affect the Electrophysiological Correlates of Statistical Learning

Statistical learning (SL), the process of extracting regularities from the environment, is a fundamental skill of our cognitive system to structure the world regularly and predictably. SL has been studied using mainly behavioral tasks under implicit conditions and with triplets presenting the same level of difficulty, i.e., a mean transitional probability (TP) of 1.00. Yet, the neural mechanisms underlying SL under other learning conditions remain largely unknown. Here, we investigated the neurofunctional correlates of SL using triplets (i.e., three-syllable nonsense words) with a mean TP of 1.00 (easy "words") and 0.50 (hard "words") in an SL task performed under incidental (implicit) and intentional (explicit) conditions, to determine whether the same core mechanisms were recruited to assist learning. Event-related potentials (ERPs) were recorded while participants listened firstly to a continuous auditory stream made of the concatenation of four easy and four hard "words" under implicit instructions, and subsequently to another auditory stream made of the concatenation of four easy and four hard "words" drawn from another artificial language under explicit instructions. The stream in each of the SL tasks was presented in two consecutive blocks of ~3.5-min each (~7-min in total) to further examine how ERP components might change over time. Behavioral measures of SL were collected after the familiarization phase of each SL task by asking participants to perform a two-alternative forced-choice (2-AFC) task. Results from the 2-AFC tasks revealed a moderate but reliable level of SL, with no differences between conditions. ERPs were, nevertheless, sensitive to the effect of TPs, showing larger amplitudes of N400 for easy "words," as well as to the effect of instructions, with a reduced N250 for "words" presented under explicit conditions. Also, significant differences in the N100 were found as a result of the interaction between TPs, instructions, and the amount of exposure to the auditory stream. Taken together, our findings suggest that triplets' predictability impacts the emergence of "words" representations in the brain both for statistical regularities extracted under incidental and intentional instructions, although the prior knowledge of the "words" seems to favor the recruitment of different SL mechanisms.

Neurocognitive Correlates of Statistical Learning of Orthographic-Semantic Connections in Chinese Adult Learners

We examined the neural correlates of the statistical learning of orthographic-semantic connections in Chinese adult learners. Visual event-related potentials (ERPs) were recorded while participants were exposed to a sequence of artificial logographic characters containing semantic radicals carrying low, moderate, or high levels of semantic consistency. The behavioral results showed that the mean accuracy of participants' recognition of previously exposed characters was 63.1% that was significantly above chance level (50%), indicating the statistical learning of the regularities of semantic radicals. The ERP data revealed a temporal sequence of the neural process of statistical learning of orthographic-semantic connections, and different brain indexes were found to be associated with this processing, i.e., a clear N170-P200-N400 pattern. For N170, the larger negative amplitudes were evoked by the high and moderate consistency than the low consistency. For P200, the mean amplitudes elicited by the moderate and low consistency were larger than the high consistency. In contrast, a larger N400 amplitude was observed in the low than moderate and high consistency; and more negative amplitude was elicited by the moderate than high consistency. We propose that the initial potential shifts (N170 and P200) may reflect orthographic or graphic form identification, while the later component (N400) may be associated with semantic information analysis.

Concurrent Statistical Learning of Ignored and Attended Sound Sequences: An MEG Study

In an auditory environment, humans are frequently exposed to overlapping sound sequences such as those made by human voices and musical instruments, and we can acquire information embedded in these sequences via attentional and nonattentional accesses. Whether the knowledge acquired by attentional accesses interacts with that acquired by nonattentional accesses is unknown, however. The present study examined how the statistical learning (SL) of two overlapping sound sequences is reflected in neurophysiological and behavioral responses, and how the learning effects are modulated by attention to each sequence. SL in this experimental paradigm was reflected in a neuromagnetic response predominantly in the right hemisphere, and the learning effects were not retained when attention to the tone streams was switched during the learning session. These results suggest that attentional and nonattentional learning scarcely interact with each other and that there may be a specific system for nonattentional learning, which is independent of attentional learning.

Statistical learning of speech regularities can occur outside the focus of attention

Statistical learning, the process of extracting regularities from the environment, plays an essential role in many aspects of cognition, including speech segmentation and language acquisition. A key component of statistical learning in a linguistic context is the perceptual binding of adjacent individual units (e.g., syllables) into integrated composites (e.g., multisyllabic words). A second, conceptually dissociable component of statistical learning is the memory storage of these integrated representations. Here we examine whether these two dissociable components of statistical learning are differentially impacted by top-down, voluntary attentional resources. Learners' attention was either focused towards or diverted from a speech stream made up of repeating nonsense words. Building on our previous findings, we quantified the online perceptual binding of individual syllables into component words using an EEG-based neural entrainment measure. Following exposure, statistical learning was assessed using offline tests, sensitive to both perceptual binding and memory storage. Neural measures verified that our manipulation of selective attention successfully reduced limited-capacity resources to the speech stream. Diverting attention away from the speech stream did not alter neural entrainment to the component words or post-exposure familiarity ratings, but did impact performance on an indirect reaction-time based memory test. We conclude that theoretically dissociable components of statistically learning are differentially impacted by attention and top-down processing resources. A reduction in attention to the speech stream may impede memory storage of the component words. In contrast, the moment-by-moment perceptual binding of speech regularities can occur even while learners' attention is focused on a demanding concurrent task, and we found no evidence that selective attention modulates this process. These results suggest that learners can acquire basic statistical properties of language without directly focusing on the speech input, potentially opening up previously overlooked opportunities for language learning, particularly in adult learners.

Neurophysiological Markers of Statistical Learning in Music and Language: Hierarchy, Entropy, and Uncertainty

Statistical learning (SL) is a method of learning based on the transitional probabilities embedded in sequential phenomena such as music and language. It has been considered an implicit and domain-general mechanism that is innate in the human brain and that functions independently of intention to learn and awareness of what has been learned. SL is an interdisciplinary notion that incorporates information technology, artificial intelligence, musicology, and linguistics, as well as psychology and neuroscience. A body of recent study has suggested that SL can be reflected in neurophysiological responses based on the framework of information theory. This paper reviews a range of work on SL in adults and children that suggests overlapping and independent neural correlations in music and language, and that indicates disability of SL. Furthermore, this article discusses the relationships between the order of transitional probabilities (TPs) (i.e., hierarchy of local statistics) and entropy (i.e., global statistics) regarding SL strategies in human's brains; claims importance of information-theoretical approaches to understand domain-general, higher-order, and global SL covering both real-world music and language; and proposes promising approaches for the application of therapy and pedagogy from various perspectives of psychology, neuroscience, computational studies, musicology, and linguistics.

Exploring the neural correlates of lexical stress perception in english among Chinese-English bilingual children with autism spectrum disorder: An ERP study

Previous studies found that individuals with autism spectrum disorder (ASD) were less sensitive to the variations of lexical stress in their native language than typically developing controls. However, no study has been conducted to explore the perception of lexical stress in the second language among individuals with ASD. Using ERPs (event-related potentials) measurement with an oddball paradigm, the current study examined and compared the neural responses by Chinese-English bilingual children with ASD and typically developing controls in the processing of English lexical stress. The results showed that when compared with typically developing controls, children with ASD manifested reduced MMN (mismatch negativity) amplitude at the left temporal-parietal and parietal sites, indicating that they were less sensitive to lexical stress. However, a more negative MMN response was found for ASD group than for typically developing group at the right central-parietal, temporal-parietal, and temporal sites. In addition, the right hemisphere was more activated than the left hemisphere for ASD group, which might be derived from the reversed asymmetry of brain activation for individuals with ASD when processing language-related stimuli.

Enhanced Neonatal Brain Responses To Sung Streams Predict Vocabulary Outcomes By Age 18 Months

Words and melodies are some of the basic elements infants are able to extract early in life from the auditory input. Whether melodic cues contained in songs can facilitate word-form extraction immediately after birth remained unexplored. Here, we provided converging neural and computational evidence of the early benefit of melodies for language acquisition. Twenty-eight neonates were tested on their ability to extract word-forms from continuous flows of sung and spoken syllabic sequences. We found different brain dynamics for sung and spoken streams and observed successful detection of word-form violations in the sung condition only. Furthermore, neonatal brain responses for sung streams predicted expressive vocabulary at 18 months as demonstrated by multiple regression and cross-validation analyses. These findings suggest that early neural individual differences in prosodic speech processing might be a good indicator of later language outcomes and could be considered as a relevant factor in the development of infants' language skills.

Experience with a second language affects the use of fundamental frequency in speech segmentation

This study investigates whether listeners’ experience with a second language learned later in life affects their use of fundamental frequency (F0) as a cue to word boundaries in the segmentation of an artificial language (AL), particularly when the cues to word boundaries conflict between the first language (L1) and second language (L2). F0 signals phrase-final (and thus word-final) boundaries in French but word-initial boundaries in English. Participants were functionally monolingual French listeners, functionally monolingual English listeners, bilingual L1-English L2-French listeners, and bilingual L1-French L2-English listeners. They completed the AL-segmentation task with F0 signaling word-final boundaries or without prosodic cues to word boundaries (monolingual groups only). After listening to the AL, participants completed a forced-choice word-identification task in which the foils were either non-words or part-words. The results show that the monolingual French listeners, but not the monolingual English listeners, performed better in the presence of F0 cues than in the absence of such cues. Moreover, bilingual status modulated listeners’ use of F0 cues to word-final boundaries, with bilingual French listeners performing less accurately than monolingual French listeners on both word types but with bilingual English listeners performing more accurately than monolingual English listeners on non-words. These findings not only confirm that speech segmentation is modulated by the L1, but also newly demonstrate that listeners’ experience with the L2 (French or English) affects their use of F0 cues in speech segmentation. This suggests that listeners’ use of prosodic cues to word boundaries is adaptive and non-selective, and can change as a function of language experience.

Musicians' Online Performance during Auditory and Visual Statistical Learning Tasks

Musicians' brains are considered to be a functional model of neuroplasticity due to the structural and functional changes associated with long-term musical training. In this study, we examined implicit extraction of statistical regularities from a continuous stream of stimuli-statistical learning (SL). We investigated whether long-term musical training is associated with better extraction of statistical cues in an auditory SL (aSL) task and a visual SL (vSL) task-both using the embedded triplet paradigm. Online measures, characterized by event related potentials (ERPs), were recorded during a familiarization phase while participants were exposed to a continuous stream of individually presented pure tones in the aSL task or individually presented cartoon figures in the vSL task. Unbeknown to participants, the stream was composed of triplets. Musicians showed advantages when compared to non-musicians in the online measure (early N1 and N400 triplet onset effects) during the aSL task. However, there were no differences between musicians and non-musicians for the vSL task. Results from the current study show that musical training is associated with enhancements in extraction of statistical cues only in the auditory domain.

Online neural monitoring of statistical learning

The extraction of patterns in the environment plays a critical role in many types of human learning, from motor skills to language acquisition. This process is known as statistical learning. Here we propose that statistical learning has two dissociable components: (1) perceptual binding of individual stimulus units into integrated composites and (2) storing those integrated representations for later use. Statistical learning is typically assessed using post-learning tasks, such that the two components are conflated. Our goal was to characterize the online perceptual component of statistical learning. Participants were exposed to a structured stream of repeating trisyllabic nonsense words and a random syllable stream. Online learning was indexed by an EEG-based measure that quantified neural entrainment at the frequency of the repeating words relative to that of individual syllables. Statistical learning was subsequently assessed using conventional measures in an explicit rating task and a reaction-time task. In the structured stream, neural entrainment to trisyllabic words was higher than in the random stream, increased as a function of exposure to track the progression of learning, and predicted performance on the reaction time (RT) task. These results demonstrate that monitoring this critical component of learning via rhythmic EEG entrainment reveals a gradual acquisition of knowledge whereby novel stimulus sequences are transformed into familiar composites. This online perceptual transformation is a critical component of learning.

An fMRI study of implicit language learning in developmental language impairment

Individuals with developmental language impairment can show deficits into adulthood. This suggests that neural networks related to their language do not normalize with time. We examined the ability of 16 adults with and without impaired language to learn individual words in an unfamiliar language. Adults with impaired language were able to segment individual words from running speech, but needed more time to do so than their normal-language peers. ICA analysis of fMRI data indicated that adults with language impairment activate a neural network that is comparable to that of adults with normal language. However, a regional analysis indicated relative hyperactivation of a collection of regions associated with language processing. These results are discussed with reference to the Statistical Learning Framework and the sub-skills thought to relate to word segmentation.

•

Adults with developmental language impairment were imaged during a word segmentation task in an unfamiliar natural language.

•

Impaired adults learned to identify individual words, although it took them longer than their typical language peers.

•

The impaired group used the same learning network as the typical group, arguing against recruitment of additional regions.

•

Hyper-activation in language regions characterized the impaired group, unless performance was equated between groups.

•

This suggests that hyper-activation for the impaired group reflects greater effort by learners at earlier stages of learning.

Infant Directed Speech Enhances Statistical Learning in Newborn Infants: An ERP Study

Statistical learning and the social contexts of language addressed to infants are hypothesized to play important roles in early language development. Previous behavioral work has found that the exaggerated prosodic contours of infant-directed speech (IDS) facilitate statistical learning in 8-month-old infants. Here we examined the neural processes involved in on-line statistical learning and investigated whether the use of IDS facilitates statistical learning in sleeping newborns. Event-related potentials (ERPs) were recorded while newborns were exposed to12 pseudo-words, six spoken with exaggerated pitch contours of IDS and six spoken without exaggerated pitch contours (ADS) in ten alternating blocks. We examined whether ERP amplitudes for syllable position within a pseudo-word (word-initial vs. word-medial vs. word-final, indicating statistical word learning) and speech register (ADS vs. IDS) would interact. The ADS and IDS registers elicited similar ERP patterns for syllable position in an early 0-100 ms component but elicited different ERP effects in both the polarity and topographical distribution at 200-400 ms and 450-650 ms. These results provide the first evidence that the exaggerated pitch contours of IDS result in differences in brain activity linked to on-line statistical learning in sleeping newborns.

Cortical encoding and neurophysiological tracking of intensity and pitch cues signaling English stress patterns in native and nonnative speakers

We examined cross-language differences in neural encoding and tracking of intensity and pitch cues signaling English stress patterns. Auditory mismatch negativities (MMNs) were recorded in English and Mandarin listeners in response to contrastive English pseudowords whose primary stress occurred either on the first or second syllable (i.e., "nocTICity" vs. "NOCticity"). The contrastive syllable stress elicited two consecutive MMNs in both language groups, but English speakers demonstrated larger responses to stress patterns than Mandarin speakers. Correlations between the amplitude of ERPs and continuous changes in the running intensity and pitch of speech assessed how well each language group's brain activity tracked these salient acoustic features of lexical stress. We found that English speakers' neural responses tracked intensity changes in speech more closely than Mandarin speakers (higher brain-acoustic correlation). Findings demonstrate more robust and precise processing of English stress (intensity) patterns in early auditory cortical responses of native relative to nonnative speakers.

Differential Gaze Patterns on Eyes and Mouth During Audiovisual Speech Segmentation

Speech is inextricably multisensory: both auditory and visual components provide critical information for all aspects of speech processing, including speech segmentation, the visual components of which have been the target of a growing number of studies. In particular, a recent study (Mitchel and Weiss, 2014) established that adults can utilize facial cues (i.e., visual prosody) to identify word boundaries in fluent speech. The current study expanded upon these results, using an eye tracker to identify highly attended facial features of the audiovisual display used in Mitchel and Weiss (2014). Subjects spent the most time watching the eyes and mouth. A significant trend in gaze durations was found with the longest gaze duration on the mouth, followed by the eyes and then the nose. In addition, eye-gaze patterns changed across familiarization as subjects learned the word boundaries, showing decreased attention to the mouth in later blocks while attention on other facial features remained consistent. These findings highlight the importance of the visual component of speech processing and suggest that the mouth may play a critical role in visual speech segmentation.

Beyond Words: How Humans Communicate Through Sound

Every day we communicate using complex linguistic and musical systems, yet these modern systems are the product of a much more ancient relationship with sound. When we speak, we communicate not only with the words we choose, but also with the patterns of sound we create and the movements that create them. From the natural rhythms of speech, to the precise timing characteristics of a consonant, these patterns guide our daily communication. By examining the principles of information processing that are common to speech and music, we peel back the layers to reveal the biological foundations of human communication through sound. Further, we consider how the brain's response to sound is shaped by experience, such as musical expertise, and implications for the treatment of communication disorders.

Prosodic cues enhance rule learning by changing speech segmentation mechanisms

Prosody has been claimed to have a critical role in the acquisition of grammatical information from speech. The exact mechanisms by which prosodic cues enhance learning are fully unknown. Rules from language often require the extraction of non-adjacent dependencies (e.g., he plays, he sings, he speaks). It has been proposed that pauses enhance learning because they allow computing non-adjacent relations helping word segmentation by removing the need to compute adjacent computations. So far only indirect evidence from behavioral and electrophysiological measures comparing learning effects after exposure to speech with and without pauses support this claim. By recording event-related potentials during the acquisition process of artificial languages with and without pauses between words with embedded non-adjacent rules we provide direct evidence on how the presence of pauses modifies the way speech is processed during learning to enhance segmentation and rule generalization. The electrophysiological results indicate that pauses as short as 25 ms attenuated the N1 component irrespective of whether learning was possible or not. In addition, a P2 enhancement was present only when learning of non-adjacent dependencies was possible. The overall results support the claim that the simple presence of subtle pauses changed the segmentation mechanism used reflected in an exogenously driven N1 component attenuation and improving segmentation at the behavioral level. This effect can be dissociated from the endogenous P2 enhancement that is observed irrespective of the presence of pauses whenever non-adjacent dependencies are learned.

ERPs recorded during early second language exposure predict syntactic learning

Millions of adults worldwide are faced with the task of learning a second language (L2). Understanding the neural mechanisms that support this learning process is an important area of scientific inquiry. However, most previous studies on the neural mechanisms underlying L2 acquisition have focused on characterizing the results of learning, relying upon end-state outcome measures in which learning is assessed after it has occurred, rather than on the learning process itself. In this study, we adopted a novel and more direct approach to investigate neural mechanisms engaged during L2 learning, in which we recorded ERPs from beginning adult learners as they were exposed to an unfamiliar L2 for the first time. Learners' proficiency in the L2 was then assessed behaviorally using a grammaticality judgment task, and ERP data acquired during initial L2 exposure were sorted as a function of performance on this task. High-proficiency learners showed a larger N100 effect to open-class content words compared with closed-class function words, whereas low-proficiency learners did not show a significant N100 difference between open- and closed-class words. In contrast, amplitude of the N400 word category effect correlated with learners' L2 comprehension, rather than predicting syntactic learning. Taken together, these results indicate that learners who spontaneously direct greater attention to open- rather than closed-class words when processing L2 input show better syntactic learning, suggesting a link between selective attention to open-class content words and acquisition of basic morphosyntactic rules. These findings highlight the importance of selective attention mechanisms for L2 acquisition.

Faster sound stream segmentation in musicians than in nonmusicians

The musician's brain is considered as a good model of brain plasticity as musical training is known to modify auditory perception and related cortical organization. Here, we show that music-related modifications can also extend beyond motor and auditory processing and generalize (transfer) to speech processing. Previous studies have shown that adults and newborns can segment a continuous stream of linguistic and non-linguistic stimuli based only on probabilities of occurrence between adjacent syllables, tones or timbres. The paradigm classically used in these studies consists of a passive exposure phase followed by a testing phase. By using both behavioural and electrophysiological measures, we recently showed that adult musicians and musically trained children outperform nonmusicians in the test following brief exposure to an artificial sung language. However, the behavioural test does not allow for studying the learning process per se but rather the result of the learning. In the present study, we analyze the electrophysiological learning curves that are the ongoing brain dynamics recorded as the learning is taking place. While musicians show an inverted U shaped learning curve, nonmusicians show a linear learning curve. Analyses of Event-Related Potentials (ERPs) allow for a greater understanding of how and when musical training can improve speech segmentation. These results bring evidence of enhanced neural sensitivity to statistical regularities in musicians and support the hypothesis of positive transfer of training effect from music to sound stream segmentation in general.

Neurocognitive mechanisms of statistical-sequential learning: what do event-related potentials tell us?

Statistical-sequential learning (SL) is the ability to process patterns of environmental stimuli, such as spoken language, music, or one's motor actions, that unfold in time. The underlying neurocognitive mechanisms of SL and the associated cognitive representations are still not well understood as reflected by the heterogeneity of the reviewed cognitive models. The purpose of this review is: (1) to provide a general overview of the primary models and theories of SL, (2) to describe the empirical research - with a focus on the event-related potential (ERP) literature - in support of these models while also highlighting the current limitations of this research, and (3) to present a set of new lines of ERP research to overcome these limitations. The review is articulated around three descriptive dimensions in relation to SL: the level of abstractness of the representations learned through SL, the effect of the level of attention and consciousness on SL, and the developmental trajectory of SL across the life-span. We conclude with a new tentative model that takes into account these three dimensions and also point to several promising new lines of SL research.

Using semantics to enhance new word learning: an ERP investigation

This study aimed to investigate whether the addition of meaning (semantics) would enhance new word learning for novel objects, and whether it would influence the neurophysiological response to new words. Twenty-five young healthy adults underwent 4 days of training to learn the names of 80 novel objects. Half of the items were learnt under a 'semantic' condition, whereby the name consisted of a legal nonword and two adjectives denoting semantic attributes. The remaining items were learnt under a 'name' condition, whereby the name consisted of a legal nonword and two proper names. Participants demonstrated superior recognition of names in the semantic condition compared to the name condition during training sessions 1-3. On the 5th day, following training, ERPs were recorded whilst participants performed a picture-word judgement task including familiar items. Analysis of the results revealed an N400 for incongruent items in the semantic condition, whilst no ERP component was observed for the name condition. These findings suggest that items learnt with semantic information form stronger associations than those trained without semantics.

Auditory evoked potentials reveal early perceptual effects of distal prosody on speech segmentation

Prosodic context several syllables prior (i.e., distal) to an ambiguous word boundary influences speech segmentation. To assess whether distal prosody influences early perceptual processing or later lexical competition, EEG was recorded while subjects listened to eight-syllable sequences with ambiguous word boundaries for the last four syllables (e.g., tie murder bee vs. timer derby). Pitch and duration of the first 5 syllables were manipulated to induce sequence segmentation with either a monosyllabic or disyllabic final word. Behavioral results confirmed a successful manipulation. Moreover, penultimate syllables (e.g., der) elicited a larger anterior positivity 200-500 ms after onset for prosodic contexts predicted to induce word-initial perception of these syllables. Final syllables (e.g. bee) elicited a similar anterior positivity in the context predicted to induce word-initial perception of these syllables. Additionally, these final syllables elicited a larger positive-to-negative deflection (P1-N1) 60-120 ms after onset, and a larger N400. The finding that prosodic characteristics of speech several syllables prior to ambiguous word boundaries modulate both early and late ERPs elicited by subsequent syllable onsets provides evidence that distal prosody influences early perceptual processing, and later lexical competition.

Predictive brain signals of linguistic development

The ability to extract word forms from continuous speech is a prerequisite for constructing a vocabulary and emerges in the first year of life. Electrophysiological (ERP) studies of speech segmentation by 9- to 12-month-old listeners in several languages have found a left-localized negativity linked to word onset as a marker of word detection. We report an ERP study showing significant evidence of speech segmentation in Dutch-learning 7-month-olds. In contrast to the left-localized negative effect reported with older infants, the observed overall mean effect had a positive polarity. Inspection of individual results revealed two participant sub-groups: a majority showing a positive-going response, and a minority showing the left negativity observed in older age groups. We retested participants at age three, on vocabulary comprehension and word and sentence production. On every test, children who at 7 months had shown the negativity associated with segmentation of words from speech outperformed those who had produced positive-going brain responses to the same input. The earlier that infants show the left-localized brain responses typically indicating detection of words in speech, the better their early childhood language skills.