Neurophysiological evidence for the interplay of speech segmentation and word-referent mapping during novel word learning

Speech Segmentation and Cross-Situational Word Learning in Parallel

Language learners track conditional probabilities to find words in continuous speech and to map words and objects across ambiguous contexts. It remains unclear, however, whether learners can leverage the structure of the linguistic input to do both tasks at the same time. To explore this question, we combined speech segmentation and cross-situational word learning into a single task. In Experiment 1, when adults (N = 60) simultaneously segmented continuous speech and mapped the newly segmented words to objects, they demonstrated better performance than when either task was performed alone. However, when the speech stream had conflicting statistics, participants were able to correctly map words to objects, but were at chance level on speech segmentation. In Experiment 2, we used a more sensitive speech segmentation measure to find that adults (N = 35), exposed to the same conflicting speech stream, correctly identified non-words as such, but were still unable to discriminate between words and part-words. Again, mapping was above chance. Our study suggests that learners can track multiple sources of statistical information to find and map words to objects in noisy environments. It also prompts questions on how to effectively measure the knowledge arising from these learning experiences.

Neural correlates of statistical learning in developmental dyslexia: An electroencephalography study

The human brain extracts statistical regularities from the surrounding environment in a process called statistical learning. Behavioural evidence suggests that developmental dyslexia affects statistical learning. However, surprisingly few studies have assessed how developmental dyslexia affects the neural processing underlying this type of learning. We used electroencephalography to explore the neural correlates of an important aspect of statistical learning - sensitivity to transitional probabilities - in individuals with developmental dyslexia. Adults diagnosed with developmental dyslexia (n = 17) and controls (n = 19) were exposed to a continuous stream of sound triplets. Every so often, a triplet ending had a low transitional probability given the triplet's first two sounds ("statistical deviants"). Furthermore, every so often a triplet ending was presented from a deviant location ("acoustic deviants"). We examined mismatch negativity elicited by statistical deviants (sMMN), and MMN elicited by location deviants (i.e., acoustic changes). Acoustic deviants elicited a MMN which was larger in the control group than in the developmental dyslexia group. Statistical deviants elicited a small, yet significant, sMMN in the control group, but not in the developmental dyslexia group. However, the difference between the groups was not significant. Our findings indicate that the neural mechanisms underlying pre-attentive acoustic change detection and implicit statistical auditory learning are both affected in developmental dyslexia.

Foreign speech sound discrimination and associative word learning lead to a fast reconfiguration of resting-state networks

Learning new words in an unfamiliar language is a complex endeavor that requires the orchestration of multiple perceptual and cognitive functions. Although the neural mechanisms governing word learning are becoming better understood, little is known about the predictive value of resting-state (RS) metrics for foreign word discrimination and word learning attainment. In addition, it is still unknown which of the multistep processes involved in word learning have the potential to rapidly reconfigure RS networks. To address these research questions, we used electroencephalography (EEG), measured forty participants, and examined scalp-based power spectra, source-based spectral density maps and functional connectivity metrics before (RS1), in between (RS2) and after (RS3) a series of tasks which are known to facilitate the acquisition of new words in a foreign language, namely word discrimination, word-referent mapping and semantic generalization. Power spectra at the scalp level consistently revealed a reconfiguration of RS networks as a function of foreign word discrimination (RS1 vs. RS2) and word learning (RS1 vs. RS3) tasks in the delta, lower and upper alpha, and upper beta frequency ranges. Otherwise, functional reconfigurations at the source level were restricted to the theta (spectral density maps) and to the lower and upper alpha frequency bands (spectral density maps and functional connectivity). Notably, scalp RS changes related to the word discrimination tasks (difference between RS2 and RS1) correlated with word discrimination abilities (upper alpha band) and semantic generalization performance (theta and upper alpha bands), whereas functional changes related to the word learning tasks (difference between RS3 and RS1) correlated with word discrimination scores (lower alpha band). Taken together, these results highlight that foreign speech sound discrimination and word learning have the potential to rapidly reconfigure RS networks at multiple functional scales.

The electrophysiological correlates of word pre-activation during associative word learning

Human beings continuously make use of learned associations to generate predictions about future occurrences in the environment. Such memory-related predictive processes provide a scaffold for learning in that mental representations of foreseeable events can be adjusted or strengthened based on a specific outcome. Learning the meaning of novel words through picture-word associations constitutes a prime example of associative learning because pictures preceding words can trigger word prediction through the pre-activation of the related mnemonic representations. In the present electroencephalography (EEG) study, we used event-related potentials (ERPs) to compare neural indices of word pre-activation between a word learning condition with maximal prediction likelihood and a non-learning control condition with low prediction. Results revealed that prediction-related N400 amplitudes in response to pictures decreased over time at central electrodes as a function of word learning, whereas late positive component (LPC) amplitudes increased. Notably, N400 but not LPC changes were also predictive of word learning performance, suggesting that the N400 component constitutes a sensitive marker of word pre-activation during associative word learning.

Electrophysiological Examination of Ambient Speech Processing in Children With Cochlear Implants

PURPOSE

This research examined the expression of cortical auditory evoked potentials in a cohort of children who received cochlear implants (CIs) for treatment of congenital deafness (n = 28) and typically hearing controls (n = 28).

METHOD

We make use of a novel electroencephalography paradigm that permits the assessment of auditory responses to ambiently presented speech and evaluates the contributions of concurrent visual stimulation on this activity.

RESULTS

Our findings show group differences in the expression of auditory sensory and perceptual event-related potential components occurring in 80- to 200-ms and 200- to 300-ms time windows, with reductions in amplitude and a greater latency difference for CI-using children. Relative to typically hearing children, current source density analysis showed muted responses to concurrent visual stimulation in CI-using children, suggesting less cortical specialization and/or reduced responsiveness to auditory information that limits the detection of the interaction between sensory systems.

CONCLUSION

These findings indicate that even in the face of early interventions, CI-using children may exhibit disruptions in the development of auditory and multisensory processing.

Oscillatory activity and EEG phase synchrony of concurrent word segmentation and meaning-mapping in 9-year-old children

When learning a new language, one must segment words from continuous speech and associate them with meanings. These complex processes can be boosted by attentional mechanisms triggered by multi-sensory information. Previous electrophysiological studies suggest that brain oscillations are sensitive to different hierarchical complexity levels of the input, making them a plausible neural substrate for speech parsing. Here, we investigated the functional role of brain oscillations during concurrent speech segmentation and meaning acquisition in sixty 9-year-old children. We collected EEG data during an audio-visual statistical learning task during which children were exposed to a learning condition with consistent word-picture associations and a random condition with inconsistent word-picture associations before being tested on their ability to recall words and word-picture associations. We capitalized on the brain dynamics to align neural activity to the same rate as an external rhythmic stimulus to explore modulations of neural synchronization and phase synchronization between electrodes during multi-sensory word learning. Results showed enhanced power at both word- and syllabic-rate and increased EEG phase synchronization between frontal and occipital regions in the learning compared to the random condition. These findings suggest that multi-sensory cueing and attentional mechanisms play an essential role in children's successful word learning.

Perceptual uncertainty modulates auditory statistical learning: A magnetoencephalography study

Statistical learning allows comprehension of structured information, such as that in language and music. The brain computes a sequence's transition probability and predicts future states to minimise sensory reaction and derive entropy (uncertainty) from sequential information. Neurophysiological studies have revealed that early event-related neural responses (P1 and N1) reflect statistical learning - when the brain encodes transition probability in stimulus sequences, it predicts an upcoming stimulus with a high transition probability and suppresses the early event-related responses to a stimulus with a high transition probability. This amplitude difference between high and low transition probabilities reflects statistical learning effects. However, how a sequence's transition probability ratio affects neural responses contributing to statistical learning effects remains unknown. This study investigated how transition-probability ratios or conditional entropy (uncertainty) in auditory sequences modulate the early event-related neuromagnetic responses of P1m and N1m. Sequence uncertainties were manipulated using three different transition-probability ratios: 90:10%, 80:20%, and 67:33% (conditional entropy: 0.47, 0.72, and 0.92 bits, respectively). Neuromagnetic responses were recorded when participants listened to sequential sounds with these three transition probabilities. Amplitude differences between lower and higher probabilities were larger in sequences with transition-probability ratios of 90:10% and smaller in sequences with those of 67:33%, compared to sequences with those of 80:20%. This suggests that the transition-probability ratio finely tunes P1m and N1m. Our study also showed larger amplitude differences between frequent- and rare-transition stimuli in P1m than in N1m. This indicates that information about transition-probability differences may be calculated in earlier cognitive processes.

Active listening

This paper introduces active listening, as a unified framework for synthesising and recognising speech. The notion of active listening inherits from active inference, which considers perception and action under one universal imperative: to maximise the evidence for our (generative) models of the world. First, we describe a generative model of spoken words that simulates (i) how discrete lexical, prosodic, and speaker attributes give rise to continuous acoustic signals; and conversely (ii) how continuous acoustic signals are recognised as words. The 'active' aspect involves (covertly) segmenting spoken sentences and borrows ideas from active vision. It casts speech segmentation as the selection of internal actions, corresponding to the placement of word boundaries. Practically, word boundaries are selected that maximise the evidence for an internal model of how individual words are generated. We establish face validity by simulating speech recognition and showing how the inferred content of a sentence depends on prior beliefs and background noise. Finally, we consider predictive validity by associating neuronal or physiological responses, such as the mismatch negativity and P300, with belief updating under active listening, which is greatest in the absence of accurate prior beliefs about what will be heard next.

Evidence for Enhanced Long-term Memory in Professional Musicians and Its Contribution to Novel Word Learning

Previous studies evidenced transfer effects from professional music training to novel word learning. However, it is unclear whether such an advantage is driven by cascading, bottom-up effects from better auditory perception to semantic processing or by top-down influences from cognitive functions on perception. Moreover, the long-term effects of novel word learning remain an open issue. To address these questions, we used a word learning design, with four different sets of novel words, and we neutralized the potential perceptive and associative learning advantages in musicians. Under such conditions, we did not observe any advantage in musicians on the day of learning (Day 1 [D1]), at neither a behavioral nor an electrophysiological level; this suggests that the previously reported advantages in musicians are likely to be related to bottom-up processes. Nevertheless, 1 month later (Day 30 [D30]) and for all types of novel words, the error increase from D1 to D30 was lower in musicians compared to nonmusicians. In addition, for the set of words that were perceptually difficult to discriminate, only musicians showed typical N400 effects over parietal sites on D30. These results demonstrate that music training improved long-term memory and that transfer effects from music training to word learning (i.e., semantic levels of speech processing) benefit from reinforced (long-term) memory functions. Finally, these findings highlight the positive impact of music training on the acquisition of foreign languages.

Not All Words Are Equally Acquired: Transitional Probabilities and Instructions Affect the Electrophysiological Correlates of Statistical Learning

Statistical learning (SL), the process of extracting regularities from the environment, is a fundamental skill of our cognitive system to structure the world regularly and predictably. SL has been studied using mainly behavioral tasks under implicit conditions and with triplets presenting the same level of difficulty, i.e., a mean transitional probability (TP) of 1.00. Yet, the neural mechanisms underlying SL under other learning conditions remain largely unknown. Here, we investigated the neurofunctional correlates of SL using triplets (i.e., three-syllable nonsense words) with a mean TP of 1.00 (easy "words") and 0.50 (hard "words") in an SL task performed under incidental (implicit) and intentional (explicit) conditions, to determine whether the same core mechanisms were recruited to assist learning. Event-related potentials (ERPs) were recorded while participants listened firstly to a continuous auditory stream made of the concatenation of four easy and four hard "words" under implicit instructions, and subsequently to another auditory stream made of the concatenation of four easy and four hard "words" drawn from another artificial language under explicit instructions. The stream in each of the SL tasks was presented in two consecutive blocks of ~3.5-min each (~7-min in total) to further examine how ERP components might change over time. Behavioral measures of SL were collected after the familiarization phase of each SL task by asking participants to perform a two-alternative forced-choice (2-AFC) task. Results from the 2-AFC tasks revealed a moderate but reliable level of SL, with no differences between conditions. ERPs were, nevertheless, sensitive to the effect of TPs, showing larger amplitudes of N400 for easy "words," as well as to the effect of instructions, with a reduced N250 for "words" presented under explicit conditions. Also, significant differences in the N100 were found as a result of the interaction between TPs, instructions, and the amount of exposure to the auditory stream. Taken together, our findings suggest that triplets' predictability impacts the emergence of "words" representations in the brain both for statistical regularities extracted under incidental and intentional instructions, although the prior knowledge of the "words" seems to favor the recruitment of different SL mechanisms.

Neurophysiological Correlates of Top-Down Phonological and Semantic Influence during the Orthographic Processing of Novel Visual Word-Forms

The acquisition of new vocabulary is usually mediated by previous experience with language. In the visual domain, the representation of orthographically unfamiliar words at the phonological or conceptual levels may facilitate their orthographic learning. The neural correlates of this advantage were investigated by recording EEG activity during reading novel and familiar words across three different experiments (n = 22 each), manipulating the availability of previous knowledge on the novel written words. A different pattern of event-related potential (ERP) responses was found depending on the previous training, resembling cross-level top-down interactive effects during vocabulary acquisition. Thus, whereas previous phonological experience caused a modulation at the post-lexical stages of the visual recognition of novel written words (~520 ms), additional semantic training influenced their processing at a lexico-semantic stage (~320 ms). Moreover, early lexical differences (~180 ms) elicited in the absence of previous training did not emerge after both phonological and semantic training, reflecting similar orthographic processing and word-form access.

Rapid Cortical Plasticity Induced by Active Associative Learning of Novel Words in Human Adults

Human speech requires that new words are routinely memorized, yet neurocognitive mechanisms of such acquisition of memory remain highly debatable. Major controversy concerns the question whether cortical plasticity related to word learning occurs in neocortical speech-related areas immediately after learning, or neocortical plasticity emerges only on the second day after a prolonged time required for consolidation after learning. The functional spatiotemporal pattern of cortical activity related to such learning also remains largely unknown. In order to address these questions, we examined magnetoencephalographic responses elicited in the cerebral cortex by passive presentations of eight novel pseudowords before and immediately after an operant conditioning task. This associative procedure forced participants to perform an active search for unique meaning of four pseudowords that referred to movements of left and right hands and feet. The other four pseudowords did not require any movement and thus were not associated with any meaning. Familiarization with novel pseudowords led to a bilateral repetition suppression of cortical responses to them; the effect started before or around the uniqueness point and lasted for more than 500 ms. After learning, response amplitude to pseudowords that acquired meaning was greater compared with response amplitude to pseudowords that were not assigned meaning; the effect was significant within 144-362 ms after the uniqueness point, and it was found only in the left hemisphere. Within this time interval, a learning-related selective response initially emerged in cortical areas surrounding the Sylvian fissure: anterior superior temporal sulcus, ventral premotor cortex, the anterior part of intraparietal sulcus and insula. Later within this interval, activation additionally spread to more anterior higher-tier brain regions, and reached the left temporal pole and the triangular part of the left inferior frontal gyrus extending to its orbital part. Altogether, current findings evidence rapid plastic changes in cortical representations of meaningful auditory word-forms occurring almost immediately after learning. Additionally, our results suggest that familiarization resulting from stimulus repetition and semantic acquisition resulting from an active learning procedure have separable effects on cortical activity.

Concurrent Statistical Learning of Ignored and Attended Sound Sequences: An MEG Study

In an auditory environment, humans are frequently exposed to overlapping sound sequences such as those made by human voices and musical instruments, and we can acquire information embedded in these sequences via attentional and nonattentional accesses. Whether the knowledge acquired by attentional accesses interacts with that acquired by nonattentional accesses is unknown, however. The present study examined how the statistical learning (SL) of two overlapping sound sequences is reflected in neurophysiological and behavioral responses, and how the learning effects are modulated by attention to each sequence. SL in this experimental paradigm was reflected in a neuromagnetic response predominantly in the right hemisphere, and the learning effects were not retained when attention to the tone streams was switched during the learning session. These results suggest that attentional and nonattentional learning scarcely interact with each other and that there may be a specific system for nonattentional learning, which is independent of attentional learning.

When the statistical MMN meets the physical MMN

How do listeners respond to prediction errors within patterned sequence of sounds? To answer this question we carried out a statistical learning study using electroencephalography (EEG). In a continuous auditory stream of sound triplets the deviations were either (a) statistical, in terms of transitional probability, (b) physical, due to a change in sound location (left or right speaker) or (c) a double deviants, i.e. a combination of the two. Statistical and physical deviants elicited a statistical mismatch negativity and a physical MMN respectively. Most importantly, we found that effects of statistical and physical deviants interacted (the statistical MMN was smaller when co-occurring with a physical deviant). Results show, for the first time, that processing of prediction errors due to statistical learning is affected by prediction errors due to physical deviance. Our findings thus show that the statistical MMN interacts with the physical MMN, implying that prediction error processing due to physical sound attributes suppresses processing of learned statistical properties of sounds.

Neurophysiological Markers of Statistical Learning in Music and Language: Hierarchy, Entropy, and Uncertainty

Statistical learning (SL) is a method of learning based on the transitional probabilities embedded in sequential phenomena such as music and language. It has been considered an implicit and domain-general mechanism that is innate in the human brain and that functions independently of intention to learn and awareness of what has been learned. SL is an interdisciplinary notion that incorporates information technology, artificial intelligence, musicology, and linguistics, as well as psychology and neuroscience. A body of recent study has suggested that SL can be reflected in neurophysiological responses based on the framework of information theory. This paper reviews a range of work on SL in adults and children that suggests overlapping and independent neural correlations in music and language, and that indicates disability of SL. Furthermore, this article discusses the relationships between the order of transitional probabilities (TPs) (i.e., hierarchy of local statistics) and entropy (i.e., global statistics) regarding SL strategies in human's brains; claims importance of information-theoretical approaches to understand domain-general, higher-order, and global SL covering both real-world music and language; and proposes promising approaches for the application of therapy and pedagogy from various perspectives of psychology, neuroscience, computational studies, musicology, and linguistics.

Parallel odor processing by mitral and middle tufted cells in the olfactory bulb

The olfactory bulb (OB) transforms sensory input into spatially and temporally organized patterns of activity in principal mitral (MC) and middle tufted (mTC) cells. Thus far, the mechanisms underlying odor representations in the OB have been mainly investigated in MCs. However, experimental findings suggest that MC and mTC may encode parallel and complementary odor representations. We have analyzed the functional roles of these pathways by using a morphologically and physiologically realistic three-dimensional model to explore the MC and mTC microcircuits in the glomerular layer and deeper plexiform layer. The model makes several predictions. MCs and mTCs are controlled by similar computations in the glomerular layer but are differentially modulated in deeper layers. The intrinsic properties of mTCs promote their synchronization through a common granule cell input. Finally, the MC and mTC pathways can be coordinated through the deep short-axon cells in providing input to the olfactory cortex. The results suggest how these mechanisms can dynamically select the functional network connectivity to create the overall output of the OB and promote the dynamic synchronization of glomerular units for any given odor stimulus.

Making predictable unpredictable with style - Behavioral and electrophysiological evidence for the critical role of prosodic expectations in the perception of prominence in speech

Perceptual prominence of linguistic units such as words has been earlier connected to the concepts of predictability and attentional orientation. One hypothesis is that low-probability prosodic or lexical content is perceived as prominent due to the surprisal and high information value associated with the stimulus. However, the existing behavioral studies have used stimulus manipulations that follow or violate typical linguistic patterns present in the listeners' native language, i.e., assuming that the listeners have already established a model for acceptable prosodic patterns in the language. In the present study, we investigated whether prosodic expectations and the resulting subjective impression of prominence is affected by brief statistical adaptation to suprasegmental acoustic features in speech, also in the case where the prosodic patterns do not necessarily follow language-typical marking for prominence. We first exposed listeners to five minutes of speech with uneven distributions of falling and rising fundamental frequency (F0) trajectories on sentence-final words, and then tested their judgments of prominence on a set of new utterances. The results show that the probability of the F0 trajectory affects the perception of prominence, a less frequent F0 trajectory making a word more prominent independently of the absolute direction of F0 change. In the second part of the study, we conducted EEG-measurements on a set of new subjects listening to similar utterances with predominantly rising or falling F0 on sentence-final words. Analysis of the resulting event-related potentials (ERP) reveals a significant difference in N200 and N400 ERP-component amplitudes between standard and deviant prosody, again independently of the F0 direction and the underlying lexical content. Since N400 has earlier been associated with semantic processing of stimuli, this suggests that listeners implicitly track probabilities at the suprasegmental level and that predictability of a prosodic pattern during a word has an impact to the semantic processing of the word. Overall, the study suggests that prosodic markers for prominence are at least partially driven by the statistical structure of recently perceived speech, and therefore prominence perception could be based on statistical learning mechanisms similar to those observed in early word learning, but in this case operating at the level of suprasegmental acoustic features.

Enhanced Neonatal Brain Responses To Sung Streams Predict Vocabulary Outcomes By Age 18 Months

Words and melodies are some of the basic elements infants are able to extract early in life from the auditory input. Whether melodic cues contained in songs can facilitate word-form extraction immediately after birth remained unexplored. Here, we provided converging neural and computational evidence of the early benefit of melodies for language acquisition. Twenty-eight neonates were tested on their ability to extract word-forms from continuous flows of sung and spoken syllabic sequences. We found different brain dynamics for sung and spoken streams and observed successful detection of word-form violations in the sung condition only. Furthermore, neonatal brain responses for sung streams predicted expressive vocabulary at 18 months as demonstrated by multiple regression and cross-validation analyses. These findings suggest that early neural individual differences in prosodic speech processing might be a good indicator of later language outcomes and could be considered as a relevant factor in the development of infants' language skills.

Fast Brain Plasticity during Word Learning in Musically-Trained Children

Children learn new words every day and this ability requires auditory perception, phoneme discrimination, attention, associative learning and semantic memory. Based on previous results showing that some of these functions are enhanced by music training, we investigated learning of novel words through picture-word associations in musically-trained and control children (8-12 year-old) to determine whether music training would positively influence word learning. Results showed that musically-trained children outperformed controls in a learning paradigm that included picture-sound matching and semantic associations. Moreover, the differences between unexpected and expected learned words, as reflected by the N200 and N400 effects, were larger in children with music training compared to controls after only 3 min of learning the meaning of novel words. In line with previous results in adults, these findings clearly demonstrate a correlation between music training and better word learning. It is argued that these benefits reflect both bottom-up and top-down influences. The present learning paradigm might provide a useful dynamic diagnostic tool to determine which perceptive and cognitive functions are impaired in children with learning difficulties.