Phonetic training and non-native speech perception — New memory traces evolve in just three days as indexed by the mismatch negativity (MMN) and behavioural measures

Native language background affects the perception of duration and pitch

Estonian is a quantity language with both a primary duration cue and a secondary pitch cue, whereas Chinese is a tonal language with a dominant pitch use. Using a mismatch negativity experiment and a behavioral discrimination experiment, we investigated how native language background affects the perception of duration only, pitch only, and duration plus pitch information. Chinese participants perceived duration in Estonian as meaningless acoustic information due to a lack of phonological use of duration in their native language; however, they demonstrated a better pitch discrimination ability than Estonian participants. On the other hand, Estonian participants outperformed Chinese participants in perceiving the non-speech pure tones that resembled the Estonian quantity (i.e., containing both duration and pitch information). Our results indicate that native language background affects the perception of duration and pitch and that such an effect is not specific to processing speech sounds.

Short-term training helps second-language learners read like native readers: An ERP study

This randomized controlled trial study aimed to examine what experience other than immersion may help adult learners read with native-like neural responses. We compared a group of 13 native Chinese English learners completing English letter-sound association training with another group of 12 completing visual symbol-sound association training and included one group of native English readers as the reference. The results showed that after three hours of training, all learners no longer showed attenuated cross-modal mismatch negativity (MMN) to English letter-sound integration as in the pretest. After six hours of training, the learners receiving English letter-sound association training showed enhanced cross-modal MMN and theta oscillations, as native English readers did. The enhanced neural responses were significantly correlated with better phonological awareness. Thus, with training specific to critical second language reading skills of appropriate dosages, adult learners can overcome the constraints of their native language background and learn to read with native-like neural responses.

Non-game like training benefits spoken foreign-language processing in children with dyslexia

Children with dyslexia often face difficulties in learning foreign languages, which is reflected as weaker neural activation. However, digital language-learning applications could support learning-induced plastic changes in the brain. Here we aimed to investigate whether plastic changes occur in children with dyslexia more readily after targeted training with a digital language-learning game or similar training without game-like elements. We used auditory event-related potentials (ERPs), specifically, the mismatch negativity (MMN), to study learning-induced changes in the brain responses. Participants were 24 school-aged Finnish-speaking children with dyslexia and 24 age-matched typically reading control children. They trained English speech sounds and words with "Say it again, kid!" (SIAK) language-learning game for 5 weeks between ERP measurements. During the game, the players explored game boards and produced English words aloud to score stars as feedback from an automatic speech recognizer. To compare the effectiveness of the training type (game vs. non-game), we embedded in the game some non-game levels stripped of all game-like elements. In the dyslexia group, the non-game training increased the MMN amplitude more than the game training, whereas in the control group the game training increased the MMN response more than the non-game training. In the dyslexia group, the MMN increase with the non-game training correlated with phonological awareness: the children with poorer phonological awareness showed a larger increase in the MMN response. Improved neural processing of foreign speech sounds as indicated by the MMN increase suggests that targeted training with a simple application could alleviate some spoken foreign-language learning difficulties that are related to phonological processing in children with dyslexia.

Event-Related Potential Correlates of Learning to Produce Novel Foreign Phonemes

Learning to pronounce a foreign phoneme requires an individual to acquire a motor program that enables the reproduction of the new acoustic target sound. This process is largely based on the use of auditory feedback to detect pronunciation errors to adjust vocalization. While early auditory evoked neural activity underlies automatic detection and adaptation to vocalization errors, little is known about the neural correlates of acquiring novel speech targets. To investigate the neural processes that mediate the learning of foreign phoneme pronunciation, we recorded event-related potentials when participants (N = 19) pronounced native or foreign phonemes. Behavioral results indicated that the participants' pronunciation of the foreign phoneme improved during the experiment. Early auditory responses (N1 and P2 waves, approximately 85-290 ms after the sound onset) revealed no differences between foreign and native phonemes. In contrast, the amplitude of the frontocentrally distributed late slow wave (LSW, 320-440 ms) was modulated by the pronunciation of the foreign phonemes, and the effect changed during the experiment, paralleling the improvement in pronunciation. These results suggest that the LSW may reflect higher-order monitoring processes that signal successful pronunciation and help learn novel phonemes.

Listen-and-repeat training in the learning of non-native consonant duration contrasts: influence of consonant type as reflected by MMN and behavioral methods

Phonological duration differences in quantity languages can be problematic for second language learners whose native language does not use duration contrastively. Recent studies have found improvement in the processing of non-native vowel duration contrasts with the use of listen-and-repeat training, and the current study explores the efficacy of similar methodology on consonant duration contrasts. 18 adult participants underwent two days of listen-and-repeat training with pseudoword stimuli containing either a sibilant or a stop consonant contrast. The results were examined with psychophysiological event-related potentials (mismatch negativity and P3), behavioral discrimination tests and a production task. The results revealed no training-related effects in the event-related potentials or the production task, but behavioral discrimination performance improved. Furthermore, differences emerged between the processing of the two consonant types. The findings suggest that stop consonants are processed more slowly than the sibilants, and the findings are discussed with regard to possible segmentation difficulties.

Gaming enhances learning-induced plastic changes in the brain

Digital games may benefit children's learning, yet the factors that induce gaming benefits to cognition are not well known. In this study, we investigated the effectiveness of digital game-based learning in children by comparing the learning of foreign speech sounds and words in a digital game or a non-game digital application. To evaluate gaming-induced plastic changes in the brain, we used the mismatch negativity (MMN) brain response that reflects the access to long-term memory representations. We recorded auditory brain responses from 37 school-aged Finnish-speaking children before and after playing a computer-based language-learning game. The MMN amplitude increased between the pre- and post-measurement for the game condition but not for the non-game condition, suggesting that the gaming intervention enhanced learning more than the non-game intervention. The results indicate that digital games can be beneficial for children's speech-sound learning and that gaming elements per se, not just practice time, support learning.

First Language Attrition: What It Is, What It Isn't, and What It Can Be

This review aims at clarifying the concept of first language attrition by tracing its limits, identifying its phenomenological and contextual constraints, discussing controversies associated with its definition, and suggesting potential directions for future research. We start by reviewing different definitions of attrition as well as associated inconsistencies. We then discuss the underlying mechanisms of first language attrition and review available evidence supporting different background hypotheses. Finally, we attempt to provide the groundwork to build a unified theoretical framework allowing for generalizable results. To this end, we suggest the deployment of a rigorous neuroscientific approach, in search of neural markers of first language attrition in different linguistic domains, putting forward hypothetical experimental ways to identify attrition's neural traces and formulating predictions for each of the proposed experimental paradigms.

Early differentiation of memory retrieval processes for newly learned voices and phonemes as indexed by the MMN

Linguistic and vocal information are thought to be differentially processed since the early stages of speech perception, but it remains unclear if this differentiation also concerns automatic processes of memory retrieval. The aim of this ERP study was to compare the automatic retrieval processes for newly learned voices vs phonemes. In a longitudinal experiment, two groups of participants were trained in learning either a new phoneme or a new voice. The MMN elicited by the presentation of the two was measured before and after the training. An enhanced MMN was elicited by the presentation of the learned phoneme, reflecting the activation of an automatic memory retrieval process. Instead, a reduced MMN was elicited by the learned voice, indicating that the voice was perceived as a typical member of the learned voice identity. This suggests that the automatic processes that retrieve linguistic and vocal information are differently affected by experience.

Perceptual Asymmetries and Auditory Processing of Estonian Quantities

Similar to visual perception, auditory perception also has a clearly described “pop-out” effect, where an element with some extra feature is easier to detect among elements without an extra feature. This phenomenon is better known as auditory perceptual asymmetry. We investigated such asymmetry between shorter or longer duration, and level or falling of pitch of linguistic stimuli that carry a meaning in one language (Estonian), but not in another (Russian). For the mismatch negativity (MMN) experiment, we created four different types of stimuli by modifying the duration of the first vowel [ɑ] (170, 290 ms) and pitch contour (level vs. falling pitch) of the stimuli words (‘SATA,’ ‘SAKI’). The stimuli were synthesized from Estonian words (‘SATA,’ ‘SAKI’) and follow the Estonian language three-way quantity system, which incorporates tonal features (falling pitch contour) together with temporal patterns. This made the meaning of the word dependent on the combination of both features and allows us to compare the relative contribution of duration and pitch contour in discrimination of language stimuli in the brain via MMN generation. The participants of the experiment were 12 Russian native speakers with little or no experience in Estonian and living in Estonia short-term, and 12 Estonian native speakers (age 18–27 years). We found that participants’ perception of the linguistic stimuli differed not only according to the physical features but also according to their native language, confirming that the meaning of the word interferes with the early automatic processing of phonological features. The GAMM and ANOVA analysis of the reversed design results showed that the deviant with longer duration among shorter standards elicited a MMN response with greater amplitude than the short deviant among long standards, while changes in pitch contour (falling vs. level pitch) produced neither strong MMN nor asymmetry. Thus, we demonstrate the effect of language background on asymmetric perception of linguistic stimuli that aligns with those of previous studies (Jaramillo et al., 2000), and contributes to the growing body of knowledge supporting auditory perceptual asymmetry.

Aging and non-native speech perception: A phonetic training study

Cognitive decline is evident in the elderly and it affects speech perception and foreign language learning. A listen-and-repeat training with a challenging speech sound contrast was earlier found to be effective in young monolingual adults and even in advanced L2 university students at the attentive and pre-attentive levels. This study investigates foreign language speech perception in the elderly with the same protocol used with the young adults. Training effects were measured with attentive behavioural measures (N = 9) and with electroencephalography measuring the pre-attentive mismatch negativity (MMN) response (N = 10). Training was effective in identification, but not in discrimination and there were no changes in the MMN. The most attention demanding perceptual functions which benefit from experience-based linguistic knowledge were facilitated through training, whereas pre-attentive processing was unaffected. The elderly would probably benefit from different training types compared to younger adults.

One-week phonemic training rebuilds the memory traces of merged phonemes in merged speakers

The phonemic merger is a unique phenomenon which is referred to as acoustically very different phonemes are recognized as the same phoneme. In our previous study, we demonstrated that the merged speakers had lost the ability to discriminate the merged phonemes pre-attentively, as revealed by their failure in mismatch negativity (MMN) elicitation in the oddball stream of the merged phonemes /n/-/l/. In this study, we investigated the recovery of the discrimination ability via phonemic training and found that the merged speakers regained the ability of discriminating merged phonemes pre-attentively, after a 7-day /n/-/l/ phonemic training, as revealed by the reactivation of MMN brain response to the /n/-/l/ phoneme categories. Our finding indicates that separate memory traces of merged phonemes could be rebuilt during the training process.

The Effect of Visual Articulatory Information on the Neural Correlates of Non-native Speech Sound Discrimination

Behavioral studies have shown that the ability to discriminate between non-native speech sounds improves after seeing how the sounds are articulated. This study examined the influence of visual articulatory information on the neural correlates of non-native speech sound discrimination. English speakers’ discrimination of the Hindi dental and retroflex sounds was measured using the mismatch negativity (MMN) event-related potential, before and after they completed one of three 8-min training conditions. In an audio-visual speech training condition (n = 14), each sound was presented with its corresponding visual articulation. In one control condition (n = 14), both sounds were presented with the same visual articulation, resulting in one congruent and one incongruent audio-visual pairing. In another control condition (n = 14), both sounds were presented with the same image of a still face. The control conditions aimed to rule out the possibility that the MMN is influenced by non-specific audio-visual pairings, or by general exposure to the dental and retroflex sounds over the course of the study. The results showed that audio-visual speech training reduced the latency of the MMN but did not affect MMN amplitude. No change in MMN amplitude or latency was observed for the two control conditions. The pattern of results suggests that a relatively short audio-visual speech training session (i.e., 8 min) may increase the speed with which the brain processes non-native speech sound contrasts. The absence of a training effect on MMN amplitude suggests a single session of audio-visual speech training does not lead to the formation of more discrete memory traces for non-native speech sounds. Longer and/or multiple sessions might be needed to influence the MMN amplitude.

NMDA Receptor Antagonist Effects on Speech-Related Mismatch Negativity and Its Underlying Oscillatory and Source Activity in Healthy Humans

Background: Previous studies in schizophrenia have consistently shown that deficits in the generation of the auditory mismatch negativity (MMN) – a pre-attentive, event-related potential (ERP) typically elicited by changes to simple sound features – are linked to N-methyl-D-aspartate (NMDA) receptor hypofunction. Concomitant with extensive language dysfunction in schizophrenia, patients also exhibit MMN deficits to changes in speech but their relationship to NMDA-mediated neurotransmission is not clear. Accordingly, our study aimed to investigate speech MMNs in healthy humans and their underlying electrophysiological mechanisms in response to NMDA antagonist treatment. We also evaluated the relationship between baseline MMN/electrocortical activity and emergent schizophrenia-like symptoms associated with NMDA receptor blockade.

Methods: In a sample of 18 healthy volunteers, a multi-feature Finnish language paradigm incorporating changes in syllables, vowels and consonant stimuli was used to assess the acute effects of the NMDA receptor antagonist ketamine and placebo on the MMN. Further, measures of underlying neural activity, including evoked theta power, theta phase locking and source-localized current density in cortical regions of interest were assessed. Subjective symptoms were assessed with the Clinician Administered Dissociative States Scale (CADSS).

Results: Participants exhibited significant ketamine-induced increases in psychosis-like symptoms and depending on temporal or frontal recording region, co-occurred with reductions in MMN generation in response to syllable frequency/intensity, vowel duration, across vowel and consonant deviants. MMN attenuation was associated with decreases in evoked theta power, theta phase locking and diminished current density in auditory and inferior frontal (language-related cortical) regions. Baseline (placebo) MMN and underlying electrophysiological features associated with the processing of changes in syllable intensity correlated with the degree of psychotomimetic response to ketamine.

Conclusion: Ketamine-induced impairments in healthy human speech MMNs and their underlying electrocortical mechanisms closely resemble those observed in schizophrenia and support a model of dysfunctional NMDA receptor-mediated neurotransmission of language processing deficits in schizophrenia.

Human Brainstem Exhibits higher Sensitivity and Specificity than Auditory-Related Cortex to Short-Term Phonetic Discrimination Learning

Phonetic discrimination learning is an active perceptual process that operates under the influence of cognitive control mechanisms by increasing the sensitivity of the auditory system to the trained stimulus attributes. It is assumed that the auditory cortex and the brainstem interact in order to refine how sounds are transcribed into neural codes. Here, we evaluated whether these two computational entities are prone to short-term functional changes, whether there is a chronological difference in malleability, and whether short-term training suffices to alter reciprocal interactions. We performed repeated cortical (i.e., mismatch negativity responses, MMN) and subcortical (i.e., frequency-following response, FFR) EEG measurements in two groups of participants who underwent one hour of phonetic discrimination training or were passively exposed to the same stimulus material. The training group showed a distinctive brainstem energy reduction in the trained frequency-range (i.e., first formant), whereas the passive group did not show any response modulation. Notably, brainstem signal change correlated with the behavioral improvement during training, this result indicating a close relationship between behavior and underlying brainstem physiology. Since we did not reveal group differences in MMN responses, results point to specific short-term brainstem changes that precede functional alterations in the auditory cortex.

Passive exposure to speech sounds induces long-term memory representations in the auditory cortex of adult rats

Experience-induced changes in the functioning of the auditory cortex are prominent in early life, especially during a critical period. Although auditory perceptual learning takes place automatically during this critical period, it is thought to require active training in later life. Previous studies demonstrated rapid changes in single-cell responses of anesthetized adult animals while exposed to sounds presented in a statistical learning paradigm. However, whether passive exposure to sounds can form long-term memory representations remains to be demonstrated. To investigate this issue, we first exposed adult rats to human speech sounds for 3 consecutive days, 12 h/d. Two groups of rats exposed to either spectrotemporal or tonal changes in speech sounds served as controls for each other. Then, electrophysiological brain responses from the auditory cortex were recorded to the same stimuli. In both the exposure and test phase statistical learning paradigm, was applied. The exposure effect was found for the spectrotemporal sounds, but not for the tonal sounds. Only the animals exposed to spectrotemporal sounds differentiated subtle changes in these stimuli as indexed by the mismatch negativity response. The results point to the occurrence of long-term memory traces for the speech sounds due to passive exposure in adult animals.

Cortical encoding and neurophysiological tracking of intensity and pitch cues signaling English stress patterns in native and nonnative speakers

We examined cross-language differences in neural encoding and tracking of intensity and pitch cues signaling English stress patterns. Auditory mismatch negativities (MMNs) were recorded in English and Mandarin listeners in response to contrastive English pseudowords whose primary stress occurred either on the first or second syllable (i.e., "nocTICity" vs. "NOCticity"). The contrastive syllable stress elicited two consecutive MMNs in both language groups, but English speakers demonstrated larger responses to stress patterns than Mandarin speakers. Correlations between the amplitude of ERPs and continuous changes in the running intensity and pitch of speech assessed how well each language group's brain activity tracked these salient acoustic features of lexical stress. We found that English speakers' neural responses tracked intensity changes in speech more closely than Mandarin speakers (higher brain-acoustic correlation). Findings demonstrate more robust and precise processing of English stress (intensity) patterns in early auditory cortical responses of native relative to nonnative speakers.