Processing of word stress related acoustic information: A multi-feature MMN study

Investigating the neurocognitive background of speech perception with a fast multi-feature MMN paradigm

The speech multi-feature MMN (Mismatch Negativity) offers a means to explore the neurocognitive background of the processing of multiple speech features in a short time, by capturing the time-locked electrophysiological activity of the brain known as event-related brain potentials (ERPs). Originating from Näätänen et al. (Clin Neurophysiol 115:140-144, 2004) pioneering work, this paradigm introduces several infrequent deviant stimuli alongside standard ones, each differing in various speech features. In this study, we aimed to refine the multi-feature MMN paradigm used previously to encompass both segmental and suprasegmental (prosodic) features of speech. In the experiment, a two-syllable long pseudoword was presented as a standard, and the deviant stimuli included alterations in consonants (deviation by place or place and mode of articulation), vowels (deviation by place or mode of articulation), and stress pattern in the first syllable of the pseudoword. Results indicated the emergence of MMN components across all segmental and prosodic contrasts, with the expected fronto-central amplitude distribution. Subsequent analyses revealed subtle differences in MMN responses to the deviants, suggesting varying sensitivity to phonetic contrasts. Furthermore, individual differences in MMN amplitudes were noted, partially attributable to participants' musical and language backgrounds. These findings underscore the utility of the multi-feature MMN paradigm for rapid and efficient investigation of the neurocognitive mechanisms underlying speech processing. Moreover, the paradigm demonstrated the potential to be used in further research to study the speech processing abilities in various populations.

Segregation and integration of sensory features by flexible temporal characteristics of independent neural representations

Segregation and integration are two fundamental yet competing computations in cognition. For example, in serial speech processing, stable perception necessitates the sequential establishment of perceptual representations to remove irrelevant features for achieving invariance. Whereas multiple features need to combine to create a coherent percept. How to simultaneously achieve seemingly contradicted computations of segregation and integration in a serial process is unclear. To investigate their neural mechanisms, we used loudness and lexical tones as a research model and employed a novel multilevel oddball paradigm with Electroencephalogram (EEG) recordings to explore the dynamics of mismatch negativity (MMN) responses to their deviants. When two types of deviants were presented separately, distinct topographies of MMNs to loudness and tones were observed at different latencies (loudness earlier), supporting the sequential dynamics of independent representations for two features. When they changed simultaneously, the latency of responses to tones became shorter and aligned with that to loudness, while the topographies remained independent, yielding the combined MMN as a linear additive of single MMNs of loudness and tones. These results suggest that neural dynamics can be temporally synchronized to distinct sensory features and balance the computational demands of segregation and integration, grounding for invariance and feature binding in serial processing.

ERP evidence for Slavic and German word stress cue sensitivity in English

Word stress is demanding for non-native learners of English, partly because speakers from different backgrounds weight perceptual cues to stress like pitch, intensity, and duration differently. Slavic learners of English and particularly those with a fixed stress language background like Czech and Polish have been shown to be less sensitive to stress in their native and non-native languages. In contrast, German English learners are rarely discussed in a word stress context. A comparison of these varieties can reveal differences in the foreign language processing of speakers from two language families. We use electroencephalography (EEG) to explore group differences in word stress cue perception between Slavic and German learners of English. Slavic and German advanced English speakers were examined in passive multi-feature oddball experiments, where they were exposed to the word impact as an unstressed standard and as deviants stressed on the first or second syllable through higher pitch, intensity, or duration. The results revealed a robust Mismatch Negativity (MMN) component of the event-related potential (ERP) in both language groups in response to all conditions, demonstrating sensitivity to stress changes in a non-native language. While both groups showed higher MMN responses to stress changes to the second than the first syllable, this effect was more pronounced for German than for Slavic participants. Such group differences in non-native English word stress perception from the current and previous studies are argued to speak in favor of customizable language technologies and diversified English curricula compensating for non-native perceptual variation.

Relationship between early neural responses to syntactic and acoustic irregularities in music

Humans can detect various anomalies in a sound sequence without attending to each dimension explicitly. Event-related potentials (ERPs) have been used to examine the processes of auditory deviance detection. Previous research has shown that music-syntactic anomalies elicit early right anterior negativity (ERAN), whereas more general acoustic irregularities elicit mismatch negativity (MMN). Although these ERP components occur in a similar latency range with a similar scalp topography, the relationship between the detection processes they reflect remains unclear. This study compared these components by manipulating music-syntactic (chord progression) and acoustic (intensity) irregularities orthogonally in two experiments. Non-musicians (Experiment 1: N = 39; Experiment 2: N = 24) were asked to listen to chord sequences, each consisting of 5 four-voice chords, as they watched a silent video clip. Standard, harmonic-deviant, intensity-deviant and double-deviant chords occurred at the final position in each sequence. Deviant stimuli were presented infrequently (p = .10) in Experiment 1 and equiprobably (p = .25) in Experiment 2. Regardless of deviance probability, both harmonic and intensity deviants elicited similar negativities, which were indistinguishable in terms of latency or scalp distribution. When the two deviant types occurred simultaneously, the negativity increased in an additive manner; that is, the amplitude of the double-deviant ERP was as large as the sum of the single-deviant ERPs. These findings suggest that the detection of music-syntactic and acoustic irregularities works independently, based on different regularity representations.

Auditory Discrimination Elicited by Nonspeech and Speech Stimuli in Children With Congenital Hearing Loss

PURPOSE

Congenital deafness not only delays auditory development but also hampers the ability to perceive nonspeech and speech signals. This study aimed to use auditory event-related potentials to explore the mismatch negativity (MMN), P3a, negative wave (Nc), and late discriminative negativity (LDN) components in children with and without hearing loss.

METHOD

Nineteen children with normal hearing (CNH) and 17 children with hearing loss (CHL) participated in this study. Two sets of pure tones (1 kHz vs. 1.1 kHz) and lexical tones (/ba2/ vs. /ba4/) were used to examine the auditory discrimination process.

RESULTS

MMN could be elicited by the pure tone and the lexical tone in both groups. The MMN latency elicited by nonspeech and speech was later in CHL than in CNH. Additionally, the MMN latency induced by speech occurred later in the left than in the right hemisphere in CNH, and the MMN amplitude elicited by speech in CHL produced a discriminative deficiency compared with that in CNH. Although the P3a latency and amplitude elicited by nonspeech in CHL and CNH were not significantly different, the Nc amplitude elicited by speech performed much lower in CHL than in CNH. Furthermore, the LDN latency elicited by nonspeech was later in CHL than in CNH, and the LDN amplitude induced by speech showed higher dominance in the right hemisphere in both CNH and CHL.

CONCLUSION

By incorporating nonspeech and speech auditory conditions, we propose using MMN, Nc, and LDN as potential indices to investigate auditory perception, memory, and discrimination.

English and Mandarin native speakers' cue-weighting of lexical stress: Results from MMN and LDN

Past research on how listeners weight stress cues such as pitch, duration and intensity has reported two inconsistent patternss: listeners' weighting conforms to 1) their native language experience (e.g., language rhythmicity, lexical tone), and 2) a general "iambic-trochaic law" (ITL), favouring innate sound groupings in cue perception. This study aims to tease apart the above effects by investigating the weighting of pitch, duration and intensity cues in stress-timed (Australian English) and non-stress-timed and tonal (Taiwan Mandarin) language speaking adults using a mismatch negativity (MMN) multi-feature paradigm. Results show effects that can be explained by language-specific rhythmic influence, but only partially by the ITL. Moreover, these findings revealed cross-linguistic differences indexed by both MMN and late discriminative negativity (LDN) responses at cue and syllable position levels, and thus call for more sophisticated perspectives for existing cue-weighting models.

Formant-invariant voice and pitch representations are pre-attentively formed from constantly varying speech and non-speech stimuli

The present study investigated whether listeners can form abstract voice representations while ignoring constantly changing phonological information and if they can use the resulting information to facilitate voice change detection. Further, the study aimed at understanding whether the use of abstraction is restricted to the speech domain or can be deployed also in non‐speech contexts. We ran an electroencephalogram (EEG) experiment including one passive and one active oddball task, each featuring a speech and a rotated speech condition. In the speech condition, participants heard constantly changing vowels uttered by a male speaker (standard stimuli) which were infrequently replaced by vowels uttered by a female speaker with higher pitch (deviant stimuli). In the rotated speech condition, participants heard rotated vowels, in which the natural formant structure of speech was disrupted. In the passive task, the mismatch negativity was elicited after the presentation of the deviant voice in both conditions, indicating that listeners could successfully group together different stimuli into a formant‐invariant voice representation. In the active task, participants showed shorter reaction times (RTs), higher accuracy and a larger P3b in the speech condition with respect to the rotated speech condition. Results showed that whereas at a pre‐attentive level the cognitive system can track pitch regularities while presumably ignoring constantly changing formant information both in speech and in rotated speech, at an attentive level the use of such information is facilitated for speech. This facilitation was also testified by a stronger synchronisation in the theta band (4–7 Hz), potentially pointing towards differences in encoding/retrieval processes.

Electrophysiological Signatures of Perceiving Alternated Tone in Mandarin Chinese: Mismatch Negativity to Underlying Tone Conflict

Although phonological alternation is prevalent in languages, the process of perceiving phonologically alternated sounds is poorly understood, especially at the neurolinguistic level. We examined the process of perceiving Mandarin 3rd tone sandhi (T3 + T3 → T2 + T3) with a mismatch negativity (MMN) experiment. Our design has two independent variables (whether the deviant undergoes tone sandhi; whether the standard and the deviant have matched underlying tone). These two independent variables modulated ERP responses in both the first and the second syllables. Notably, despite the apparent segmental conflict between the standard and the deviant in all conditions, MMN is only observed when neither the standard nor the deviant undergoes tone sandhi, suggesting that discovering the underlying representation of an alternated sound could interfere with the generation of MMN. A tentative model with three hypothesized underlying processing mechanisms is proposed to explain the observed latency and amplitude differences across conditions. The results are also discussed in light of the potential electrophysiological signatures involved in the process of perceiving alternated sounds.

Improved Auditory Function Caused by Music Versus Foreign Language Training at School Age: Is There a Difference?

In adults, music and speech share many neurocognitive functions, but how do they interact in a developing brain? We compared the effects of music and foreign language training on auditory neurocognition in Chinese children aged 8–11 years. We delivered group-based training programs in music and foreign language using a randomized controlled trial. A passive control group was also included. Before and after these year-long extracurricular programs, auditory event-related potentials were recorded (n = 123 and 85 before and after the program, respectively). Through these recordings, we probed early auditory predictive brain processes.

To our surprise, the language program facilitated the children’s early auditory predictive brain processes significantly more than did the music program. This facilitation was most evident in pitch encoding when the experimental paradigm was musically relevant. When these processes were probed by a paradigm more focused on basic sound features, we found early predictive pitch encoding to be facilitated by music training.

Thus, a foreign language program is able to foster auditory and music neurocognition, at least in tonal language speakers, in a manner comparable to that by a music program. Our results support the tight coupling of musical and linguistic brain functions also in the developing brain.

Deficient sensory and cognitive processing in children with cochlear implants: An event-related potential study

Compared with children having normal hearing (NH), those with cochlear implants (CIs) perform poorly in spoken language comprehension which involves both low-level acoustic encoding and higher-level cognitive processing. Here, we performed an electroencephalography study to portray this brain dynamics of speech perception in CI children. We presented a Mandarin Chinese monosyllable or four-syllable idiom to CI and NH children, and infrequently varied its lexical tone to form a novel monosyllable or pseudo-idiom in an oddball paradigm. The tone contrast embedded in the monosyllables evoked similar mismatch negativities (MMNs) in CI and NH children at an early stage (~200 ms). However, the amplitude of the MMN evoked by the tone contrast in the idiom context was significantly lower in CI children than in NH children. Furthermore, robust late discriminative negativity (LDN) at a late stage (~500 ms) was found only in NH children, but not in CI children. The MMN and LDN findings indicate deficits of low-level acoustic encoding in a complex context (such as an idiom) and higher-level cognitive processing in CI children, respectively. Both deficient sensory and cognitive processing may contribute to the speech perception difficulties in CI children.

Near native-like stress pattern perception in English-French bilinguals as indexed by the mismatch negativity

We examined lexical stress processing in English-French bilinguals. Auditory mismatch negativity (MMN) responses were recorded in response to English and French pseudowords, whose primary stress occurred either on a language-consistent "usual" or language-inconsistent "unusual" syllable. In most conditions, the pseudowords elicited two consecutive MMNs, and somewhat surprisingly, these MMNs were not systematically modulated by bilingual experience. This suggests that it is possible to achieve native-like pre-attentive processing of lexical stress, even in a language that one has not learned since birth.