Effects of prosodic familiarity on the automatic processing of words in the human brain

Investigating the neurocognitive background of speech perception with a fast multi-feature MMN paradigm

The speech multi-feature MMN (Mismatch Negativity) offers a means to explore the neurocognitive background of the processing of multiple speech features in a short time, by capturing the time-locked electrophysiological activity of the brain known as event-related brain potentials (ERPs). Originating from Näätänen et al. (Clin Neurophysiol 115:140-144, 2004) pioneering work, this paradigm introduces several infrequent deviant stimuli alongside standard ones, each differing in various speech features. In this study, we aimed to refine the multi-feature MMN paradigm used previously to encompass both segmental and suprasegmental (prosodic) features of speech. In the experiment, a two-syllable long pseudoword was presented as a standard, and the deviant stimuli included alterations in consonants (deviation by place or place and mode of articulation), vowels (deviation by place or mode of articulation), and stress pattern in the first syllable of the pseudoword. Results indicated the emergence of MMN components across all segmental and prosodic contrasts, with the expected fronto-central amplitude distribution. Subsequent analyses revealed subtle differences in MMN responses to the deviants, suggesting varying sensitivity to phonetic contrasts. Furthermore, individual differences in MMN amplitudes were noted, partially attributable to participants' musical and language backgrounds. These findings underscore the utility of the multi-feature MMN paradigm for rapid and efficient investigation of the neurocognitive mechanisms underlying speech processing. Moreover, the paradigm demonstrated the potential to be used in further research to study the speech processing abilities in various populations.

Predictions about prosody facilitate lexical access: Evidence from P50/N100 and MMN components

Research into the neural foundation of perception asserts a model where top-down predictions modulate the bottom-up processing of sensory input. Despite becoming increasingly influential in cognitive neuroscience, the precise account of this predictive coding framework remains debated. In this study, we aim to contribute to this debate by investigating how predictions about prosody facilitate speech perception, and to shed light especially on lexical access influenced by simultaneous predictions in different domains, inter alia, prosodic and semantic. Using a passive auditory oddball paradigm, we examined neural responses to prosodic changes, leading to a semantic change as in Dutch nouns canon ['kaːnɔn] 'canon' vs kanon [kaː'nɔn] 'cannon', and used acoustically identical pseudowords as controls. Results from twenty-eight native speakers of Dutch (age range 18-32 years) indicated an enhanced P50/N100 complex to prosodic change in pseudowords as well as an MMN response to both words and pseudowords. The enhanced P50/N100 response to pseudowords is claimed to indicate that all relevant auditory information is still processed by the brain, whereas the reduced response to words might reflect the suppression of information that has already been encoded. The MMN response to pseudowords and words, on the other hand, is best justified by the unification of previously established prosodic representations with sensory and semantic input respectively. This pattern of results is in line with the predictive coding framework acting on multiple levels and is of crucial importance to indicate that predictions about linguistic prosodic information are utilized by the brain as early as 50 ms.

ERP evidence for Slavic and German word stress cue sensitivity in English

Word stress is demanding for non-native learners of English, partly because speakers from different backgrounds weight perceptual cues to stress like pitch, intensity, and duration differently. Slavic learners of English and particularly those with a fixed stress language background like Czech and Polish have been shown to be less sensitive to stress in their native and non-native languages. In contrast, German English learners are rarely discussed in a word stress context. A comparison of these varieties can reveal differences in the foreign language processing of speakers from two language families. We use electroencephalography (EEG) to explore group differences in word stress cue perception between Slavic and German learners of English. Slavic and German advanced English speakers were examined in passive multi-feature oddball experiments, where they were exposed to the word impact as an unstressed standard and as deviants stressed on the first or second syllable through higher pitch, intensity, or duration. The results revealed a robust Mismatch Negativity (MMN) component of the event-related potential (ERP) in both language groups in response to all conditions, demonstrating sensitivity to stress changes in a non-native language. While both groups showed higher MMN responses to stress changes to the second than the first syllable, this effect was more pronounced for German than for Slavic participants. Such group differences in non-native English word stress perception from the current and previous studies are argued to speak in favor of customizable language technologies and diversified English curricula compensating for non-native perceptual variation.

Processing of Grammatical Agreement in the Face of Variation in Lexical Stress: A Mismatch Negativity Study

Previous electroencephalography studies have yielded evidence for automatic processing of syntax and lexical stress. However, these studies looked at both effects in isolation, limiting their generalizability to everyday language comprehension. In the current study, we investigated automatic processing of grammatical agreement in the face of variation in lexical stress. Using an oddball paradigm, we measured the Mismatch Negativity (MMN) in Dutch-speaking participants while they listened to Dutch subject-verb sequences (linguistic context) or acoustically similar sequences in which the subject was replaced by filtered noise (nonlinguistic context). The verb forms differed in the inflectional suffix, rendering the subject-verb sequences grammatically correct or incorrect, and leading to a difference in the stress pattern of the verb forms. We found that the MMNs were modulated in both the linguistic and nonlinguistic condition, suggesting that the processing load induced by variation in lexical stress can hinder early automatic processing of grammatical agreement. However, as the morphological differences between the verb forms correlated with differences in number of syllables, an interpretation in terms of the prosodic structure of the sequences cannot be ruled out. Future research is needed to determine which of these factors (i.e., lexical stress, syllabic structure) most strongly modulate early syntactic processing.

Learning words without trying: Daily second language podcasts support word-form learning in adults

Spoken language contains overlapping patterns across different levels, from syllables to words to phrases. The discovery of these structures may be partially supported by statistical learning (SL), the unguided, automatic extraction of regularities from the environment through passive exposure. SL supports word learning in artificial language experiments, but few studies have examined whether it scales up to support natural language learning in adult second language learners. Here, adult English speakers (n = 70) listened to daily podcasts in either Italian or English for 2 weeks while going about their normal routines. To measure word knowledge, participants provided familiarity ratings of Italian words and nonwords both before and after the listening period. Critically, compared with English controls, Italian listeners significantly improved in their ability to discriminate Italian words and nonwords. These results suggest that unguided exposure to natural, foreign language speech supports the extraction of relevant word features and the development of nascent word forms. At a theoretical level, these findings indicate that SL may effectively scale up to support real-world language acquisition. These results also have important practical implications, suggesting that adult learners may be able to acquire relevant speech patterns and initial word forms simply by listening to the language. This form of learning can occur without explicit effort, formal instruction or focused study.

Vocal melody and musical background are simultaneously processed by the brain for musical predictions

Musical pleasure is related to the capacity to predict and anticipate the music. By recording early cerebral responses of 16 participants with electroencephalography during periods of silence inserted in known and unknown songs, we aimed to measure the contribution of different musical attributes to musical predictions. We investigated the mismatch between past encoded musical features and the current sensory inputs when listening to lyrics associated with vocal melody, only background instrumental material, or both attributes grouped together. When participants were listening to chords and lyrics for known songs, the brain responses related to musical violation produced event-related potential responses around 150-200 ms that were of a larger amplitude than for chords or lyrics only. Microstate analysis also revealed that for chords and lyrics, the global field power had an increased stability and a longer duration. The source localization identified that the right superior temporal and frontal gyri and the inferior and medial frontal gyri were activated for a longer time for chords and lyrics, likely caused by the increased complexity of the stimuli. We conclude that grouped together, a broader integration and retrieval of several musical attributes at the same time recruit larger neuronal networks that lead to more accurate predictions.

Mismatch negativity (MMN) as an index of asymmetric processing of consonant duration in fake Mandarin geminates

Unlike languages where consonant duration is used contrastively to distinguish word meanings, long consonants in Mandarin Chinese only occur across morpheme boundaries as a result of concatenation and are referred to as fake geminates. To investigate whether Mandarin speakers employ duration contrast to differentiate fake Mandarin geminates and corresponding singletons as well as the underlying pattern of the processing, two auditory oddball tasks were carried out to measure the component of MMN, an index of the automatic detection of deviant stimulus. Mandarin pseudoword pairs which differ only in the duration of the medial consonant ([an1 an1] ∼ [an1 nan1] vs. [an2 an2] ∼ [an2 nan2]) were used as stimuli. An asymmetric pattern of brain activation was observed where the singleton deviant in the context of geminate words elicited higher MMNs than in the reversed condition. These findings are in line with earlier research suggesting that the singleton is unspecified for a moraic representation, while the geminate is specified. Mandarin speakers can employ duration contrast to distinguish fake geminates and corresponding singletons; furthermore, the processing of fake concatenated geminates in contrast to singletons is similar to that of real geminates and corresponding singletons.

•

Mandarin speakers use durational cues to differentiate fake geminate/singleton contrast.

•

The contrast is represented asymmetrically with a geminate specified for its length with a mora while a singleton is not.

•

Asymmetric processing pattern between fake geminate/singleton contrast was found for word pairs with different tones.

The effect of lexical status on prosodic processing in infants learning a fixed stress language

In speech processing, in the first year of life, prosody and phoneme-relevant aspects serve different functions. Recent studies have assumed that the two aspects become integrated at around 9 months of age. The present study investigates the effect of lexical status on stress processing in a fixed stress language. We hypothesize that lexicality modulates stress processing, and where the stress cue is in conflict with the lexical status (legal deviant condition), we will observe differences in age indicating the stage of integration. We tested 69 6 and 10 month-old infants in an acoustic oddball event-related potential paradigm. A frequent word stimulus (baba) and a pseudoword (bebe) were used with legal versus illegal stress. We systematically swapped the standard and deviant roles of the different stress variants in two conditions. In the illegal deviant condition in the case of the word stimulus, the response pattern typical for the pseudoword (an MMR to the absence of the stress cue) was missing. This implies the suppression effect of lexicality. In the legal deviant condition, negative MMR (N-MMR) in the second time window indicated a facilitation effect of lexicality in both age groups. As only the 6-month-olds produced an N-MMR in the first time window, we concluded that in a fixed stress language, integration starts at 6 months but is only completed by the age of 10 months. Our results show that lexical status modulates stress processing at word level in a highly regularly stressed language in which stable, long-term language-specific stress representation exists from early infancy.

Mismatch Negativity Is Not Always Modulated by Lexicality

One factor that is commonly thought to influence MMN amplitude is lexicality; multiple studies have shown that real-word deviants elicit larger MMNs than pseudoword deviants. Here, however, we report data from two experiments challenging this assumption. In the first experiment (N = 48), real-word deviants did not elicit more negative MMNs than pseudoword deviants; the acoustic difference between standard and deviant was identical across these comparisons. In this experiment, the pseudoword deviant [pʰa˨˩] differed from a real-word [pʰa˧˥] in tone only; therefore, to test the possibility that the lexicality effect is real but is restricted to pseudowords that differ from real words by at least one segment, we ran a second experiment which included different items and participants, and also included a control comparison in which the pseudoword ([tsʰei˨˩]) differs from all real words by at least one segment (there is no existing Mandarin morpheme pronounced [tsʰei] in any tone). In the second experiment (N = 36), both types of pseudowords failed to elicit less negative MMNs than real words. These findings, together with other recent studies failing to show lexicality effects in MMN, challenge the assumption that wordhood reliably influences MMN amplitude.

ERP mismatch response to phonological and temporal regularities in speech

Predictions of our sensory environment facilitate perception across domains. During speech perception, formal and temporal predictions may be made for phonotactic probability and syllable stress patterns, respectively, contributing to the efficient processing of speech input. The current experiment employed a passive EEG oddball paradigm to probe the neurophysiological processes underlying temporal and formal predictions simultaneously. The component of interest, the mismatch negativity (MMN), is considered a marker for experience-dependent change detection, where its timing and amplitude are indicative of the perceptual system’s sensitivity to presented stimuli. We hypothesized that more predictable stimuli (i.e. high phonotactic probability and first syllable stress) would facilitate change detection, indexed by shorter peak latencies or greater peak amplitudes of the MMN. This hypothesis was confirmed for phonotactic probability: high phonotactic probability deviants elicited an earlier MMN than low phonotactic probability deviants. We do not observe a significant modulation of the MMN to variations in syllable stress. Our findings confirm that speech perception is shaped by formal and temporal predictability. This paradigm may be useful to investigate the contribution of implicit processing of statistical regularities during (a)typical language development.

Brain Processing (Auditory Event-Related Potential) of Stressed Versus Unstressed Words in Thai Speech

This study investigated the effect of the stressed word in Thai language on auditory event-related potential (aERP) in unattended conditions. We presented 30 healthy participants with monosyllabic Thai words consisting of either stressed or unstressed words. We instructed them not to attend to the sound stimuli, but rather to watch and memorize the contents of a silent natural documentary without subtitles. The two listening conditions consisted of 20% deviant stimuli (70 stressed and 70 unstressed words, respectively) and 80% standard stimuli (other 280 unstressed words) presented pseudorandomly and binaurally via a pair of earphones. Participants' aERPs from the two conditions were evaluated by the mismatch negativity (MMN) component of aERP. The mismatch negativity amplitudes in the stressed word condition were significantly higher than those in the unstressed word condition, especially in frontal and left fronto-central brain areas. Therefore, these data show the role of the frontal and left fronto-central brain regions in auditory preattentive processing of stressed word perception among native Thai speakers. This is the first study demonstration that stressed meaningful monosyllable words in tonal language facilitate word perception in this preattentive stage. This result has implications for developing clinical tests evaluating preattentive speech perception.

Human Novelty Response to Emotional Animal Vocalizations: Effects of Phylogeny and Familiarity

Darwin (1872) postulated that emotional expressions contain universals that are retained across species. We recently showed that human rating responses were strongly affected by a listener's familiarity with vocalization types, whereas evidence for universal cross-taxa emotion recognition was limited. To disentangle the impact of evolutionarily retained mechanisms (phylogeny) and experience-driven cognitive processes (familiarity), we compared the temporal unfolding of event-related potentials (ERPs) in response to agonistic and affiliative vocalizations expressed by humans and three animal species. Using an auditory oddball novelty paradigm, ERPs were recorded in response to task-irrelevant novel sounds, comprising vocalizations varying in their degree of phylogenetic relationship and familiarity to humans. Vocalizations were recorded in affiliative and agonistic contexts. Offline, participants rated the vocalizations for valence, arousal, and familiarity. Correlation analyses revealed a significant correlation between a posteriorly distributed early negativity and arousal ratings. More specifically, a contextual category effect of this negativity was observed for human infant and chimpanzee vocalizations but absent for other species vocalizations. Further, a significant correlation between the later and more posteriorly P3a and P3b responses and familiarity ratings indicates a link between familiarity and attentional processing. A contextual category effect of the P3b was observed for the less familiar chimpanzee and tree shrew vocalizations. Taken together, these findings suggest that early negative ERP responses to agonistic and affiliative vocalizations may be influenced by evolutionary retained mechanisms, whereas the later orienting of attention (positive ERPs) may mainly be modulated by the prior experience.

Lexical influence on stress processing in a fixed-stress language

In the present study, we investigate how lexicality affects the processing of suprasegmental features at the word level. In contrast to earlier studies which analyzed the role of either segmental or suprasegmental feature in language processing our aim was to investigate the effect of the lexical status on the processing of violated stress pattern defined by linguistic rules. We have conducted a passive oddball ERP experiment, presenting a frequent CVCV word with legal (familiar) and illegal (unfamiliar) stress patterns. Former results obtained with pseudo-words in a similar paradigm enabled to assess the influence of lexical information on stress processing. The presence of lexically relevant information resulted in different ERP patterns compared to those obtained with pseudo-words. We obtained two consecutive MMN responses to the illegally stressed words while violating the illegal stress pattern with a legal one the deviant stimulus elicited two consecutive MMN responses as well. In the latter condition lexicality clearly enhanced the comparison of prosodic information between standard and deviant stimuli, as these components very completely missing when presenting pseudo-words. We interpret the results that lexicality acts as a filter since in the absence of lexical familiarity unfamiliar stress patterns are discriminated better. Our results highlight that even when stress is fully predictable, it is taken into account during pre-attentive processing of linguistic input.

Don't forget the lyrics! Spatiotemporal dynamics of neural mechanisms spontaneously evoked by gaps of silence in familiar and newly learned songs

The vast majority of people experience musical imagery, the sensation of reliving a song in absence of any external stimulation. Internal perception of a song can be deliberate and effortful, but also may occur involuntarily and spontaneously. Moreover, musical imagery is also involuntarily used for automatically completing missing parts of music or lyrics from a familiar song. The aim of our study was to explore the onset of musical imagery dynamics that leads to the automatic completion of missing lyrics. High-density electroencephalography was used to record the cerebral activity of twenty healthy volunteers while they were passively listening to unfamiliar songs, very familiar songs, and songs previously listened to for two weeks. Silent gaps inserted into these songs elicited a series of neural activations encompassing perceptual, attentional and cognitive mechanisms (range 100-500ms). Familiarity and learning effects emerged as early as 100ms and lasted 400ms after silence occurred. Although participants reported more easily mentally imagining lyrics in familiar rather than passively learnt songs, the onset of neural mechanisms and the power spectrum underlying musical imagery were similar for both types of songs. This study offers new insights into the musical imagery dynamics evoked by gaps of silence and on the role of familiarity and learning processes in the generation of these dynamics. The automatic and effortless method presented here is a potentially useful tool to understand failure in the familiarity and learning processes of pathological populations.

Neural correlates of lexical stress: mismatch negativity reflects fundamental frequency and intensity

Neural correlates of lexical stress were studied using the mismatch negativity (MMN) component in event-related potentials. The MMN responses were expected to reveal the encoding of stress information into long-term memory and the contributions of prosodic features such as fundamental frequency (F0) and intensity toward lexical access. In a passive oddball paradigm, neural responses to changes in F0, intensity, and in both features together were recorded for words and pseudowords. The findings showed significant differences not only between words and pseudowords but also between prosodic features. Early processing of prosodic information in words was indexed by an intensity-related MMN and an F0-related P200. These effects were stable at right-anterior and mid-anterior regions. At a later latency, MMN responses were recorded for both words and pseudowords at the mid-anterior and posterior regions. The P200 effect observed for F0 at the early latency for words developed into an MMN response. Intensity elicited smaller MMN for pseudowords than for words. Moreover, a larger brain area was recruited for the processing of words than for the processing of pseudowords. These findings suggest earlier and higher sensitivity to prosodic changes in words than in pseudowords, reflecting a language-related process. The present study, therefore, not only establishes neural correlates of lexical stress but also confirms the presence of long-term memory traces for prosodic information in the brain.

Cortical encoding and neurophysiological tracking of intensity and pitch cues signaling English stress patterns in native and nonnative speakers

We examined cross-language differences in neural encoding and tracking of intensity and pitch cues signaling English stress patterns. Auditory mismatch negativities (MMNs) were recorded in English and Mandarin listeners in response to contrastive English pseudowords whose primary stress occurred either on the first or second syllable (i.e., "nocTICity" vs. "NOCticity"). The contrastive syllable stress elicited two consecutive MMNs in both language groups, but English speakers demonstrated larger responses to stress patterns than Mandarin speakers. Correlations between the amplitude of ERPs and continuous changes in the running intensity and pitch of speech assessed how well each language group's brain activity tracked these salient acoustic features of lexical stress. We found that English speakers' neural responses tracked intensity changes in speech more closely than Mandarin speakers (higher brain-acoustic correlation). Findings demonstrate more robust and precise processing of English stress (intensity) patterns in early auditory cortical responses of native relative to nonnative speakers.

In the eye of the listener: pupil dilation elucidates discourse processing

The current study investigated cognitive resource allocation in discourse processing by means of pupil dilation and behavioral measures. Short question-answer dialogs were presented to listeners. Either the context question queried a new information focus in the successive answer, or else the context query was corrected in the answer sentence (correction information). The information foci contained in the answer sentences were either adequately highlighted by prosodic means or not. Participants had to judge the adequacy of the focus prosody with respect to the preceding context question. Prosodic judgment accuracy was higher in the conditions bearing adequate focus prosody than in the conditions with inadequate focus prosody. Latency to peak pupil dilation was longer when new information foci were perceived compared to correction foci. Moreover, for the peak dilation, an interaction of focus type and prosody was found. Post hoc statistical tests revealed that prosodically adequate correction focus positions were processed with smaller peak dilation in comparison to all other dialog conditions. Thus, pupil dilation and results of a principal component analysis suggest an interaction of focus type and focus prosody in discourse processing.

Schizophrenia and language--shall we look for a deficit of deviance detection?

In this article, we consider the view on schizophrenia that asserts this disease originates from a deficit in the hemispheric specialization for language. We suggest that a deficit in the hemispheric specialization for language may be a consequence of the other recently shown neurophysiological deficit of schizophrenia, namely deviance detection. We hypothesise that a deficit of deviance detection related to the dysfunction of NMDA receptors in schizophrenia leads to the abnormal interaction between the parallel and sequential streams of speech processing in the brain. This hypothesis opens perspectives for genetic, molecular and pharmacological studies of the deficit of deviance detection in schizophrenia, as reflected by event-related potentials and neuroimaging during speech processing.