An electrophysiological response to different pitch contours in words

Spoken Word Recognition across Language Boundary: ERP Evidence of Prosodic Transfer Driven by Pitch

Extensive research has explored the perception of English lexical stress by Chinese EFL learners and tried to unveil the underlying mechanism of the prosodic transfer from a native tonal language to a non-native stress language. However, the role of the pitch as the shared cue by lexical stress and lexical tone during the transfer remains controversial when the segmental cue (i.e., reduced vowel) is absent. By employing event-related potential (ERP) measurements, the current study aimed to further investigate the role of the pitch during the prosodic transfer from L1 lexical tone to L2 lexical stress and the underlying neural responses. Two groups of adult Chinese EFL learners were compared, as both Mandarin and Cantonese are tonal languages with different levels of complexity. The results showed that Cantonese speakers relied more than Mandarin speakers on pitch cues, not only in their processing of English lexical stress but also in word recognition. Our findings are consistent with the arguments of Cue Weighting and attest to the influence of native tonal language experience on second language acquisition. The results may have implications on pedagogical methods that pitch could be an important clue in second language teaching.

Jellybeans… or Jelly, Beans…? 5-6-year-olds can identify the prosody of compounds but not lists

Learning to use word versus phrase level prosody to identify compounds from lists is thought to be a protracted process, only acquired by 11 years (Vogel & Raimy, 2002). However, a recent study has shown that 5-year-olds can use prosodic cues other than stress for these two structures in production, at least for early-acquired noun-noun compounds (Yuen et al., 2021). This raises the question of whether children this age can also use naturally-produced prosody to identify noun-noun compounds from their list forms in comprehension. The results show that 5-6-year-olds (N = 28) can only identify compounds. Unlike adults, children as a group could not use boundary cues to identify lists and were significantly slower in their processing compared to adults. This suggests that the acquisition of word level prosody may precede the acquisition of phrase level prosody, i.e., some higher-level aspects of phrasal prosody may take longer to acquire.

Perception of lexical neutral tones in mandarin compounds: Electroencephalographic evidence from an oddball paradigm

The present electroencephalographic (EEG) study was designed to investigate the perception of Mandarin neutral tone (T0) by adult Mandarin speakers. For this purpose, we examined the event-related brain potential (ERP) correlates of T0 processing in two-character Mandarin compounds. Eighteen native Mandarin speakers were tested using a modified oddball paradigm. Sixty strong-strong and 60 strong-weak (T0) disyllabic Mandarin compounds were selected from the Contemporary Chinese Dictionary. Participants performed one explicit task of stress discrimination (i.e., deciding whether the fourth word in sequences of 4 spoken words had the same stress pattern as the previous words or a different one) and one implicit task of passive listening. Behaviorally, accuracy showed that Mandarin listeners were able to correctly discriminate T0 from the canonical strong-strong stress pattern in 87.2% of cases. Neurophysiologically, larger N200 and P200 were observed for the infrequent strong-weak stress pattern compared to the canonical strong-strong one. Critically, a N325 was replicated in Mandarin, with a larger N325 for strong-weak than for strong-strong compounds. Consistently with a previous interpretation proposed by Böcker et al. (1999) for Dutch, we argue that the N325 might be a manifestation of the extraction of stress pattern in Mandarin also. Taken together, the present data on T0 perception in Mandarin are discussed in the context of the Prosody-Assisted-Processing (PAP) model (Isel et al., 2003), a cognitive model of spoken compound processing based on stress-timed languages, which postulates an early involvement of prosody in order to guide the morphological analysis at the lexical level.

When correction turns positive: processing corrective prosody in Dutch

Current research on spoken language does not provide a consistent picture as to whether prosody, the melody and rhythm of speech, conveys a specific meaning. Perception studies show that English listeners assign meaning to prosodic patterns, and, for instance, associate some accents with contrast, whereas Dutch listeners behave more controversially. In two ERP studies we tested how Dutch listeners process words carrying two types of accents, which either provided new information (new information accents) or corrected information (corrective accents), both in single sentences (experiment 1) and after corrective and new information questions (experiment 2). In both experiments corrective accents elicited a sustained positivity as compared to new information accents, which started earlier in context than in single sentences. The positivity was not modulated by the nature of the preceding question, suggesting that the underlying neural mechanism likely reflects the construction of an interpretation to the accented word, either by identifying an alternative in context or by inferring it when no context is present. Our experimental results provide strong evidence for inferential processes related to prosodic contours in Dutch.

The relevance of rhythmical alternation in language processing: an ERP study on English compounds

This study investigates the influence of rhythmic expectancies on language processing. It is assumed that language rhythm involves an alternation of strong and weak beats within a linguistic domain. Hence, in some contexts rhythmically induced stress shifts occur in order to comply with the Rhythm Rule. In English, this rule operates to prevent clashes of stressed adjacent syllables or lapses of adjacent unstressed syllables. While previous studies investigated effects on speech production and perception, this study focuses on brain responses to structures either obeying or deviating from this rule. Event-related potentials show that rhythmic regularity is relevant for language processing: rhythmic deviations evoked different ERP components reflecting the deviance from rhythmic expectancies. An N400 effect found for shifted items reflects higher costs in lexical processing due to stress deviation. The overall results disentangle lexical and rhythmical influences on language processing and complement the findings of previous studies on rhythmical processing.

Good, bad and ugly word stress--fMRI evidence for foot structure driven processing of prosodic violations

Using a stress violation paradigm, we investigated whether metrical feet constrain the way prosodic patterns are processed and evaluated. Processing of correctly versus incorrectly stressed words was associated with activation in left posterior angular and retrosplenial cortex, indicating the recognition of an expected and familiar pattern, whereas the inverse contrast yielded enhanced bilateral activation in the superior temporal gyrus, reflecting higher costs in auditory (re-)analysis. More fine-grained analyses of severe versus mild stress violations revealed activations of the left superior temporal and left anterior angular gyrus whereas the opposite contrast led to frontal activations including Broca's area and its right-hemisphere homologue, suggesting that detection of mild violations lead to increased effort in working memory and deeper phonological processing. Our results provide first evidence that different incorrect stress patterns are processed in a qualitatively different way and that the underlying foot structure seems to determine potential stress positions in German words.

Perception and bias in the processing of compound versus phrasal stress: evidence from event-related brain potentials

Previous research using picture/word matching tasks has demonstrated a tendency to incorrectly interpret phrasally stressed strings as compounds. Using event-related potentials, we sought to determine whether this pattern stems from poor perceptual sensitivity to the compound/phrasal stress distinction, or from a post-perceptual bias in behavioral response selection. A secondary aim was to gain insight into the role played by contrastive stress patterns in online sentence comprehension. The behavioral results replicated previous findings of a preference for compounds, but the electrophysiological data suggested a robust sensitivity to both stress patterns. When incongruent with the context, both compound and phrasal stress elicited a sustained left-lateralized negativity. Moreover, incongruent compound stress elicited a centro-parietal negativity (N400), while incongruent phrasal stress elicited a late posterior positivity (P600). We conclude that the previous findings of a preference for compounds are due to response selection bias, and not a lack of perceptual sensitivity. The present results complement previous evidence for the immediate use of meter in semantic processing, as well as evidence for late interactions between prosodic and syntactic information.

Detection of pitch violations depends upon the familiarity of intonational contour of sentences

INTRODUCTION

Former studies have shown that a pitch change in utterances in speech was detected accurately in both native (French) and unfamiliar (Portuguese) language and produced an early negativity and a late positivity in the event-related brain potentials (ERPs), more clearly marked in the native language (Schön et al., 2004; Marques et al., 2007). The present study used the same design to further investigate the influence of the familiarity of the language context on pitch perception with Italian participants. The aim was to examine the effects of pitch change in the native (Italian) and foreign (French) languages, and in meaningless sentences preserving the intonational contour of the mother tongue (jabberwocky).

METHOD

Weak and strong pitch changes incongruous with the preceding context were compared to a control congruous condition. Participants had to decide as fast and as accurately as possible if they perceived a pitch change. Both behavioral (accuracy and reaction times - RTs) and ERP measures were analyzed.

RESULTS

Results showed optimal accuracy and faster RTs in Italian, followed by jabberwocky, and then French. The same trend was present in ERP data, with an early negativity over temporal sites and a late positivity over parietal sites. These effects developed earlier and were more pronounced for the strong incongruity in Italian and in jabberwocky than in French.

CONCLUSIONS

The similarity of results for Italian (congruous) and jabberwocky sentences on one hand, and the difference of results for French sentences, on the other hand, show that familiarity with intonational contour of utterances/speech provided essential cues to perform the task.

Beneficial effects of word final stress in segmenting a new language: evidence from ERPs

Background

How do listeners manage to recognize words in an unfamiliar language? The physical continuity of the signal, in which real silent pauses between words are lacking, makes it a difficult task. However, there are multiple cues that can be exploited to localize word boundaries and to segment the acoustic signal. In the present study, word-stress was manipulated with statistical information and placed in different syllables within trisyllabic nonsense words to explore the result of the combination of the cues in an online word segmentation task.

Results

The behavioral results showed that words were segmented better when stress was placed on the final syllables than when it was placed on the middle or first syllable. The electrophysiological results showed an increase in the amplitude of the P2 component, which seemed to be sensitive to word-stress and its location within words.

Conclusion

The results demonstrated that listeners can integrate specific prosodic and distributional cues when segmenting speech. An ERP component related to word-stress cues was identified: stressed syllables elicited larger amplitudes in the P2 component than unstressed ones.

Musicians detect pitch violation in a foreign language better than nonmusicians: behavioral and electrophysiological evidence

The aim of this study was to determine whether musical expertise influences the detection of pitch variations in a foreign language that participants did not understand. To this end, French adults, musicians and nonmusicians, were presented with sentences spoken in Portuguese. The final words of the sentences were prosodically congruous (spoken at normal pitch height) or incongruous (pitch was increased by 35% or 120%). Results showed that when the pitch deviations were small and difficult to detect (35%: weak prosodic incongruities), the level of performance was higher for musicians than for nonmusicians. Moreover, analysis of the time course of pitch processing, as revealed by the event-related brain potentials to the prosodically congruous and incongruous sentence-final words, showed that musicians were, on average, 300 msec faster than nonmusicians to categorize prosodically congruous and incongruous endings. These results are in line with previous ones showing that musical expertise, by increasing discrimination of pitch--a basic acoustic parameter equally important for music and speech prosody--does facilitate the processing of pitch variations not only in music but also in language. Finally, comparison with previous results [Schön, D., Magne, C., & Besson, M. The music of speech: Music training facilitates pitch processing in both music and language. Psychophysiology, 41, 341-349, 2004] points to the influence of semantics on the perception of acoustic prosodic cues.

It's Early: Event-related Potential Evidence for Initial Interaction of Syntax and Prosody in Speech Comprehension

Psycholinguistic theories assume an interaction between prosody and syntax during language processing. Based on studies using mostly off-line methods, it is unclear whether an interaction occurs at later or initial processing stages. Using event-related potentials, the present study provides neurophysiological evidence for a prosody and syntax interaction in initial processing. The sentence material contained mere prosodic and syntactic as well as combined prosodic-syntactic violations. For the syntax violation, the critical word appeared after a preposition. The suffix of the critical word either indicated a noun fulfilling the syntactic requirements of the preceding preposition or a verb causing a word category violation. For the prosodic manipulation, congruent critical words were normally intonated (signaling sentence continuation) while prosodically incongruent critical words signaled sentence end. For the mere prosodic incongruity, a broadly distributed negativity was observed at the critical word-stem (300–500 msec aligned to word onset). In response to a mere syntactic error, a left temporal negativity was elicited in an early time window (200–400 msec aligned to suffix onset), taken to reflect initial phrase structure building processes. In contrast, in response to the combined prosodic-syntactic violation, an early temporal negativity showed up bilaterally at the suffix in the same time window. Our interpretation is that the process of initial structure building as reflected in the early left anterior negativity recruits additional right hemispheric neural resources when the critical word contains both syntactic and prosodic violations. This suggests the immediate influence of phrasal prosody during the initial parsing stage in speech processing.

Late interaction of syntactic and prosodic processes in sentence comprehension as revealed by ERPs

The present study was designed to examine the processing of prosodic and syntactic information in spoken language. The aim was to investigate the long discussed relationship between prosody and syntax in online speech comprehension to reveal direct evidence about whether the two information types are interactive or independent from each other. The method of event-related potentials allowed us to sheet light on the precise time course of this relationship. Our experimental manipulation involved two prosodically different positions in German sentences, i.e., the critical noun in penultimate vs. final position. In syntactically correct sentences, a prosodic manipulation of the penultimate word gave rise to a late centroparietal negativity that resembled the classical N400 component. We interpreted the negativity as a correlate of lexical integration costs for the prosodically unexpected sentence-final word. Comparisons with syntactically incorrect sentences revealed that this effect was dependent on the sentences' grammatical correctness. When the prosodic manipulation was realized at the final word, we observed a right anterior negativity followed by a late positivity (P600). The right anterior negativity was present independent of the sentences' grammatical correctness. However, the P600 was not, as a late positivity was present for straightforward prosodic and syntactic violations but increased for the combined violations. This suggests that the right anterior negativity, and not the P600, should be considered as a pure prosodic effect. The combined data moreover suggest an interaction between prosody and syntax in a later time window during sentence comprehension.

On-line Processing of “Pop-Out” Words in Spoken French Dialogues

Highlighting relevant information in a discourse context is a major aim of spoken language communication. Prosodic cues such as focal prominences are used to fulfill this aim through the pragmatic function of prosody. To determine whether listeners make on-line use of focal prominences to build coherent representations of the informational structure of the utterances, we used the brain event-related potential (ERP) method. Short dialogues composed of a question and an answer were presented auditorily. The design of the experiment allowed us to examine precisely the time course of the processing of prosodic patterns of sentence-medial or -final words in the answer. These patterns were either congruous or incongruous with regard to the pragmatic context introduced by the question. Furthermore, the ERP effects were compared for words with or without focal prominences. Results showed that pragmatically congruous and incongruous prosodic patterns elicit clear differences in the ERPs, which were largely modulated in latency and polarity by their position within the answer. By showing that prosodic patterns are processed on-line by listeners in order to understand the informational structure of the message, the present results demonstrate the psychobiological validity of the pragmatic concept of focus, expressed via prosodic cues. Moreover, the functional significance of the positive-going effects found sentence medially and negative-going effects found sentence finally is discussed. Whereas the former may reflect the processing of surprising and task-relevant prosodic patterns, the latter may reflect the integration problems encountered in extracting the overall informational structure of the sentence.

MEG responses to rippled noise and Huggins pitch reveal similar cortical representations

The onset of pitch within an ongoing noise signal evokes a particular brain activity, the pitch onset response (POR). Using whole-head MEG, PORs to iterated rippled noise (IRN) and Huggins pitch (HP), representing prototypical pitch-in-noise signals, were measured in twenty subjects during a pitch identification task (333 Hz, 400 Hz, randomized). HP and IRN yielded similar responses, lateralized to the left hemisphere and peaking about 180 ms after pitch onset. The initial phase (140 ms) showed stronger activations to 400 than to 333 Hz whereas later stages (200-300 ms) showed target vs nontarget effects. These results suggest, first, that different pitches converge into a common cortical representation and, second, that the POR encompasses various successive processing stages.

The role of the insular cortex in pitch pattern perception: the effect of linguistic contexts

Auditory pitch patterns are significant ecological features to which nervous systems have exquisitely adapted. Pitch patterns are found embedded in many contexts, enabling different information-processing goals. Do the psychological functions of pitch patterns determine the neural mechanisms supporting their perception, or do all pitch patterns, regardless of function, engage the same mechanisms? This issue is pursued in the present study by using 150-water positron emission tomography to study brain activations when two subject groups discriminate pitch patterns in their respective native languages, one of which is a tonal language and the other of which is not. In a tonal language, pitch patterns signal lexical meaning. Native Mandarin-speaking and English-speaking listeners discriminated pitch patterns embedded in Mandarin and English words and also passively listened to the same stimuli. When Mandarin listeners discriminated pitch embedded in Mandarin lexical tones, the left anterior insular cortex was the most active. When they discriminated pitch patterns embedded in English words, the homologous area in the right hemisphere activated as it did in English-speaking listeners discriminating pitch patterns embedded in either Mandarin or English words. These results support the view that neural responses to physical acoustic stimuli depend on the function of those stimuli and implicate anterior insular cortex in auditory processing, with the left insular cortex especially responsive to linguistic stimuli.

Brain Responses to Segmentally and Tonally Induced Semantic Violations in Cantonese

The present event-related potential (ERP) study examined the role of tone and segmental information in Cantonese word processing. To this end, participants listened to sentences that were either semantically correct or contained a semantically incorrect word. Semantically incorrect words differed from the most expected sentence completion at the tone level, at the segmental level, or at both levels. All semantically incorrect words elicited an increased frontal negativity that was maximal 300 msec following word onset and an increased centroparietal positivity that was maximal 650 msec following word onset. There were differences between completely incongruous words and the other two violation conditions with respect to the latency and amplitude of the ERP effects. These differences may be due to differences in the onset of acoustic deviation of the presented from the expected word and different mechanisms involved in the processing of complete as compared to partial acoustic deviations. Most importantly, however, tonally and segmentally induced semantic violations were comparable. This suggests that listeners access tone and segmental information at a similar point in time and that both types of information play comparable roles during word processing in Cantonese.

Time course and hemispheric lateralization effects of complex pitch processing: evoked magnetic fields in response to rippled noise stimuli

To delineate the time course and processing stages of pitch encoding at the level of the supratemporal plane, the present study recorded evoked magnetic fields in response to rippled noise (RN) stimuli. RN largely masks simple tonotopic representations and addresses pitch processing within the temporal domain (periodicity encoding). Four dichotic stimulus types (111 or 133 Hz RN at one ear, white noise to the other one) were applied in randomized order during either visual distraction or selective auditory attention. Strictly periodic signals, noise-like events, and mixtures of both signals served as control conditions. (1) Attention-dependent ear x hemisphere interactions were observed within the time domain of the M50 field, indicating early streaming of auditory information. (2) M100 responses to strictly periodic stimuli were found lateralized to the right hemisphere. Furthermore, the higher-pitched stimuli yielded enhanced activation as compared to the lower-pitch signals (pitch scaling), conceivably reflecting sensory memory operations. (3) Besides right-hemisphere pitch scaling, the relatively late M100 component in association with the RN condition (latency = 136 ms) showed significantly stronger field strengths over the left hemisphere. Control experiments revealed this lateralization effect to be related to noise rather than pitch processing. Furthermore, subtle noise variations interacted with signal periodicity. Obviously, thus, complex task demands such as RN encoding give rise to functional segregation of auditory processing across the two hemispheres (left hemisphere: noise, right hemisphere: periodicity representation). The observed noise/periodicity interactions, furthermore, might reflect pitch-synchronous spectral evaluation at the level of the left supratemporal plane, triggered by right-hemisphere representation of signal periodicity.

ERPs reflect lexical identification in word fragment priming

Behavioral evidence suggests that spoken word recognition involves the temporary activation of multiple entries in a listener's mental lexicon. This phenomenon can be demonstrated in cross-modal word fragment priming (CMWP). In CMWP, an auditory word fragment (prime) is immediately followed by a visual word or pseudoword (target). Experiment 1 investigated ERPs for targets presented in this paradigm. Half of the targets were congruent with the prime (e.g., in the prime-target pair: AM-AMBOSS [anvil]), half were not (e.g., AM-PENSUM [pensum]). Lexical entries of the congruent targets should receive activation from the prime. Thus, lexical identification of these targets should be facilitated. An ERP effect named P350, two frontal negative ERP deflections, and the N400 were sensitive to prime-target congruency. In Experiment 2, the relation of the formerly observed ERP effects to processes in a modality-independent mental lexicon was investigated by presenting primes visually. Only the P350 effect could be replicated across different fragment lengths. Therefore, the P350 is discussed as a correlate of lexical identification in a modality-independent mental lexicon.

Pitch modulates lexical identification in spoken word recognition: ERP and behavioral evidence

Event-related potentials (ERPs) were recorded in cross-modal word fragment priming (CMWP) to address the function of pitch for the identification of spoken words. In CMWP fragments of spoken words (e.g., re taken from Regal [Engl. shelves]) are immediately followed by visual targets. Together with reduced reaction times (RTs), an ERP deflection named P350 has been found to be reduced for targets, which match the primes (e.g., in the prime-target pair re-REGAL) as compared to unrelated targets (e.g., re-WIRBEL [Engl. burble]). The P350 has been related to facilitated lexical identification [Friedrich, Kotz, Friederici and Gunter (in press), ERPs reflect lexical identification in word fragment priming, JOCN]. In the present study, we presented syllable primes with different pitch contours. One version of each prime bore a stressed pitch contour (e.g., re_1), the other an unstressed pitch contour (e.g., re_2). Primes were combined with targets being either stressed on the first syllable (e.g., REgel [Engl. rule]) or on the second syllable (e.g., reGAL [Engl. shelves]). We found a reduced amplitude of the P350 and slightly faster reactions for targets with a stress pattern that matched the pitch of the primes (e.g., re_1-REgel) as compared to targets with a stress pattern that did not match the pitch of the primes (e.g., re_1-reGAL). The present study replicates the P350 effect with different material, and indicates that pitch is used for lexical identification in spoken word recognition.

Non-voluntary and voluntary processing of emotional prosody: an event-related potentials study

The present study investigated whether event-related potentials (ERPs) reflect non-voluntary vs voluntary processing of emotional prosody. ERPs were obtained while participants processed emotional information non-voluntarily (i.e. while evaluating semantic characteristics of a stimulus) and voluntarily (i.e. while evaluating emotional characteristics of a stimulus). Results suggest that emotional prosody is processed around 160 ms after stimulus onset under non-voluntary processing conditions (when the attention is diverted from the emotional meaning of the tone of voice); and around 360 ms under voluntary processing conditions. The findings support the notion that emotional prosody is processed non-voluntarily in the comprehension of a spoken message.