ERP evidence for the online processing of rhythmic pattern during Chinese sentence reading

Frequency effects can modulate the neural correlates of prosodic processing in Mandarin

In tonal languages, tone perception involves the processing of both acoustic and phonological information conveyed by tonal signals. In Mandarin, in addition to four canonical full tones, there exists a group of weak syllables known as neutral tones. This study aims to investigate the impact of lexical frequency effects and prosodic information associated with neutral tones on the auditory representation of Mandarin compounds. We initially selected disyllabic compounds as targets, manipulating their lexical frequencies and prosodic structures. Subsequently, these target compounds were embedded into selected sentences and auditorily presented to native speakers. During the experiments, participants engaged in lexical decision tasks while their event-related potentials were recorded. The results showed that the auditory lexical representation of disyllabic compounds was modulated by lexical frequency effects. Rare compounds and compounds with rare first constituents elicited larger N400 effects compared to frequent compounds. Furthermore, neutral tones were found to play a role in the processing, resulting in larger N400 effects. Our findings showed significantly increased amplitudes of the N400 component, suggesting that the processing of rare compounds and compounds with neutral tones may require more cognitive resources. Additionally, we observed an interaction effect between lexical frequency and neutral tones, indicating that they could serve as determining cues in the auditory processing of disyllabic compounds.

Perceptual intake explains variability in statistical word segmentation

One of the first problems in language learning is to segment words from continuous speech. Both prosodic and distributional information can be useful, and it is an important question how the two types of information are integrated. In this paper, we propose that the distinction between input (the statistical properties of the syllable sequence), and intake (how learners perceptually represent the syllable sequence) is a useful framework to integrate different sources of information. We took a novel approach, observing how a large number of syllable sequences were segmented. These sequences had the same transitional probability information for finding word boundaries but different syllables in them. We found large variability in the performance of the segmentation task, suggesting that factors other than the statistical properties of sequences were at play. This variability was explored using the input/intake asymmetry framework, which predicted that factors that shaped the representation of different syllable sequences could explain the variability of learning. We examined two factors, the saliency of the rhythm in these syllable sequences and how familiar the novel word forms in the sequence were to the existing lexicon. Both factors explained the variance in the learnability of different sequences, suggesting that processing of the sequences shaped learning. The implications of these results to computational models of statistical learning and broader implications to language learning were discussed.

Theta Band (4-8 Hz) Oscillations Reflect Online Processing of Rhythm in Speech Production

How speech prosody is processed in the brain during language production remains an unsolved issue. The present work used the phrase-recall paradigm to analyze brain oscillation underpinning rhythmic processing in speech production. Participants were told to recall target speeches aloud consisting of verb-noun pairings with a common (e.g., [2+2], the numbers in brackets represent the number of syllables) or uncommon (e.g., [1+3]) rhythmic pattern. Target speeches were preceded by rhythmic musical patterns, either congruent or incongruent, created by using pure tones at various temporal intervals. Electroencephalogram signals were recorded throughout the experiment. Behavioral results in 2+2 target speeches showed a rhythmic priming effect when comparing congruent and incongruent conditions. Cerebral-acoustic coherence analysis showed that neural activities synchronized with the rhythmic patterns of primes. Furthermore, target phrases that had congruent rhythmic patterns with a prime rhythm were associated with increased theta-band (4-8 Hz) activity in the time window of 400-800 ms in both the 2+2 and 1+3 target conditions. These findings suggest that rhythmic patterns can be processed online. Neural activities synchronize with the rhythmic input and speakers create an abstract rhythmic pattern before and during articulation in speech production.

Sentential position of VN combination modulates the rhythmic pattern effect during Chinese sentence reading: Evidence from eye movements

Though previous research has examined how implicit meter can facilitate the processing of stress-timed languages, syllable-timed languages, such as Chinese, remain under studied. Past research has shown that among verb-noun combinations in Chinese, the processing of [2 + 2] (two disyllabic words) combination rhythmic pattern is easier than that of [2 + 1] (a disyllabic word and a monosyllabic word) pattern, though it is unclear whether this effect is modulated by the sentential position of the verb-noun combination. The present study uses eye-tracking to examine the influence of position on rhythmic pattern during silent reading. In Experiment 1, participants read sentences with [2 + 1] versus [2 + 2] VN phrases embedded in different sentential positions. Results show that the fixation duration of [2 + 1] VN phrases is significantly longer than that of [2 + 2] and that the fixation duration of VN phrases is shorter at the sentence-middle position than it is at the sentence-final position, suggesting that the rhythmic pattern effect at the sentence-middle position exhibits a reduced magnitude compared to the sentence-final position. In Experiment 2, participants read sentences with either mono- or disyllabic words after the VN phrases to further explore whether the reduction of the rhythmic pattern effect is related to the number of succeeding syllables. Results show that while the fixation duration of the [2 + 1] VN pattern is significantly longer than that of the [2 + 2] pattern, there is no significant difference between the monosyllabic versus the disyllabic conditions, nor is there a significant interaction between rhythmic pattern and syllable length post VN phrases, thus ruling out the rhythmic effect from succeeding context. Together, these patterns suggest that the reduction of the rhythmic pattern effect is caused by position rather than number of syllables after phrases.

Rhythmic pattern facilitates speech production: An ERP study

Rhythm affects the speech perception of events unfolding over time. However, it is not clear to what extent the rhythm could affect the processes of sentence speech production. In this event-related potential (ERP) study, we examined whether a particular rhythmic pattern could affect the planning of speech production before articulation. We recorded electrophysiological (EEG) and behavioural (reaction time) data while participants read aloud a target speech in Chinese. Target speeches were sentences or phrases consisting four characters, with regular (e.g., the 2 + 2 pattern; numbers in the brackets represent the number of syllables) or irregular (e.g., 1 + 3) rhythmic patterns, which were preceded by congruent or incongruent musical rhythmic patterns formed by simple pure tones with different temporal intervals. Behavioural and ERP findings indicated a rhythmic priming effect in comparing congruent and incongruent conditions in the regular target speeches, but not in the irregular ones. An early component (N100) that was elicited in response to target speeches that were rhythmically mismatched to primes was linked to the detection of hierarchical linguistic units, which did not conform to expectations. A later negative component (N400) was thought to reflect the violation of expectation on rhythmic pattern in speech production. These findings suggest that rhythmic pattern constrains grammatical and prosodic encoding during speech production, and support the hypothesis that speakers form a grammatical or a prosodic abstract frame before articulation.

Sentence-Level Effects of Literary Genre: Behavioral and Electrophysiological Evidence

The current study used event-related brain potentials (ERPs) and behavioral measures to examine effects of genre awareness on sentence processing and evaluation. We hypothesized that genre awareness modulates effects of genre-typical manipulations. We manipulated instructions between participants, either specifying a genre (poetry) or not (neutral). Sentences contained genre-typical variations of semantic congruency (congruent/incongruent) and morpho-phonological features (archaic/contemporary inflections). Offline ratings of meaningfulness (n = 64/group) showed higher average ratings for semantically incongruent sentences in the poetry vs. neutral condition. ERPs during sentence reading (n = 24/group; RSVP presentation at a fixed per-constituent rate; probe task) showed a left-lateralized N400-like effect for contemporary vs. archaic inflections. Semantic congruency elicited a bilateral posterior N400 effect for incongruent vs. congruent continuations followed by a centro-parietal positivity (P600). While N400 amplitudes were insensitive to the genre, the latency of the P600 was delayed by the poetry instruction. From these results, we conclude that during real-time sentence comprehension, readers are sensitive to subtle morphological manipulations and the implicit prosodic differences that accompany them. By contrast, genre awareness affects later stages of comprehension.

Aesthetic appreciation of poetry correlates with ease of processing in event-related potentials

Rhetorical theory suggests that rhythmic and metrical features of language substantially contribute to persuading, moving, and pleasing an audience. A potential explanation of these effects is offered by "cognitive fluency theory," which stipulates that recurring patterns (e.g., meter) enhance perceptual fluency and can lead to greater aesthetic appreciation. In this article, we explore these two assertions by investigating the effects of meter and rhyme in the reception of poetry by means of event-related brain potentials (ERPs). Participants listened to four versions of lyrical stanzas that varied in terms of meter and rhyme, and rated the stanzas for rhythmicity and aesthetic liking. The behavioral and ERP results were in accord with enhanced liking and rhythmicity ratings for metered and rhyming stanzas. The metered and rhyming stanzas elicited smaller N400/P600 ERP responses than their nonmetered, nonrhyming, or nonmetered and nonrhyming counterparts. In addition, the N400 and P600 effects for the lyrical stanzas correlated with aesthetic liking effects (metered-nonmetered), implying that modulation of the N400 and P600 has a direct bearing on the aesthetic appreciation of lyrical stanzas. We suggest that these effects are indicative of perceptual-fluency-enhanced aesthetic liking, as postulated by cognitive fluency theory.

Speech rhythm sensitivity and musical aptitude: ERPs and individual differences

This study investigated the electrophysiological markers of rhythmic expectancy during speech perception. In addition, given the large literature showing overlaps between cognitive and neural resources recruited for language and music, we considered a relation between musical aptitude and individual differences in speech rhythm sensitivity. Twenty adults were administered a standardized assessment of musical aptitude, and EEG was recorded as participants listened to sequences of four bisyllabic words for which the stress pattern of the final word either matched or mismatched the stress pattern of the preceding words. Words with unexpected stress patterns elicited an increased fronto-central mid-latency negativity. In addition, rhythm aptitude significantly correlated with the size of the negative effect elicited by unexpected iambic words, the least common type of stress pattern in English. The present results suggest shared neurocognitive resources for speech rhythm and musical rhythm.

Processing Rhythmic Pattern during Chinese Sentence Reading: An Eye Movement Study

Prosodic constraints play a fundamental role during both spoken sentence comprehension and silent reading. In Chinese, the rhythmic pattern of the verb-object (V-O) combination has been found to rapidly affect the semantic access/integration process during sentence reading (Luo and Zhou, 2010). Rhythmic pattern refers to the combination of words with different syllabic lengths, with certain combinations disallowed (e.g., [2 + 1]; numbers standing for the number of syllables of the verb and the noun respectively) and certain combinations preferred (e.g., [1 + 1] or [2 + 2]). This constraint extends to the situation in which the combination is used to modify other words. A V-O phrase could modify a noun by simply preceding it, forming a V-O-N compound; when the verb is disyllabic, however, the word order has to be O-V-N and the object is preferred to be disyllabic. In this study, we investigated how the reader processes the rhythmic pattern and word order information by recording the reader's eye-movements. We created four types of sentences by crossing rhythmic pattern and word order in compounding. The compound, embedding a disyllabic verb, could be in the correct O-V-N or the incorrect V-O-N order; the object could be disyllabic or monosyllabic. We found that the reader spent more time and made more regressions on and after the compounds when either type of anomaly was detected during the first pass reading. However, during re-reading (after all the words in the sentence have been viewed), less regressive eye movements were found for the anomalous rhythmic pattern, relative to the correct pattern; moreover, only the abnormal rhythmic pattern, not the violated word order, influenced the regressive eye movements. These results suggest that while the processing of rhythmic pattern and word order information occurs rapidly during the initial reading of the sentence, the process of recovering from the rhythmic pattern anomaly may ease the reanalysis processing at the later stage of sentence integration. Thus, rhythmic pattern in Chinese can dynamically affect both local phrase analysis and global sentence integration during silent reading.

Do metrical incongruities disrupt semantic processing in modern Chinese? An event-related potential study

The present study examined the relationship between prosody and semantic processing in the written form of modern Chinese by analysing behavioural data and event-related potential data. By manipulating the number of noun syllables in verb-objection, we compare the dynamic neural mechanisms of the structure bisyllabic verb (V2)+monosyllabic noun (N1) (i.e. V2+N1) with V2+N1 (N2, bisyllabic noun). In Chinese, the rhythmic pattern V2+N1 is considered to be a metrical incongruity, whereas V2+N2 is considered to be a metrical congruity. For example, the verb yunshu (to transport) can be followed by liangshi (cereals). However, if yunshu is followed by liang (cereals), yunshu liang is usually considered to be metrically incongruous. This paper shows that (i) V2+N1 elicited more negative amplitudes than V2+N2 in the 90-170 ms and 450-500 ms windows, which indicates that metrical incongruities affect semantic processing in Chinese, and (ii) the acceptance rate for V2+N1 is significantly lower than that of V2+N2, which implies that metrical incongruities disrupt semantic processing in modern Chinese. These results are in agreement with previous studies. This is the first study to find that metrical incongruities disrupt semantic processing in Chinese. This study provides convergent evidence that metrical congruities facilitate semantic processing, whereas metrical incongruities disrupt semantic processing. Video abstract available (Supplemental digital content 1, http://links.lww.com/WNR/A340).

Rhetorical features facilitate prosodic processing while handicapping ease of semantic comprehension

Studies on rhetorical features of language have reported both enhancing and adverse effects on ease of processing. We hypothesized that two explanations may account for these inconclusive findings. First, the respective gains and losses in ease of processing may apply to different dimensions of language processing (specifically, prosodic and semantic processing) and different types of fluency (perceptual vs. conceptual) and may well allow for an integration into a more comprehensive framework. Second, the effects of rhetorical features may be sensitive to interactions with other rhetorical features; employing a feature separately or in combination with others may then predict starkly different effects. We designed a series of experiments in which we expected the same rhetorical features of the very same sentences to exert adverse effects on semantic (conceptual) fluency and enhancing effects on prosodic (perceptual) fluency. We focused on proverbs that each employ three rhetorical features: rhyme, meter, and brevitas (i.e., artful shortness). The presence of these target features decreased ease of conceptual fluency (semantic comprehension) while enhancing perceptual fluency as reflected in beauty and succinctness ratings that were mainly driven by prosodic features. The rhetorical features also predicted choices for persuasive purposes, yet only for the sentence versions featuring all three rhetorical features; the presence of only one or two rhetorical features had an adverse effect on the choices made. We suggest that the facilitating effects of a combination of rhyme, meter, and rhetorical brevitas on perceptual (prosodic) fluency overcompensated for their adverse effects on conceptual (semantic) fluency, thus resulting in a total net gain both in processing ease and in choices for persuasive purposes.

Perception and bias in the processing of compound versus phrasal stress: evidence from event-related brain potentials

Previous research using picture/word matching tasks has demonstrated a tendency to incorrectly interpret phrasally stressed strings as compounds. Using event-related potentials, we sought to determine whether this pattern stems from poor perceptual sensitivity to the compound/phrasal stress distinction, or from a post-perceptual bias in behavioral response selection. A secondary aim was to gain insight into the role played by contrastive stress patterns in online sentence comprehension. The behavioral results replicated previous findings of a preference for compounds, but the electrophysiological data suggested a robust sensitivity to both stress patterns. When incongruent with the context, both compound and phrasal stress elicited a sustained left-lateralized negativity. Moreover, incongruent compound stress elicited a centro-parietal negativity (N400), while incongruent phrasal stress elicited a late posterior positivity (P600). We conclude that the previous findings of a preference for compounds are due to response selection bias, and not a lack of perceptual sensitivity. The present results complement previous evidence for the immediate use of meter in semantic processing, as well as evidence for late interactions between prosodic and syntactic information.

Aesthetic and emotional effects of meter and rhyme in poetry

Metrical patterning and rhyme are frequently employed in poetry but also in infant-directed speech, play, rites, and festive events. Drawing on four line-stanzas from nineteenth and twentieth German poetry that feature end rhyme and regular meter, the present study tested the hypothesis that meter and rhyme have an impact on aesthetic liking, emotional involvement, and affective valence attributions. Hypotheses that postulate such effects have been advocated ever since ancient rhetoric and poetics, yet they have barely been empirically tested. More recently, in the field of cognitive poetics, these traditional assumptions have been readopted into a general cognitive framework. In the present experiment, we tested the influence of meter and rhyme as well as their interaction with lexicality in the aesthetic and emotional perception of poetry. Participants listened to stanzas that were systematically modified with regard to meter and rhyme and rated them. Both rhyme and regular meter led to enhanced aesthetic appreciation, higher intensity in processing, and more positively perceived and felt emotions, with the latter finding being mediated by lexicality. Together these findings clearly show that both features significantly contribute to the aesthetic and emotional perception of poetry and thus confirm assumptions about their impact put forward by cognitive poetics. The present results are explained within the theoretical framework of cognitive fluency, which links structural features of poetry with aesthetic and emotional appraisal.

Electroencephalogram oscillations differentiate semantic and prosodic processes during sentence reading

How prosodic information is processed at the neural level during silent sentence reading is an unsolved issue. In this study, we investigate whether and how the processing of prosodic constraints can be distinguished from the processing of semantic constraints by measuring changes in event-related electroencephalogram (EEG) power. We visually presented Chinese sentences containing verb-noun combinations that were semantically congruent or incongruent and that had normal or abnormal rhythmic patterns and asked participants to judge whether the sentences were semantically and rhythmically acceptable. In Chinese, the rhythmic pattern refers to the combination of words with different syllable lengths. While the [1+1] pattern is normal for a verb-noun combination, the [2+1] pattern is abnormal. With the critical nouns, we found that the violation of semantic constraints was associated with the low beta (16-20 Hz) decrease in the early window (0-200 ms post onset) and the alpha (10-15 Hz) and low beta decrease in the later window (400-657 ms) while the processing of the abnormal rhythmic pattern was associated with the theta (4-6 Hz) and the alpha increase in the early window and the alpha and upper beta (20-24 Hz) decrease in the later window. These findings suggest that although the processing of semantic constraints and the processing of rhythmic pattern may partially share neuro-cognitive processes, as reflected by the similar decreases in alpha band power, they can nevertheless be differentiated in EEG responses during sentence reading.