The influence of information structure on the depth of semantic processing: How focus and pitch accent determine the size of the N400 effect

Auditory Processing of Intonational Rises and Falls in German: Rises Are Special in Attention Orienting

This article investigates the processing of intonational rises and falls when presented unexpectedly in a stream of repetitive auditory stimuli. It examines the neurophysiological correlates (ERPs) of attention to these unexpected stimuli through the use of an oddball paradigm where sequences of repetitive stimuli are occasionally interspersed with a deviant stimulus, allowing for elicitation of an MMN. Whereas previous oddball studies on attention toward unexpected sounds involving pitch rises were conducted on nonlinguistic stimuli, the present study uses as stimuli lexical items in German with naturalistic intonation contours. Results indicate that rising intonation plays a special role in attention orienting at a pre-attentive processing stage, whereas contextual meaning (here a list of items) is essential for activating attentional resources at a conscious processing stage. This is reflected in the activation of distinct brain responses: Rising intonation evokes the largest MMN, whereas falling intonation elicits a less pronounced MMN followed by a P3 (reflecting a conscious processing stage). Subsequently, we also find a complex interplay between the phonological status (i.e., accent/head marking vs. boundary/edge marking) and the direction of pitch change in their contribution to attention orienting: Attention is not oriented necessarily toward a specific position in prosodic structure (head or edge). Rather, we find that the intonation contour itself and the appropriateness of the contour in the linguistic context are the primary cues to two core mechanisms of attention orienting, pre-attentive and conscious orientation respectively, whereas the phonological status of the pitch event plays only a supplementary role.

Electrophysiological alterations during action semantic processing in Parkinson's disease

Assessments of action semantics consistently reveal markers of Parkinson's disease (PD). However, neurophysiological signatures of the domain remain under-examined in this population, especially under conditions that allow patients to process stimuli without stringent time constraints. Here we assessed event-related potentials and time-frequency modulations in healthy individuals (HPs) and PD patients during a delayed-response semantic judgment task involving related and unrelated action-picture pairs. Both groups had shorter response times for related than for unrelated trials, but they exhibited discrepant electrophysiological patterns. HPs presented significantly greater N400 amplitudes as well as theta enhancement and mu desynchronization for unrelated relative to related trials. Conversely, N400 and theta modulations were abolished in the patients, who further exhibited a contralateralized cluster in the mu range. None of these patterns were associated with the participants' cognitive status. Our results suggest that PD involves multidimensional neurophysiological disruptions during action-concept processing, even under task conditions that elicit canonical behavioral effects. New constraints thus emerge for translational neurocognitive models of the disease.

Effects of Healthy Aging and Gender on the Electrophysiological Correlates of Semantic Sentence Comprehension: The Development of Dutch Normative Data

PURPOSE

The clinical use of event-related potentials in patients with language disorders is increasingly acknowledged. For this purpose, normative data should be available. Within this context, healthy aging and gender effects on the electrophysiological correlates of semantic sentence comprehension were investigated.

METHOD

One hundred and ten healthy subjects (55 men and 55 women), divided among three age groups (young, middle aged, and elderly), performed a semantic sentence congruity task in the visual modality during electroencephalographic recording.

RESULTS

The early visual complex was affected by increasing age as shown by smaller P2 amplitudes in the elderly compared to the young. Moreover, the N400 effect in the elderly was smaller than in the young and was delayed compared to latency measures in both middle-aged and young subjects. The topography of age-related amplitude changes of the N400 effect appeared to be gender specific. The late positive complex effect was increased at frontal electrode sites from middle age on, but this was not statistically significant. No gender effects were detected regarding the early P1, N1, and P2, or the late positive complex effect.

CONCLUSION

Especially aging effects were found during semantic sentence comprehension, and this from the level of perceptual processing on. Normative data are now available for clinical use.

Focus Effects on Immediate and Delayed Recognition of Referents in Samoan

This paper looks at the effect of focus-marking on the immediate and delayed recognition of referents in Samoan. Focus-marking on a word can imply the presence of alternatives to that word which are relevant to the interpretation of the utterance. Consistent with this, psycholinguistic evidence is growing that alternatives to focus-marked words are selectively activated in the immediate processing of an utterance and longer term memory. However, most of this research is on Western Germanic languages which primarily use prosodic prominence to mark focus. We explore this in two experiments using immediate and delayed probe recognition tasks in the under-studied language Samoan, which primarily uses syntactic focus-marking. Participants heard short narratives ending in a critical sentence in which the object word was either focused or not, using a cleft-like construction. In the first experiment, probe recognition, alternatives to the object word which were either mentioned or unmentioned in the narrative were responded to more slowly if the object was focus-marked. In the second experiment, delayed recognition, participants were faster to correctly recognize mentioned alternatives, and slower to reject unmentioned, if the object was focus-marked. Both results are consistent with immediate and longer-term activation of focus alternatives. There was no significant effect of focus-marking on recognition of the object word itself.

Rapid adaptation of predictive models during language comprehension: Aperiodic EEG slope, individual alpha frequency and idea density modulate individual differences in real-time model updating

Predictive coding provides a compelling, unified theory of neural information processing, including for language. However, there is insufficient understanding of how predictive models adapt to changing contextual and environmental demands and the extent to which such adaptive processes differ between individuals. Here, we used electroencephalography (EEG) to track prediction error responses during a naturalistic language processing paradigm. In Experiment 1, 45 native speakers of English listened to a series of short passages. Via a speaker manipulation, we introduced changing intra-experimental adjective order probabilities for two-adjective noun phrases embedded within the passages and investigated whether prediction error responses adapt to reflect these intra-experimental predictive contingencies. To this end, we calculated a novel measure of speaker-based, intra-experimental surprisal (“speaker-based surprisal”) as defined on a trial-by-trial basis and by clustering together adjectives with a similar meaning. N400 amplitude at the position of the critical second adjective was used as an outcome measure of prediction error. Results showed that N400 responses attuned to speaker-based surprisal over the course of the experiment, thus indicating that listeners rapidly adapt their predictive models to reflect local environmental contingencies (here: the probability of one type of adjective following another when uttered by a particular speaker). Strikingly, this occurs in spite of the wealth of prior linguistic experience that participants bring to the laboratory. Model adaptation effects were strongest for participants with a steep aperiodic (1/f) slope in resting EEG and low individual alpha frequency (IAF), with idea density (ID) showing a more complex pattern. These results were replicated in a separate sample of 40 participants in Experiment 2, which employed a highly similar design to Experiment 1. Overall, our results suggest that individuals with a steep aperiodic slope adapt their predictive models most strongly to context-specific probabilistic information. Steep aperiodic slope is thought to reflect low neural noise, which in turn may be associated with higher neural gain control and better cognitive control. Individuals with a steep aperiodic slope may thus be able to more effectively and dynamically reconfigure their prediction-related neural networks to meet current task demands. We conclude that predictive mechanisms in language are highly malleable and dynamic, reflecting both the affordances of the present environment as well as intrinsic information processing capabilities of the individual.

The Inhibition Effect of Affordances in Action Picture Naming: An ERP Study

How quickly are different kinds of conceptual knowledge activated in action picture naming? Using a masked priming paradigm, we manipulated the prime category type (artificial vs. natural), prime action type (precision, power, vs. neutral grip), and target action type (precision vs. power grip) in action picture naming, while electrophysiological signals were measured concurrently. Naming latencies showed an inhibition effect in the congruent action type condition compared with the neutral condition. ERP results showed that artificial and natural category primes induced smaller waveforms in precision or power action primes than neutral primes in the time window of 100-200 msec. Time-frequency results consistently presented a power desynchronization of the mu rhythm in the time window of 0-210 msec with precision action type artificial objects compared with neutral primes, which localized at the supplementary motor, precentral and postcentral areas in the left hemisphere. These findings suggest an inhibitory effect of affordances arising at conceptual preparation in action picture naming and provide evidence for embodied cognition.

How accentuation influences the processing of emotional words in spoken language: An ERP study

Pitch accent marks information structure in utterances in many languages but little is known about the effects of accent on the perception of emotional word meaning. The present study explored the processing of accentuation and its influence on the semantic integration of emotional words during spoken sentence comprehension. Twenty-five participants were presented with sets of spoken Chinese sentences while accentuation and emotional meaning of the adjectives were orthogonally manipulated. An implicit task required the recognition of words contained in the sentences, whereas an explicit task required judging the presence of an accented or emotional word. In the ERPs to the adjectives, accentuation induced a long-lasting anterior negativity starting around 150-250 ms and a late posterior positivity. More importantly, emotionally negative words elicited larger negativities between 300 and 700 ms as compared to neutral words but only when they were accented. Interestingly, these negativities showed a parietal N400-typical distribution when accent was implicit but strongly overlapped with the accent-induced anterior negativity when accent was task-relevant. Hence, accentuation may enable the processing of emotional meaning by directing attention towards the accented words. When accent and emotion are explicit parts of the task, similar frontal attentional networks are activated by emotion as by accent alone. In contrast, when accent and emotion are implicit to the task, emotion appears to merely activate parietal networks, typical for semantic integration effects. Together, these results suggest that accentuation plays an important role in spoken sentence comprehension, deserving further study.

Contrastive intonation effects on word recall for information-structural alternatives across the sexes

Focus highlights the fact that contextual alternatives are relevant for the interpretation of an utterance. For example, if someone says: “The meeting is on TUESDAY,” with focus marked by a pitch accent on “Tuesday,” the speaker might want to correct the assumption that the meeting is on Monday (an alternative date). Intonation as one way to signal focus was manipulated in a delayed-recall paradigm. Recall of contextual alternatives was tested in a condition where a set of alternatives was evoked by contrastive intonation. A control condition used intonation contours reported for broad focus in German. It was hypothesized that contrastive intonation improves recall, just as focus-sensitive particles (e.g., ‘only’) do, compared to sentences without such particles. Participants listened to short texts introducing a list of three elements from taxonomic categories. One of the three elements was re-mentioned in a subsequent critical sentence, realized with either a broad (H+!H*) or with a contrastive intonation contour (L+H*). Cued recall of the list elements was tested block-wise. Results show that contrastive intonation enhances recall for focus alternatives. In addition, it was found that the observed recall benefit is predominantly driven by females. The results support the assumption that contextual alternatives are better encoded in memory irrespective of whether focus is expressed prosodically or by a focus-sensitive particle. The results further show that females are more sensitive to pragmatic information conveyed through prosody than males.

Situating language in a minimal social context: how seeing a picture of the speaker's face affects language comprehension

Natural use of language involves at least two individuals. Some studies have focused on the interaction between senders in communicative situations and how the knowledge about the speaker can bias language comprehension. However, the mere effect of a face as a social context on language processing remains unknown. In the present study, we used event-related potentials to investigate the semantic and morphosyntactic processing of speech in the presence of a photographic portrait of the speaker. In Experiment 1, we show that the N400, a component related to semantic comprehension, increased its amplitude when processed within this minimal social context compared to a scrambled face control condition. Hence, the semantic neural processing of speech is sensitive to the concomitant perception of a picture of the speaker's face, even if irrelevant to the content of the sentences. Moreover, a late posterior negativity effect was found to the presentation of the speaker's face compared to control stimuli. In contrast, in Experiment 2, we found that morphosyntactic processing, as reflected in left anterior negativity and P600 effects, is not notably affected by the presence of the speaker's portrait. Overall, the present findings suggest that the mere presence of the speaker's image seems to trigger a minimal communicative context, increasing processing resources for language comprehension at the semantic level.

Cortical Tracking of Speech: Toward Collaboration between the Fields of Signal and Sentence Processing

In recent years, a growing number of studies have used cortical tracking methods to investigate auditory language processing. Although most studies that employ cortical tracking stem from the field of auditory signal processing, this approach should also be of interest to psycholinguistics-particularly the subfield of sentence processing-given its potential to provide insight into dynamic language comprehension processes. However, there has been limited collaboration between these fields, which we suggest is partly because of differences in theoretical background and methodological constraints, some mutually exclusive. In this paper, we first review the theories and methodological constraints that have historically been prioritized in each field and provide concrete examples of how some of these constraints may be reconciled. We then elaborate on how further collaboration between the two fields could be mutually beneficial. Specifically, we argue that the use of cortical tracking methods may help resolve long-standing debates in the field of sentence processing that commonly used behavioral and neural measures (e.g., ERPs) have failed to adjudicate. Similarly, signal processing researchers who use cortical tracking may be able to reduce noise in the neural data and broaden the impact of their results by controlling for linguistic features of their stimuli and by using simple comprehension tasks. Overall, we argue that a balance between the methodological constraints of the two fields will lead to an overall improved understanding of language processing as well as greater clarity on what mechanisms cortical tracking of speech reflects. Increased collaboration will help resolve debates in both fields and will lead to new and exciting avenues for research.

Concurrent use of animacy and event-knowledge during comprehension: Evidence from event-related potentials

In two ERP experiments, we investigated whether readers prioritize animacy over real-world event-knowledge during sentence comprehension. We used the paradigm of Paczynski and Kuperberg (2012), who argued that animacy is prioritized based on the observations that the 'related anomaly effect' (reduced N400s for context-related anomalous words compared to unrelated words) does not occur for animacy violations, and that animacy violations but not relatedness violations elicit P600 effects. Participants read passive sentences with plausible agents (e.g., The prescription for the mental disorder was written by the psychiatrist) or implausible agents that varied in animacy and semantic relatedness (schizophrenic/guard/pill/fence). In Experiment 1 (with a plausibility judgment task), plausible sentences elicited smaller N400s relative to all types of implausible sentences. Crucially, animate words elicited smaller N400s than inanimate words, and related words elicited smaller N400s than unrelated words, but Bayesian analysis revealed substantial evidence against an interaction between animacy and relatedness. Moreover, at the P600 time-window, we observed more positive ERPs for animate than inanimate words and for related than unrelated words at anterior regions. In Experiment 2 (without judgment task), we observed an N400 effect with animacy violations, but no other effects. Taken together, the results of our experiments fail to support a prioritized role of animacy information over real-world event-knowledge, but they support an interactive, constraint-based view on incremental semantic processing.

Gender-dependent variations in optical illusions: evidence from N400 waveforms

OBJECTIVE

The cognitive mechanisms (especially the gender-related ones) underlying optical illusion processing remain elusive. Since the N400 component of event-related potentials (ERPs) is an index of the semantic integration of information processing tasks, the present study focuses on gender-related differences in N400 waveforms elicited during the reasoning process applied to reach a valid conclusion of optical illusions engaging working memory (WM).

APPROACH

Fifty-one healthy participants (28 males, age = 34.25 years ± 10.25, years of education = 16.00 years ± 1.78; and 23 females, age = 33.43 ± 7.93, years of education = 15.56 ± 1.82) were measured. The N400 ERP component was evoked by 39 optical illusions adjusted to induce WM. We compared brain activation patterns while participants maintained conclusions of the optical illusions in WM. The N400 of ERPs was recorded during the WM phase, during which participants were required to draw a logical conclusion regarding the correctness of the optical illusions.

MAIN RESULTS

Analysis revealed that females compared to males exhibited significantly increased N400 amplitudes located at parietal and occipital sites, whereas males exhibited significantly higher N400 amplitudes located at frontal areas. Furthermore, females compared to males demonstrated significantly prolonged latencies of the N400 component located at right frontotemporal abductions.

SIGNIFICANCE

These results suggest that coupling of optical illusions with WM engages distinct gender-related variations of brain semantic processing as reflected by the N400 ERP component. Based on the dual process account, our study gives support to the notion that women tend to employ a more deliberate and slower semantic reasoning than the men who tend to employ an automatic and fast one. Topographically, within the network sub-serving the semantic operation, the posterior brain areas responsible for sensorimotor integration-related processes elicit a greater brain activation among females while the anterior brain areas responsible for control and storage/retrieval operation elicit a greater brain activation among males.

Similarity between referents constrains the processing of contrastive focus during reading

Contrastive focus implies a contrast between two elements. However, it is unclear whether and how any interplay between such a contrast and similarity between potentially contrasting elements might affect focus processing. Accordingly, we report an eye movement experiment investigating this issue. The experiment used a background story to introduce eight characters whose social identities were manipulated to be similar or dissimilar. Participants first read this background story, then a series of two-sentence discourses while their eye movements were recorded. Each discourse referred to two characters from the passage who had either similar or dissimilar identities, with one (the target character) either focused using the Chinese particle zhiyou (meaning only) or unfocused. The results showed a typical focus facilitation effect, such that target character names were processed more quickly when focused than unfocused. We also observed a main effect of the similarity/dissimilarly of characters and, crucially, an interaction between this variable and focus. This interaction was due to slower processing of a post-target region when the target character was focused and the two characters had similar rather than dissimilar identities, but no such effect when the target character was unfocused. The findings suggest that establishing a contrast between referents is effortful during reading when these have similar rather than dissimilar social identities and so are more difficult to differentiate. The distinctiveness of referents in a discourse context may therefore constrain the establishment of contrastive focus during reading. We discuss these findings in relation to current theories of focus interpretation.

Attention allocation in a language with post-focal prominences

Accentuation influences selective attention and the depth of semantic processing during online speech comprehension. We investigated the processing of semantically congruent and incongruent words in a language that presents cues to prosodic prominences in the region of the utterance occurring after the focussed information (the post-focal region). This language is Italian, in particular the variety spoken in Bari. In this variety, questions have a compressed, post-focal accent, whereas in statements there is a low-level pitch in this position. Using event-related potentials, we investigated the processing of congruent and incongruent target words with two prosodic realizations (focussed with accentuation, post-focal realization) and in two-sentence modalities (statement, question). Results indicate an N400 congruence effect that was modulated by position (focal, post-focal) and modality (statement, question): processing was deeper for questions in narrow focus than in post-focal position, while statements showed similar pronounced N400 effects across positions. The attenuated N400 difference for post-focal targets in questions was accompanied by a more enhanced late positivity when they were incongruent, indicating that attentional resources are allocated during updating of speech act information.

The neurobiology of language beyond single-word processing

In this Review, I propose a multiple-network view for the neurobiological basis of distinctly human language skills. A much more complex picture of interacting brain areas emerges than in the classical neurobiological model of language. This is because using language is more than single-word processing, and much goes on beyond the information given in the acoustic or orthographic tokens that enter primary sensory cortices. This requires the involvement of multiple networks with functionally nonoverlapping contributions.

The neurocognitive signature of focus alternatives

Focus alternatives are words/phrases that can substitute for the focused constituent of an utterance. In "Carsten has picked [CHERRIES]_F from the tree.", (marked by pitch focus on cherries), the speaker wants to not only convey the fact that Carsten has picked cherries, but also to contrast cherries with other fruit that could have been picked, such as plums. Although focus alternatives are key to understanding the implicit aspects of an utterance, nothing is known about their neural representation. We directly contrasted neural representations of lexico-semantic similarity and focus alternative status using fMRI. Semantic relatedness was reflected in decreased activation in the bilateral superior temporal gyri. By contrast, processing of focus alternatives induced increased activations in the precuneus and the fronto-median wall, two regions previously implicated in discourse processing. These results suggest that focus alternative status is processed separately from semantic relatedness, at the level of discourse integration.

Relationships between alpha oscillations during speech preparation and the listener N400 ERP to the produced speech

While previous studies separately demonstrate EEG spectral modulations during speech preparation and ERP responses to the listened speech, it is unclear whether these responses are related on a trial-by-trial basis between a speaker and listener. In order to determine whether these responses are related in real-time, Electroencephalography (EEG) responses were measured simultaneously within a speaker and listener using a 24 electrode Mobile EEG system (18 participants; 9 pairs) during a sentence completion task. Each trial consisted of a sentence prompt with an incomplete ending (e.g. “I took my dog for a ____”). The speaker was instructed to fill in the ending with something expected (e.g. “walk”) (40 trials) or unexpected (e.g. “drink”) (40 trials). The other participant listened to the speaker throughout the block. We found that lower alpha band activity was reduced when individuals prepared unexpected sentence endings compared to expected sentence endings. Greater reductions in the speaker’s lower alpha activity during response preparation were correlated with a more negative N400 response in the listener to the unexpected word. These findings demonstrate that alpha suppression and the N400 ERP effect are present within a hyperscanning context and they are correlated between the speaker and listener during sentence completion.

Is infant-directed speech interesting because it is surprising? - Linking properties of IDS to statistical learning and attention at the prosodic level

The exaggerated intonation and special rhythmic properties of infant-directed speech (IDS) have been hypothesized to attract infants' attention to the speech stream. However, there has been little work actually connecting the properties of IDS to models of attentional processing or perceptual learning. A number of such attention models suggest that surprising or novel perceptual inputs attract attention, where novelty can be operationalized as the statistical (un)predictability of the stimulus in the given context. Since prosodic patterns such as F0 contours are accessible to young infants who are also known to be adept statistical learners, the present paper investigates a hypothesis that F0 contours in IDS are less predictable than those in adult-directed speech (ADS), given previous exposure to both speaking styles, thereby potentially tapping into basic attentional mechanisms of the listeners in a similar manner that relative probabilities of other linguistic patterns are known to modulate attentional processing in infants and adults. Computational modeling analyses with naturalistic IDS and ADS speech from matched speakers and contexts show that IDS intonation has lower overall temporal predictability even when the F0 contours of both speaking styles are normalized to have equal means and variances. A closer analysis reveals that there is a tendency of IDS intonation to be less predictable at the end of short utterances, whereas ADS exhibits more stable average predictability patterns across the full extent of the utterances. The difference between IDS and ADS persists even when the proportion of IDS and ADS exposure is varied substantially, simulating different relative amounts of IDS heard in different family and cultural environments. Exposure to IDS is also found to be more efficient for predicting ADS intonation contours in new utterances than exposure to the equal amount of ADS speech. This indicates that the more variable prosodic contours of IDS also generalize to ADS, and may therefore enhance prosodic learning in infancy. Overall, the study suggests that one reason behind infant preference for IDS could be its higher information value at the prosodic level, as measured by the amount of surprisal in the F0 contours. This provides the first formal link between the properties of IDS and the models of attentional processing and statistical learning in the brain. However, this finding does not rule out the possibility that other differences between the IDS and ADS also play a role.

Making predictable unpredictable with style - Behavioral and electrophysiological evidence for the critical role of prosodic expectations in the perception of prominence in speech

Perceptual prominence of linguistic units such as words has been earlier connected to the concepts of predictability and attentional orientation. One hypothesis is that low-probability prosodic or lexical content is perceived as prominent due to the surprisal and high information value associated with the stimulus. However, the existing behavioral studies have used stimulus manipulations that follow or violate typical linguistic patterns present in the listeners' native language, i.e., assuming that the listeners have already established a model for acceptable prosodic patterns in the language. In the present study, we investigated whether prosodic expectations and the resulting subjective impression of prominence is affected by brief statistical adaptation to suprasegmental acoustic features in speech, also in the case where the prosodic patterns do not necessarily follow language-typical marking for prominence. We first exposed listeners to five minutes of speech with uneven distributions of falling and rising fundamental frequency (F0) trajectories on sentence-final words, and then tested their judgments of prominence on a set of new utterances. The results show that the probability of the F0 trajectory affects the perception of prominence, a less frequent F0 trajectory making a word more prominent independently of the absolute direction of F0 change. In the second part of the study, we conducted EEG-measurements on a set of new subjects listening to similar utterances with predominantly rising or falling F0 on sentence-final words. Analysis of the resulting event-related potentials (ERP) reveals a significant difference in N200 and N400 ERP-component amplitudes between standard and deviant prosody, again independently of the F0 direction and the underlying lexical content. Since N400 has earlier been associated with semantic processing of stimuli, this suggests that listeners implicitly track probabilities at the suprasegmental level and that predictability of a prosodic pattern during a word has an impact to the semantic processing of the word. Overall, the study suggests that prosodic markers for prominence are at least partially driven by the statistical structure of recently perceived speech, and therefore prominence perception could be based on statistical learning mechanisms similar to those observed in early word learning, but in this case operating at the level of suprasegmental acoustic features.

Planning versus comprehension in turn-taking: Fast responders show reduced anticipatory processing of the question

Rapid response latencies in conversation suggest that responders start planning before the ongoing turn is finished. Indeed, an earlier EEG study suggests that listeners start planning their responses to questions as soon as they can (Bögels et al., 2015a). The present study aimed to (1) replicate this early planning effect and (2) investigate whether such early response planning incurs a cost on participants' concurrent comprehension of the ongoing turn. During the experiment participants answered questions from a confederate partner. To address aim (1), the questions were designed such that response planning could start either early or late in the turn. Our results largely replicate Bögels et al. (2015a), showing a large positive ERP effect and an oscillatory alpha/beta reduction right after participants could have first started planning their verbal response, again suggesting an early start of response planning. To address aim (2), the confederate's questions also contained either an expected word or an unexpected one to elicit a differential N400 effect, either before or after the start of response planning. We hypothesized an attenuated N400 effect after response planning had started. In contrast, the N400 effects before and after planning did not differ. There was, however, a positive correlation between participants' response time and their N400 effect size after planning had started; quick responders showed a smaller N400 effect, suggesting reduced attention to comprehension and possibly reduced anticipatory processing. We conclude that early response planning can indeed impact comprehension processing.

The Interplay between Topic Shift and Focus in the Dynamic Construction of Discourse Representations

Previous studies have suggested that focusing an element can enhance the activation of the focused element and bring about a number of processing benefits. However, whether and how this local prominence of information interacts with global discourse organization remains unclear. In the present study, we addressed this issue in two experiments. Readers were presented with four-sentence discourses. The first sentence of each discourse contained a critical word that was either focused or unfocused in relation to a wh-question preceding the discourse. The second sentence either maintained or shifted the topic of the first sentence. Participants were told to read for comprehension and for a probe recognition task in which the memory of the critical words was tested. In Experiment 1, when the probe words were tested immediately after the point of topic shift, we found shorter response times for the focused critical words than the unfocused ones regardless of topic manipulation. However, in Experiment 2, when the probe words were tested two sentences away from the point of topic shift, we found the facilitation effect of focus only in the topic-maintained discourses, but not in the topic-shifted discourses. This suggests that the facilitation effect of focus was not immediately suppressed at the point of topic shifting, but when additional information was added to the new topic. Our findings provide evidence for the dynamic interplay between global topic structure and local salience of information and have important implications on how activation of information fluctuates in mental representation.

Beat that Word: How Listeners Integrate Beat Gesture and Focus in Multimodal Speech Discourse

Communication is facilitated when listeners allocate their attention to important information (focus) in the message, a process called “information structure.” Linguistic cues like the preceding context and pitch accent help listeners to identify focused information. In multimodal communication, relevant information can be emphasized by nonverbal cues like beat gestures, which represent rhythmic nonmeaningful hand movements. Recent studies have found that linguistic and nonverbal attention cues are integrated independently in single sentences. However, it is possible that these two cues interact when information is embedded in context, because context allows listeners to predict what information is important. In an ERP study, we tested this hypothesis and asked listeners to view videos capturing a dialogue. In the critical sentence, focused and nonfocused words were accompanied by beat gestures, grooming hand movements, or no gestures. ERP results showed that focused words are processed more attentively than nonfocused words as reflected in an N1 and P300 component. Hand movements also captured attention and elicited a P300 component. Importantly, beat gesture and focus interacted in a late time window of 600–900 msec relative to target word onset, giving rise to a late positivity when nonfocused words were accompanied by beat gestures. Our results show that listeners integrate beat gesture with the focus of the message and that integration costs arise when beat gesture falls on nonfocused information. This suggests that beat gestures fulfill a unique focusing function in multimodal discourse processing and that they have to be integrated with the information structure of the message.

"Who" should be focused? The influence of focus status on pronoun resolution

Focus is assumed to be able to enhance the salience of a focused constituent and thereby facilitate the interpretation of a pronoun that refers to a focused antecedent relative to an unfocused antecedent. To assess how discourse-based focus structure influences the interpretation of a pronoun and whether this process is modulated by the grammatical role of the antecedent, we conducted an ERP study in which the focus status of a pronoun's potential antecedents was manipulated by means of a wh-question-answer structure. We found that, relative to those in the focused position, pronouns referring to antecedents in the unfocused position evoked enhanced positive responses in both early (180-230 ms) and late time windows (400-800 ms). Moreover, while a larger positivity was evoked by object-referring pronouns compared to subject-referring pronouns in the 400-800 ms time window over the right hemisphere, there was no effect of grammatical role in the 180-230 ms time window. These findings indicate that, while the initial stage of pronoun resolution is modulated by focus information assigned via a wh-question structure, integration of the pronoun and its antecedent into a coherent discourse representation at the later stage could be constrained by various factors, including the focus status and possibly the grammatical role of the antecedent.

The "exaptation" of linguistic implicit strategies

Implicit strategies are known to increase persuasion performances. Implicits of content (vagueness, implicatures) and implicits of responsibility (presuppositions, topics) will be compared semiotically to non-linguistic implicits such as images and sounds. The results of psycholinguistic and neurolinguistic experiments will be used to propose that presuppositions and topics arose in language as means to spare addressees processing effort on already known contents, but they were subsequently “exapted” to spare effort on unknown marginal contents, and eventually to reduce the probability for doubtful contents to be processed thoroughly and rejected. This will be shown by many examples from commercial advertising and political propaganda.

How Distance Affects Semantic Integration in Discourse: Evidence from Event-Related Potentials

Event-related potentials were used to investigate whether semantic integration in discourse is influenced by the number of intervening sentences between the endpoints of integration. Readers read discourses in which the last sentence contained a critical word that was either congruent or incongruent with the information introduced in the first sentence. Furthermore, for the short discourses, the first and last sentence were intervened by only one sentence while for the long discourses, they were intervened by three sentences. We found that the incongruent words elicited an N400 effect for both the short and long discourses. However, a P600 effect was only observed for the long discourses, but not for the short ones. These results suggest that although readers can successfully integrate upcoming words into the existing discourse representation, the effort required for this integration process is modulated by the number of intervening sentences. Thus, discourse distance as measured by the number of intervening sentences should be taken as an important factor for semantic integration in discourse.

Sex Differences in Neurophysiological Activation Patterns During Phonological Input Processing: An Influencing Factor for Normative Data

In the context of neurophysiological normative data, it has been established that aging has a significant impact on neurophysiological correlates of auditory phonological input processes, such as phoneme discrimination (PD) and word recognition (WR). Besides age, sex is another demographic factor that influences several language processes. We aimed to disentangle whether sex has a similar effect on PD and WR. Event-related potentials (ERPs) were recorded in 20 men and 24 women. During PD, three phonemic contrasts (place and manner of articulation and voicing) were compared using the attentive P300 and pre-attentive Mismatch Negativity. To investigate WR, real words were contrasted with pseudowords in a pre-attentive oddball task. Women demonstrated a larger sensitivity to spectrotemporal differences, as evidenced by larger P300 responses to the place of articulation (PoA) contrast and larger P300 and MMN responses than men in PoA-based PD. Men did not display such sensitivity. Attention played an important role, considering that women needed more attentional resources to differentiate between PoA and the other phonemic contrasts. During WR, pseudowords evoked larger amplitudes already 100 ms post-stimulus independent of sex. However, women had decreased P200 latencies, but longer N400 latencies in response to pseudowords, whereas men showed increased N400 latencies compared to women in response to real words. The current results demonstrate significant sex-related influences on phonological input processes. Therefore, existing neurophysiological normative data for age should be complemented for the factor sex.

How information structure influences the processing of rhythmic irregularities: ERP evidence from German phrases

This study explores the influence of focus and givenness on the cognitive processing of rhythmic irregularities occurring in natural speech. Previous ERP studies showed that even subtle rhythmic deviations are detected by the brain if attention is directed towards the rhythmic structure. By using question-answer pairs, it was investigated whether subtle rhythmic irregularities in form of stress clashes (two adjacent stressed syllables) and stress lapses (two adjacent unstressed syllables) are still perceived when presented in post-focus position in an answer sentence and attention is directed away from them, towards the meaning of the element in narrow focus position by the preceding wh-question. Moreover, by visually presenting the lexical-semantic input of the deviating structure in the question, the influence of rhythmical and lexical properties in these two forms of rhythmic deviations are disentangled. While words in the present stress clash condition do not deviate from lexical stress, stress lapses contain deviations from metrical and lexical stress. The data reveal an early negativity effect for stress clashes but not for stress lapses, supporting the assumption that they are processed differently. The absence of a negative component for stress lapses indicates that the metrical deviation alone is not salient enough to be registered in non-focus position. Moreover, the lack of a late positive component suggests that subtle rhythmic deviations are less perceivable and hence more acceptable when presented in non-focus position. Thus, these results show that attentional shift induced by information structure influences the degree of the processing of rhythm.

When correction turns positive: processing corrective prosody in Dutch

Current research on spoken language does not provide a consistent picture as to whether prosody, the melody and rhythm of speech, conveys a specific meaning. Perception studies show that English listeners assign meaning to prosodic patterns, and, for instance, associate some accents with contrast, whereas Dutch listeners behave more controversially. In two ERP studies we tested how Dutch listeners process words carrying two types of accents, which either provided new information (new information accents) or corrected information (corrective accents), both in single sentences (experiment 1) and after corrective and new information questions (experiment 2). In both experiments corrective accents elicited a sustained positivity as compared to new information accents, which started earlier in context than in single sentences. The positivity was not modulated by the nature of the preceding question, suggesting that the underlying neural mechanism likely reflects the construction of an interpretation to the accented word, either by identifying an alternative in context or by inferring it when no context is present. Our experimental results provide strong evidence for inferential processes related to prosodic contours in Dutch.

Seriality of semantic and phonological processes during overt speech in Mandarin as revealed by event-related brain potentials

How is information transmitted across semantic and phonological levels in spoken word production? Recent evidence from speakers of Western languages such as English and Dutch suggests non-discrete transmission, but it is not clear whether this view can be generalized to other languages such as Mandarin, given potential differences in phonological encoding across languages. The present study used Mandarin speakers and combined a behavioral picture-word interference task with event-related potentials. The design factorially crossed semantic and phonological relatedness. Results showed semantic and phonological effects both in behavioral and electrophysiological measurements, with statistical additivity in latencies, and discrete time signatures (250-450 ms and 450-600 ms after picture onset for the semantic and phonological condition, respectively). Overall, results suggest that in Mandarin spoken production, information is transmitted from semantic to phonological levels in a sequential fashion. Hence, temporal signatures associated with spoken word production might differ depending on target language.

The effect of discourse structure on depth of semantic integration in reading

A coherent discourse exhibits certain structures in that subunits of discourses are related to one another in various ways and in that subunits that contribute to the same discourse purpose are joined to create a larger unit so as to produce an effect on the reader. To date, this crucial aspect of discourse has been largely neglected in the psycholinguistic literature. In two experiments, we examined whether semantic integration in discourse context was influenced by the difference of discourse structure. Readers read discourses in which the last sentence was locally congruent but either semantically congruent or incongruent when interpreted with the preceding sentence. Furthermore, the last sentence was either in the same discourse unit or not in the same discourse unit as the preceding sentence, depending on whether they shared the same discourse purpose. Results from self-paced reading (Experiment 1) and eye tracking (Experiment 2) showed that discourse-incongruous words were read longer than discourse-congruous words only when the critical sentence and the preceding sentence were in the same discourse unit, but not when they belonged to different discourse units. These results establish discourse structure as a new factor in semantic integration and suggest that discourse effects depend both on the content of what is being said and on the way that the contents are organized.

Conversation electrified: ERP correlates of speech act recognition in underspecified utterances

The ability to recognize speech acts (verbal actions) in conversation is critical for everyday interaction. However, utterances are often underspecified for the speech act they perform, requiring listeners to rely on the context to recognize the action. The goal of this study was to investigate the time-course of auditory speech act recognition in action-underspecified utterances and explore how sequential context (the prior action) impacts this process. We hypothesized that speech acts are recognized early in the utterance to allow for quick transitions between turns in conversation. Event-related potentials (ERPs) were recorded while participants listened to spoken dialogues and performed an action categorization task. The dialogues contained target utterances that each of which could deliver three distinct speech acts depending on the prior turn. The targets were identical across conditions, but differed in the type of speech act performed and how it fit into the larger action sequence. The ERP results show an early effect of action type, reflected by frontal positivities as early as 200 ms after target utterance onset. This indicates that speech act recognition begins early in the turn when the utterance has only been partially processed. Providing further support for early speech act recognition, actions in highly constraining contexts did not elicit an ERP effect to the utterance-final word. We take this to show that listeners can recognize the action before the final word through predictions at the speech act level. However, additional processing based on the complete utterance is required in more complex actions, as reflected by a posterior negativity at the final word when the speech act is in a less constraining context and a new action sequence is initiated. These findings demonstrate that sentence comprehension in conversational contexts crucially involves recognition of verbal action which begins as soon as it can.

The processing difference between person names and common nouns in sentence contexts: an ERP study

Person names and common nouns differ in how they are stored in the mental lexicon. Using event-related potentials, this study compared the integration of names and nouns into sentence contexts. Both person names and common nouns were highly related in meaning and either congruent or incongruent within the previous contexts. Name incongruence elicited an N400 effect, suggesting that people were able to rapidly retrieve the semantic meaning of names from long-term memory even when this process was mediated by person identification. Conversely, participants showed a "good enough" processing of the nouns due to their low specificity level and, thus, rich semantic associations, leading to a P600 effect. These distinctive ERP effects provide clear evidence for the distinctive semantic representations of these word categories by showing that the activation of a name's meaning is mediated by a single connection between identity-specific information and person identity, whereas multiple connections exist between nouns and their meanings.

A review on the cognitive function of information structure during language comprehension

During communication, information structure can be used to highlight the most relevant piece of information, so that sufficient amount of attention can be allocated to the most important information. This paper aims to review the cognitive function of information structure during language comprehension from a neurocognitve perspective. First, we gave a brief introduction to the concept of information structure that has been studied mostly in linguistic field. Then we introduced recent studies on information structure using electrophysiological and neuroimaging techniques. After that, we discussed the relationship between attention and language processing more generally. Finally, we discussed potential directions for future studies.

Topic structure affects semantic integration: evidence from event-related potentials

This study investigated whether semantic integration in discourse context could be influenced by topic structure using event-related brain potentials. Participants read discourses in which the last sentence contained a critical word that was either congruent or incongruent with the topic established in the first sentence. The intervening sentences between the first and the last sentence of the discourse either maintained or shifted the original topic. Results showed that incongruent words in topic-maintained discourses elicited an N400 effect that was broadly distributed over the scalp while those in topic-shifted discourses elicited an N400 effect that was lateralized to the right hemisphere and localized over central and posterior areas. Moreover, a late positivity effect was only elicited by incongruent words in topic-shifted discourses, but not in topic-maintained discourses. This suggests an important role for discourse structure in semantic integration, such that compared with topic-maintained discourses, the complexity of discourse structure in topic-shifted condition reduces the initial stage of semantic integration and enhances the later stage in which a mental representation is updated.

ERP evidence on the interaction between information structure and emotional salience of words

Both emotional words and words focused by information structure can capture attention. This study examined the interplay between emotional salience and information structure in modulating attentional resources in the service of integrating emotional words into sentence context. Event-related potentials (ERPs) to affectively negative, neutral, and positive words, which were either focused or nonfocused in question-answer pairs, were evaluated during sentence comprehension. The results revealed an early negative effect (90-200 ms), a P2 effect, as well as an effect in the N400 time window, for both emotional salience and information structure. Moreover, an interaction between emotional salience and information structure occurred within the N400 time window over right posterior electrodes, showing that information structure influences the semantic integration only for neutral words, but not for emotional words. This might reflect the fact that the linguistic salience of emotional words can override the effect of information structure on the integration of words into context. The interaction provides evidence for attention-emotion interactions at a later stage of processing. In addition, the absence of interaction in the early time window suggests that the processing of emotional information is highly automatic and independent of context. The results suggest independent attention capture systems of emotional salience and information structure at the early stage but an interaction between them at a later stage, during the semantic integration of words.

MUC (Memory, Unification, Control) and beyond

A neurobiological model of language is discussed that overcomes the shortcomings of the classical Wernicke-Lichtheim-Geschwind model. It is based on a subdivision of language processing into three components: Memory, Unification, and Control. The functional components as well as the neurobiological underpinnings of the model are discussed. In addition, the need for extension of the model beyond the classical core regions for language is shown. The attention network and the network for inferential processing are crucial to realize language comprehension beyond single word processing and beyond decoding propositional content. It is shown that this requires the dynamic interaction between multiple brain regions.

Comprehension of marked pronouns in Spanish and English: object anaphors cross-linguistically

Previous research on pronoun resolution has identified several individual factors that are deemed to be important for resolving reference. In this paper, we argue that of these factors, as tested here, plausibility is the most important, but interacts with form markedness and structural parallelism. We investigated how listeners resolved object pronouns that were ambiguous in the sense of having more than one possible antecedent. We manipulated the form of the anaphoric expression in terms of accentuation (English: Experiments 1a and 2a) and morphology (Spanish: Experiments 1b and 2b). We looked at sentences where both antecedents were equally plausible, or where only one of the antecedents was plausible. Listeners generally resolved toward the (parallel) grammatical object of the previous clause. When the pronouns were marked due to accentuation (English) or use of specific morphology (Spanish), preference switched to the alternative antecedent, the grammatical subject of the previous clause. In contrast, when one of the two antecedents was a much more plausible antecedent than the other, antecedent choice was almost wholly dictated by plausibility, although reference form prominence did significantly attenuate the strength of the preference.

Less Is Not More: Neural Responses to Missing and Superfluous Accents in Context

Prosody, particularly accent, aids comprehension by drawing attention to important elements such as the information that answers a question. A study using ERP registration investigated how the brain deals with the interpretation of prosodic prominence. Sentences were embedded in short dialogues and contained accented elements that were congruous or incongruous with respect to a preceding question. In contrast to previous studies, no explicit prosodic judgment task was added. Robust effects of accentuation were evident in the form of an “accent positivity” (200–500 msec) for accented elements irrespective of their congruity. Our results show that incongruously accented elements, that is, superfluous accents, activate a specific set of neural systems that is inactive in case of incongruously unaccented elements, that is, missing accents. Superfluous accents triggered an early positivity around 100 msec poststimulus, followed by a right-lateralized negative effect (N400). This response suggests that redundant information is identified immediately and leads to the activation of a neural system that is associated with semantic processing (N400). No such effects were found when contextually expected accents were missing. In a later time window, both missing and superfluous accents triggered a late positivity on midline electrodes, presumably related to making sense of both kinds of mismatching stimuli. These results challenge previous findings of greater processing for missing accents and suggest that the natural processing of prosody involves a set of distinct, temporally organized neural systems.

Information structure influences depth of syntactic processing: event-related potential evidence for the Chomsky illusion

Information structure facilitates communication between interlocutors by highlighting relevant information. It has previously been shown that information structure modulates the depth of semantic processing. Here we used event-related potentials to investigate whether information structure can modulate the depth of syntactic processing. In question-answer pairs, subtle (number agreement) or salient (phrase structure) syntactic violations were placed either in focus or out of focus through information structure marking. P600 effects to these violations reflect the depth of syntactic processing. For subtle violations, a P600 effect was observed in the focus condition, but not in the non-focus condition. For salient violations, comparable P600 effects were found in both conditions. These results indicate that information structure can modulate the depth of syntactic processing, but that this effect depends on the salience of the information. When subtle violations are not in focus, they are processed less elaborately. We label this phenomenon the Chomsky illusion.

The interface between language and attention: prosodic focus marking recruits a general attention network in spoken language comprehension

In spoken language, pitch accent can mark certain information as focus, whereby more attentional resources are allocated to the focused information. Using functional magnetic resonance imaging, this study examined whether pitch accent, used for marking focus, recruited general attention networks during sentence comprehension. In a language task, we independently manipulated the prosody and semantic/pragmatic congruence of sentences. We found that semantic/pragmatic processing affected bilateral inferior and middle frontal gyrus. The prosody manipulation showed bilateral involvement of the superior/inferior parietal cortex, superior and middle temporal cortex, as well as inferior, middle, and posterior parts of the frontal cortex. We compared these regions with attention networks localized in an auditory spatial attention task. Both tasks activated bilateral superior/inferior parietal cortex, superior temporal cortex, and left precentral cortex. Furthermore, an interaction between prosody and congruence was observed in bilateral inferior parietal regions: for incongruent sentences, but not for congruent ones, there was a larger activation if the incongruent word carried a pitch accent, than if it did not. The common activations between the language task and the spatial attention task demonstrate that pitch accent activates a domain general attention network, which is sensitive to semantic/pragmatic aspects of language. Therefore, attention and language comprehension are highly interactive.