Neuroimaging studies of language production and comprehension

The 1990s were dubbed the "Decade of the Brain." During this time there was a marked increase in the amount of neuroimaging work observing how the brain accomplishes many tasks, including the processing of language. In this chapter we review the past 15 years of neuroimaging research on language production and comprehension. The findings of these studies indicate that the processing involved in language use occurs in diffuse brain regions. These regions include Broca's and Wernicke's areas, primary auditory and visual cortex, and frontal regions in the left hemisphere, as well as in the right hemisphere homologues to these regions. We conclude the chapter by discussing the future of neuroimaging research into language production and comprehension.

Comprehending expository texts: the dynamic neurobiological correlates of building a coherent text representation

Little is known about the neural correlates of expository text comprehension. In this study, we sought to identify neural networks underlying expository text comprehension, how those networks change over the course of comprehension, and whether information central to the overall meaning of the text is functionally distinct from peripheral information. Seventeen adult subjects read expository passages while being scanned using functional magnetic resonance imaging (fMRI). By convolving phrase onsets with the hemodynamic response function (HRF), we were able to identify regions that increase and decrease in activation over the course of passage comprehension. We found that expository text comprehension relies on the co-activation of the semantic control network and regions in the posterior midline previously associated with mental model updating and integration [posterior cingulate cortex (PCC) and precuneus (PCU)]. When compared to single word comprehension, left PCC and left Angular Gyrus (AG) were activated only for discourse-level comprehension. Over the course of comprehension, reliance on the same regions in the semantic control network increased, while a parietal region associated with attention [intraparietal sulcus (IPS)] decreased. These results parallel previous findings in narrative comprehension that the initial stages of mental model building require greater visuospatial attention processes, while maintenance of the model increasingly relies on semantic integration regions. Additionally, we used an event-related analysis to examine phrases central to the text's overall meaning vs. peripheral phrases. It was found that central ideas are functionally distinct from peripheral ideas, showing greater activation in the PCC and PCU, while over the course of passage comprehension, central and peripheral ideas increasingly recruit different parts of the semantic control network. The finding that central information elicits greater response in mental model updating regions than peripheral ideas supports previous behavioral models on the cognitive importance of distinguishing textual centrality.

Discourse prominence effects on 2.5-year-old children's interpretation of pronouns

Three experiments examined 2.5-year-olds' sensitivity to discourse structure in pronoun interpretation. Children heard simple two-character stories illustrated by pictures on two video screens. In Experiments 1 and 2, one character in each story was established as more prominent than the other in several context sentences because it was mentioned first, appeared in subject position, was mentioned more often, and was pronominalized once. In Experiment 3, one character was singled out as more prominent only by being mentioned first and placed in subject position. In all three experiments, after hearing a pronoun subject in the final (test) sentence of each story, children looked longer at the character established as more prominent in the preceding sentences. These experiments show that 2.5-year-olds, like older children and adults, interpret pronouns relative to a discourse representation in which referents are ranked in prominence, and that the prominence of discourse referents is influenced by some of the same factors that guide pronoun interpretation in adulthood.

Searching for Judy: how small mysteries affect narrative processes and memory

Current theories of text processing say little about how authors' narrative choices, including the introduction of small mysteries, can affect readers' narrative experiences. Gerrig, Love, and McKoon (2009) provided evidence that 1 type of small mystery-a character introduced without information linking him or her to the story-affects readers' moment-by-moment processing. For that project, participants read stories that introduced characters by proper name alone (e.g., "Judy") or with information connecting the character to the rest of the story (e.g., "our principal Judy"). In an online recognition probe task, responses to the character's name 3 lines after his or her introduction were faster when the character had not been introduced with connecting information, suggesting that the character remained accessible awaiting resolution. In the 4 experiments in this article, we extend our theoretical analysis of small mysteries. In Experiments 1 and 2, we found evidence that trait information (e.g., "daredevil Judy") is not sufficient to connect a character to a text. In Experiments 3 and 4, we found evidence that the moment-by-moment processing effects of such small mysteries also affect readers' memory for the stories. We interpret the results in terms of Kintsch's (1988) construction-integration model of discourse processing.

Effects of Psychological Attention on Pronoun Comprehension

Pronoun comprehension is facilitated for referents that are focused in the discourse context. Discourse focus has been described as a function of attention, especially shared attention, but few studies have explicitly tested this idea. Two experiments used an exogenous capture cue paradigm to demonstrate that listeners' visual attention at the onset of a story influences their preferences during pronoun resolution later in the story. In both experiments trial-initial attention modulated listeners' transitory biases while considering referents for the pronoun, whether it was in response to the capture cue or not. These biases even had a small influence on listeners' final interpretation of the pronoun. These results provide independently-motivated evidence that the listener's attention influences the on-line processes of pronoun comprehension. Trial-initial attentional shifts were made on the basis of non-shared, private information, demonstrating that attentional effects on pronoun comprehension are not restricted to shared attention among interlocutors.

Lexical and Structural Cues to Discourse Processing in First and Second Language

Discourse connectives are lexical items like "but" and "so" that are well-known to influence the online processing of the discourse relations they convey. Yet, discourse relations like causality or contrast can also be signaled by other means than connectives, such as syntactic structures. So far, the influence of these alternative signals for discourse processing has been comparatively under-researched. In particular, their processing in a second language remains entirely unexplored. In a series of three self-paced reading experiments, we compare the reading patterns of contrastive relations by native French-speakers and non-native speakers of French with English as a first language. We focus on the effect of syntactic parallelism and how it interacts with different types of connectives. We test whether native and non-native readers equally recruit parallelism to process contrast in combination with or without a connective (Experiment 1), with a frequent vs. infrequent connective (Experiment 2) and with an ambiguous vs. unambiguous connective (Experiment 3), thus varying the explicitness and ease of retrieval of the contrast relation. Our results indicate that parallelism plays an important role for both groups of readers, but that it is a more prominent cue for non-native speakers, while its effect is modulated by task difficulty for native participants.

Spatial Inferences in Narrative Comprehension: the Role of Verbal and Spatial Working Memory

During the comprehension of narrative texts, readers keep a mental representation of the location of protagonists and objects; a breach in spatial coherence is detected by longer online reading times (consistency effect). We addressed whether these spatial inferences involve verbal or spatial working memory in two experiments, combining the consistency paradigm with selective verbal and spatial working memory concurrent tasks. The first experiment found longer reading times with a concurrent spatial task under imagery instructions (t33 = 2.87, p = .021). The second experiment, under comprehension reading instructions, found effects of verbal interference on reading times and accuracy. With a verbal secondary task, reading times for the target sentence were shorter (t45 = 3.60, p = .004) and the error rate was significantly higher (t47 = 2.95, p = .005) than without interference. This pattern of results suggests that spatial inferences in narrative comprehension rely mainly on verbal resources, and spatial working memory resources are recruited when imagery is required.

Impact of Respectfulness on Semantic Integration During Discourse Processing

Linguistic expressions of respectful terms are shaped by social status. Previous studies have shown respectful term usage affects online language processing. This study investigates its impact on semantic integration through three self-pace reading experiments, manipulating Respect Consistency (Respect vs. Disrespect) and Semantic Consistency (Semantic Consistent vs. Semantic Inconsistent). In Experiment 1, disrespect was manipulated by using the plain form of pronouns instead of the respectful form when addressing individuals of higher social status. The results showed longer reading times for semantically inconsistent sentences compared to consistent ones, reflecting the classic semantic integration effect. Nevertheless, this effect was only detected when respectful pronouns were employed. For Experiments 2 and 3, disrespect was operationalized by directly addressing individuals of higher social status by their personal names. A comparable interaction to that in Experiment 1 was identified solely in Experiment 3, which involved an appropriateness judgment task. In contrast, no such interaction was observed in Experiment 2, which involved a reading comprehension task. These results indicated that both disrespectful pronouns and addressing individuals by their personal names hinder semantic integration, but through different mechanisms. These findings provide important insights into the role of respectful term usage on semantic integration during discourse comprehension.

Electrophysiological indices of brain activity to content and function words in discourse

BACKGROUND

An increase in positivity of event-related potentials (ERPs) at the lateral anterior sites has been hypothesized to be an index of semantic and discourse processing, with the right lateral anterior positivity (LAP) showing particular sensitivity to discourse factors. However, the research investigating the LAP is limited; it is unclear whether the effect is driven by word class (function word versus content word) or by a more general process of structure building triggered by elements of a determiner phrase (DP).

AIMS

To examine the neurophysiological indices of semantic/discourse integration using two different word categories (function versus content word) in the discourse contexts and to contrast processing of these word categories in meaningful versus nonsense contexts.

METHODS & PROCEDURES

Planned comparisons of ERPs time locked to a function word stimulus 'the' and a content word stimulus 'cats' in sentence-initial position were conducted in both discourse and nonsense contexts to examine the time course of processing following these word forms.

OUTCOMES & RESULTS

A repeated-measures analysis of variance (ANOVA) for the Discourse context revealed a significant interaction of condition and site due to greater positivity for 'the' relative to 'cats' at anterior and superior sites. In the Nonsense context, there was a significant interaction of condition, time and site due to greater positivity for 'the' relative to 'cats' at anterior sites from 150 to 350 ms post-stimulus offset and at superior sites from 150 to 200 ms post-stimulus offset. Overall, greater positivity for both 'the' and 'cats' was observed in the discourse relative to the nonsense context beginning approximately 150 ms post-stimulus offset. Additionally, topographical analyses were highly correlated for the two word categories when processing meaningful discourse. This topographical pattern could be characterized as a prominent right LAP. The LAP was attenuated when the target stimulus word initiated a nonsense context.

CONCLUSIONS & IMPLICATIONS

The results of this study support the view that the right LAP is an index of general discourse processing rather than an index of word class. These findings demonstrate that the LAP can be used to study discourse processing in populations with compromised metalinguistic skills, such as adults with aphasia or traumatic brain injury.

Focus Effects on Immediate and Delayed Recognition of Referents in Samoan

This paper looks at the effect of focus-marking on the immediate and delayed recognition of referents in Samoan. Focus-marking on a word can imply the presence of alternatives to that word which are relevant to the interpretation of the utterance. Consistent with this, psycholinguistic evidence is growing that alternatives to focus-marked words are selectively activated in the immediate processing of an utterance and longer term memory. However, most of this research is on Western Germanic languages which primarily use prosodic prominence to mark focus. We explore this in two experiments using immediate and delayed probe recognition tasks in the under-studied language Samoan, which primarily uses syntactic focus-marking. Participants heard short narratives ending in a critical sentence in which the object word was either focused or not, using a cleft-like construction. In the first experiment, probe recognition, alternatives to the object word which were either mentioned or unmentioned in the narrative were responded to more slowly if the object was focus-marked. In the second experiment, delayed recognition, participants were faster to correctly recognize mentioned alternatives, and slower to reject unmentioned, if the object was focus-marked. Both results are consistent with immediate and longer-term activation of focus alternatives. There was no significant effect of focus-marking on recognition of the object word itself.

How Robust Is Discourse Processing for Native Readers? The Role of Connectives and the Coherence Relations They Convey

While corpus studies have shown that discourse connectives that convey the same coherence relation can display subtle differences, research on online discourse processing has only focused on a rather limited set of connectives. Yet, different connectives - for example, rare or polyfunctional ones - might elicit different reading patterns. In order to explore this assumption, we test the robustness of discourse processing for French native speakers by measuring the way they process causal and concessive sentences that are conveyed by either an appropriate or inappropriate connective. Throughout three experiments, we change important characteristics of the connectives: we first test frequently used connectives (Experiment 1), secondly less frequent ones (Experiment 2), and finally less frequent connectives that are polyfunctional and for which different functions clearly compete (Experiment 3). Our results show that the processing for incoherent items was affected for all connectives, however readers showed altered reading fluency when infrequent connectives were used. We conclude that discourse processing is quite robust and that readers are able to insert meaning conveyed by rare connectives while still showing the highest reading ease with frequent connectives.

Extended Perspective Shift and Discourse Economy in Language Processing

Research spanning linguistics, psychology, and philosophy suggests that speakers and hearers are finely attuned to perspectives and viewpoints that are not their own, even though perspectival information is not encoded directly in the morphosyntax of languages like English. While some terms seem to require a perspective or a judge for interpretation (e.g., epithets, evaluative adjectives, locational PPs, etc.), perspective may also be determined on the basis of subtle information spanning multiple sentences, especially in vivid styles of narrative reporting. In this paper, I develop an account of the cues that are involved in evaluating and maintaining non-speaker perspectives, and present an economy-based discourse processing model of perspective that embodies two core principles. First, perspectives are subject to a “speaker-default,” but may shift to a non-speaker perspective if sufficient contextual cues are provided. Second, the processor follows the path of least resistance to maintaining perspective, opting to maintain the current perspective across sentences as long as the shifted perspective continues to be coherent. The predictions of the model are tested in a series of offline and online studies, manipulating the form of an attitude report and the tense of the sentence that follows. Implications for processing perspective and viewpoint in speech and narrative forms are explored.

Limits to the Rational Production of Discourse Connectives

Rational accounts of language use such as the uniform information density hypothesis, which asserts that speakers distribute information uniformly across their utterances, and the rational speech act (RSA) model, which suggests that speakers optimize the formulation of their message by reasoning about what the comprehender would understand, have been hypothesized to account for a wide range of language use phenomena. We here specifically focus on the production of discourse connectives. While there is some prior work indicating that discourse connective production may be governed by RSA, that work uses a strongly gamified experimental setting. In this study, we aim to explore whether speakers reason about the interpretation of their conversational partner also in more realistic settings. We thereby systematically vary the task setup to tease apart effects of task instructions and effects of the speaker explicitly seeing the interpretation alternatives for the listener. Our results show that the RSA-predicted effect of connective choice based on reasoning about the listener is only found in the original setting where explicit interpretation alternatives of the listener are available for the speaker. The effect disappears when the speaker has to reason about listener interpretations. We furthermore find that rational effects are amplified by the gamified task setting, indicating that meta-reasoning about the specific task may play an important role and potentially limit the generalizability of the found effects to more naturalistic every-day language use.