Pronouns reactivate conceptual representations in human hippocampal neurons

During discourse comprehension, every new word adds to an evolving representation of meaning that accumulates over consecutive sentences and constrains the next words. To minimize repetition and utterance length, languages use pronouns, like the word "she," to refer to nouns and phrases that were previously introduced. It has been suggested that language comprehension requires that pronouns activate the same neuronal representations as the nouns themselves. We recorded from individual neurons in the human hippocampus during a reading task. Cells that were selective to a particular noun were later reactivated by pronouns that refer to the cells' preferred noun. These results imply that concept cells contribute to a rapid and dynamic semantic memory network that is recruited during language comprehension.

Hierarchical neural processing in γ oscillations for syntactic and semantic operations accounts for first- and second-language epistemology

Introduction

We discuss event-related power differences (ERPDs) in low- and broadband-γ oscillations as the embedded-clause edge is processed in wh-dependencies such as Which decision regarding/about him/her did Paul say that Lydie rejected without hesitation? in first (L1) and second language (L2) French speakers.

Methods

The experimental conditions manipulated whether pronouns appeared in modifiers (Mods; regarding him/her) or in noun complements (Comps; about him/her) and whether they matched or mismatched a matrix-clause subject in gender.

Results

Across L1 and L2 speakers, we found that anaphora-linked ERPDs for Mods vs. Comps in evoked power first arose in low γ and then in broadband γ. Referential elements first seem to be retrieved from working memory by narrowband processes in low γ and then referential identification seems to be computed in broadband-γ output. Interactions between discourse- and syntax-based referential processes for the Mods vs. Comps in these ERPDs furthermore suggest that multidomain γ-band processing enables a range of elementary operations for discourse and semantic interpretation.

Discussion

We argue that a multidomain mechanism enabling operations conditioned by the syntactic and semantic nature of the elements processed interacts with local brain microcircuits representing features and feature sets that have been established in L1 or L2 acquisition, accounting for a single language epistemology across learning contexts.

Delta-band Activity Underlies Referential Meaning Representation during Pronoun Resolution

Human language offers a variety of ways to create meaning, one of which is referring to entities, objects, or events in the world. One such meaning maker is understanding to whom or to what a pronoun in a discourse refers to. To understand a pronoun, the brain must access matching entities or concepts that have been encoded in memory from previous linguistic context. Models of language processing propose that internally stored linguistic concepts, accessed via exogenous cues such as phonological input of a word, are represented as (a)synchronous activities across a population of neurons active at specific frequency bands. Converging evidence suggests that delta band activity (1-3 Hz) is involved in temporal and representational integration during sentence processing. Moreover, recent advances in the neurobiology of memory suggest that recollection engages neural dynamics similar to those which occurred during memory encoding. Integrating from these two research lines, we here tested the hypothesis that neural dynamic patterns, especially in delta frequency range, underlying referential meaning representation, would be reinstated during pronoun resolution. By leveraging neural decoding techniques (i.e., representational similarity analysis) on a magnetoencephalogram data set acquired during a naturalistic story-listening task, we provide evidence that delta-band activity underlies referential meaning representation. Our findings suggest that, during spoken language comprehension, endogenous linguistic representations such as referential concepts may be proactively retrieved and represented via activation of their underlying dynamic neural patterns.

Neural correlates of pronoun processing: An activation likelihood estimation meta-analysis

Pronouns are unique linguistic devices that allow for the expression of referential relationships. Despite their communicative utility, the neural correlates of the operations involved in reference assignment and/or resolution, are not well-understood. The present study synthesized the neuroimaging literature on pronoun processing to test extant theories of pronoun comprehension. Following the PRISMA guidelines and thebest-practice recommendations for neuroimaging meta-analyses, a systematic literature search and record assessment were performed. As a result, 16 fMRI studies were included in the meta-analysis, and were coded in Scribe 3.6 for inclusion in the BrainMap database. The activation coordinates for the contrasts of interest were transformed into Talairach space and submitted to an Activation Likelihood Estimation (ALE) meta-analysis in GingerALE 3.0.1. The results indicated that pronoun processing had functional convergence in the left posterior middle and superior temporal gyri, potentially reflecting the retrieval, prediction and integration roles of these areas for pronoun processing.

The timing versus resource problem in nonnative sentence processing: Evidence from a time-frequency analysis of anaphora resolution in successive wh-movement in native and nonnative speakers of French

Nonnative processing has been argued to reflect either reduced processing capacity or delayed timing of structural analysis compared to the extraction of lexical/semantic information. The current study simultaneously investigates timing and resource allocation through a time-frequency analysis of the intrinsic neural activity during syntactic processing in native and English-speaking nonnative speakers of French. It involved structurally constrained anaphora resolution in bi-clausal wh-filler-gap dependencies such as Quelle décision à propos de lui est-ce que Paul a dit que Lydie avait rejetée sans hésitation? 'Which decision about him did Paul say that Lydie rejected without hesitation?'. We tested the hypothesis that nonnative speakers may allocate greater resources than native speakers to the computation of syntactic representations based on the grammatical specifications encoded in lexical entries, though both native and nonnative processing involves the immediate application of structural constraints. This distinct resource allocation is likely to arise in response to higher activation thresholds for nonnative knowledge acquired after the first language grammar has been fully acquired. To examine this bias in nonnative neurocognitive processing, we manipulated the wh-filler to contain either a lexically specified noun complement such as à propos de lui 'about him' or a non-lexcially specified noun phrase modifier such as le concernant 'concerning him'. We focused on processing at the intermediate gap site, that is, the point of information exchange between the matrix and the embedded clauses by adopting a measurement window corresponding to the bridge verb dit 'said' and subordinator que 'that' introducing the embedded clause. Our results showed that structural constraints on anaphora produced event-related spectral perturbations at 13-14Hz early into the presentation of the bridge verb across groups. An interaction of structural constraints on anaphora with group was found at 18-19Hz early into the presentation of the bridge verb. In this interaction, the nonnative-speaker activity at 18-19Hz echoed the concurrent general patterns at 13-14Hz, whereas the native-speaker activity revealed distinct power at 18-19Hz and at 13-14Hz. There was no evidence of delay of structural constraints on intermediate gaps with respect to lexical access to the bridge verb and subordinator. However, nonnative speakers' allocation of power in cell assembly synchronizations of fillers and gaps at the intermediate gap site reflected the grammatical specifications lexically encoded in the fillers.

Age-related differences in the retrieval of phonologically similar words during sentence processing: Evidence from ERPs

We investigated how phonologically similar words are encoded and retrieved from memory during sentence processing across younger and older adults. Critical sentences included two phonologically similar or dissimilar noun phrases (henceforth NPs) followed by a pronoun. We examined brain activity time-locked to the onsets of the second NP, and the pronoun to investigate the encoding and retrieval of the NPs, respectively. Encoding the second NP resulted in smaller N400 amplitudes when the preceding NP was phonologically similar, for both younger and older adults, suggesting age-invariant encoding facilitation with increasing phonological similarity. However, when processing the pronoun, younger adults exhibited greater negativity following phonologically similar NPs, suggesting retrieval difficulty, whereas older adults showed greater negativity for pronouns following dissimilar NPs, suggesting an apparent retrieval facilitation. A post-hoc behavioral experiment suggested that older adults perform shallow processing during retrieval. The results suggest age-related decline in retrieval, but not encoding, of phonological information.

Functional brain plasticity during L1 training on complex sentences: Changes in gamma-band oscillatory activity

The adult human brain remains plastic even after puberty. However, whether first language (L1) training in adults can alter the language network is yet largely unknown. Thus, we conducted a longitudinal training experiment on syntactically complex German sentence comprehension. Sentence complexity was varied by the depth of the center embedded relative clauses (i.e., single or double embedded). Comprehension was tested after each sentence with a question on the thematic role assignment. Thirty adult, native German speakers were recruited for 4 days of training. Magnetoencephalography (MEG) data were recorded and subjected to spectral power analysis covering the classical frequency bands (i.e., theta, alpha, beta, low gamma, and gamma). Normalized spectral power, time‐locked to the final closure of the relative clause, was subjected to a two‐factor analysis (“sentence complexity” and “training days”). Results showed that for the more complex sentences, the interaction of sentence complexity and training days was observed in Brodmann area 44 (BA 44) as a decrease of gamma power with training. Moreover, in the gamma band (55–95 Hz) functional connectivity between BA 44 and other brain regions such as the inferior frontal sulcus and the inferior parietal cortex were correlated with behavioral performance increase due to training. These results show that even for native speakers, complex L1 sentence training improves language performance and alters neural activities of the left hemispheric language network. Training strengthens the use of the dorsal processing stream with working‐memory‐related brain regions for syntactically complex sentences, thereby demonstrating the brain's functional plasticity for L1 training.

The limited role of hippocampal declarative memory in transient semantic activation during online language processing

Recent findings point to a role for hippocampus in the moment-by-moment processing of language, including the use and generation of semantic features in certain contexts. What role the hippocampus might play in the processing of semantic relations in spoken language comprehension, however, is unknown. Here we test patients with bilateral hippocampal damage and dense amnesia in order to examine the necessity of hippocampus for lexico-semantic mapping processes in spoken language understanding. In two visual-world eye-tracking experiments, we monitor eye movements to images that are semantically related to spoken words and sentences. We find no impairment in amnesia, relative to matched healthy comparison participants. These findings suggest, at least for close semantic links and simple language comprehension tasks, a lack of necessity for hippocampus in lexico-semantic mapping between spoken words and simple pictures.

The effect of repetition on pronoun resolution in patients with memory impairment

Referring to things in the world - that woman, her idea, she - is a central component of language. Understanding reference requires the listener to keep track of the unfolding discourse history while integrating multiple sources of information to interpret the speech stream as it unfolds in time. Pronouns are a common way to establish reference. But due to their impoverished form, to understand them listeners must relate features of the pronoun (e.g., gender, animacy) with existing representations of potential discourse referents. Successful referential processing seems to place demands on memory. In a previous study, patients with hippocampal amnesia and healthy participants listened to short stories as their eye movements were monitored. When interpreting ambiguous pronouns, healthy participants demonstrated order-of-mention effects, whereby ambiguous pronouns are interpreted as referring to the first-mentioned referent in the story. By contrast, memory-impaired patients exhibited significant disruptions in their ability to use information about which character had been mentioned first to interpret pronouns. Repetition of the most salient information is a common clinical recommendation for improving pronoun resolution and communication in individuals with memory disorders (e.g., Alzheimer's disease) but this recommendation lacks an evidentiary basis. The present study seeks to determine whether the pronoun resolution performance of hippocampal patients can be improved, by repetition of the target referent, increasing its salience. Results indicate that patients with hippocampal damage demonstrate improved processing of pronouns following repetition of the target referent, but benefit from this repetition to a significantly smaller degree compared to healthy participants. These results provide further evidence for the role of the hippocampal-dependent memory system in language processing and point to the need for empirically tested communication interventions.

Distinguishing Old From New Referents During Discourse Comprehension: Evidence From ERPs and Oscillations

In this EEG study, we used pre-registered and exploratory ERP and time-frequency analyses to investigate the resolution of anaphoric and non-anaphoric noun phrases during discourse comprehension. Participants listened to story contexts that described two antecedents, and subsequently read a target sentence with a critical noun phrase that lexically matched one antecedent ('old'), matched two antecedents ('ambiguous'), partially matched one antecedent in terms of semantic features ('partial-match'), or introduced another referent (non-anaphoric, 'new'). After each target sentence, participants judged whether the noun referred back to an antecedent (i.e., an 'old/new' judgment), which was easiest for ambiguous nouns and hardest for partially matching nouns. The noun-elicited N400 ERP component demonstrated initial sensitivity to repetition and semantic overlap, corresponding to repetition and semantic priming effects, respectively. New and partially matching nouns both elicited a subsequent frontal positivity, which suggested that partially matching anaphors may have been processed as new nouns temporarily. ERPs in an even later time window and ERPs time-locked to sentence-final words suggested that new and partially matching nouns had different effects on comprehension, with partially matching nouns incurring additional processing costs up to the end of the sentence. In contrast to the ERP results, the time-frequency results primarily demonstrated sensitivity to noun repetition, and did not differentiate partially matching anaphors from new nouns. In sum, our results show the ERP and time-frequency effects of referent repetition during discourse comprehension, and demonstrate the potentially demanding nature of establishing the anaphoric meaning of a novel noun.

THE POWER OF NEURAL OSCILLATIONS TO INFORM SENTENCE COMPREHENSION: A LINGUISTIC PERSPECTIVE

The field of psycholinguistics is currently experiencing an explosion of interest in the analysis of neural oscillations - rhythmic brain activity synchronized at different temporal and spatial levels. Given that language comprehension relies on a myriad of processes, which are carried out in parallel in distributed brain networks, there is hope that this methodology might bring the field closer to understanding some of the more basic (spatially and temporally distributed, yet at the same time often overlapping) neural computations that support language function. In this review we discuss existing proposals linking oscillatory dynamics in different frequency bands to basic neural computations, and review relevant theories suggesting associations between band-specific oscillations and higher-level cognitive processes. More or less consistent patterns of oscillatory activity related to certain types of linguistic processing can already be derived from the evidence that has accumulated over the past few decades. The centerpiece of the current review is a synthesis of such patterns grouped by linguistic phenomenon. We restrict our review to evidence linking measures of oscillatory power to the comprehension of sentences, as well as linguistically (and/or pragmatically) more complex structures. For each grouping, we provide a brief summary and a table of associated oscillatory signatures that a psycholinguist might expect to find when employing a particular linguistic task. Summarizing across different paradigms, we conclude that a handful of basic neural oscillatory mechanisms are likely recruited in different ways and at different times for carrying out a variety of linguistic computations.

Cue integration during sentence comprehension: Electrophysiological evidence from ellipsis

Language processing requires us to integrate incoming linguistic representations with representations of past input, often across intervening words and phrases. This computational situation has been argued to require retrieval of the appropriate representations from memory via a set of features or representations serving as retrieval cues. However, even within in a cue-based retrieval account of language comprehension, both the structure of retrieval cues and the particular computation that underlies direct-access retrieval are still underspecified. Evidence from two event-related brain potential (ERP) experiments that show cue-based interference from different types of linguistic representations during ellipsis comprehension are consistent with an architecture wherein different cue types are integrated, and where the interaction of cue with the recent contents of memory determines processing outcome, including expression of the interference effect in ERP componentry. I conclude that retrieval likely includes a computation where cues are integrated with the contents of memory via a linear weighting scheme, and I propose vector addition as a candidate formalization of this computation. I attempt to account for these effects and other related phenomena within a broader cue-based framework of language processing.

What does "it" mean, anyway? Examining the time course of semantic activation in reference resolution

Pronouns serve a critical referential function, yet the cognitive processes engaged during pronoun comprehension remain incompletely understood. One view is that encountering a pronoun leads the comprehender to reactivate the semantic features of its antecedent. We examined this by manipulating the concreteness of a noun antecedent and assessing whether an Event Related Potential (ERP) concreteness effect was elicited at a downstream pronoun. We observed a robust concreteness effect at the noun, but no similar effect at the pronoun. We also examined whether N400 semantic priming from the antecedent would increase on content words shortly following the pronoun, relative to those preceding it. Again, although we observed semantic priming following the noun, it did not increase following the pronoun. Our data suggest that pronouns do not induce the activation of (much) new semantic information in long-term memory, perhaps instead triggering an attentional shift towards their antecedents' extant representations within working memory.

The Role of Gamma Oscillations During Integration of Metaphoric Gestures and Abstract Speech

Metaphoric (MP) co-speech gestures are commonly used during daily communication. They communicate about abstract information by referring to gestures that are clearly concrete (e.g., raising a hand for “the level of the football game is high”). To understand MP co-speech gestures, a multisensory integration at semantic level is necessary between abstract speech and concrete gestures. While semantic gesture-speech integration has been extensively investigated using functional magnetic resonance imaging, evidence from electroencephalography (EEG) is rare. In the current study, we set out an EEG experiment, investigating the processing of MP vs. iconic (IC) co-speech gestures in different contexts, to reveal the oscillatory signature of MP gesture integration. German participants (n = 20) viewed video clips with an actor performing both types of gestures, accompanied by either comprehensible German or incomprehensible Russian (R) speech, or speaking German sentences without any gestures. Time-frequency analysis of the EEG data showed that, when gestures were accompanied by comprehensible German speech, MP gestures elicited decreased gamma band power (50–70 Hz) between 500 and 700 ms in the parietal electrodes when compared to IC gestures, and the source of this effect was localized to the right middle temporal gyrus. This difference is likely to reflect integration processes, as it was reduced in the R language and no-gesture conditions. Our findings provide the first empirical evidence suggesting the functional relationship between gamma band oscillations and higher-level semantic processes in a multisensory setting.

Oscillatory Brain Responses Reflect Anticipation during Comprehension of Speech Acts in Spoken Dialog

Everyday conversation requires listeners to quickly recognize verbal actions, so-called speech acts, from the underspecified linguistic code and prepare a relevant response within the tight time constraints of turn-taking. The goal of this study was to determine the time-course of speech act recognition by investigating oscillatory EEG activity during comprehension of spoken dialog. Participants listened to short, spoken dialogs with target utterances that delivered three distinct speech acts (Answers, Declinations, Pre-offers). The targets were identical across conditions at lexico-syntactic and phonetic/prosodic levels but differed in the pragmatic interpretation of the speech act performed. Speech act comprehension was associated with reduced power in the alpha/beta bands just prior to Declination speech acts, relative to Answers and Pre-offers. In addition, we observed reduced power in the theta band during the beginning of Declinations, relative to Answers. Based on the role of alpha and beta desynchronization in anticipatory processes, the results are taken to indicate that anticipation plays a role in speech act recognition. Anticipation of speech acts could be critical for efficient turn-taking, allowing interactants to quickly recognize speech acts and respond within the tight time frame characteristic of conversation. The results show that anticipatory processes can be triggered by the characteristics of the interaction, including the speech act type.