Keeping track of who knows what in multiparty conversation despite severe memory impairment

Language use has long been understood to be tailored to the intended addressee, a process termed audience design. Audience design is reflected in multiple aspects of language use, including adjustments based on the addressee's knowledge about the topic at hand. In group settings, audience design depends on representations of multiple individuals, each of whom may have different knowledge about the conversational topic. A central question, then, concerns how these representations are encoded and retrieved in multiparty conversation where successful conversation requires keeping track of who knows what. In the present research, we probe the biological memory systems that are involved in this process of multiparty audience design. We present the results of two experiments that compare language use in persons with bilateral hippocampal damage and severe declarative memory impairment (amnesia), and demographically matched neurotypical comparison participants. Participants played a game in which they discussed abstract images with one partner in conversation, and then discussed the images again with the same partner or with a new partner in a three-party conversation. Neurotypical participants' language use reflected newly formed representations of which partner was familiar with which images. Participants with amnesia showed evidence of partner-specific audience design in multiparty conversation but it was attenuated, especially when success required rapid alternations between representations of common ground. The findings suggest partial independence of the formation and use of partner-specific representations from the hippocampal-dependent declarative memory system and highlight the unique contributions of the declarative memory system to flexible and dynamic language use.

Alcohol and common ground: The effects of intoxication on linguistic markers of shared understanding during social exchange

OBJECTIVE

Most alcohol consumption takes place in social contexts, and the belief that alcohol enhances social interactions has been identified as among the more robust predictors of alcohol use disorder (AUD) development. Yet, we know little of how alcohol affects mental representations of others-what we share and do not share-nor the extent to which intoxication might impact the development of shared understanding (i.e., common ground) between interaction partners. Employing a randomized experimental design and objective linguistic outcome measures, we present two studies examining the impact of alcohol consumption on the development and use of common ground.

METHOD

In Study 1, groups of strangers or friends were administered either alcohol (target Breath Alcohol Content = .08%) or a control beverage, following which they completed a task requiring them to develop a shared language to describe ambiguous images and then describe those images to either a knowledgeable or a naïve partner. The same procedures were completed in Study 2 using a within-subjects alcohol administration design and all-stranger groups.

RESULTS

Study 1 findings did not reach significance but suggested that alcohol may facilitate common ground development selectively among stranger groups. This effect emerged as significant in the context of the within-subjects design of Study 2, b = -0.19, p = .007, with participants demonstrating greater facility in establishing common ground during alcohol versus control sessions.

CONCLUSIONS

Results suggest that alcohol facilitates the development of shared linguistic understanding in novel social spaces, indicating common ground as one potential mechanism to consider in our broader examination of alcohol reinforcement and AUD etiology. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Linguistic features of spontaneous speech predict conversational recall

Empirical studies of conversational recall show that the amount of conversation that can be recalled after a delay is limited and biased in favor of one's own contributions. What aspects of a conversational interaction shape what will and will not be recalled? This study aims to predict the contents of conversation that will be recalled based on linguistic features of what was said. Across 59 conversational dyads, we observed that two linguistic features that are hallmarks of interactive language use-disfluency (um/uh) and backchannelling (ok, yeah)-promoted recall. Two other features-disagreements between the interlocutors and use of "like"-were not predictive of recall. While self-generated material was better remembered overall, both hearing and producing disfluency and backchannels improved memory for the associated utterances. Finally, the disfluency-related memory boost was similar regardless of the number of disfluencies in the utterance. Overall, we conclude that interactional linguistic features of conversation are predictive of what is and is not recalled following conversation.

Memory for social media images following traumatic brain injury

PRIMARY OBJECTIVE

An emerging body of research examines the role of computer-mediated communication in supporting social connection in persons with traumatic brain injury (TBI). We examine the cognitive impacts of engaging with images posted to social media for persons with moderate-severe TBI.

RESEARCH DESIGN

Prior work shows that after viewing social media posts, adults have better memory for posts when they generate a comment about the post. We examined if persons with TBI experience a memory benefit for commented-upon social media images similar to non-injured comparison participants.

METHODS AND PROCEDURES

53 persons with moderate-to-severe TBI and 52 non-injured comparison participants viewed arrays of real social media images and were prompted to comment on some of them. After a brief delay, a surprise two-alternative forced choice recognition memory test measured memory for these images.

MAIN OUTCOMES AND RESULTS

Persons with TBI remembered social media images at above-chance levels and experienced a commenting-related memory boost much like non-injured comparison participants.

CONCLUSIONS

These findings add to a growing literature on the potential benefits of social media use in individuals with TBI and point to the benefits of active engagement for memory in social media contexts in TBI.

Intact speech-gesture integration in narrative recall by adults with moderate-severe traumatic brain injury

PURPOSE

Real-world communication is situated in rich multimodal contexts, containing speech and gesture. Speakers often convey unique information in gesture that is not present in the speech signal (e.g., saying "He searched for a new recipe" while making a typing gesture). We examine the narrative retellings of participants with and without moderate-severe traumatic brain injury across three timepoints over two online Zoom sessions to investigate whether people with TBI can integrate information from co-occurring speech and gesture and if information from gesture persists across delays.

METHODS

60 participants with TBI and 60 non-injured peers watched videos of a narrator telling four short stories. On key details, the narrator produced complementary gestures that conveyed unique information. Participants retold the stories at three timepoints: immediately after, 20-min later, and one-week later. We examined the words participants used when retelling these key details, coding them as a Speech Match (e.g., "He searched for a new recipe"), a Gesture Match (e.g., "He searched for a new recipe online), or Other ("He looked for a new recipe"). We also examined whether participants produced representative gestures themselves when retelling these details.

RESULTS

Despite recalling fewer story details, participants with TBI were as likely as non-injured peers to report information from gesture in their narrative retellings. All participants were more likely to report information from gesture and produce representative gestures themselves one-week later compared to immediately after hearing the story.

CONCLUSION

We demonstrated that speech-gesture integration is intact after TBI in narrative retellings. This finding has exciting implications for the utility of gesture to support comprehension and memory after TBI and expands our understanding of naturalistic multimodal language processing in this population.

Incorporating Functional Response Time Effects into a Signal Detection Theory Model

Signal detection theory (SDT; Tanner & Swets in Psychological Review 61:401-409, 1954) is a dominant modeling framework used for evaluating the accuracy of diagnostic systems that seek to distinguish signal from noise in psychology. Although the use of response time data in psychometric models has increased in recent years, the incorporation of response time data into SDT models remains a relatively underexplored approach to distinguishing signal from noise. Functional response time effects are hypothesized in SDT models, based on findings from other related psychometric models with response time data. In this study, an SDT model is extended to incorporate functional response time effects using smooth functions and to include all sources of variability in SDT model parameters across trials, participants, and items in the experimental data. The extended SDT model with smooth functions is formulated as a generalized linear mixed-effects model and implemented in the gamm4 R package. The extended model is illustrated using recognition memory data to understand how conversational language is remembered. Accuracy of parameter estimates and the importance of modeling variability in detecting the experimental condition effects and functional response time effects are shown in conditions similar to the empirical data set via a simulation study. In addition, the type 1 error rate of the test for a smooth function of response time is evaluated.

Items Outperform Adjectives in a Computational Model of Binary Semantic Classification

Semantic memory encompasses one's knowledge about the world. Distributional semantic models, which construct vector spaces with embedded words, are a proposed framework for understanding the representational structure of human semantic knowledge. Unlike some classic semantic models, distributional semantic models lack a mechanism for specifying the properties of concepts, which raises questions regarding their utility for a general theory of semantic knowledge. Here, we develop a computational model of a binary semantic classification task, in which participants judged target words for the referent's size or animacy. We created a family of models, evaluating multiple distributional semantic models, and mechanisms for performing the classification. The most successful model constructed two composite representations for each extreme of the decision axis (e.g., one averaging together representations of characteristically big things and another of characteristically small things). Next, the target item was compared to each composite representation, allowing the model to classify more than 1,500 words with human-range performance and to predict response times. We propose that when making a decision on a binary semantic classification task, humans use task prompts to retrieve instances representative of the extremes on that semantic dimension and compare the probe to those instances. This proposal is consistent with the principles of the instance theory of semantic memory.

The Multiple Perspectives Theory of Mental States in Communication

Inspired by early proposals in philosophy, dominant accounts of language posit a central role for mutual knowledge, either encoded directly in common ground, or approximated through other cognitive mechanisms. Using existing empirical evidence from language and memory, we challenge this tradition, arguing that mutual knowledge captures only a subset of the mental states needed to support communication. In a novel theoretical proposal, we argue for a cognitive architecture that includes separate, distinct representations of the self and other, and a cognitive process that compares these representations continuously during conversation, outputting both similarities and differences in perspective. Our theory accounts for existing data, interfaces with findings from other cognitive domains, and makes novel predictions about the role of perspective in language use. We term this new account the Multiple Perspectives Theory of mental states in communication.

MEMCONS: How Contemporaneous Note-Taking Shapes Memory for Conversation

Written memoranda of conversations, or memcons, provide a near-contemporaneous record of what was said in conversation, and offer important insights into the activities of high-profile individuals. We assess the impact of writing a memcon on memory for conversation. Pairs of participants engaged in conversation and were asked to recall the contents of that conversation 1 week later. One participant in each pair memorialized the content of the interaction in a memcon shortly after the conversation. Participants who generated memcons recalled more details of the conversations than participants who did not, but the content of recall was equally and largely accurate for both participants. Remarkably, only 4.7% of the details of the conversation were recalled by both of the partners after a week delay. Contemporaneous note-taking appears to enhance memory for conversation by increasing the amount of information remembered but not the accuracy of that information. These findings have implications for evaluating the testimony of participants on conversations with major political or legal ramifications.

Inaugural editorial

In this editorial, the new journal editor introduces herself and then describes her goals for the Journal of Experimental Psychology: General. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Evidence of Audience Design in Amnesia: Adaptation in Gesture but Not Speech

Speakers design communication for their audience, providing more information in both speech and gesture when their listener is naïve to the topic. We test whether the hippocampal declarative memory system contributes to multimodal audience design. The hippocampus, while traditionally linked to episodic and relational memory, has also been linked to the ability to imagine the mental states of others and use language flexibly. We examined the speech and gesture use of four patients with hippocampal amnesia when describing how to complete everyday tasks (e.g., how to tie a shoe) to an imagined child listener and an adult listener. Although patients with amnesia did not increase their total number of words and instructional steps for the child listener, they did produce representational gestures at significantly higher rates for the imagined child compared to the adult listener. They also gestured at similar frequencies to neurotypical peers, suggesting that hand gesture can be a meaningful communicative resource, even in the case of severe declarative memory impairment. We discuss the contributions of multiple memory systems to multimodal audience design and the potential of gesture to act as a window into the social cognitive processes of individuals with neurologic disorders.

The effect of disfluency on memory for what was said

Disfluencies such as pauses, "um"s, and "uh"s are common interruptions in the speech stream. Previous work probing memory for disfluent speech shows memory benefits for disfluent compared to fluent materials. Complementary evidence from studies of language production and comprehension have been argued to show that different disfluency types appear in distinct contexts and, as a result, serve as a meaningful cue. If the disfluency-memory boost is a result of sensitivity to these form-meaning mappings, forms of disfluency that cue new upcoming information (fillers and pauses) may produce a stronger memory boost compared to forms that reflect speaker difficulty (repetitions). If the disfluency-memory boost is simply due to the attentional-orienting properties of a disruption to fluent speech, different disfluency forms may produce similar memory benefit. Experiments 1 and 2 compared the relative mnemonic benefit of three types of disfluent interruptions. Experiments 3 and 4 examined the scope of the disfluency-memory boost to probe its cognitive underpinnings. Across the four experiments, we observed a disfluency-memory boost for three types of disfluency that were tested. This boost was local and position dependent, only manifesting when the disfluency immediately preceded a critical memory probe word at the end of the sentence. Our findings reveal a short-lived disfluency-memory boost that manifests at the end of the sentence but is evoked by multiple types of disfluent forms, consistent with the idea that disfluencies bring attentional focus to immediately upcoming material. The downstream consequence of this localized memory benefit is better understanding and encoding of the speaker's message. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Space-time modeling of intensive binary time series eye-tracking data using a generalized additive logistic regression model

Eye-tracking has emerged as a popular method for empirical studies of cognitive processes across multiple substantive research areas. Eye-tracking systems are capable of automatically generating fixation-location data over time at high temporal resolution. Often, the researcher obtains a binary measure of whether or not, at each point in time, the participant is fixating on a critical interest area or object in the real world or in a computerized display. Eye-tracking data are characterized by spatial-temporal correlations and random variability, driven by multiple fine-grained observations taken over small time intervals (e.g., every 10 ms). Ignoring these data complexities leads to biased inferences for the covariates of interest such as experimental condition effects. This article presents a novel application of a generalized additive logistic regression model for intensive binary time series eye-tracking data from a between- and within-subjects experimental design. The model is formulated as a generalized additive mixed model (GAMM) and implemented in the mgcv R package. The generalized additive logistic regression model was illustrated using an empirical data set aimed at understanding the accommodation of regional accents in spoken language processing. Accuracy of parameter estimates and the importance of modeling the spatial-temporal correlations in detecting the experimental condition effects were shown in conditions similar to our empirical data set via a simulation study. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Referential Form and Memory for the Discourse History

The way we refer to things in the world is shaped by the immediate physical context as well as the discourse history. But what part of the discourse history is relevant to language use in the present? In four experiments, we combine the study of task-based conversation with measures of recognition memory to examine the role of physical contextual cues that shape what speakers perceive to be a part of the relevant discourse history. Our studies leverage the differentiation effect, a phenomenon in which speakers are more likely to use a modified expression to refer to an object (e.g., dotted sock) if they had previously described a similar object (e.g., striped sock) than when they had not described a similar object. Two physical cues-the background that framed the to-be-described pictures and the position of the pictures in the display-were manipulated to alter perceptions about the relevant discourse context. We measured the rate with which speakers modify referring expressions to differentiate current from past referents. Recognition memory measures following the conversation probed what was and was not remembered about past discourse referents and contexts. Analysis of modification rates indicated that these contextual grouping cues shaped perceptions about the relevant discourse context. The contextual cues did not affect memory for the referents, but the memory for past referents was better for speakers than for listeners. Our findings show that perceptions about the relevant discourse history are a key determinant of how language is used in the moment but also that conversational partners form asymmetric representations of the discourse history.

The limited role of hippocampal declarative memory in transient semantic activation during online language processing

Recent findings point to a role for hippocampus in the moment-by-moment processing of language, including the use and generation of semantic features in certain contexts. What role the hippocampus might play in the processing of semantic relations in spoken language comprehension, however, is unknown. Here we test patients with bilateral hippocampal damage and dense amnesia in order to examine the necessity of hippocampus for lexico-semantic mapping processes in spoken language understanding. In two visual-world eye-tracking experiments, we monitor eye movements to images that are semantically related to spoken words and sentences. We find no impairment in amnesia, relative to matched healthy comparison participants. These findings suggest, at least for close semantic links and simple language comprehension tasks, a lack of necessity for hippocampus in lexico-semantic mapping between spoken words and simple pictures.

What's New to You? Preschoolers' Partner-Specific Online Processing of Disfluency

Speech disfluencies (e.g., “Point to thee um turtle”) can signal that a speaker is about to refer to something difficult to name. In two experiments, we found evidence that 4-year-olds, like adults, flexibly interpret a particular partner’s disfluency based on their estimate of that partner’s knowledge, derived from the preceding conversation. In entrainment trials, children established partner-specific shared knowledge of names for tangram pictures with one or two adult interlocutors. In each test trial, an adult named one of two visible tangrams either fluently or disfluently while children’s eye-movements were monitored. We manipulated speaker knowledge in the test trials. In Experiment 1, the test-trial speaker was the same speaker from entrainment or a naïve experimenter; in Experiment 2, the test-trial speaker had been one of the child’s partners in entrainment and had seen half of the tangrams (either animal or vehicle tangrams). When hearing disfluent expressions, children looked more at a tangram that was unfamiliar from the speaker’s perspective; this systematic disfluency effect disappeared in Experiment 1 when the speaker was entirely naïve, and depended on each speaker’s entrainment experience in Experiment 2. These findings show that 4-year-olds can keep track of two different partners’ knowledge states, and use this information to determine what should be difficult for a particular partner to name, doing so efficiently enough to guide online interpretation of disfluent speech.

Referential context and executive functioning influence children's resolution of syntactic ambiguity

Classic studies reveal two striking differences between preschoolers and adults in online sentence comprehension. Adults (a) recruit referential context cues to guide syntactic parsing, interpreting an ambiguous phrase as a modifier if a modifier is needed to single out the intended referent among multiple options, and (b) use late-arriving information to recover from misinterpretation. Five-year-olds fail on both counts, appearing insensitive to the referential context and often failing to recover from parsing errors (Trueswell, Sekerina, Hill, & Logrip, 1999). But other findings suggest that 5-year-olds show delayed rather than absent sensitivity to the referential context, and that individual differences in executive functioning predict children's ability to recover from garden-path errors. In 2 experiments, we built on these findings, focusing on whether children recruit referential-context cues if given time to do so. Children heard temporarily ambiguous instructions (e.g., Put the frog on the pond into the tent), while we monitored their eye-gaze and actions. We used a slow speech rate, and manipulated referential context between rather than within subjects, to give children time to bring referential context cues into play. Across experiments, eye-movement and action analyses revealed emerging sensitivity to the referential context. Moreover, error rates and eye-movement patterns indicating failures to revise were predicted by individual differences in executive function (scores in Simon Says and Flanker tasks). These data suggest that children, like adults, use referential context information in syntactic processing under some circumstances; the findings are also consistent with a role for domain-general executive function in resolution of syntactic ambiguity. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

#foodie: Implications of interacting with social media for memory

BACKGROUND

Social media is an increasingly popular outlet for leisure and social interaction. On many social media platforms, the user experience involves commenting on or responding to user-generated content, such as images of cats, food, and people. In two experiments, we examined how the act of commenting on social media images impacts subsequent memory of those images, using Instagram posts as a test case. This project was inspired by recent findings of laboratory studies of conversation which found that describing a picture for a conversational partner boosts recognition memory for those images. Here we aimed to understand how this finding translates to the more ecologically valid realm of social media interactions. A second motivation for the study was the popularity of food- and dieting-related content on Instagram and prior findings that use of Instagram in particular is associated with disordered eating behaviors.

RESULTS

Across two experiments, we observed that commenting on Instagram posts consistently boosted subsequent recognition and that correct recognition increased with comment length. Stable individual differences in recognition memory were observed, and "unhealthy" food images such as chocolates were particularly well remembered; however, these memory findings did not relate to self-reported eating behavior.

CONCLUSIONS

Taken together, our findings show that the way in which we engage with social media content shapes subsequent memory of it, raising new questions about how our online lives persist in memory over time, potentially shaping future behavior.

Modeling Intensive Polytomous Time-Series Eye-Tracking Data: A Dynamic Tree-Based Item Response Model

This paper presents a dynamic tree-based item response (IRTree) model as a novel extension of the autoregressive generalized linear mixed effect model (dynamic GLMM). We illustrate the unique utility of the dynamic IRTree model in its capability of modeling differentiated processes indicated by intensive polytomous time-series eye-tracking data. The dynamic IRTree was inspired by but is distinct from the dynamic GLMM which was previously presented by Cho, Brown-Schmidt, and Lee (Psychometrika 83(3):751-771, 2018). Unlike the dynamic IRTree, the dynamic GLMM is suitable for modeling intensive binary time-series eye-tracking data to identify visual attention to a single interest area over all other possible fixation locations. The dynamic IRTree model is a general modeling framework which can be used to model change processes (trend and autocorrelation) and which allows for decomposing data into various sources of heterogeneity. The dynamic IRTree model was illustrated using an experimental study that employed the visual-world eye-tracking technique. The results of a simulation study showed that parameter recovery of the model was satisfactory and that ignoring trend and autoregressive effects resulted in biased estimates of experimental condition effects in the same conditions found in the empirical study.

The effect of repetition on pronoun resolution in patients with memory impairment

Referring to things in the world - that woman, her idea, she - is a central component of language. Understanding reference requires the listener to keep track of the unfolding discourse history while integrating multiple sources of information to interpret the speech stream as it unfolds in time. Pronouns are a common way to establish reference. But due to their impoverished form, to understand them listeners must relate features of the pronoun (e.g., gender, animacy) with existing representations of potential discourse referents. Successful referential processing seems to place demands on memory. In a previous study, patients with hippocampal amnesia and healthy participants listened to short stories as their eye movements were monitored. When interpreting ambiguous pronouns, healthy participants demonstrated order-of-mention effects, whereby ambiguous pronouns are interpreted as referring to the first-mentioned referent in the story. By contrast, memory-impaired patients exhibited significant disruptions in their ability to use information about which character had been mentioned first to interpret pronouns. Repetition of the most salient information is a common clinical recommendation for improving pronoun resolution and communication in individuals with memory disorders (e.g., Alzheimer's disease) but this recommendation lacks an evidentiary basis. The present study seeks to determine whether the pronoun resolution performance of hippocampal patients can be improved, by repetition of the target referent, increasing its salience. Results indicate that patients with hippocampal damage demonstrate improved processing of pronouns following repetition of the target referent, but benefit from this repetition to a significantly smaller degree compared to healthy participants. These results provide further evidence for the role of the hippocampal-dependent memory system in language processing and point to the need for empirically tested communication interventions.

Talker-specific predictions during language processing

Language comprehension is shaped by world knowledge. After hearing about "a farm animal," meanings of typical ("cow") versus atypical exemplars ("ox") are more accessible, as evidenced by N400 responses. Moreover, atypical exemplars elicit a larger post-N400 frontal positivity than typical and incongruous ("ivy") exemplars, indexing the integration of unexpected information. Do listeners adapt this category knowledge to specific talkers? We first replicated typicality effects in the auditory modality. Then, we extended the design to a two-talker context: talkers alternated cueing (Bob: "Susan, name a farm animal") and answering (Susan: "cow"). Critically, participants first heard interviews in which one talker revealed strong associations with atypical exemplars (Susan works on an ox farm). We observed increased frontal positivity to a typical exemplar ("cow") said by Susan compared to Bob, indicating participants appreciated that the typical exemplar was atypical for Susan. These results suggest that comprehenders can tailor their expectations to the talker.

Gesture height reflects common ground status even in patients with amnesia

When we communicate, we alter our language and gesture based on the mutually shared knowledge - common ground - that we have with our listener. How memory supports these alterations remain unclear. We asked healthy adults and patients with hippocampal amnesia to engage in a referential communication task. Previous work suggests that common ground can be encoded by distinct memory systems; Amnesic patients show normal learning and referential label use as common ground increases, but inconsistently mark these labels with definite determiners (e.g., the vs. a windmill). Which memory systems support the ability to mark common ground via hand gesture? We found that gestures of both healthy participants and amnesic patients reflected common ground status. Both groups produced high gestures when common ground was lacking, and were less likely to do so as common ground increased. These findings suggest that gesture can reflect common ground status during conversation, potentially via non-declarative memory.

Autoregressive Generalized Linear Mixed Effect Models with Crossed Random Effects: An Application to Intensive Binary Time Series Eye-Tracking Data

As a method to ascertain person and item effects in psycholinguistics, a generalized linear mixed effect model (GLMM) with crossed random effects has met limitations in handing serial dependence across persons and items. This paper presents an autoregressive GLMM with crossed random effects that accounts for variability in lag effects across persons and items. The model is shown to be applicable to intensive binary time series eye-tracking data when researchers are interested in detecting experimental condition effects while controlling for previous responses. In addition, a simulation study shows that ignoring lag effects can lead to biased estimates and underestimated standard errors for the experimental condition effects.

Compensating for an Inattentive Audience

The common ground that conversational partners share is thought to form the basic context for language use. According to the classic view, inferences about common ground, or mutual knowledge, are guided by beliefs about the physical, cognitive, and attentional states of one's communicative partners. Here, we provide a first test of the attention assumption for common ground, the proposal that common ground for a co-present entity-such as an object or an utterance-can only be formed if a person has evidence that his or her partner has also attended to it. In three experiments, a participant speaker and two partners learned names for novel monster pictures as a group. The speaker was then asked to describe the monsters to each partner separately in a referential communication task. The critical manipulation was the (in)attentiveness of one partner at different points in the study. Analysis of the speaker's referring expressions revealed that speakers assumed their partner shared common ground for the monster names only when that partner exhibited engaged attention as the names were learned. These findings provide key and novel support for the classic proposal that formation of common ground critically depends on assumptions about the attentional state of one's conversational partner.

Knowledge and learning of verb biases in amnesia

Verb bias-the co-occurrence frequencies between a verb and the syntactic structures it may appear with-is a critical and reliable linguistic cue for online sentence processing. In particular, listeners use this information to disambiguate sentences with multiple potential syntactic parses (e.g., Feel the frog with the feather.). Further, listeners dynamically update their representations of specific verbs in the face of new evidence about verb-structure co-occurrence. Yet, little is known about the biological memory systems that support the use and dynamic updating of verb bias. We propose that hippocampal-dependent declarative (relational) memory represents a likely candidate system because it has been implicated in the flexible binding of relational co-occurrences and in statistical learning. We explore this question by testing patients with severe and selective deficits in declarative memory (anterograde amnesia), and demographically matched healthy participants, in their on-line interpretation of ambiguous sentences and the ability to update their verb bias with experience. We find that (1) patients and their healthy counterparts use existing verb bias to successfully interpret on-line ambiguity, however (2) unlike healthy young adults, neither group updated these biases in response to recent exposure. These findings demonstrate that using existing representations of verb bias does not necessitate involvement of the declarative memory system, but leave open the question of whether the ability to update representations of verb-specific biases requires hippocampal engagement.

The Necessity of the Hippocampus for Statistical Learning

Converging evidence points to a role for the hippocampus in statistical learning, but open questions about its necessity remain. Evidence for necessity comes from Schapiro and colleagues who report that a single patient with damage to hippocampus and broader medial temporal lobe cortex was unable to discriminate new from old sequences in several statistical learning tasks. The aim of the current study was to replicate these methods in a larger group of patients who have either damage localized to hippocampus or broader medial temporal lobe damage, to ascertain the necessity of the hippocampus in statistical learning. Patients with hippocampal damage consistently showed less learning overall compared with healthy comparison participants, consistent with an emerging consensus for hippocampal contributions to statistical learning. Interestingly, lesion size did not reliably predict performance. However, patients with hippocampal damage were not uniformly at chance and demonstrated above-chance performance in some task variants. These results suggest that hippocampus is necessary for statistical learning levels achieved by most healthy comparison participants but significant hippocampal pathology alone does not abolish such learning.

Learning and using knowledge about what other people do and don't know despite amnesia

Successful communication requires keeping track of what other people do and do not know, and how this differs from our own knowledge. Here we ask how knowledge of what others know is stored in memory. We take a neuropsychological approach, comparing healthy adults to patients with severe declarative memory impairment (amnesia). We evaluate whether this memory impairment disrupts the ability to successfully acquire and use knowledge about what other people know when communicating with them. We tested participants in a referential communication task in which the participants described a series of abstract "tangram" images for a partner. Participants then repeated the task with the same partner or a new partner. Findings show that much like healthy individuals, individuals with amnesia successfully tailored their communicative language to the knowledge shared with their conversational partner-their common ground. They produced brief descriptions of the tangram images for the familiar partner and provided more descriptive, longer expressions for the new partner. These findings demonstrate remarkable sparing in amnesia of the acquisition and use of partner-specific knowledge that underlies common ground, and have important implications for understanding the memory systems that support conversational language.

Memory and Common Ground Processes in Language Use

During communication, we form assumptions about what our communication partners know and believe. Information that is mutually known between the discourse partners-their common ground-serves as a backdrop for successful communication. Here we present an introduction to the focus of this topic, which is the role of memory in common ground and language use. Two types of questions emerge as central to understanding the relationship between memory and common ground, specifically questions having to do with the representation of common ground in memory, and the use of common ground during language processing.

Verb biases are shaped through lifelong learning

Verbs often participate in more than 1 syntactic structure, but individual verbs can be biased in terms of whether they are used more often with 1 structure or the other. For instance, in a sentence such as "Bop the bunny with the flower," the phrase "with the flower" is more likely to indicate an instrument with which to "bop," rather than which "bunny" to bop. Conversely, in a sentence such as "Choose the cow with the flower," the phrase "with the flower" is more likely to indicate which "cow" to choose. An open question is where these biases come from and whether they continue to be shaped in adulthood in a way that has lasting consequences for real-time processing of language. In Experiment 1 we replicated previous findings that these language-wide biases guide online syntactic processing in a computer-based visual-world paradigm. In Experiment 2, we tested the malleability of these biases by exposing adults to initially unbiased verbs situated in unambiguous contexts that led to either instrument or modifier interpretations. During test, participants interpreted sentences containing either modifier- or instrument-trained verbs in ambiguous contexts. Eye-movement and action data show that participants' considerations of the candidate interpretations of the ambiguous with-phrases were guided by the newly learned verb biases. These results suggest that co-occurrence information about specific verbs and syntactic structures embedded in language experiences plays a role in forming, and continuously shaping, the verb biases that constitute a part of the broader representation of the language. (PsycINFO Database Record

The influence of partner-specific memory associations on picture naming: a failure to replicate Horton (2007)

The results of two experiments by Horton (2007) show that speakers name a pictured object faster when in the presence of another person with whom the speaker has previously associated that object name. The first of those two experiments (Horton, 2007, Experiment 1) is the focus of the present research. This paper presents the results of three experiments designed to replicate and extend Horton's (2007) Experiment 1. The original findings were not replicated. These findings do not support the hypothesis that partner-specific memory associations facilitate object naming.

Do adults show a curse of knowledge in false-belief reasoning? A robust estimate of the true effect size

Seven experiments use large sample sizes to robustly estimate the effect size of a previous finding that adults are more likely to commit egocentric errors in a false-belief task when the egocentric response is plausible in light of their prior knowledge. We estimate the true effect size to be less than half of that reported in the original findings. Even though we found effects in the same direction as the original, they were substantively smaller; the original study would have had less than 33% power to detect an effect of this magnitude. The influence of plausibility on the curse of knowledge in adults appears to be small enough that its impact on real-life perspective-taking may need to be reevaluated.

Talker-specific learning in amnesia: Insight into mechanisms of adaptive speech perception

A hallmark of human speech perception is the ability to comprehend speech quickly and effortlessly despite enormous variability across talkers. However, current theories of speech perception do not make specific claims about the memory mechanisms involved in this process. To examine whether declarative memory is necessary for talker-specific learning, we tested the ability of amnesic patients with severe declarative memory deficits to learn and distinguish the accents of two unfamiliar talkers by monitoring their eye-gaze as they followed spoken instructions. Analyses of the time-course of eye fixations showed that amnesic patients rapidly learned to distinguish these accents and tailored perceptual processes to the voice of each talker. These results demonstrate that declarative memory is not necessary for this ability and points to the involvement of non-declarative memory mechanisms. These results are consistent with findings that other social and accommodative behaviors are preserved in amnesia and contribute to our understanding of the interactions of multiple memory systems in the use and understanding of spoken language.

Limitations on adaptation to foreign accents

Although foreign accents can be highly dissimilar to native speech, existing research suggests that listeners readily adapt to foreign accents after minimal exposure. However, listeners often report difficulty understanding non-native accents, and the time-course and specificity of adaptation remain unclear. Across five experiments, we examined whether listeners could use a newly learned feature of a foreign accent to eliminate lexical competitors during online speech perception. Participants heard the speech of a native English speaker and a native speaker of Québec French who, in English, pronounces /i/ as [i] (e.g., weak as wick) before all consonants except voiced fricatives. We examined whether listeners could learn to eliminate a shifted /i/-competitor (e.g., weak) when interpreting the accented talker produce an unshifted word (e.g., wheeze). In four experiments, adaptation was strikingly limited, though improvement across the course of the experiment and with stimulus variations indicates learning was possible. In a fifth experiment, adaptation was not improved when a native English talker produced the critical vowel shift, demonstrating that the limitation is not simply due to the fact the accented talker was non-native. These findings suggest that although listeners can arrive at the correct interpretation of a foreign accent, this process can pose significant difficulty.

Hippocampal contributions to language: evidence of referential processing deficits in amnesia

A growing body of work suggests the hippocampus contributes to a variety of cognitive domains beyond its traditional role in memory. We propose that the hippocampus, in its capacity for relational binding, representational flexibility, and online maintenance and integration of multimodal relational representations, is a key contributor to language processing. Here we test the hypothesis that the online interpretation of pronouns is hippocampus-dependent. We combined eye tracking with neuropsychological methods, where participants (4 patients with bilateral hippocampal damage and severe declarative memory impairment, 4 patients with ventromedial prefrontal cortex [vmPFC] damage, and healthy comparison participants) viewed a scene while listening to short dialogues introducing 2 characters; for example, Melissa is playing violin for Debbie/Danny as the sun is shining overhead. She is wearing a blue/purple dress. Consistent with previous work, analysis of eye gaze showed that younger and older healthy comparison participants and the vmPFC patients rapidly identified the intended referent of the pronoun when gender uniquely identified the referent, and when it did not, they showed a preference to interpret the pronoun as referring to the first-mentioned character. By contrast, hippocampal patients, while exhibiting a similar gender effect, exhibited significant disruptions in their ability to use information about which character had been mentioned first to interpret the pronoun. This finding suggests that the hippocampus plays a role in maintaining and integrating information even over a very short discourse history. These observed disruptions in referential processing demonstrate how promiscuously the hallmark processing features of the hippocampus are used in service of a variety of cognitive domains including language.

The hippocampus and the flexible use and processing of language

Fundamental to all human languages is an unlimited expressive capacity and creative flexibility that allow speakers to rapidly generate novel and complex utterances. In turn, listeners interpret language “on-line,” incrementally integrating multiple sources of information as words unfold over time. A challenge for theories of language processing has been to understand how speakers and listeners generate, gather, integrate, and maintain representations in service of language processing. We propose that many of the processes by which we use language place high demands on and receive contributions from the hippocampal declarative memory system. The hippocampal declarative memory system is long known to support relational binding and representational flexibility. Recent findings demonstrate that these same functions are engaged during the real-time processes that support behavior in-the-moment. Such findings point to the hippocampus as a potentially key contributor to cognitive functions that require on-line integration of multiple sources of information, such as on-line language processing. Evidence supporting this view comes from findings that individuals with hippocampal amnesia show deficits in the use of language flexibly and on-line. We conclude that the relational binding and representational flexibility afforded by the hippocampal declarative memory system positions the hippocampus as a key contributor to language use and processing.

How do I remember that I know you know that I know?

Communication is aided greatly when speakers and listeners take advantage of mutually shared knowledge (i.e., common ground). How such information is represented in memory is not well known. Using a neuropsychological-psycholinguistic approach to real-time language understanding, we investigated the ability to form and use common ground during conversation in memory-impaired participants with hippocampal amnesia. Analyses of amnesics' eye fixations as they interpreted their partner's utterances about a set of objects demonstrated successful use of common ground when the amnesics had immediate access to common-ground information, but dramatic failures when they did not. These findings indicate a clear role for declarative memory in maintenance of common-ground representations. Even when amnesics were successful, however, the eye movement record revealed subtle deficits in resolving potential ambiguity among competing intended referents; this finding suggests that declarative memory may be critical to more basic aspects of the on-line resolution of linguistic ambiguity.

Real-time investigation of referential domains in unscripted conversation: a targeted language game approach

Two experiments examined the restriction of referential domains during unscripted conversation by analyzing the modification and on-line interpretation of referring expressions. Experiment 1 demonstrated that from the earliest moments of processing, addressees interpreted referring expressions with respect to referential domains constrained by the conversation. Analysis of eye movements during the conversation showed elimination of standard competition effects seen with scripted language. Results from Experiment 2 pinpointed two pragmatic factors responsible for restriction of the referential domains used by speakers to design referential expressions and demonstrated that the same factors predict whether addressees consider local competitors to be potential referents during on-line interpretation of the same expressions. These experiments demonstrate for the first time that on-line interpretation of referring expressions in conversation is facilitated by referential domains constrained by pragmatic factors which predict when addressees are likely to encounter temporary ambiguity in language processing.

Partner-specific interpretation of maintained referential precedents during interactive dialog

In dialog settings, conversational partners converge on similar names for referents. These lexically entrained terms (Garrod & Anderson, 1987) are part of the common ground between the particular individuals who established the entrained term (Brennan & Clark, 1996), and are thought to be encoded in memory with a partner-specific cue. Thus far, analyses of the time-course of interpretation suggest that partner-specific information may not constrain the initial interpretation of referring expressions (Kronmüller & Barr, 2007; Barr & Keysar, 2002). However, these studies used non-interactive paradigms, which may limit the use of partner-specific representations. This article presents the results of three eye-tracking experiments. Experiment 1a used an interactive conversation methodology in which the experimenter and participant jointly established entrained terms for various images. On critical trials, the same experimenter, or a new experimenter described a critical image using an entrained term, or a new term. The results demonstrated an early, on-line partner-specific effect for interpretation of entrained terms, as well as preliminary evidence for an early, partner-specific effect for new terms. Experiment 1b used a non-interactive paradigm in which participants completed the same task by listening to image descriptions recorded during Experiment 1a; the results showed that partner-specific effects were eliminated. Experiment 2 replicated the partner-specific findings of Experiment 1a with an interactive paradigm and scenes that contained previously unmentioned images. The results suggest that partner-specific interpretation is most likely to occur in interactive dialog settings; the number of critical trials and stimulus characteristics may also play a role. The results are consistent with a large body of work demonstrating that the language processing system uses a rich source of contextual and pragmatic representations to guide on-line processing decisions.

Little houses and casas pequeñas: message formulation and syntactic form in unscripted speech with speakers of English and Spanish

During unscripted speech, speakers coordinate the formulation of pre-linguistic messages with the linguistic processes that implement those messages into speech. We examine the process of constructing a contextually appropriate message and interfacing that message with utterance planning in English (the small butterfly) and Spanish (la mariposa pequeña) during an unscripted, interactive task. The coordination of gaze and speech during formulation of these messages is used to evaluate two hypotheses regarding the lower-limit on the size of message planning units, namely whether messages are planned in units isomorphous to entire phrases or units isomorphous to single lexical items. Comparing the planning of fluent pre-nominal adjectives in English and post-nominal adjectives in Spanish showed that size information is added to the message later in Spanish than English, suggesting that speakers can prepare pre-linguistic messages in lexically-sized units. The results also suggest that speakers can use disfluency to coordinate the transition from thought to speech.

Language processing in the natural world

The authors argue that a more complete understanding of how people produce and comprehend language will require investigating real-time spoken-language processing in natural tasks, including those that require goal-oriented unscripted conversation. One promising methodology for such studies is monitoring eye movements as speakers and listeners perform natural tasks. Three lines of research that adopt this approach are reviewed: (i) spoken word recognition in continuous speech, (ii) reference resolution in real-world contexts, and (iii) real-time language processing in interactive conversation. In each domain, results emerge that provide insights which would otherwise be difficult to obtain. These results extend and, in some cases, challenge standard assumptions about language processing.