What can narratives tell us about the neural bases of human memory?

"All the Stars Will Be Wells with a Rusty Pulley": Neural Processing of the Social and Pragmatic Content in a Narrative

Making sense of natural language and narratives requires building and manipulating a situation model by adding incoming information to the model and using the context stored in the model to comprehend subsequent details and events. Situation model maintenance is supported by the default mode network (DMN), but comprehension of the individual moments in the narrative relies on access to the conceptual store within the semantic system. The present study examined how these systems are engaged by different narrative content to investigate whether highly informative, or semantic, content is a particularly strong driver of semantic system activation compared with contextually driven content that requires using the situation model, which might instead engage DMN regions. The study further investigated which subregions of the graded semantic hub in the left anterior temporal lobe (ATL) were engaged by the type of narrative content. To do this, we quantified the semantic, pragmatic, social, ambiguous, and emotional content for each sentence in a complete narrative, the English translation of The Little Prince. Increased activation in the transmodal hub in the ventral ATL was only observed for high semantic (i.e., informative) relative to low semantic sentences. Activation in the dorsolateral and ventrolateral ATL subregions was observed for both high relative to low semantic and social content sentences, but the ventrolateral ATL effects were more extensive in the social condition. There was high correspondence between the social and pragmatic content results, particularly in the ventrolateral ATL. We argue that the ventrolateral ATL may be particularly engaged by internal, or endogenous, processing demands, aided by functional connections between the anterior middle temporal gyrus and the DMN. Pragmatic and social content may have driven endogenous processing given the pervasive and plot-progressing nature of this content in the narrative. We put forward a revised account of how the semantic system is engaged in naturalistic contexts, a critical step toward better understanding real-world semantic and social processing.

Towards an ecologically valid naturalistic cognitive neuroscience of memory and event cognition

The landscape of human memory and event cognition research has witnessed a transformative journey toward the use of naturalistic contexts and tasks. In this review, we track this progression from abrupt, artificial stimuli used in extensively controlled laboratory experiments to more naturalistic tasks and stimuli that present a more faithful representation of the real world. We argue that in order to improve ecological validity, naturalistic study designs must consider the complexity of the cognitive phenomenon being studied. Then, we review the current state of "naturalistic" event segmentation studies and critically assess frequently employed movie stimuli. We evaluate recently developed tools like lifelogging and other extended reality technologies to help address the challenges we identified with existing naturalistic approaches. We conclude by offering some guidelines that can be used to design ecologically valid cognitive neuroscience studies of memory and event cognition.

Culturally and linguistically diverse children's retention of spoken narratives encoded in quiet and in babble noise

Multi-talker noise impedes children's speech processing and may affect children listening to their second language more than children listening to their first language. Evidence suggests that multi-talker noise also may impede children's memory retention and learning. A total of 80 culturally and linguistically diverse children aged 7 to 9 years listened to narratives in two listening conditions: quiet and multi-talker noise (signal-to-noise ratio +6 dB). Repeated recall (immediate and delayed recall), was measured with a 1-week retention interval. Retention was calculated as the difference in recall accuracy per question between immediate and delayed recall. Working memory capacity was assessed, and the children's degree of school language (Swedish) exposure was quantified. Immediate narrative recall was lower for the narrative encoded in noise than in quiet. During delayed recall, narrative recall was similar for both listening conditions. Children with higher degrees of school language exposure and higher working memory capacity had better narrative recall overall, but these factors were not associated with an effect of listening condition or retention. Multi-talker babble noise does not impair culturally and linguistically diverse primary school children's retention of spoken narratives as measured by multiple-choice questions. Although a quiet listening condition allows for a superior encoding compared with a noisy listening condition, details are likely lost during memory consolidation and re-consolidation.

Brain decoding of spontaneous thought: Predictive modeling of self-relevance and valence using personal narratives

The contents and dynamics of spontaneous thought are important factors for personality traits and mental health. However, assessing spontaneous thoughts is challenging due to their unconstrained nature, and directing participants' attention to report their thoughts may fundamentally alter them. Here, we aimed to decode two key content dimensions of spontaneous thought-self-relevance and valence-directly from brain activity. To train functional MRI-based predictive models, we used individually generated personal stories as stimuli in a story-reading task to mimic narrative-like spontaneous thoughts (n = 49). We then tested these models on multiple test datasets (total n = 199). The default mode, ventral attention, and frontoparietal networks played key roles in the predictions, with the anterior insula and midcingulate cortex contributing to self-relevance prediction and the left temporoparietal junction and dorsomedial prefrontal cortex contributing to valence prediction. Overall, this study presents brain models of internal thoughts and emotions, highlighting the potential for the brain decoding of spontaneous thought.

The neural basis of naturalistic semantic and social cognition

Decoding social environments and engaging meaningfully with other people are critical aspects of human cognition. Multiple cognitive systems, including social and semantic cognition, work alongside each other to support these processes. This study investigated shared processing between social and semantic systems using neuroimaging data collected during movie-viewing, which captures the multimodal environment in which social knowledge is exchanged. Semantic and social content from movie events (event-level) and movie transcripts (word-level) were used in parametric modulation analyses to test (1) the degree to which semantic and social information is processed within each respective network and (2) engagement of the same cross-network regions or the same domain-general hub located within the semantic network during semantic and social processing. Semantic word and event-level content engaged the same fronto-temporo-parietal network and a portion of the semantic hub in the anterior temporal lobe (ATL). Social word and event-level content engaged the supplementary motor area and right angular gyrus within the social network, but only social words engaged the domain-general semantic hub in left ATL. There was evidence of shared processing between the social and semantic systems in the dorsolateral portion of right ATL which was engaged by word and event-level semantic and social content. Overlap between the semantic and social word and event results was highly variable within and across participants, with the most consistent loci of overlap occurring in left inferior frontal, bilateral precentral and supramarginal gyri for social and semantic words and in bilateral superior temporal gyrus extending from ATL posteriorly into supramarginal gyri for social and semantic events. These results indicate a complex pattern of shared and distinct regions for social and semantic cognition during naturalistic processing. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on October 11, 2022. The protocol, as accepted by the journal, can be found at: https://doi.org/10.17605/OSF.IO/ACWQY .

Hippocampal-cortical interactions during event boundaries support retention of complex narrative events

According to most memory theories, encoding involves continuous communication between the hippocampus and neocortex, but recent work has shown that key moments at the end of an event, called event boundaries, may be especially critical for memory formation. We sought to determine how communication between the hippocampus and neocortical regions during the encoding of naturalistic events related to subsequent retrieval of those events and whether this was particularly important at event boundaries. Participants encoded and recalled two cartoon movies during fMRI scanning. Higher functional connectivity between the hippocampus and the posterior medial network (PMN) at an event's offset is related to the subsequent successful recall of that event. Furthermore, hippocampal-PMN offset connectivity also predicted the amount of detail retrieved after a 2-day delay. These data demonstrate that the relationship between memory encoding and hippocampal-neocortical interaction is dynamic and biased toward boundaries.

Long-term memory for movie details: selective decay for verbal information at one week

Mnemonic representations of complex events are multidimensional, incorporating information about objects and characters, their interactions and their spatial-temporal context. The present study investigated the degree to which detailed verbal information (i.e., dialogues), as well as semantic and spatiotemporal (i.e., "what", "where", and "when") elements of episodic memories for movies, are forgotten over the course of a week. Moreover, we tested whether the amount of dimension-specific forgetting differed as a function of the participant's age. In a mixed design, younger and middle-aged participants were asked to watch a ∼90 min movie and provide yes/no answers to detailed questions about different dimensions of the presented material after 1, 3 days, and 1 week. The results indicate that memory decay mainly affects the verbal dimension, both in terms of response accuracy and confidence. Instead, detailed information about objects/characters' features and spatiotemporal context seems to be relatively preserved, despite a general decrease in response confidence. Furthermore, younger adults were in general more accurate and confident than middle-aged participants, although, again, the verbal dimension exhibited a significant age-related difference. We propose that this selective forgetting depends on the progressive advantage of visual compared to auditory/verbal information in memory for complex events.

Exploring the Neural Processes behind Narrative Engagement: An EEG Study

Past cognitive neuroscience studies using naturalistic stimuli have considered narratives holistically and focused on cognitive processes. In this study, we incorporated the narrative structure, the dramatic arc, as an object of investigation, to examine how engagement levels fluctuate across a narrative-aligned dramatic arc. We explored the possibility of predicting self-reported engagement ratings from neural activity and investigated the idiosyncratic effects of each phase of the dramatic arc on brain responses as well as the relationship between engagement and brain responses. We presented a movie excerpt following the six-phase narrative arc structure to female and male participants while collecting EEG signals. We then asked this group of participants to recall the excerpt, another group to segment the video based on the dramatic arc model, and a third to rate their engagement levels while watching the movie. The results showed that the self-reported engagement ratings followed the pattern of the narrative dramatic arc. Moreover, while EEG amplitude could not predict group-averaged engagement ratings, other features comprising dynamic intersubject correlation (dISC), including certain frequency bands, dynamic functional connectivity patterns and graph features were able to achieve this. Furthermore, neural activity in the last two phases of the dramatic arc significantly predicted engagement patterns. This study is the first to explore the cognitive processes behind the dramatic arc and its phases. By demonstrating how neural activity predicts self-reported engagement, which itself aligns with the narrative structure, this study provides insights on the interrelationships between narrative structure, neural responses, and viewer engagement.

Social reappraisal of emotions is linked with the social presence effect in the default mode network

Introduction

Social reappraisal, during which one person deliberately tries to regulate another's emotions, is a powerful cognitive form of social emotion regulation, crucial for both daily life and psychotherapy. The neural underpinnings of social reappraisal include activity in the default mode network (DMN), but it is unclear how social processes influence the DMN and thereby social reappraisal functioning. We tested whether the mere presence of a supportive social regulator had an effect on the DMN during rest, and whether this effect in the DMN was linked with social reappraisal-related neural activations and effectiveness during negative emotions.

Methods

A two-part fMRI experiment was performed, with a psychotherapist as the social regulator, involving two resting state (social, non-social) and two task-related (social reappraisal, social no-reappraisal) conditions.

Results

The psychotherapist's presence enhanced intrinsic functional connectivity of the dorsal anterior cingulate (dACC) within the anterior medial DMN, with the effect positively related to participants' trust in psychotherapists. Secondly, the social presence-induced change in the dACC was related with (a) the social reappraisal-related activation in the bilateral dorsomedial/dorsolateral prefrontal cortex and the right temporoparietal junction and (b) social reappraisal success, with the latter relationship moderated by trust in psychotherapists.

Conclusion

Results demonstrate that a psychotherapist's supportive presence can change anterior medial DMN's intrinsic connectivity even in the absence of stimuli and that this DMN change during rest is linked with social reappraisal functioning during negative emotions. Data suggest that trust-dependent social presence effects on DMN states are relevant for social reappraisal-an idea important for daily-life and psychotherapy-related emotion regulation.

Representational dynamics of memories for real-life events

The continuous flow of experience that characterizes real-life events is not recorded as such in episodic memory but is condensed as a succession of event segments separated by temporal discontinuities. To unravel the neural basis of this representational structure, we recorded real-life events using wearable camera technology and used fMRI to investigate brain activity during their temporal unfolding in memory. We found that, compared to the representation of static scenes in memory, dynamically unfolding memory representations were associated with greater activation of the posterior medial episodic network. Strikingly, by analyzing the autocorrelation of brain activity patterns at successive time points throughout the retrieval period, we found that this network showed higher temporal dynamics when recalling events that included a higher density of event segments. These results reveal the key role of the posterior medial network in representing the dynamic unfolding of the event segments that constitute real-world memories.

A partially nested cortical hierarchy of neural states underlies event segmentation in the human brain

A fundamental aspect of human experience is that it is segmented into discrete events. This may be underpinned by transitions between distinct neural states. Using an innovative data-driven state segmentation method, we investigate how neural states are organized across the cortical hierarchy and where in the cortex neural state boundaries and perceived event boundaries overlap. Our results show that neural state boundaries are organized in a temporal cortical hierarchy, with short states in primary sensory regions, and long states in lateral and medial prefrontal cortex. State boundaries are shared within and between groups of brain regions that resemble well-known functional networks. Perceived event boundaries overlap with neural state boundaries across large parts of the cortical hierarchy, particularly when those state boundaries demarcate a strong transition or are shared between brain regions. Taken together, these findings suggest that a partially nested cortical hierarchy of neural states forms the basis of event segmentation.

Narrative thinking lingers in spontaneous thought

Some experiences linger in mind, spontaneously returning to our thoughts for minutes after their conclusion. Other experiences fall out of mind immediately. It remains unclear why. We hypothesize that an input is more likely to persist in our thoughts when it has been deeply processed: when we have extracted its situational meaning rather than its physical properties or low-level semantics. Here, participants read sequences of words with different levels of coherence (word-, sentence-, or narrative-level). We probe participants’ spontaneous thoughts via free word association, before and after reading. By measuring lingering subjectively (via self-report) and objectively (via changes in free association content), we find that information lingers when it is coherent at the narrative level. Furthermore, and an individual’s feeling of transportation into reading material predicts lingering better than the material’s objective coherence. Thus, our thoughts in the present moment echo prior experiences that have been incorporated into deeper, narrative forms of thinking.

Some experiences linger in our minds, while others quickly fade. Here, the authors show that the extent to which our recent experiences linger into subsequent thought increases as a function of processing depth.

Predicting memory from the network structure of naturalistic events

When we remember events, we often do not only recall individual events, but also the connections between them. However, extant research has focused on how humans segment and remember discrete events from continuous input, with far less attention given to how the structure of connections between events impacts memory. Here we conduct a functional magnetic resonance imaging study in which participants watch and recall a series of realistic audiovisual narratives. By transforming narratives into networks of events, we demonstrate that more central events-those with stronger semantic or causal connections to other events-are better remembered. During encoding, central events evoke larger hippocampal event boundary responses associated with memory formation. During recall, high centrality is associated with stronger activation in cortical areas involved in episodic recollection, and more similar neural representations across individuals. Together, these results suggest that when humans encode and retrieve complex real-world experiences, the reliability and accessibility of memory representations is shaped by their location within a network of events.

Charting cognition: Mapping public understanding of COVID-19

The COVID-19 pandemic of the last 2 years (and counting) disrupted commerce, travel, workplaces, habits, and-of course-health, the world over. This study aimed to capture snapshots of the perceptions and misperceptions of COVID-19 among 27 participants from three US municipalities. These perspectives are analyzed through thematic analyses and concept maps. Such snapshots, particularly as viewed through the lens of narrative sense-making theory, capture a sample of cognitions at this unique moment in history: a little over 1 year into the COVID-19 pandemic. Findings suggest that the (mis)perceptions captured are predominantly conveyed via narratives of participants' personal experiences, and that the themes of attitudes toward precautionary measures, uncertainty, and the muddied science communication environment are prevalent. These themes suggest several salient targets for future research and current science communication, such as a focus on basic explainers, vaccinations' safety and effectiveness and the necessity of uncertainty in the practice of science.

Rethinking the distinction between episodic and semantic memory: Insights from the past, present, and future

On the 50th anniversary of Tulving's introduction of the celebrated distinction between episodic and semantic memory, it seems more than fitting to revisit his proposal in light of recent conceptual and methodological advances in the field. This Special Issue of Memory & Cognition brings together researchers doing cutting-edge work at the intersection between episodic and semantic memory to showcase studies directly probing this psychological distinction, as well as articles that seek to provide conceptual and theoretical accounts to understand their interaction. The 14 articles presented here highlight the need to critically examine the way in which we conceptualize not only the relationship between episodic and semantic memory, but also the interplay between declarative and non-declarative memory, and the myriad implications of such conceptual changes. In many ways, we suggest this Special Issue might serve as a call to action for our field, inspiring future work to challenge pre-existing conceptions and stimulate new directions in this fast-moving field.

Fractal-Based Analysis of fMRI BOLD Signal During Naturalistic Viewing Conditions

Background: Temporal fractals are characterized by prominent scale-invariance and self-similarity across time scales. Monofractal analysis quantifies this scaling behavior in a single parameter, the Hurst exponent (H). Higher H reflects greater correlation in the signal structure, which is taken as being more fractal. Previous fMRI studies have observed lower H during conventional tasks relative to resting state conditions, and shown that H is negatively correlated with task difficulty and novelty. To date, no study has investigated the fractal dynamics of BOLD signal during naturalistic conditions. Methods: We performed fractal analysis on Human Connectome Project 7T fMRI data (n = 72, 41 females, mean age 29.46 ± 3.76 years) to compare H across movie-watching and rest. Results: In contrast to previous work using conventional tasks, we found higher H values for movie relative to rest (mean difference = 0.014; p = 5.279 × 10^-7; 95% CI [0.009, 0.019]). H was significantly higher in movie than rest in the visual, somatomotor and dorsal attention networks, but was significantly lower during movie in the frontoparietal and default networks. We found no cross-condition differences in test-retest reliability of H. Finally, we found that H of movie-derived stimulus properties (e.g., luminance changes) were fractal whereas H of head motion estimates were non-fractal. Conclusions: Overall, our findings suggest that movie-watching induces fractal signal dynamics. In line with recent work characterizing connectivity-based brain state dynamics during movie-watching, we speculate that these fractal dynamics reflect the configuring and reconfiguring of brain states that occurs during naturalistic processing, and are markedly different than dynamics observed during conventional tasks.

The effect of encoding task on the forgetting of object gist and details

One important feature of episodic memory is that it contains fine-grained and vividly recollected details. How to improve and maintain detailed information over time has been one of the central issues in memory research. Previous studies have inconsistent findings on whether detailed memory is forgotten more rapidly than gist memory. In this study, we investigated to what extent different encoding tasks modulated forgetting of gist and detailed information. In three experiments, participants were presented pictures of common objects and were asked to name them (Experiment 1), describe the details about them (Experiment 2) or imagine scenes associated with them (Experiment 3). After intervals of 10 minutes, one day, one week and one month, gist and detailed memories of the pictures were tested and assessed using a remember/know/guess judgement. The results showed that after the naming task, gist and detailed memories were forgotten at a similar rate, but after the description and the imagination tasks, detailed memory was forgotten at a slower rate than gist memory. The forgetting rate of gist memory was the slowest after the naming task, while that of detailed memory was the slowest after the description task. In addition, when three experiments were compared, the naming task enhanced the contributions of recollection and familiarity for gist memory, while the description task enhanced the contribution of familiarity for detailed memory. These results reveal the importance of the encoding task in the forgetting of gist and detailed information, and suggest a possible way to maintain perceptual details of objects at longer intervals.

Relating the Past with the Present: Information Integration and Segregation during Ongoing Narrative Processing

This study examined how the brain dynamically updates event representations by integrating new information over multiple minutes while segregating irrelevant input. A professional writer custom-designed a narrative with two independent storylines, interleaving across minute-long segments (ABAB). In the last (C) part, characters from the two storylines meet and their shared history is revealed. Part C is designed to induce the spontaneous recall of past events, upon the recurrence of narrative motifs from A/B, and to shed new light on them. Our fMRI results showed storyline-specific neural patterns, which were reinstated (i.e., became more active) during storyline transitions. This effect increased along the processing timescale hierarchy, peaking in the default mode network. Similarly, the neural reinstatement of motifs was found during Part C. Furthermore, participants showing stronger motif reinstatement performed better in integrating A/B and C events, demonstrating the role of memory reactivation in information integration over intervening irrelevant events.

The default mode network: where the idiosyncratic self meets the shared social world

The default mode network (DMN) is classically considered an 'intrinsic' system, specializing in internally oriented cognitive processes such as daydreaming, reminiscing and future planning. In this Perspective, we suggest that the DMN is an active and dynamic 'sense-making' network that integrates incoming extrinsic information with prior intrinsic information to form rich, context-dependent models of situations as they unfold over time. We review studies that relied on naturalistic stimuli, such as stories and movies, to demonstrate how an individual's DMN neural responses are influenced both by external information accumulated as events unfold over time and by the individual's idiosyncratic past memories and knowledge. The integration of extrinsic and intrinsic information over long timescales provides a space for negotiating a shared neural code, which is necessary for establishing shared meaning, shared communication tools, shared narratives and, above all, shared communities and social networks.