Representation of Real-World Event Schemas during Narrative Perception

The role of surface and structural similarities in the retrieval of realistic perceptual events

This study investigated whether structural similarities (i.e. abstract frames, e.g. once bitten twice shy) can prevail over surface similarities (i.e. contexts, e.g. restaurant) in driving the retrieval of realistic events involving dynamic, multimodal and perceptually crowded data. After watching an initial set of video clips, participants had to indicate whether a new video clip, that shared surface similarities with an initial event and structural similarities with another one, elicited a retrieval. The results of Experiment 1A showed that retrieval was more likely to be elicited by structural rather than by surface similarities. Experiment 1B confirmed that the surface similarities manipulated in this study were strong enough to elicit substantial retrievals when the competing structural match was neutralized. The pattern of results obtained in Experiment 1A remained unchanged when the number of unrelated video clips within the initial set was increased. The findings suggest that structurally based retrievals still prevail when familiar structures underlie realistic perceptual events. They open new perspectives regarding the settings that promote structurally based retrievals in educational contexts where unfamiliar principles are introduced.

Uncovering narrative aging: an underlying neural mechanism compensated through spatial constructional ability

"The narrative" is a complex cognitive process that has sparked a debate on whether its features age through maintenance or decline. To address this question, we attempted to uncover the narrative aging and its underlying neural characteristics with a cross-validation based cognitive neuro-decoding statistical framework. This framework used a total of 740 healthy older participants with completed narrative and extensive neuropsychological tests and MRI scans. The results indicated that narrative comprises macro and micro structures, with the macrostructure involving complex cognitive processes more relevant to aging. For the brain functional basis, brain hub nodes contributing to macrostructure were predominantly found in the angular gyrus and medial frontal lobe, while microstructure hub nodes were located in the supramarginal gyrus and middle cingulate cortex. Moreover, networks enriched by macrostructure included the default mode network and fronto-parietal network, indicating a higher functional gradient compared to the microstructure-enriched dorsal attention network. Additionally, an interesting finding showed that macrostructure increases in spatial contribution with age, suggesting a compensatory interaction where brain regions related to spatial-constructional ability have a greater impact on macrostructure. These results, supported by neural-level validation and multimodal structural MRI, provide detailed insights into the compensatory effect in the narrative aging process.

Schemas, reinforcement learning and the medial prefrontal cortex

Schemas are rich and complex knowledge structures about the typical unfolding of events in a context; for example, a schema of a dinner at a restaurant. In this Perspective, we suggest that reinforcement learning (RL), a computational theory of learning the structure of the world and relevant goal-oriented behaviour, underlies schema learning. We synthesize literature about schemas and RL to offer that three RL principles might govern the learning of schemas: learning via prediction errors, constructing hierarchical knowledge using hierarchical RL, and dimensionality reduction through learning a simplified and abstract representation of the world. We then suggest that the orbitomedial prefrontal cortex is involved in both schemas and RL due to its involvement in dimensionality reduction and in guiding memory reactivation through interactions with posterior brain regions. Last, we hypothesize that the amount of dimensionality reduction might underlie gradients of involvement along the ventral-dorsal and posterior-anterior axes of the orbitomedial prefrontal cortex. More specific and detailed representations might engage the ventral and posterior parts, whereas abstraction might shift representations towards the dorsal and anterior parts of the medial prefrontal cortex.

A core set of neural states underlying memory reactivation of naturalistic events in the posterior medial cortex

In sensory and mid-level regions of the brain, stimulus information is often topographically organized; functional responses are arranged in maps according to features such as retinal coordinates, auditory pitch, and object animacy or size. However, such organization is typically measured during stimulus input, e.g., when subjects are viewing gratings or images. Much less is known about the possible spatial organization of function during episodic recall of real-world events, which seems to drive higher-order cortical regions in the default mode network, particularly in posterior midline areas. Prior studies have shown that when multiple people remember a common experience, event-specific activity patterns in the posterior medial cortex are similar across individuals. This indicates that spatially organized functional responses underlying episodic recall do exist. In this paper we leverage fMRI data collected during recall of naturalistic movies to identify a core set of neural states in the posterior medial cortex. These states are stimulus-locked, reactivated during recall, and have a shared spatial organization across brains. We show that a surprisingly small number of these states (16 states across hemispheres) is sufficient to achieve the same levels of reactivation in the posterior medial cortex as when using the standard methods of the field. Furthermore, these states are significantly related to actions and social-affective features of events in the movies. Together, these findings elucidate the properties of a spatially organized code within the posterior default mode network which appears during natural recollection of memories.

Cultural life scripts as schema: recalling schema congruent and incongruent events from a hypothetical life story

This study, involving 34 Malaysian adults, investigated if the memory of cultural life script congruent and incongruent events was better than life script neutral events. We created a 1500-word hypothetical life story of a Malaysian girl comprising six life script congruent, six life script incongruent, and six life script neutral events. Half of those events were high-frequent, and half were low-frequent life script events. Participants first read the story on a computer screen and then completed a free recall test ten minutes later and again one week later. They recalled as many details as they could from the 18 events presented in the story. The results revealed that participants retrieved more information from the life script incongruent events than from life script congruent events at both recall points. The memory for high-frequent life script events was better than for low-frequent and life script-neutral events. Overall, the recall rate was higher in the immediate than in the late phase. The results confirm the isolation effects or Von Restorff effect, which predicts that unusual and distinctive events are more memorable than typical, commonly expected events.

Effects of Language Proficiency on Selective Attention Patterns at Segmenting Boundaries in English Audio Sentences

BACKGROUND/OBJECTIVES

Normative perceptual segmentation facilitates event perception, comprehension, and memory. Given that native English listeners' normative perceptual segmentation of English speech streams coexists with a highly selective attention pattern at segmentation boundaries, it is significant to test whether Chinese learners of English have a different attention pattern at boundaries, thereby checking whether they perform a normative segmentation.

METHODS

Thirty Chinese learners of English with relatively higher language proficiency (CLH) and 26 with relatively lower language proficiency (CLL) listened to a series of English audio sentences. Meanwhile, they were asked to press the key whenever a phonetic probe "ba" occurred. Response time to "ba" reflects the attention where "ba" is located at segmentation boundaries.

RESULTS

The results revealed that, (1) relative to native English listeners' highly selective attention pattern, the CLH group showed a relatively selective attention pattern, while the CLL group displayed a non-selective attention pattern. (2) Both the CLH group and natives had better recognition memory than the CLL group. (3) Both the CLH group and natives' attention at segmentation boundaries was not correlated with their memory for sentences, while the CLL group's attention at boundaries was correlated with memory.

CONCLUSIONS

These findings suggest that (1) Chinese learners of English did not perform a normative segmentation, which shows the effect of English proficiency on perceptual segmentation; (2) English proficiency has a superior effect on memory for sentences, while perceptual segmentation would come next to support memory by providing structure for memory construction if English proficiency is not high; (3) a comparison of attention patterns between Chinese learners and natives can provide a reference for potential intervention to rectify non-natives, thereby improving their perception of English speech streams.

How sturdy is your memory palace? Reliable room representations predict subsequent reinstatement of placed objects

Our autobiographical experiences typically occur within the context of familiar spatial locations. When we encode these experiences into memory, we can use our spatial map of the world to help organize these memories and later retrieve their episodic details. However, it is still not well understood what psychological and neural factors make spatial contexts an effective scaffold for storing and accessing memories. We hypothesized that spatial locations with distinctive and stable neural representations would best support the encoding and robust reinstatement of new episodic memories. We developed a novel paradigm that allowed us to quantify the within-participant reliability of a spatial context ("room reliability") prior to memory encoding, which could then be used to predict the degree of successful re-activation of item memories. To do this, we constructed a virtual reality (VR) "memory palace", a custom-built environment made up of 23 distinct rooms that participants explored using a head-mounted VR display. The day after learning the layout of the environment, participants underwent whole-brain fMRI while being presented with videos of the rooms in the memory palace, allowing us to measure the reliability of the neural activity pattern associated with each room. Participants were taken back to VR and asked to memorize the locations of 23 distinct objects randomly placed within each of the 23 rooms, and then returned to the scanner as they recalled the objects and the rooms in which they appeared. We found that our room reliability measure was predictive of object reinstatement across cortex, and further showed that this was driven not only by the group-level reliability of a room across participants, but also the idiosyncratic reliability of rooms within each participant. Together, these results showcase how the quality of the neural representation of a spatial context can be quantified and used to 'audit' its utility as a memory scaffold for future experiences.

Narrative predicts cardiac synchrony in audiences

Audio-visual media possesses a remarkable ability to synchronise audiences' neural, behavioural, and physiological responses. This synchronisation is considered to reflect some dimension of collective attention or engagement with the stimulus. But what is it about these stimuli that drives such strong engagement? There are several properties of media stimuli which may lead to synchronous audience response: from low-level audio-visual features, to the story itself. Here, we present a study which separates low-level features from narrative by presenting participants with the same content but in separate modalities. In this way, the presentations shared no low-level features, but participants experienced the same narrative. We show that synchrony in participants' heart rate can be driven by the narrative information alone. We computed both visual and auditory perceptual saliency for the content and found that narrative was approximately 10 times as predictive of heart rate as low-level saliency, but that low-level audio-visual saliency has a small additive effect towards heart rate. Further, heart rate synchrony was related to a separate cohorts' continuous ratings of immersion, and that synchrony is likely to be higher at moments of increased narrative importance. Our findings demonstrate that high-level narrative dominates in the alignment of physiology across viewers.

Goal Shifts Structure Memories and Prioritize Event-defining Information in Memory

Every day, we encounter far more information than we could possibly remember. Thus, our memory systems must organize and prioritize the details from an experience that can adaptively guide the storage and retrieval of specific episodic events. Prior work has shown that shifts in internal goal states can function as event boundaries, chunking experiences into distinct and memorable episodes. In addition, at short delays, memory for contextual information at boundaries has been shown to be enhanced compared with items within each event. However, it remains unclear if these memory enhancements are limited to features that signal a meaningful transition between events. To determine how changes in dynamic goal states influence the organization and content of long-term memory, we designed a 2-day experiment in which participants viewed a series of black-and-white objects surrounded by a color border on a two-by-two grid. The location of the object on the grid determined which of two tasks participants performed on a given trial. To examine if distinct types of goal shifts modulate the effects of event segmentation, we changed the border color, the task, or both after every four items in a sequence. We found that goal shifts influenced temporal memory in a manner consistent with the formation of distinct events. However, for subjective memory representations in particular, these effects differed by the type of event boundary. Furthermore, to examine if goal shifts lead to the prioritization of goal-relevant features in longer lasting memories, we tested source memory for each object's color and grid location both immediately and after a 24-hr delay. On the immediate test, boundaries enhanced the memory for all concurrent source features compared with nonboundary items, but only if those boundaries involved a goal shift. In contrast, after a delay, the source memory was selectively enhanced for the feature relevant to the goal shift. These findings suggest that goals can adaptively structure memories by prioritizing contextual features that define a unique episode in memory.

Causal and Chronological Relationships Predict Memory Organization for Nonlinear Narratives

While recounting an experience, one can employ multiple strategies to transition from one part to the next. For instance, if the event was learned out of linear order, one can recall events according to the time they were learned (temporal), similar events (semantic), events occurring nearby in time (chronological), or events produced by the current event (causal). To disentangle the importance of these factors, we had participants watch the nonlinear narrative, Memento, under different task instructions and presentation orders. For each scene of the film, we also separately computed semantic and causal networks. We then contrasted the evidence for temporal, semantic, chronological, or causal strategies during recall. Critically, there was stronger evidence for the causal and chronological strategies than semantic or temporal strategies. Moreover, the causal and chronological strategies outperformed the temporal one even when we asked participants to recall the film in the presented order, underscoring the fundamental nature of causal structure in scaffolding understanding and organizing recall. Nevertheless, time still marginally predicted recall transitions, suggesting it operates as a weak signal in the presence of more salient forms of structure. In addition, semantic and causal network properties predicted scene memorability, including a stronger role for incoming causes to an event than its outgoing effects. In summary, these findings highlight the importance of accounting for complex, causal networks in knowledge building and memory.

Top-down attention shifts behavioral and neural event boundaries in narratives with overlapping event scripts

Understanding and remembering the complex experiences of everyday life relies critically on prior schematic knowledge about how events in our world unfold over time. How does the brain construct event representations from a library of schematic scripts, and how does activating a specific script impact the way that events are segmented in time? We developed a novel set of 16 audio narratives, each of which combines one of four location-relevant event scripts (restaurant, airport, grocery store, and lecture hall) with one of four socially relevant event scripts (breakup, proposal, business deal, and meet cute), and presented them to participants in an fMRI study and a separate online study. Responses in the angular gyrus, parahippocampal gyrus, and subregions of the medial prefrontal cortex (mPFC) were driven by scripts related to both location and social information, showing that these regions can track schematic sequences from multiple domains. For some stories, participants were primed to attend to one of the two scripts by training them to listen for and remember specific script-relevant episodic details. Activating a location-related event script shifted the timing of subjective event boundaries to align with script-relevant changes in the narratives, and this behavioral shift was mirrored in the timing of neural responses, with mPFC event boundaries (identified using a hidden Markov model) aligning to location-relevant rather than socially relevant boundaries when participants were location primed. Our findings demonstrate that neural event dynamics are actively modulated by top-down goals and provide new insight into how narrative event representations are constructed through the activation of temporally structured prior knowledge.

Distinct neural pathway and its information flow for blind individual's Braille reading

Natural Braille reading presents significant challenges to the brain networks of late blind individuals, yet its underlying neural mechanisms remain largely unexplored. Using natural Braille texts in behavioral assessments and functional MRI, we sought to pinpoint the neural pathway and information flow crucial for Braille reading performance in late blind individuals. In the resting state, we discovered a unique neural connection between the higher-order 'visual' cortex, the lateral occipital cortex (LOC), and the inferior frontal cortex (IFC) in late blind individuals, but not in sighted controls. The left-lateralized LOC-IFC connectivity was correlated with individual Braille reading proficiency. Prolonged Braille reading practice led to increased strength of this connectivity. During a natural Braille reading task, bidirectional information flow between the LOC and the IFC was positively modulated, with a predominantly stronger top-down modulation from the IFC to the LOC. This stronger top-down modulation contributed to higher Braille reading proficiency. We thus proposed a two-predictor multiple regression model to predict individual Braille reading proficiency, incorporating both static connectivity and dynamic top-down communication between the LOC-IFC link. This work highlights the dual contributions of the occipito-frontal neural pathway and top-down cognitive strategy to superior natural Braille reading performance, offering guidance for training late blind individuals.

Temporal asymmetries in inferring unobserved past and future events

Unlike temporally symmetric inferences about simple sequences, inferences about our own lives are asymmetric: we are better able to infer the past than the future, since we remember our past but not our future. Here we explore whether there are asymmetries in inferences about the unobserved pasts and futures of other people's lives. In two experiments (analyses of the replication experiment were pre-registered), our participants view segments of two character-driven television dramas and write out what they think happens just before or after each just-watched segment. Participants are better at inferring unseen past (versus future) events. This asymmetry is driven by participants' reliance on characters' conversational references in the narrative, which tend to favor the past. This tendency is also replicated in a large-scale analysis of conversational references in natural conversations. Our work reveals a temporal asymmetry in how observations of other people's behaviors can inform inferences about the past and future.

Effective Connectivity Network of Aberrant Prediction Error Processing in Auditory Phantom Perception

Introduction: Prediction error (PE) is key to perception in the predictive coding framework. However, previous studies indicated the varied neural activities evoked by PE in tinnitus patients. Here, we aimed to reconcile the conflict by (1) a more nuanced view of PE, which could be driven by changing stimulus (stimulus-driven PE [sPE]) and violation of current context (context-driven PE [cPE]) and (2) investigating the aberrant connectivity networks that are engaged in the processing of the two types of PEs in tinnitus patients. Methods: Ten tinnitus patients with normal hearing and healthy controls were recruited, and a local-global auditory oddball paradigm was applied to measure the electroencephalographic difference between the two groups during sPE and cPE conditions. Results: Overall, the sPE condition engaged bottom-up and top-down connections, whereas the cPE condition engaged mostly top-down connections. The tinnitus group showed decreased sensitivity to the sPE and increased sensitivity to the cPE condition. Particularly, the auditory cortex and posterior cingulate cortex were the hubs for processing cPE in the control and tinnitus groups, respectively, showing the orientation to an internal state in tinnitus. Furthermore, tinnitus patients showed stronger connectivity to the parahippocampus and pregenual anterior cingulate cortex for the establishment of the prediction during the cPE condition. Conclusion: These results begin to dissect the role of changes in stimulus characteristics versus changes in the context of processing the same stimulus in mechanisms of tinnitus generation. Impact Statement This study delves into the number dynamics of prediction error (PE) in tinnitus, proposing a dual framework distinguishing between stimulus-driven PE (sPE) and context-driven PE (cPE). Electroencephalographic data from tinnitus patients and controls revealed distinct connectivity patterns during sPE and cPE conditions. Tinnitus patients exhibited reduced sensitivity to sPE and increased sensitivity to cPE. The auditory cortex and posterior cingulate cortex emerged as pivotal regions for cPE processing in controls and tinnitus patients, indicative of an internal state orientation in tinnitus. Enhanced connectivity to the parahippocampus and pregenual anterior cingulate cortex underscores the role of context in tinnitus pathophysiology.

How a speaker herds the audience: multibrain neural convergence over time during naturalistic storytelling

Storytelling-an ancient way for humans to share individual experiences with others-has been found to induce neural alignment among listeners. In exploring the dynamic fluctuations in listener-listener (LL) coupling throughout stories, we uncover a significant correlation between LL coupling and lagged speaker-listener (lag-SL) coupling over time. Using the analogy of neural pattern (dis)similarity as distances between participants, we term this phenomenon the "herding effect." Like a shepherd guiding a group of sheep, the more closely listeners mirror the speaker's preceding brain activity patterns (higher lag-SL similarity), the more tightly they cluster (higher LL similarity). This herding effect is particularly pronounced in brain regions where neural alignment among listeners tracks with moment-by-moment behavioral ratings of narrative content engagement. By integrating LL and SL neural coupling, this study reveals a dynamic, multibrain functional network between the speaker and the audience, with the unfolding narrative content playing a mediating role in network configuration.

What we mean when we say semantic: Toward a multidisciplinary semantic glossary

Tulving characterized semantic memory as a vast repository of meaning that underlies language and many other cognitive processes. This perspective on lexical and conceptual knowledge galvanized a new era of research undertaken by numerous fields, each with their own idiosyncratic methods and terminology. For example, "concept" has different meanings in philosophy, linguistics, and psychology. As such, many fundamental constructs used to delineate semantic theories remain underspecified and/or opaque. Weak construct specificity is among the leading causes of the replication crisis now facing psychology and related fields. Term ambiguity hinders cross-disciplinary communication, falsifiability, and incremental theory-building. Numerous cognitive subdisciplines (e.g., vision, affective neuroscience) have recently addressed these limitations via the development of consensus-based guidelines and definitions. The project to follow represents our effort to produce a multidisciplinary semantic glossary consisting of succinct definitions, background, principled dissenting views, ratings of agreement, and subjective confidence for 17 target constructs (e.g., abstractness, abstraction, concreteness, concept, embodied cognition, event semantics, lexical-semantic, modality, representation, semantic control, semantic feature, simulation, semantic distance, semantic dimension). We discuss potential benefits and pitfalls (e.g., implicit bias, prescriptiveness) of these efforts to specify a common nomenclature that other researchers might index in specifying their own theoretical perspectives (e.g., They said X, but I mean Y).

Partisans process policy-based and identity-based messages using dissociable neural systems

Political partisanship is often conceived as a lens through which people view politics. Behavioral research has distinguished two types of "partisan lenses"-policy-based and identity-based-that may influence peoples' perception of political events. Little is known, however, about the mechanisms through which partisan discourse appealing to policy beliefs or targeting partisan identities operate within individuals. We addressed this question by collecting neuroimaging data while participants watched videos of speakers expressing partisan views. A "partisan lens effect" was identified as the difference in neural synchrony between each participant's brain response and that of their partisan ingroup vs. outgroup. When processing policy-based messaging, a partisan lens effect was observed in socio-political reasoning and affective responding brain regions. When processing negative identity-based attacks, a partisan lens effect was observed in mentalizing and affective responding brain regions. These data suggest that the processing of political discourse that appeals to different forms of partisanship is supported by related but distinguishable neural-and therefore psychological-mechanisms, which may have implications for how we characterize partisanship and ameliorate its deleterious impacts.

Replay-triggered brain-wide activation in humans

The consolidation of discrete experiences into a coherent narrative shapes the cognitive map, providing structured mental representations of our experiences. In this process, past memories are reactivated and replayed in sequence, fostering hippocampal-cortical dialogue. However, brain-wide engagement coinciding with sequential reactivation (or replay) of memories remains largely unexplored. In this study, employing simultaneous EEG-fMRI, we capture both the spatial and temporal dynamics of memory replay. We find that during mental simulation, past memories are replayed in fast sequences as detected via EEG. These transient replay events are associated with heightened fMRI activity in the hippocampus and medial prefrontal cortex. Replay occurrence strengthens functional connectivity between the hippocampus and the default mode network, a set of brain regions key to representing the cognitive map. On the other hand, when subjects are at rest following learning, memory reactivation of task-related items is stronger than that of pre-learning rest, and is also associated with heightened hippocampal activation and augmented hippocampal connectivity to the entorhinal cortex. Together, our findings highlight a distributed, brain-wide engagement associated with transient memory reactivation and its sequential replay.

Updating predictions in a complex repertoire of actions and its neural representation

Even though actions we observe in everyday life seem to unfold in a continuous manner, they are automatically divided into meaningful chunks, that are single actions or segments, which provide information for the formation and updating of internal predictive models. Specifically, boundaries between actions constitute a hub for predictive processing since the prediction of the current action comes to an end and calls for updating of predictions for the next action. In the current study, we investigated neural processes which characterize such boundaries using a repertoire of complex action sequences with a predefined probabilistic structure. Action sequences consisted of actions that started with the hand touching an object (T) and ended with the hand releasing the object (U). These action boundaries were determined using an automatic computer vision algorithm. Participants trained all action sequences by imitating demo videos. Subsequently, they returned for an fMRI session during which the original action sequences were presented in addition to slightly modified versions thereof. Participants completed a post-fMRI memory test to assess the retention of original action sequences. The exchange of individual actions, and thus a violation of action prediction, resulted in increased activation of the action observation network and the anterior insula. At U events, marking the end of an action, increased brain activation in supplementary motor area, striatum, and lingual gyrus was indicative of the retrieval of the previously encoded action repertoire. As expected, brain activation at U events also reflected the predefined probabilistic branching structure of the action repertoire. At T events, marking the beginning of the next action, midline and hippocampal regions were recruited, reflecting the selected prediction of the unfolding action segment. In conclusion, our findings contribute to a better understanding of the various cerebral processes characterizing prediction during the observation of complex action repertoires.

The causal structure and computational value of narratives

Many human behavioral and brain imaging studies have used narratively structured stimuli (e.g., written, audio, or audiovisual stories) to better emulate real-world experience in the laboratory. However, narratives are a special class of real-world experience, largely defined by their causal connections across time. Much contemporary neuroscience research does not consider this key property. We review behavioral and neuroscientific work that speaks to how causal structure shapes comprehension of and memory for narratives. We further draw connections between this work and reinforcement learning, highlighting how narratives help link causes to outcomes in complex environments. By incorporating the plausibility of causal connections between classes of actions and outcomes, reinforcement learning models may become more ecologically valid, while simultaneously elucidating the value of narratives.

The Complex Role Played by the Default Mode Network during Sexual Stimulation: A Cluster-Based fMRI Meta-Analysis

The default mode network (DMN) is a complex network that plays a significant and active role during naturalistic stimulation. Previous studies that have used naturalistic stimuli, such as real-life stories or silent or sonorous films, have found that the information processing involved a complex hierarchical set of brain regions, including the DMN nodes. The DMN is not involved in low-level features and is only associated with high-level content-related incoming information. The human sexual experience involves a complex set of processes related to both external context and inner processes. Since the DMN plays an active role in the integration of naturalistic stimuli and aesthetic perception with beliefs, thoughts, and episodic autobiographical memories, we aimed at quantifying the involvement of the nodes of the DMN during visual sexual stimulation. After a systematic search in the principal electronic databases, we selected 83 fMRI studies, and an ALE meta-analysis was calculated. We performed conjunction analyses to assess differences in the DMN related to stimulus modalities, sex differences, and sexual orientation. The results show that sexual stimulation alters the topography of the DMN and highlights the DMN's active role in the integration of sexual stimuli with sexual schemas and beliefs.

Familiarity enhances functional connectivity between visual and nonvisual regions of the brain during natural viewing

We explored the neural correlates of familiarity with people and places using a naturalistic viewing paradigm. Neural responses were measured using functional magnetic resonance imaging, while participants viewed a movie taken from Game of Thrones. We compared inter-subject correlations and functional connectivity in participants who were either familiar or unfamiliar with the TV series. Higher inter-subject correlations were found between familiar participants in regions, beyond the visual brain, that are typically associated with the processing of semantic, episodic, and affective information. However, familiarity also increased functional connectivity between face and scene regions in the visual brain and the nonvisual regions of the familiarity network. To determine whether these regions play an important role in face recognition, we measured responses in participants with developmental prosopagnosia (DP). Consistent with a deficit in face recognition, the effect of familiarity was significantly attenuated across the familiarity network in DP. The effect of familiarity on functional connectivity between face regions and the familiarity network was also attenuated in DP. These results show that the neural response to familiarity involves an extended network of brain regions and that functional connectivity between visual and nonvisual regions of the brain plays an important role in the recognition of people and places during natural viewing.

Shared functional specialization in transformer-based language models and the human brain

When processing language, the brain is thought to deploy specialized computations to construct meaning from complex linguistic structures. Recently, artificial neural networks based on the Transformer architecture have revolutionized the field of natural language processing. Transformers integrate contextual information across words via structured circuit computations. Prior work has focused on the internal representations ("embeddings") generated by these circuits. In this paper, we instead analyze the circuit computations directly: we deconstruct these computations into the functionally-specialized "transformations" that integrate contextual information across words. Using functional MRI data acquired while participants listened to naturalistic stories, we first verify that the transformations account for considerable variance in brain activity across the cortical language network. We then demonstrate that the emergent computations performed by individual, functionally-specialized "attention heads" differentially predict brain activity in specific cortical regions. These heads fall along gradients corresponding to different layers and context lengths in a low-dimensional cortical space.

Linguistic inputs must be syntactically parsable to fully engage the language network

Human language comprehension is remarkably robust to ill-formed inputs (e.g., word transpositions). This robustness has led some to argue that syntactic parsing is largely an illusion, and that incremental comprehension is more heuristic, shallow, and semantics-based than is often assumed. However, the available data are also consistent with the possibility that humans always perform rule-like symbolic parsing and simply deploy error correction mechanisms to reconstruct ill-formed inputs when needed. We put these hypotheses to a new stringent test by examining brain responses to a) stimuli that should pose a challenge for syntactic reconstruction but allow for complex meanings to be built within local contexts through associative/shallow processing (sentences presented in a backward word order), and b) grammatically well-formed but semantically implausible sentences that should impede semantics-based heuristic processing. Using a novel behavioral syntactic reconstruction paradigm, we demonstrate that backward-presented sentences indeed impede the recovery of grammatical structure during incremental comprehension. Critically, these backward-presented stimuli elicit a relatively low response in the language areas, as measured with fMRI. In contrast, semantically implausible but grammatically well-formed sentences elicit a response in the language areas similar in magnitude to naturalistic (plausible) sentences. In other words, the ability to build syntactic structures during incremental language processing is both necessary and sufficient to fully engage the language network. Taken together, these results provide strongest to date support for a generalized reliance of human language comprehension on syntactic parsing.

The Language Network Reliably "Tracks" Naturalistic Meaningful Nonverbal Stimuli

The language network, comprised of brain regions in the left frontal and temporal cortex, responds robustly and reliably during language comprehension but shows little or no response during many nonlinguistic cognitive tasks (e.g., Fedorenko & Blank, 2020). However, one domain whose relationship with language remains debated is semantics-our conceptual knowledge of the world. Given that the language network responds strongly to meaningful linguistic stimuli, could some of this response be driven by the presence of rich conceptual representations encoded in linguistic inputs? In this study, we used a naturalistic cognition paradigm to test whether the cognitive and neural resources that are responsible for language processing are also recruited for processing semantically rich nonverbal stimuli. To do so, we measured BOLD responses to a set of ∼5-minute-long video and audio clips that consisted of meaningful event sequences but did not contain any linguistic content. We then used the intersubject correlation (ISC) approach (Hasson et al., 2004) to examine the extent to which the language network "tracks" these stimuli, that is, exhibits stimulus-related variation. Across all the regions of the language network, meaningful nonverbal stimuli elicited reliable ISCs. These ISCs were higher than the ISCs elicited by semantically impoverished nonverbal stimuli (e.g., a music clip), but substantially lower than the ISCs elicited by linguistic stimuli. Our results complement earlier findings from controlled experiments (e.g., Ivanova et al., 2021) in providing further evidence that the language network shows some sensitivity to semantic content in nonverbal stimuli.

Integrating and fragmenting memories under stress and alcohol

Stress can powerfully influence the way we form memories, particularly the extent to which they are integrated or situated within an underlying spatiotemporal and broader knowledge architecture. These different representations in turn have significant consequences for the way we use these memories to guide later behavior. Puzzlingly, although stress has historically been argued to promote fragmentation, leading to disjoint memory representations, more recent work suggests that stress can also facilitate memory binding and integration. Understanding the circumstances under which stress fosters integration will be key to resolving this discrepancy and unpacking the mechanisms by which stress can shape later behavior. Here, we examine memory integration at multiple levels: linking together the content of an individual experience, threading associations between related but distinct events, and binding an experience into a pre-existing schema or sense of causal structure. We discuss neural and cognitive mechanisms underlying each form of integration as well as findings regarding how stress, aversive learning, and negative affect can modulate each. In this analysis, we uncover that stress can indeed promote each level of integration. We also show how memory integration may apply to understanding effects of alcohol, highlighting extant clinical and preclinical findings and opportunities for further investigation. Finally, we consider the implications of integration and fragmentation for later memory-guided behavior, and the importance of understanding which type of memory representation is potentiated in order to design appropriate interventions.

Taking time to compose thoughts with prefrontal schemata

Under what conditions can prefrontal cortex direct the composition of brain states, to generate coherent streams of thoughts? Using a simplified Potts model of cortical dynamics, crudely differentiated into two halves, we show that once activity levels are regulated, so as to disambiguate a single temporal sequence, whether the contents of the sequence are mainly determined by the frontal or by the posterior half, or by neither, depends on statistical parameters that describe its microcircuits. The frontal cortex tends to lead if it has more local attractors, longer lasting and stronger ones, in order of increasing importance. Its guidance is particularly effective to the extent that posterior cortices do not tend to transition from state to state on their own. The result may be related to prefrontal cortex enforcing its temporally-oriented schemata driving coherent sequences of brain states, unlike the atemporal "context" contributed by the hippocampus. Modelling a mild prefrontal (vs. posterior) lesion offers an account of mind-wandering and event construction deficits observed in prefrontal patients.

The language network as a natural kind within the broader landscape of the human brain

Language behaviour is complex, but neuroscientific evidence disentangles it into distinct components supported by dedicated brain areas or networks. In this Review, we describe the 'core' language network, which includes left-hemisphere frontal and temporal areas, and show that it is strongly interconnected, independent of input and output modalities, causally important for language and language-selective. We discuss evidence that this language network plausibly stores language knowledge and supports core linguistic computations related to accessing words and constructions from memory and combining them to interpret (decode) or generate (encode) linguistic messages. We emphasize that the language network works closely with, but is distinct from, both lower-level - perceptual and motor - mechanisms and higher-level systems of knowledge and reasoning. The perceptual and motor mechanisms process linguistic signals, but, in contrast to the language network, are sensitive only to these signals' surface properties, not their meanings; the systems of knowledge and reasoning (such as the system that supports social reasoning) are sometimes engaged during language use but are not language-selective. This Review lays a foundation both for in-depth investigations of these different components of the language processing pipeline and for probing inter-component interactions.

Remembering what we imagine: the role of event schemas in shaping how imagined autobiographical events are recalled

Much like recalling autobiographical memories, constructing imagined autobiographical events depends on episodic memory processes. The ability to imagine events contributes to several future-oriented behaviors (e.g., decision-making, problem solving), which relies, in part, on the ability to remember the imagined events. A factor affecting the memorability of such events is their adherence to event schemas-conceptualizations of how events generally unfold. In the current study, we examined how two aspects of event schemas-event expectancy and familiarity-affect the ability to recall imagined events. Participants first imagined and described in detail autobiographical events that either aligned with or deviated from an event, expected to occur in a context (e.g., a kitchen) that was either familiar or unfamiliar. This resulted in imaginations ranging from maximally schema-congruent (expected events in a familiar context) to maximally novel (unexpected events in an unfamiliar context). Twenty-four hours later, participants recalled these imagined events. Recollections were scored for the number of reinstated details from the imaginations and the number of newly added details. We found greater reinstatement of details for both the maximally congruent and maximally novel events, while maximally novel events were recalled more precisely than other events (i.e., fewer added details). Our results indicate a complementary benefit to remembering schematic and novel imagined events, which may guide equally important but distinct future-oriented behaviors.

Dissociable Neural Mechanisms for Human Inference Processing Predicted by Static and Contextual Language Models

Language models (LMs) continue to reveal non-trivial relations to human language performance and the underlying neurophysiology. Recent research has characterized how word embeddings from an LM can be used to generate integrated discourse representations in order to perform inference on events. The current research investigates how such event knowledge may be coded in distinct manners in different classes of LMs and how this maps onto different forms of human inference processing. To do so, we investigate inference on events using two well-documented human experimental protocols from Metusalem et al. (2012) and McKoon and Ratcliff (1986), compared with two protocols for simpler semantic processing. Interestingly, this reveals a dissociation in the relation between local semantics versus event-inference depending on the LM. In a series of experiments, we observed that for the static LMs (word2vec/GloVe), there was a clear dissociation in the relation between semantics and inference for the two inference tasks. In contrast, for the contextual LMs (BERT/RoBERTa), we observed a correlation between semantic and inference processing for both inference tasks. The experimental results suggest that inference as measured by Metusalem and McKoon rely on dissociable processes. While the static models are able to perform Metusalem inference, only the contextual models succeed in McKoon inference. Interestingly, these dissociable processes may be linked to well-characterized automatic versus strategic inference processes in the psychological literature. This allows us to make predictions about dissociable neurophysiological markers that should be found during human inference processing with these tasks.

The dorsomedial prefrontal cortex prioritizes social learning during rest

Sociality is a defining feature of the human experience: We rely on others to ensure survival and cooperate in complex social networks to thrive. Are there brain mechanisms that help ensure we quickly learn about our social world to optimally navigate it? We tested whether portions of the brain's default network engage "by default" to quickly prioritize social learning during the memory consolidation process. To test this possibility, participants underwent functional MRI (fMRI) while viewing scenes from the documentary film, Samsara. This film shows footage of real people and places from around the world. We normed the footage to select scenes that differed along the dimension of sociality, while matched on valence, arousal, interestingness, and familiarity. During fMRI, participants watched the "social" and "nonsocial" scenes, completed a rest scan, and a surprise recognition memory test. Participants showed superior social (vs. nonsocial) memory performance, and the social memory advantage was associated with neural pattern reinstatement during rest in the dorsomedial prefrontal cortex (DMPFC), a key node of the default network. Moreover, it was during early rest that DMPFC social pattern reinstatement was greatest and predicted subsequent social memory performance most strongly, consistent with the "prioritization" account. Results simultaneously update 1) theories of memory consolidation, which have not addressed how social information may be prioritized in the learning process, and 2) understanding of default network function, which remains to be fully characterized. More broadly, the results underscore the inherent human drive to understand our vastly social world.

Driving and suppressing the human language network using large language models

Transformer models such as GPT generate human-like language and are predictive of human brain responses to language. Here, using functional-MRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of the brain response associated with each sentence. We then use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress the activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also non-invasively control neural activity in higher-level cortical areas, such as the language network.

How a speaker herds the audience: Multi-brain neural convergence over time during naturalistic storytelling

Storytelling-an ancient way for humans to share individual experiences with others-has been found to induce neural synchronization among listeners. In our exploration of the dynamic fluctuations in listener-listener (LL) coupling throughout stories, we uncover a significant correlation between LL and lag-speaker-listener (lag-SL) couplings over time. Using the analogy of neural pattern (dis)similarity as distances between participants, we term this phenomenon the "herding effect": like a shepherd guiding a group of sheep, the more closely listeners follow the speaker's prior brain activity patterns (higher lag-SL similarity), the more tightly they cluster together (higher LL similarity). This herding effect is particularly pronounced in brain regions where neural synchronization among listeners tracks with behavioral ratings of narrative engagement, highlighting the mediating role of narrative content in the observed multi-brain neural coupling dynamics. By integrating LL and SL neural couplings, this study illustrates how unfolding stories shape a dynamic multi-brain functional network and how the configuration of this network may be associated with moment-by-moment efficacy of communication.

Hippocampal-cortical interactions during event boundaries support retention of complex narrative events

According to most memory theories, encoding involves continuous communication between the hippocampus and neocortex, but recent work has shown that key moments at the end of an event, called event boundaries, may be especially critical for memory formation. We sought to determine how communication between the hippocampus and neocortical regions during the encoding of naturalistic events related to subsequent retrieval of those events and whether this was particularly important at event boundaries. Participants encoded and recalled two cartoon movies during fMRI scanning. Higher functional connectivity between the hippocampus and the posterior medial network (PMN) at an event's offset is related to the subsequent successful recall of that event. Furthermore, hippocampal-PMN offset connectivity also predicted the amount of detail retrieved after a 2-day delay. These data demonstrate that the relationship between memory encoding and hippocampal-neocortical interaction is dynamic and biased toward boundaries.

A role for the serotonin 2A receptor in the expansion and functioning of human transmodal cortex

Integrating independent but converging lines of research on brain function and neurodevelopment across scales, this article proposes that serotonin 2A receptor (5-HT2AR) signalling is an evolutionary and developmental driver and potent modulator of the macroscale functional organization of the human cerebral cortex. A wealth of evidence indicates that the anatomical and functional organization of the cortex follows a unimodal-to-transmodal gradient. Situated at the apex of this processing hierarchy-where it plays a central role in the integrative processes underpinning complex, human-defining cognition-the transmodal cortex has disproportionately expanded across human development and evolution. Notably, the adult human transmodal cortex is especially rich in 5-HT2AR expression and recent evidence suggests that, during early brain development, 5-HT2AR signalling on neural progenitor cells stimulates their proliferation-a critical process for evolutionarily-relevant cortical expansion. Drawing on multimodal neuroimaging and cross-species investigations, we argue that, by contributing to the expansion of the human cortex and being prevalent at the apex of its hierarchy in the adult brain, 5-HT2AR signalling plays a major role in both human cortical expansion and functioning. Owing to its unique excitatory and downstream cellular effects, neuronal 5-HT2AR agonism promotes neuroplasticity, learning and cognitive and psychological flexibility in a context-(hyper)sensitive manner with therapeutic potential. Overall, we delineate a dual role of 5-HT2ARs in enabling both the expansion and modulation of the human transmodal cortex.

Theta EEG neurofeedback promotes early consolidation of real life-like episodic memory

Theta oscillations are believed to coordinate neuronal activity related to human cognition, especially for memory functions. Theta power during learning and retrieval has been found to correlate with memory performance success. Additionally, up-regulating theta oscillations during a post-encoding epoch crucial for memory consolidation was previously shown to benefit long-term memory for acquired motor sequences, pictures, and object-location associations. However, it remains to be determined whether such effects would be found for more ecological aspects of long-term episodic memory. Therefore, the current study assessed neurofeedback-based theta upregulation effects on movie memory. After viewing a 15-minute silent, narrative movie, participants engaged in neurofeedback-based theta/beta up-regulation, neurofeedback beta/theta up-regulation as an active control condition, or an unrelated passive control task. Memory was tested three times: once immediately after watching the movie (as baseline); 24 hours thereafter; and once again 1 week later. Memory performance 1 week after encoding was significantly enhanced in the theta/beta up-regulation group compared with the other groups. Additionally, changes in neurofeedback theta/beta ratio from baseline EEG recordings correlated with long-term memory gains in retrieving the movie's content. These findings highlight the relationship between post-learning theta oscillations and the consolidation of episodic memory for a naturalistic event.

Behavioral representational similarity analysis reveals how episodic learning is influenced by and reshapes semantic memory

While semantic and episodic memory have been shown to influence each other, uncertainty remains as to how this interplay occurs. We introduce a behavioral representational similarity analysis approach to assess whether semantic space can be subtly re-sculpted by episodic learning. Eighty participants learned word pairs that varied in semantic relatedness, and learning was bolstered via either testing or restudying. Next-day recall is superior for semantically related pairs, but there is a larger benefit of testing for unrelated pairs. Analyses of representational change reveal that successful recall is accompanied by a pulling together of paired associates, with cue words in semantically related (but not unrelated) pairs changing more across learning than target words. Our findings show that episodic learning is associated with systematic and asymmetrical distortions of semantic space which improve later recall by making cues more predictive of targets, reducing interference from potential lures, and establishing novel connections within pairs.

Depth and hierarchies in the predictive brain: From reaction to action

The human brain is a prediction device, a view widely accepted in neuroscience. Prediction is a rational and efficient response that relies on the brain's ability to create and employ generative models to optimize actions over unpredictable time horizons. We argue that extant predictive frameworks while compelling, have not explicitly accounted for the following: (a) The brain's generative models must incorporate predictive depth (i.e., rely on degrees of abstraction to enable predictions over different time horizons); (b) The brain's implementation scheme to account for varying predictive depth relies on dynamic predictive hierarchies formed using the brain's functional networks. We show that these hierarchies incorporate the ascending processes (driven by reaction), and the descending processes (related to prediction), eventually driving action. Because they are dynamically formed, predictive hierarchies allow the brain to address predictive challenges in virtually any domain. By way of application, we explain how this framework can be applied to heretofore poorly understood processes of human behavioral thermoregulation. Although mammalian thermoregulation has been closely tied to deep brain structures engaged in autonomic control such as the hypothalamus, this narrow conception does not translate well to humans. In addition to profound differences in evolutionary history, the human brain is bestowed with substantially increased functional complexity (that itself emerged from evolutionary differences). We argue that behavioral thermoregulation in humans is possible because, (a) ascending signals shaped by homeostatic sub-networks, interject with (b) descending signals related to prediction (implemented in interoceptive and executive sub-networks) and action (implemented in executive sub-networks). These sub-networks cumulatively form a predictive hierarchy for human thermoregulation, potentiating a range of viable responses to known and unknown thermoregulatory challenges. We suggest that our proposed extensions to the predictive framework provide a set of generalizable principles that can further illuminate the many facets of the predictive brain. This article is categorized under: Neuroscience > Behavior Philosophy > Action Psychology > Prediction.

Age differences in the neural correlates of recollection: transient versus sustained fMRI effects

Prior functional magnetic resonance imaging findings in young adults indicate that recollection-sensitive neural regions dissociate according to the time courses of their respective recollection effects. Here, we examined whether such dissociations are also evident in older adults. Young and older participants encoded a series of word-image pairs, judging which of the denoted objects was the smaller. At the test, participants judged whether each of a series of test words was old or new. If a word was old, the requirement was to recall the associated image and maintain it over a variable delay period. Older adults demonstrated significantly lower associative memory performance than young adults. Transient recollection effects were identified in the left hippocampus, medial prefrontal cortex, and posterior cingulate, while sustained effects were widespread across left lateral cortex and were also evident in the bilateral striatum. Except for those in the left insula, all effects were age-invariant. These findings suggest that both transient and sustained recollection effects are largely stable across much of the healthy adult life span.

Prediction underlying comprehension of human motion: an analysis of Deaf signer and non-signer EEG in response to visual stimuli

Introduction

Sensory inference and top-down predictive processing, reflected in human neural activity, play a critical role in higher-order cognitive processes, such as language comprehension. However, the neurobiological bases of predictive processing in higher-order cognitive processes are not well-understood.

Methods

This study used electroencephalography (EEG) to track participants' cortical dynamics in response to Austrian Sign Language and reversed sign language videos, measuring neural coherence to optical flow in the visual signal. We then used machine learning to assess entropy-based relevance of specific frequencies and regions of interest to brain state classification accuracy.

Results

EEG features highly relevant for classification were distributed across language processing-related regions in Deaf signers (frontal cortex and left hemisphere), while in non-signers such features were concentrated in visual and spatial processing regions.

Discussion

The results highlight functional significance of predictive processing time windows for sign language comprehension and biological motion processing, and the role of long-term experience (learning) in minimizing prediction error.

Predictions transform memories: How expected versus unexpected events are integrated or separated in memory

Our brains constantly generate predictions about the environment based on prior knowledge. Many of the events we experience are consistent with these predictions, while others might be inconsistent with prior knowledge and thus violate our predictions. To guide future behavior, the memory system must be able to strengthen, transform, or add to existing knowledge based on the accuracy of our predictions. We synthesize recent evidence suggesting that when an event is consistent with our predictions, it leads to neural integration between related memories, which is associated with enhanced associative memory, as well as memory biases. Prediction errors, in turn, can promote both neural integration and separation, and lead to multiple mnemonic outcomes. We review these findings and how they interact with factors such as memory reactivation, prediction error strength, and task goals, to offer insight into what determines memory for events that violate our predictions. In doing so, this review brings together recent neural and behavioral research to advance our understanding of how predictions shape memory, and why.

The multi-angle extended three-dimensional activities (META) stimulus set: A tool for studying event cognition

To study complex human activity and how it is perceived and remembered, it is valuable to have large-scale, well-characterized stimuli that are representative of such activity. We present the Multi-angle Extended Three-dimensional Activities (META) stimulus set, a structured and highly instrumented set of extended event sequences performed in naturalistic settings. Performances were captured with two color cameras and a Kinect v2 camera with color and depth sensors, allowing the extraction of three-dimensional skeletal joint positions. We tracked the positions and identities of objects for all chapters using a mixture of manual coding and an automated tracking pipeline, and hand-annotated the timings of high-level actions. We also performed an online experiment to collect normative event boundaries for all chapters at a coarse and fine grain of segmentation, which allowed us to quantify event durations and agreement across participants. We share these materials publicly to advance new discoveries in the study of complex naturalistic activity.

A common neural code for meaning in discourse production and comprehension

How does the brain code the meanings conveyed by language? Neuroimaging studies have investigated this by linking neural activity patterns during discourse comprehension to semantic models of language content. Here, we applied this approach to the production of discourse for the first time. Participants underwent fMRI while producing and listening to discourse on a range of topics. We used a distributional semantic model to quantify the similarity between different speech passages and identified where similarity in neural activity was predicted by semantic similarity. When people produced discourse, speech on similar topics elicited similar activation patterns in a widely distributed and bilateral brain network. This network was overlapping with, but more extensive than, the regions that showed similarity effects during comprehension. Critically, cross-task neural similarities between comprehension and production were also predicted by similarities in semantic content. This result suggests that discourse semantics engages a common neural code that is shared between comprehension and production. Effects of semantic similarity were bilateral in all three RSA analyses, even while univariate activation contrasts in the same data indicated left-lateralised BOLD responses. This indicates that right-hemisphere regions encode semantic properties even when they are not activated above baseline. We suggest that right-hemisphere regions play a supporting role in processing the meaning of discourse during both comprehension and production.

Dissecting unsupervised learning through hidden Markov modeling in electrophysiological data

Unsupervised, data-driven methods are commonly used in neuroscience to automatically decompose data into interpretable patterns. These patterns differ from one another depending on the assumptions of the models. How these assumptions affect specific data decompositions in practice, however, is often unclear, which hinders model applicability and interpretability. For instance, the hidden Markov model (HMM) automatically detects characteristic, recurring activity patterns (so-called states) from time series data. States are defined by a certain probability distribution, whose state-specific parameters are estimated from the data. But what specific features, from all of those that the data contain, do the states capture? That depends on the choice of probability distribution and on other model hyperparameters. Using both synthetic and real data, we aim to better characterize the behavior of two HMM types that can be applied to electrophysiological data. Specifically, we study which differences in data features (such as frequency, amplitude, or signal-to-noise ratio) are more salient to the models and therefore more likely to drive the state decomposition. Overall, we aim at providing guidance for the appropriate use of this type of analysis on one- or two-channel neural electrophysiological data and an informed interpretation of its results given the characteristics of the data and the purpose of the analysis.NEW & NOTEWORTHY Compared with classical supervised methods, unsupervised methods of analysis have the advantage to be freer of subjective biases. However, it is not always clear what aspects of the data these methods are most sensitive to, which complicates interpretation. Focusing on the hidden Markov model, commonly used to describe electrophysiological data, we explore in detail the nature of its estimates through simulations and real data examples, providing important insights about what to expect from these models.

Cinematographic continuity edits across shot scales and camera angles: an ERP analysis

Film editing has attracted great theoretical and practical interest since the beginnings of cinematography. In recent times, the neural correlates of visual transitions at edit cuts have been at the focus of attention in neurocinematics. Many Event Related Potential (ERP) studies studies have reported the consequences of cuts involving narrative discontinuities, and violations of standard montage rules. However, less is known about edits that are meant to induce continuity. Here, we addressed the neural correlates of continuity editing involving scale, and angle variations across the cut within the same scene, two of the most popular devices used for continuity editing. We recorded the electroencephalographic signal obtained from 20 viewers as they watched four different cinematographic excerpts to extract ERPs at edit points. First, we were able to reproduce the general time and scalp distribution of the typical ERPs to filmic cuts in prior studies. Second, we found significant ERP modulations triggered by scale changes (scale out, scale in, or maintaining the same scale). Edits involving an increase in scale (scale out) led to amplification of the ERP deflection, and scale reduction (scale in) led to decreases, compared to edits that kept scale across the cut. These modulations coincide with the time window of the N300 and N400 components and, according to previous findings, their amplitude has been associated with the likelihood of consciously detecting the edit. Third, we did not detect similar modulations as a function of angle variations across the cut. Based on these findings, we suggest that cuts involving reduction of scale are more likely to go unnoticed, than ones that scale out. This relationship between scale in/out and visibility is documented in film edition manuals. Specifically, in order to achieve fluidity in a scene, the edition is designed from the most opened shots to the most closed ones.

Memory reactivation and suppression modulate integration of the semantic features of related memories in hippocampus

Encoding an event that overlaps with a previous experience may involve reactivating an existing memory and integrating it with new information or suppressing the existing memory to promote formation of a distinct, new representation. We used fMRI during overlapping event encoding to track reactivation and suppression of individual, related memories. We further used a model of semantic knowledge based on Wikipedia to quantify both reactivation of semantic knowledge related to a previous event and formation of integrated memories containing semantic features of both events. Representational similarity analysis revealed that reactivation of semantic knowledge related to a prior event in posterior medial prefrontal cortex (pmPFC) supported memory integration during new learning. Moreover, anterior hippocampus (aHPC) formed integrated representations combining the semantic features of overlapping events. We further found evidence that aHPC integration may be modulated on a trial-by-trial basis by interactions between ventrolateral PFC and anterior mPFC, with suppression of item-specific memory representations in anterior mPFC inhibiting hippocampal integration. These results suggest that PFC-mediated control processes determine the availability of specific relevant memories during new learning, thus impacting hippocampal memory integration.

Attention and schema violations of real world scenes differentially modulate time perception

In the real world, object arrangement follows a number of rules. Some of the rules pertain to the spatial relations between objects and scenes (i.e., syntactic rules) and others about the contextual relations (i.e., semantic rules). Research has shown that violation of semantic rules influences interval timing with the duration of scenes containing such violations to be overestimated as compared to scenes with no violations. However, no study has yet investigated whether both semantic and syntactic violations can affect timing in the same way. Furthermore, it is unclear whether the effect of scene violations on timing is due to attentional or other cognitive accounts. Using an oddball paradigm and real-world scenes with or without semantic and syntactic violations, we conducted two experiments on whether time dilation will be obtained in the presence of any type of scene violation and the role of attention in any such effect. Our results from Experiment 1 showed that time dilation indeed occurred in the presence of syntactic violations, while time compression was observed for semantic violations. In Experiment 2, we further investigated whether these estimations were driven by attentional accounts, by utilizing a contrast manipulation of the target objects. The results showed that an increased contrast led to duration overestimation for both semantic and syntactic oddballs. Together, our results indicate that scene violations differentially affect timing due to violation processing differences and, moreover, their effect on timing seems to be sensitive to attentional manipulations such as target contrast.

The Agent Preference in Visual Event Apprehension

A central aspect of human experience and communication is understanding events in terms of agent ("doer") and patient ("undergoer" of action) roles. These event roles are rooted in general cognition and prominently encoded in language, with agents appearing as more salient and preferred over patients. An unresolved question is whether this preference for agents already operates during apprehension, that is, the earliest stage of event processing, and if so, whether the effect persists across different animacy configurations and task demands. Here we contrast event apprehension in two tasks and two languages that encode agents differently; Basque, a language that explicitly case-marks agents ('ergative'), and Spanish, which does not mark agents. In two brief exposure experiments, native Basque and Spanish speakers saw pictures for only 300 ms, and subsequently described them or answered probe questions about them. We compared eye fixations and behavioral correlates of event role extraction with Bayesian regression. Agents received more attention and were recognized better across languages and tasks. At the same time, language and task demands affected the attention to agents. Our findings show that a general preference for agents exists in event apprehension, but it can be modulated by task and language demands.

Generalization of cognitive maps across space and time

Prominent theories posit that associative memory structures, known as cognitive maps, support flexible generalization of knowledge across cognitive domains. Here, we evince a representational account of cognitive map flexibility by quantifying how spatial knowledge formed one day was used predictively in a temporal sequence task 24 hours later, biasing both behavior and neural response. Participants learned novel object locations in distinct virtual environments. After learning, hippocampus and ventromedial prefrontal cortex (vmPFC) represented a cognitive map, wherein neural patterns became more similar for same-environment objects and more discriminable for different-environment objects. Twenty-four hours later, participants rated their preference for objects from spatial learning; objects were presented in sequential triplets from either the same or different environments. We found that preference response times were slower when participants transitioned between same- and different-environment triplets. Furthermore, hippocampal spatial map coherence tracked behavioral slowing at the implicit sequence transitions. At transitions, predictive reinstatement of virtual environments decreased in anterior parahippocampal cortex. In the absence of such predictive reinstatement after sequence transitions, hippocampus and vmPFC responses increased, accompanied by hippocampal-vmPFC functional decoupling that predicted individuals' behavioral slowing after a transition. Collectively, these findings reveal how expectations derived from spatial experience generalize to support temporal prediction.

A shared neural code for the physics of actions and object events

Observing others' actions recruits frontoparietal and posterior temporal brain regions - also called the action observation network. It is typically assumed that these regions support recognizing actions of animate entities (e.g., person jumping over a box). However, objects can also participate in events with rich meaning and structure (e.g., ball bouncing over a box). So far, it has not been clarified which brain regions encode information specific to goal-directed actions or more general information that also defines object events. Here, we show a shared neural code for visually presented actions and object events throughout the action observation network. We argue that this neural representation captures the structure and physics of events regardless of animacy. We find that lateral occipitotemporal cortex encodes information about events that is also invariant to stimulus modality. Our results shed light onto the representational profiles of posterior temporal and frontoparietal cortices, and their roles in encoding event information.