Processing Causality in Narrative Events: Temporal Order Matters

Weaving a story: Narrative formation over prolonged time scales engages social cognition and frontoparietal networks

Forming narratives is of key importance to human experience, enabling one to render large amounts of information into relatively compacted stories for future retrieval, giving meaning to otherwise fragmented occurrences. The neural mechanisms that underlie coherent narrative construction of causally connected information over prolonged temporal periods are yet unclear. Participants in this fMRI study observed consecutive scenes from a full-length movie either in their original order, enabling causal inferences over time, or in reverse order, impeding a key component of coherent narratives-causal inference. In between scenes, we presented short periods of blank screens for examining post-encoding processing effects. Using multivariate pattern analysis (MVPA) followed by seed-base correlation analysis, we hypothesized that networks involved in online monitoring of incoming information on the one hand, and offline processing of previous occurrences on the other would differ between the groups. We found that despite the exposure to the same scenes, the chronological-order condition exhibited enhanced functional connectivity in frontoparietal regions associated with information integration and working memory. The reverse-order condition yielded offline, post-scene coactivation of neural networks involved in social cognition and particularly theory of mind and action comprehension. These findings shed light on offline processes of narrative construction efforts, highlighting the role of social cognition networks in seeking for narrative coherence.

Neural unscrambling of temporal information during a nonlinear narrative

Although we must experience our lives chronologically, storytellers often manipulate the order in which they relay events. How the brain processes temporal information while encoding a nonlinear narrative remains unclear. Here, we use functional magnetic resonance imaging during movie watching to investigate which brain regions are sensitive to information about time in a narrative and test whether the representation of temporal context across a narrative is more influenced by the order in which events are presented or their underlying chronological sequence. Results indicate that medial parietal regions are sensitive to cued jumps through time over and above other changes in context (i.e., location). Moreover, when processing non-chronological narrative information, the precuneus and posterior cingulate engage in on-the-fly temporal unscrambling to represent information chronologically. Specifically, days that are closer together in chronological time are represented more similarly regardless of when they are presented in the movie, and this representation is consistent across participants. Additional analyses reveal a strong spatial signature associated with higher magnitude jumps through time. These findings are consistent with prior theorizing on medial parietal regions as central to maintaining and updating narrative situation models, and suggest the priority of chronological information when encoding narrative events.

Temporal-order iconicity bias in narrative event understanding and memory

Incongruence between the narrated (encoded) order and the actual chronological order of events is ubiquitous in various kinds of narratives and information modalities. The iconicity assumption in text comprehension proposes that readers will by default assume the chronological order to match the narrated order. However, it is not clear whether this iconicity assumption would directly bias inferred chronology of events and memory of their narrated order. In the current study, using non-linearly narrated video narratives as encoding materials, we dissociated the narrated order and the underlying chronological order of events. In Experiment 1, we found that participants' judgments of the chronological order of events were biased by the narrated order, but not vice versa. In Experiment 2, when the chronological positions of events were provided during encoding, participants' judgments of the chronological order were not biased by the narrated order, rather, their memory of the narrated order of events was biased by the chronological order. Interpreting the bias under a descriptive Bayesian framework, we offer a new perspective on the role of the iconicity assumption as prior belief, apart from prior knowledge about event sequences, in event understanding as well as memory.

Time and Causation in Discourse: Temporal Proximity, Implicit Causality, and Re-mention Biases

Using referential processing in discourse featuring implicit causality verbs as a test case, we demonstrate how a discourse's causal and temporal dimensions interact. We show that referential processing is affected by multiple discourse biases, and that these biases do not have uniform effects. In three discourse continuation experiments, we show that the bias to re-mention a particular referent in discourse involving implicit causality verbs is not only affected by the verb's implicit causality bias, but also by the discourse's temporal structure, which at times, can even override the implicit causality bias. Our results add to the growing number of studies that show how various discourse dimensions interact in discourse processing.

You may now kiss the bride: Interpretation of social situations by individuals with right or left hemisphere injury

While left hemisphere damage (LHD) has been clearly shown to cause a range of language impairments, patients with right hemisphere damage (RHD) also exhibit communication deficits, such as difficulties processing prosody, discourse, and social contexts. In the current study, individuals with RHD and LHD were directly compared on their ability to interpret what a character in a cartoon might be saying or thinking, in order to better understand the relative role of the right and left hemisphere in social communication. The cartoon stimuli were manipulated so as to elicit more or less formulaic responses (e.g., a scene of a couple being married by a priest vs. a scene of two people talking, respectively). Participants' responses were scored by blind raters on how appropriately they captured the gist of the social situation, as well as how formulaic and typical their responses were. Results showed that RHD individuals' responses were rated as significantly less appropriate than controls and were also significantly less typical than controls and individuals with LHD. Individuals with RHD produced a numerically lower proportion of formulaic expressions than controls, but this difference was only a trend. Counter to prediction, the pattern of performance across participant groups was not affected by how constrained/formulaic the social situation was. The current findings expand our understanding of the roles that the right and left hemispheres play in social processing and communication and have implications for the potential treatment of social communication deficits in individuals with RHD.

A set of 265 pictures standardized for studies of the cognitive processing of temporal and causal order information

Planning, predicting, reasoning, and acting often depend crucially on the correct encoding and application of knowledge concerning the temporal and causal ordering of events. Yet no pictorial stimulus set is optimized for investigating the processing of temporal and causal order information. We introduce a novel stimulus set of 265 black-and-white line drawings depicting a diverse array of recognizable events. Most of the images in the stimulus set (N = 222) share a thematic or conceptual association with one other image in the set, and the stimuli were created and extensively normed such that the image pairs vary in the degrees to which they share a causal, ordered relation with one another. The stimuli were standardized in a series of normative tasks, including concept/noun/verb agreement, perceived frequency, visual similarity, and indexes of three features of causal associations between events (i.e., temporal proximity, exclusivity, and priority). Both younger adults (ages 18-30 years) and older adults (ages 60-80 years) contributed normative data, allowing for broad applications of the stimuli to the study of normal and age-related changes in the encoding, retention, and retrieval of information regarding temporal and causal order. Complete normative data sets are available in the online supplemental materials, and the full stimulus set is available by contacting the first author.