Narrative theory and the dynamics of popular movies

Attention in hindsight: Using stimulated recall to capture dynamic fluctuations in attentional engagement

Attentional engagement is known to vary on a moment-to-moment basis. However, few self-report methods can effectively capture dynamic fluctuations in attentional engagement over time. In the current paper, we evaluated the utility of stimulated recall, a method wherein individuals are asked to remember their subjective states while using a mnemonic cue, for the measurement of temporal changes in attentional engagement. Participants were asked to watch a video lecture, during which we assessed their in-the-moment levels of attentional engagement using intermittent thought probes. Then, we used stimulated recall by cueing participants with short video clips from the lecture to retrospectively assess the levels of attentional engagement they had experienced when they first watched those clips within the lecture. Experiment 1 assessed the statistical overlap between in-the-moment and video-stimulated ratings. Experiment 2 assessed the generalizability of video-stimulated recall across different types of lectures. Experiment 3 assessed the impact of presenting video-stimulated probe clips in non-chronological order. Experiment 4 assessed the effect of video-stimulated recall on its own. Across all experiments, we found statistically robust correspondence between in-the-moment and video-stimulated ratings of attentional engagement, illustrating a strong convergence between these two methods of assessment. Taken together, our findings indicate that stimulated recall provides a new and practical methodological approach that can accurately capture dynamic fluctuations in subjective attentional states over time.

Inter- and intra-subject similarity in network functional connectivity across a full narrative movie

Naturalistic paradigms, such as watching movies during functional magnetic resonance imaging, are thought to prompt the emotional and cognitive processes typically elicited in real life situations. Therefore, naturalistic viewing (NV) holds great potential for studying individual differences. Previous studies have primarily focused on using shorter movie clips, geared toward eliciting specific and often isolated emotions, while the potential behind using full narratives depicted in commercial movies as a proxy for real-life experiences has barely been explored. Here, we offer preliminary evidence that a full narrative movie (FNM), that is, a movie covering a complete narrative arc, can capture complex socio-affective dynamics and their links to individual differences. Using the studyforrest dataset, we investigated inter- and intra-subject similarity in network functional connectivity (NFC) of 14 meta-analytically defined networks across a full narrative, audio-visual movie split into eight consecutive movie segments. We characterized the movie segments by valence and arousal portrayed within the sequences, before utilizing a linear mixed model to analyze which factors explain inter- and intra-subject similarity. Our results show that the model best explaining inter-subject similarity comprised network, movie segment, valence and a movie segment by valence interaction. Intra-subject similarity was influenced significantly by the same factors and an additional three-way interaction between movie segment, valence and arousal. Overall, inter- and intra-subject similarity in NFC were sensitive to the ongoing narrative and emotions in the movie. We conclude that FNMs offer complex content and dynamics that might be particularly valuable for studying individual differences. Further characterization of movie features, such as the overarching narratives, that enhance individual differences is needed for advancing the potential of NV research.

Time perception in film viewing: A modulation of scene's duration estimates as a function of film editing

Filmmakers and editors have empirically developed techniques to ensure the spatiotemporal continuity of a film's narration. In terms of time, editing techniques (e.g., elliptical, overlapping, or cut minimization) allow for the manipulation of the perceived duration of events as they unfold on screen. More specifically, a scene can be edited to be time compressed, expanded, or real-time in terms of its perceived duration. Despite the consistent application of these techniques in filmmaking, their perceptual outcomes have not been experimentally validated. Given that viewing a film is experienced as a precise simulation of the physical world, the use of cinematic material to examine aspects of time perception allows for experimentation with high ecological validity, while filmmakers gain more insight on how empirically developed techniques influence viewers' time percept. Here, we investigated how such time manipulation techniques of an action affect a scene's perceived duration. Specifically, we presented videos depicting different actions (e.g., a woman talking on the phone), edited according to the techniques applied for temporal manipulation and asked participants to make verbal estimations of the presented scenes' perceived durations. Analysis of data revealed that the duration of expanded scenes was significantly overestimated as compared to that of compressed and real-time scenes, as was the duration of real-time scenes as compared to that of compressed scenes. Therefore, our results validate the empirical techniques applied for the modulation of a scene's perceived duration. We also found interactions on time estimates of scene type and editing technique as a function of the characteristics and the action of the scene presented. Thus, these findings add to the discussion that the content and characteristics of a scene, along with the editing technique applied, can also modulate perceived duration. Our findings are discussed by considering current timing frameworks, as well as attentional saliency algorithms measuring the visual saliency of the presented stimuli.

Effects of a narrative template on memory for the time of movie scenes: automatic reshaping is independent of consolidation

Memory for time is influenced by reconstructive processes, but the underlying mechanisms remain unclear. The present study investigated whether the effect of schematic prior knowledge on temporal memory for movie scenes, produced by the incomplete presentation (cut) of the movie at encoding, is modulated by cut position, retention interval, and task repetition. In a timeline positioning task, participants were asked to indicate when short video clips extracted from a previously encoded movie occurred on a horizontal timeline that represented the video duration. In line with previous findings, removing the final part of the movie resulted in a systematic underestimation of clips' position as a function of their proximity to the missing part. Further experiments demonstrate that the direction of this automatic effect depends on which part of the movie is deleted from the encoding session, consistent with the inferential structure of the schema, and does not depend on consolidation nor reconsolidation processes, at least within the present experimental conditions. We propose that the observed bias depends on the automatic influence of reconstructive processes on judgments about the time of occurrence, based on prior schematic knowledge.

Movie editing influences spectators' time perception

Filmmakers use different techniques (e.g., camera movements, editing) to shape viewers' experience. In particular, editing can be used to handle the temporal unfolding of events represented in a movie. Nevertheless, little is known about how different editing types impact viewers' time perception. In an exploratory on-line study (90 participants) and a pre-registered conceptual replication study (60 participants), we asked participants to judge (Study 1) or reproduce (Study 2) the duration of 45 excerpts of the movie "Le Ballon Rouge" containing either continuous editing, action discontinuity editing or no editing. Each excerpt was formatted in three durations (2000, 2500 or 3000 ms). In both studies, we reported that scenes containing continuous editing were perceived as longer than the other two scene types. Moreover, scenes containing action discontinuity editing were perceived as longer than scenes with no editing. This study contributes to the emerging field of psycho-cinematics which could ultimately develop the dialog between arts and science.

Predicting memory from the network structure of naturalistic events

When we remember events, we often do not only recall individual events, but also the connections between them. However, extant research has focused on how humans segment and remember discrete events from continuous input, with far less attention given to how the structure of connections between events impacts memory. Here we conduct a functional magnetic resonance imaging study in which participants watch and recall a series of realistic audiovisual narratives. By transforming narratives into networks of events, we demonstrate that more central events-those with stronger semantic or causal connections to other events-are better remembered. During encoding, central events evoke larger hippocampal event boundary responses associated with memory formation. During recall, high centrality is associated with stronger activation in cortical areas involved in episodic recollection, and more similar neural representations across individuals. Together, these results suggest that when humans encode and retrieve complex real-world experiences, the reliability and accessibility of memory representations is shaped by their location within a network of events.

Leveraging the power of media to drive cognition: a media-informed approach to naturalistic neuroscience

So-called 'naturalistic' stimuli have risen in popularity in cognitive, social and affective neuroscience over the last 15 years. However, a critical property of these stimuli is frequently overlooked: Media-like film, television, books and podcasts-are 'fundamentally not natural'. They are deliberately crafted products meant to elicit particular human thought, emotion and behavior. Here, we argue for a more informed approach to adopting media stimuli in experimental paradigms. We discuss the pitfalls of combining stimuli that are designed for research with those that are designed for other purposes (e.g. entertainment) under the umbrella term of 'naturalistic' and present strategies to improve rigor in the stimulus selection process. We assert that experiencing media should be considered a task akin to any other experimental task(s) and explain how this shift in perspective will compel more nuanced and generalizable research using these stimuli. Throughout, we offer theoretical and practical knowledge from multidisciplinary media research to raise the standard for the treatment of media stimuli in neuroscience research.

A Design Space for Applying the Freytag's Pyramid Structure to Data Stories

Data stories integrate compelling visual content to communicate data insights in the form of narratives. The narrative structure of a data story serves as the backbone that determines its expressiveness, and it can largely influence how audiences perceive the insights. Freytag's Pyramid is a classic narrative structure that has been widely used in film and literature. While there are continuous recommendations and discussions about applying Freytag's Pyramid to data stories, little systematic and practical guidance is available on how to use Freytag's Pyramid for creating structured data stories. To bridge this gap, we examined how existing practices apply Freytag's Pyramid by analyzing stories extracted from 103 data videos. Based on our findings, we proposed a design space of narrative patterns, data flows, and visual communications to provide practical guidance on achieving narrative intents, organizing data facts, and selecting visual design techniques through story creation. We evaluated the proposed design space through a workshop with 25 participants. Results show that our design space provides a clear framework for rapid storyboarding of data stories with Freytag's Pyramid.

CineScale: A dataset of cinematic shot scale in movies

We provide a database containing shot scale annotations (i.e., the apparent distance of the camera from the subject of a filmed scene) for more than 792,000 image frames. Frames belong to 124 full movies from the entire filmographies by 6 important directors: Martin Scorsese, Jean-Luc Godard, Béla Tarr, Federico Fellini, Michelangelo Antonioni, and Ingmar Bergman. Each frame, extracted from videos at 1 frame per second, is annotated on the following scale categories: Extreme Close Up (ECU), Close Up (CU), Medium Close Up (MCU), Medium Shot (MS), Medium Long Shot (MLS), Long Shot (LS), Extreme Long Shot (ELS), Foreground Shot (FS), and Insert Shots (IS). Two independent coders annotated all frames from the 124 movies, whilst a third one checked their coding and made decisions in cases of disagreement. The CineScale database enables AI-driven interpretation of shot scale data and opens to a large set of research activities related to the automatic visual analysis of cinematic material, such as the automatic recognition of the director's style, or the unfolding of the relationship between shot scale and the viewers' emotional experience. To these purposes, we also provide the model and the code for building a Convolutional Neural Network (CNN) architecture for automated shot scale recognition. All this material is provided through the project website, where video frames can also be requested to authors, for research purposes under fair use.

Reconstructive nature of temporal memory for movie scenes

Remembering when events took place is a key component of episodic memory. Using a sensitive behavioral measure, the present study investigates whether spontaneous event segmentation and script-based prior knowledge affect memory for the time of movie scenes. In three experiments, different groups of participants were asked to indicate when short video clips extracted from a previously encoded movie occurred on a horizontal timeline that represented the video duration. When participants encoded the entire movie, they were more precise at judging the temporal occurrence of clips extracted from the beginning and the end of the film compared to its middle part, but also at judging clips that were closer to event boundaries. Removing the final part of the movie from the encoding session resulted in a systematic bias in memory for time. Specifically, participants increasingly underestimated the time of occurrence of the video clips as a function of their proximity to the missing part of the movie. An additional experiment indicated that such an underestimation effect generalizes to different audio-visual material and does not necessarily reflect poor temporal memory. By showing that memories are moved in time to make room for missing information, the present study demonstrates that narrative time can be adapted to fit a standard template regardless of what has been effectively encoded, in line with reconstructive theories of memory.

Augmenting training–learning experience by design: a conceptual framework

Purpose

The purpose of this paper is to explore the possibility of designing training sessions on the basis of a dramatic structure and inclusion of contextual narrative for facilitating transfer of learning. The role of stimulating curiosity in this process has also been scanned under the lens of neurobiological insights.

Design/methodology/approach

The paper theoretically develops a framework through an integrative literature review to examine the prospect of enhanced learner engagement through structure, narrative and some interdisciplinary theories, namely, the gap theory of curiosity and the peak end theory.

Findings

A contextual story-based training method designed on the basis of a dramatic plot structure can leverage on the learners’ emotions for engaging learning sessions and retention of content. Such design offers potential to improvise and strengthen the overall training module design and delivery mechanism. The outcome is seen through enhanced peak moments of curiosity and satisfaction, thus enriching the overall training–learning process.

Research limitations/implications

The study is theory based and non-empirical which does not give it a ground to make generalised statements and conclusions. This factor, however, paves the way for future research in the allied areas such as empirical testing of the framework by identifying and testing the variables and other contextual and causal factors.

Practical implications

The suggested framework has practical implications for Learning and Development managers as well as academies. The conceptual framework provided in this work can lend some unique insights towards strengthening the training–learning process.

Originality/value

Although the concept of using stories for training is not new, this study contributes by proposing a new theoretical framework that examines together the elements of sound dramatic structure and a story-based training method. Such a design, conceived by taking into account an understanding of the working mind, can influence the overall experience of achieving positive training–learning results.

Introduction to the special issue on physics of mind

In recent years, both fields of physics and psychology have made important scientific advances. The emergence of new instruments gave rise to a data-driven neuroscience allowing us to learn about the state of the brain supporting known mental functions and conversely. In parallel, the appearance of new mathematics allowed the development of computational models describing fundamental brain functions and implementing them in technological applications. While emphasizing the methodology of physics, the special issue aims to bring together these trends in both the experimental and theoretical sciences in order to explain some of the most basic mental processes such as perception, cognition, emotion, consciousness, and learning. In this editorial, we define unsolved problems for brain and psychological sciences, discuss possible means toward their respective solutions, and outline some collaborative initiatives aiming toward these goals. The following problems are defined in gradual order of difficulty: what are the universal properties of human behavior across conditions and cultures? What have each culture learned over historical times and why should specific elements of knowledge be accumulated over cultural evolution? Can computational psychiatry help predict, understand, and cure mental disorders? What is the function of art and cultural artifacts such as music, fiction, or poetry for the cognitive system? How to explain the relation between first-person subjective experience and third-person objective physiological data? What neural mechanisms operate on which mental content at the highest levels of organization of the hierarchical brain? How do abstract ideas emerge from sensory-motor contingencies and what are the conditions for the birth of a new concept? Could symmetry play a role in psychogenesis and support the emergence of new hierarchical layers in cognition? How can we start addressing the question of meaning scientifically, and what does it entail for the physical sciences?

Large-scale narrative events in popular cinema

Most experiments in event perception and cognition have focused on events that are only a few minutes in length, and the previous research on popular movies is consistent with this temporal scope. Scenes are generally between a few seconds and a few minutes in duration. But popular movies also offer an opportunity to explore larger events-variously called acts, major parts, or large-scale parts by film theorists-in which the boundaries often have few if any unique physical attributes. These units tend to be between about 20 to 35 min in duration. The present study had observers watch seven movies they had not seen before and, over the course of several days and with ample justifications, reflect on them, and then segment them into two to six parts with the aid of a running description of the narrative. Results showed consistency across viewers' segmentations, consistency with film-theoretic segmentations, and superiority over internet subjects who had access to only the scenarios used by the movie viewers. Thus, these results suggest that there are large scale events in movies; they support a view that their events are organized meronomically, layered with units of different sizes and with boundaries shared across layers; and they suggest that these larger-scale events can be discerned through cognitive, not perceptual, means.

Temporal-order iconicity bias in narrative event understanding and memory

Incongruence between the narrated (encoded) order and the actual chronological order of events is ubiquitous in various kinds of narratives and information modalities. The iconicity assumption in text comprehension proposes that readers will by default assume the chronological order to match the narrated order. However, it is not clear whether this iconicity assumption would directly bias inferred chronology of events and memory of their narrated order. In the current study, using non-linearly narrated video narratives as encoding materials, we dissociated the narrated order and the underlying chronological order of events. In Experiment 1, we found that participants' judgments of the chronological order of events were biased by the narrated order, but not vice versa. In Experiment 2, when the chronological positions of events were provided during encoding, participants' judgments of the chronological order were not biased by the narrated order, rather, their memory of the narrated order of events was biased by the chronological order. Interpreting the bias under a descriptive Bayesian framework, we offer a new perspective on the role of the iconicity assumption as prior belief, apart from prior knowledge about event sequences, in event understanding as well as memory.

Your Brain on Comics: A Cognitive Model of Visual Narrative Comprehension

The past decade has seen a rapid growth of cognitive and brain research focused on visual narratives like comics and picture stories. This paper will summarize and integrate this emerging literature into the Parallel Interfacing Narrative‐Semantics Model (PINS Model)—a theory of sequential image processing characterized by an interaction between two representational levels: semantics and narrative structure. Ongoing semantic processes build meaning into an evolving mental model of a visual discourse. Updating of spatial, referential, and event information then incurs costs when they are discontinuous with the growing context. In parallel, a narrative structure organizes semantic information into coherent sequences by assigning images to categorical roles, which are then embedded within a hierarchic constituent structure. Narrative constructional schemas allow for specific predictions of structural sequencing, independent of semantics. Together, these interacting levels of representation engage in an iterative process of retrieval of semantic and narrative information, prediction of upcoming information based on those assessments, and subsequent updating based on discontinuity. These core mechanisms are argued to be domain‐general—spanning across expressive systems—as suggested by similar electrophysiological brain responses (N400, P600, anterior negativities) generated in response to manipulation of sequential images, music, and language. Such similarities between visual narratives and other domains thus pose fundamental questions for the linguistic and cognitive sciences.

Visual narratives like comics involve a range of complex cognitive operations in order to be understood. The Parallel Interfacing Narrative‐Semantics (PINS) Model integrates an emerging literature showing that comprehension of wordless image sequences balances two representational levels of semantic and narrative structure. The neurocognitive mechanisms that guide these processes are argued to overlap with other domains, such as language and music.

Genre-typical narrative arcs in films are less appealing to lay audiences and professional film critics

People tend to like stimuli-ranging from human faces to text-that are prototypical, and thus easily processed. However, recent research has suggested that less typical stimuli may be preferred in creative contexts, such as fine art or music lyrics. In an archival sample of movie scripts, we tested whether genre-typicality predicted film ratings as a function of rater role (novice audience member or expert film critic). Genre-typicality was operationalized as the profile correlations between linguistic arcs (across five segments, or acts) for each script and within-genre averages. We predicted (1) that critics would prefer more disfluent (genre-atypical) films and general audiences would prefer fluent (genre-typical) films, and (2) that these differences would be most pronounced for genres expected to be more entertaining (e.g., action/adventure) than challenging (e.g., tragedy). Partly consistent with our hypotheses, the results showed that critics gave higher ratings to action/adventure films with less typical positive emotion arcs. However, regardless of audience-member or professional-critic status, higher ratings were attributed to films that were more genre-atypical (or disfluent), in terms of analytic thinking, narrative action, and emotional tone, across all genres except family/kids films. Such findings support the growing literature on the appeal of disfluency in the arts and have relevance for researchers in psychology and computer science who are interested in computational linguistic approaches to attitudes, film, and literature.

Chaotic and Fast Audiovisuals Increase Attentional Scope but Decrease Conscious Processing

Audiovisual cuts involve spatial, temporal, and action narrative leaps. They can even change the meaning of the narrative through film editing. Many cuts are not consciously perceived, others are, just as we perceive or not the changes in real events. In this paper, we analyze the effects of cuts and different editing styles on 36 subjects, using electroencephalographic (EEG) techniques and the projection of stimuli with different audiovisual style of edition but the same narrative. Eyeblinks, event-related potentials (ERPs), EEG spectral power and disturbances, and the functional and effective connectivity before and after the cuts were analyzed. Cuts decreased blink frequency in the first second following them. Cuts also caused an increase of the alpha rhythm, with a cortical evolution from visual toward rostral areas. There were marked differences between a video-clip editing style, with greater activities evoked in visual areas, and the classic continuous style of editing, which presented greater activities in the frontal zones. This was reflected by differences in the theta rhythm between 200 and 400 ms, in visual and frontal zones, and can be connected to the different demands that each style of edition makes on working memory and conscious processing after cutting. Also, at the time of cuts, the causality between visual, somatosensory, and frontal networks is altered in any editing style. Our findings suggest that cuts affect media perception and chaotic and fast audiovisuals increase attentional scope but decrease conscious processing.

Temporal fractals in movies and mind

Fractal patterns are seemingly everywhere. They can be analyzed through Fourier and power analyses, and other methods. Cutting, DeLong, and Nothelfer (2010) analyzed as time-series data the fluctuations of shot durations in 150 popular movies released over 70 years. They found that these patterns had become increasingly fractal-like and concluded that they might be linked to those found in the results of psychological tasks involving attention. To explore this possibility further, we began by analyzing the shot patterns of almost twice as many movies released over a century. The increasing fractal-like nature of shot patterns is affirmed, as determined by both a slope measure and a long-range dependence measure, neither of which is sensitive to the vector lengths of their inputs within the ranges explored here. But the main reason for increased long-range dependence is related to, but not caused by, the increasing vector length of the shot-series samples. It appears that, in generating increasingly fractal-like patterns, filmmakers have systematically explored dimensions that are important for holding our attention-shot durations, scene durations, motion, and sound amplitude-and have crafted fluctuations in them like those of our endogenous attention patterns. Other dimensions-luminance, clutter, and shot scale-are important to film style but their variations seem not to be important to holding viewers' moment-to-moment attention and have not changed in their fractional dimension over time.

Cryptic Emotions and the Emergence of a Metatheory of Mind in Popular Filmmaking

Hollywood movies can be deeply engaging and easy to understand. To succeed in this manner, feature‐length movies employ many editing techniques with strong psychological underpinnings. We explore the origins and development of one of these, the reaction shot. This shot typically shows a single, unspeaking character with modest facial expression in response to an event or to the behavior or speech of another character. In a sample of movies from 1940 to 2010, we show that the prevalence of one type of these shots—which we call the cryptic reaction shot—has grown dramatically. These shots are designed to enhance viewers’ emotional involvement with characters. They depict a facial gesture that reflects a slightly negative and slightly aroused emotional state. Their use at the end of conversations, and typically at the end of scenes, helps to leave viewers in a state of speculation about what the character is thinking and what her thoughts may mean for the ongoing narrative.

Considering the filmmaker: Intensified continuity, narrative structure, and the Distancing-Embracing model

Menninghaus et al. pose two open-ended questions: To what extent do formal elements of art elicit negative affect, and do artists try to elicit this response in a theory-based or intuitive manner? For popular movies, we argue that the consideration of their construction is prior to the consideration of the experience that they evoke.

The evolution of pace in popular movies

Movies have changed dramatically over the last 100 years. Several of these changes in popular English-language filmmaking practice are reflected in patterns of film style as distributed over the length of movies. In particular, arrangements of shot durations, motion, and luminance have altered and come to reflect aspects of the narrative form. Narrative form, on the other hand, appears to have been relatively unchanged over that time and is often characterized as having four more or less equal duration parts, sometimes called acts - setup, complication, development, and climax. The altered patterns in film style found here affect a movie's pace: increasing shot durations and decreasing motion in the setup, darkening across the complication and development followed by brightening across the climax, decreasing shot durations and increasing motion during the first part of the climax followed by increasing shot durations and decreasing motion at the end of the climax. Decreasing shot durations mean more cuts; more cuts mean potentially more saccades that drive attention; more motion also captures attention; and brighter and darker images are associated with positive and negative emotions. Coupled with narrative form, all of these may serve to increase the engagement of the movie viewer.

EEG Spectral Dynamics of Video Commercials: Impact of the Narrative on the Branding Product Preference

Neuromarketing has become popular and received a lot of attention. The quality of video commercials and the product information they convey to consumers is a hotly debated topic among advertising agencies and product advertisers. This study explored the impact of advertising narrative and the frequency of branding product exposures on the preference for the commercial and the branding product. We performed electroencephalography (EEG) experiments on 30 subjects while they watched video commercials. The behavioral data indicated that commercials with a structured narrative and containing multiple exposures of the branding products had a positive impact on the preference for the commercial and the branding product. The EEG spectral dynamics showed that the narratives of video commercials resulted in higher theta power of the left frontal, bilateral occipital region, and higher gamma power of the limbic system. The narratives also induced significant cognitive integration-related beta and gamma power of the bilateral temporal regions and the parietal region. It is worth noting that the video commercials with a single exposure of the branding products would be indicators of attention. These new findings suggest that the presence of a narrative structure in video commercials has a critical impact on the preference for branding products.