A tradeoff between musical tension perception and declarative memory

Successful formation of long-term declarative memory is influenced, among other things, by attention, emotion, and deviation from expectations. A unique form of expectation can be elicited through musical tension, evoked by the prolongation of certain musical progressions. We examined the effect that musical tension exerts on the formation of declarative memory, by composing three original music pieces that contained tension segments, achieved by delays in release from dominant to tonic harmonies. Music-evoked tension was validated using music information retrieval (MIR) analysis, as well as skin conductance response (SCR) measures. Indeed, tension-evoking musical excerpts were associated with heightened SCR, corroborated by increased subjective ratings of tension, as compared to neutral excerpts. In the main experiment, 50 participants listened to the three musical pieces while they were presented with unique images that were randomly assigned to four conditions: tension, tension-release, neutral music, and silence. One day later, their memory for the images was examined using a recognition test. We found that memory performance was enhanced for images presented during both neutral and tense music compared to silence. Moreover, we observed a tradeoff effect between post-experiment tension perception and memory, such that individuals who perceived musical tension as such displayed reduced memory performance for images encoded during musical tension, whereas tense music benefited memory for those with lower musical tension perception. Understanding the interrelations between musical components, which exert powerful and fundamental responses in humans, and cognitive faculties, may provide insights as to the basic features of memory formation.

Music and Time Perception in Audiovisuals: Arousing Soundtracks Lead to Time Overestimation No Matter Their Emotional Valence

One of the most tangible effects of music is its ability to alter our perception of time. Research on waiting times and time estimation of musical excerpts has attested its veritable effects. Nevertheless, there exist contrasting results regarding several musical features’ influence on time perception. When considering emotional valence and arousal, there is some evidence that positive affect music fosters time underestimation, whereas negative affect music leads to overestimation. Instead, contrasting results exist with regard to arousal. Furthermore, to the best of our knowledge, a systematic investigation has not yet been conducted within the audiovisual domain, wherein music might improve the interaction between the user and the audiovisual media by shaping the recipients’ time perception. Through the current between-subjects online experiment (n = 565), we sought to analyze the influence that four soundtracks (happy, relaxing, sad, scary), differing in valence and arousal, exerted on the time estimation of a short movie, as compared to a no-music condition. The results reveal that (1) the mere presence of music led to time overestimation as opposed to the absence of music, (2) the soundtracks that were perceived as more arousing (i.e., happy and scary) led to time overestimation. The findings are discussed in terms of psychological and phenomenological models of time perception.

Emotion elicitation during music listening: Subjective self-reports, facial expression, and autonomic reactivity

The use of music as emotional stimuli in experimental studies has grown in recent years. However, prior studies have mainly focused on self-reports and central measures, with a few works exploring the time course of psychophysiological correlates. Moreover, most of the previous research has been carried out either from the dimensional or categorical model but not combining both approaches to emotions. This study aimed to investigate subjective and physiological correlates of emotion elicitation through music, following the three-dimensional and the discrete emotion model. A sample of 50 healthy volunteers (25 women) took part in this experiment by listening to 42 film music excerpts (14 pleasant, 14 unpleasant, 14 neutral) presented during 8 s, while peripheral measures were continuously recorded. After music offset, affective dimensions (valence, energy arousal, and tension arousal) as well as discrete emotions (happiness, sadness, tenderness, fear, and anger) were collected using a 9-point scale. Results showed an effect of the music category on subjective and psychophysiological measures. In peripheral physiology, greater electrodermal activity, heart rate acceleration, and zygomatic responses, besides lower corrugator amplitude, were observed for pleasant excerpts in comparison to neutral and unpleasant music, from 2 s after stimulus onset until the end of its duration. Overall, our results add evidence for the efficacy of standardized film music excerpts to evoke powerful emotions in laboratory settings; thus, opening a path to explore interventions based on music in pathologies with underlying emotion deregulatory processes.

Soundtrack design: The impact of music on visual attention and affective responses

Through music, film composers attempt to affect the audience's emotions and visual attention; however, little empirical evidence exists characterizing the mechanisms whereby music affects viewers. We conducted a mixed-design experiment with 60 participants to explore the effect of music on viewers' visual attention and affective responses to a film. Music led to quicker first fixations on film objects and supplied emotional content, increasing positive sentiment for the film's story. However, music did not influence viewers' attitudes to filmed events. Within this context, music has limited impact on attitudes toward filmed events but can accentuate the saliency of film objects and supply emotional information altering viewers' sentiment towards the film. Understanding the mechanisms of music's physiological and behavioral effects can inform content delivery strategies. This study offers insights for composition and production of musical soundtracks for feature-length films, advertising videos, educational videos, and video games.

Affective rating of audio and video clips using the EmojiGrid

Background: In this study we measured the affective appraisal of sounds and video clips using a newly developed graphical self-report tool: the EmojiGrid. The EmojiGrid is a square grid, labeled with emoji that express different degrees of valence and arousal. Users rate the valence and arousal of a given stimulus by simply clicking on the grid. Methods: In Experiment I, observers (N=150, 74 males, mean age=25.2±3.5) used the EmojiGrid to rate their affective appraisal of 77 validated sound clips from nine different semantic categories, covering a large area of the affective space. In Experiment II, observers (N=60, 32 males, mean age=24.5±3.3) used the EmojiGrid to rate their affective appraisal of 50 validated film fragments varying in positive and negative affect (20 positive, 20 negative, 10 neutral). Results: The results of this study show that for both sound and video, the agreement between the mean ratings obtained with the EmojiGrid and those obtained with an alternative and validated affective rating tool in previous studies in the literature, is excellent for valence and good for arousal. Our results also show the typical universal U-shaped relation between mean valence and arousal that is commonly observed for affective sensory stimuli, both for sound and video. Conclusions: We conclude that the EmojiGrid can be used as an affective self-report tool for the assessment of sound and video-evoked emotions.

Affective rating of audio and video clips using the EmojiGrid

Background: In this study we measured the affective appraisal of sounds and video clips using a newly developed graphical self-report tool: the EmojiGrid. The EmojiGrid is a square grid, labeled with emoji that express different degrees of valence and arousal. Users rate the valence and arousal of a given stimulus by simply clicking on the grid. Methods: In Experiment I, observers (N=150, 74 males, mean age=25.2±3.5) used the EmojiGrid to rate their affective appraisal of 77 validated sound clips from nine different semantic categories, covering a large area of the affective space. In Experiment II, observers (N=60, 32 males, mean age=24.5±3.3) used the EmojiGrid to rate their affective appraisal of 50 validated film fragments varying in positive and negative affect (20 positive, 20 negative, 10 neutral). Results: The results of this study show that for both sound and video, the agreement between the mean ratings obtained with the EmojiGrid and those obtained with an alternative and validated affective rating tool in previous studies in the literature, is excellent for valence and good for arousal. Our results also show the typical universal U-shaped relation between mean valence and arousal that is commonly observed for affective sensory stimuli, both for sound and video. Conclusions: We conclude that the EmojiGrid can be used as an affective self-report tool for the assessment of sound and video-evoked emotions.

Framing and bias in CO2 capture and storage communication films: Reflections from a CO2 capture and storage research group

There has been a growing trend towards incorporating short, educational films as part of research funding and project proposals. Researchers and developers in CO₂ capture and storage are using films to communicate outcomes, but such films can be influenced by experiences and values of the producers. We document the content and presentation of seven online CO₂ capture and storage films to determine how framing occurs and its influence on the tone of films. The core frame presents CO₂ capture and storage as a potential solution to an imminent crisis in climatic warming and lack of a sustainable energy supply. Three subsidiary frames represent CO₂ capture and storage as (1) the only option, (2) a partial option or (3) a scientific curiosity. The results demonstrate that an understanding of the nuanced explicit and implicit messages portrayed by films is essential both for effective framing according to one's intention and for wider public understanding of a field.

Slow motion in films and video clips: Music influences perceived duration and emotion, autonomic physiological activation and pupillary responses

Slow motion scenes are ubiquitous in screen-based audiovisual media and are typically accompanied by emotional music. The strong effects of slow motion on observers are hypothetically related to heightened emotional states in which time seems to pass more slowly. These states are simulated in films and video clips, and seem to resemble such experiences in daily life. The current study investigated time perception and emotional response to media clips containing decelerated human motion, with or without music using psychometric and psychophysiological testing methods. Participants were presented with slow-motion scenes taken from commercial films, ballet and sports footage, as well as the same scenes converted to real-time. Results reveal that slow-motion scenes, compared to adapted real-time scenes, led to systematic underestimations of duration, lower perceived arousal but higher valence, lower respiration rates and smaller pupillary diameters. The presence of music compared to visual-only presentations strongly affected results in terms of higher accuracy in duration estimates, higher perceived arousal and valence, higher physiological activation and larger pupillary diameters, indicating higher arousal. Video genre affected responses in addition. These findings suggest that perceiving slow motion is not related to states of high arousal, but rather affects cognitive dimensions of perceived time and valence. Music influences these experiences profoundly, thus strengthening the impact of stretched time in audiovisual media.

Narrative theory and the dynamics of popular movies

Popular movies grab and hold our attention. One reason for this is that storytelling is culturally important to us, but another is that general narrative formulae have been honed over millennia and that a derived but specific filmic form has developed and has been perfected over the last century. The result is a highly effective format that allows rapid processing of complex narratives. Using a corpus analysis I explore a physical narratology of popular movies-narrational structure and how it impacts us-to promote a theory of popular movie form. I show that movies can be divided into 4 acts-setup, complication, development, and climax-with two optional subunits of prolog and epilog, and a few turning points and plot points. In 12 studies I show that normative aspects in patterns of shot durations, shot transitions, shot scale, shot motion, shot luminance, character introduction, and distributions of conversations, music, action shots, and scene transitions reduce to 5 correlated stylistic dimensions of movies and can litigate among theories of movie structure. In general, movie narratives have roughly the same structure as narratives in any other domain-plays, novels, manga, folktales, even oral histories-but with particular runtime constraints, cadences, and constructions that are unique to the medium.

The Effect of Background Music in Shark Documentaries on Viewers' Perceptions of Sharks

Despite the ongoing need for shark conservation and management, prevailing negative sentiments marginalize these animals and legitimize permissive exploitation. These negative attitudes arise from an instinctive, yet exaggerated fear, which is validated and reinforced by disproportionate and sensationalistic news coverage of shark 'attacks' and by highlighting shark-on-human violence in popular movies and documentaries. In this study, we investigate another subtler, yet powerful factor that contributes to this fear: the ominous background music that often accompanies shark footage in documentaries. Using three experiments, we show that participants rated sharks more negatively and less positively after viewing a 60-second video clip of swimming sharks set to ominous background music, compared to participants who watched the same video clip set to uplifting background music, or silence. This finding was not an artifact of soundtrack alone because attitudes toward sharks did not differ among participants assigned to audio-only control treatments. This is the first study to demonstrate empirically that the connotative attributes of background music accompanying shark footage affect viewers' attitudes toward sharks. Given that nature documentaries are often regarded as objective and authoritative sources of information, it is critical that documentary filmmakers and viewers are aware of how the soundtrack can affect the interpretation of the educational content.

Time course of EEG oscillations during repeated listening of a well-known aria

While previous studies have analyzed mean neurophysiological responses to musical stimuli, the current study aimed to identify specific time courses of electroencephalography (EEG) oscillations, which are associated with dynamic changes in the acoustic features of the musical stimulus. In addition, we were interested in whether these time courses change during a repeated presentation of the same musical piece. A total of 16 subjects repeatedly listened to the well-known aria “Nessun dorma,” sung by Paul Potts, while continuous 128-channel EEG and heart rate, as well as electrodermal responses, were recorded. The time courses for the EEG oscillations were calculated using a time resolution of 1 second for several frequency bands, on the basis of individual alpha-peak frequencies (theta, low alpha-1, low alpha-2, upper alpha, and beta). For all frequency bands, we identified a more or less continuous increase in power relative to a baseline period, indicating strong event-related synchronization (ERS) during music listening. The ERS time courses, however, did not correlate strongly with the time courses of the acoustic features of the aria. In addition, we did not observe changes in EEG oscillations after repeated presentation of the same musical piece. Aside from this distinctive feature, we identified a remarkable variability in EEG oscillations, both within and between the repeated presentations of the aria. We interpret the continuous increase in ERS observed in all frequency bands during music listening as an indicator of a particular neurophysiological and psychological state evoked by music listening. We suggest that this state is characterized by increased internal attention (accompanied by reduced external attention), increased inhibition of brain networks not involved in the generation of this internal state, the maintenance of a particular level of general alertness, and a type of brain state that can be described as “mind wandering.” The overall state can be categorized as a psychological process that may be seen as a “drawing in” to the musical piece. However, this state is not stable and varies considerably throughout the music listening session and across subjects. Most important, however, is the finding that the neurophysiological activations occurring during music listening are dynamic and not stationary.

A New Set of Standardised Verbal and Non-verbal Contemporary Film Stimuli for the Elicitation of Emotions

This study responded to a need for an updated set of film clips for the elicitation of emotion. Previous such stimuli are not only dated, but are limited in that they have been validated using participants from a narrow age range, which is problematic given conflicting reports regarding age differences in self-reported emotion elicited in a laboratory setting. The present study aimed to develop a new film stimulus set standardised on a wide age range of participants, while investigating age differences in self-reported emotion in response to film clips. An ancillary aim was to standardise film clips that were with and without dialogue. Two researchers composed a shortlist of 60 clips that were chosen to elicit six target emotions of amusement, anger, disgust, fear, happiness and sadness, as well as a neutral state. Each of the 109 participants (55 males, 54 females), aged between 18 and 88 years (M = 44.94) viewed all 60 clips. Self-reported emotional experience was measured using a computerised Visual Analogue Scale. Age differences were observed whereby participants aged 46–88 reported more intense positive and negative emotions in response to film clips than participants aged 18–45. Consistent with previous research, difficulties were found in eliciting the target emotion of anger. A success index was calculated for each film that took into consideration the mean rating on the target emotion and the degree to which the film elicited only the target emotion. We present a list of film clips recommended for future research.

Watching emotional movies: affective reactions and gender differences

Previous studies have found that unpleasant film clips depicting mutilated bodies or injuries evoke a sustained heart rate deceleration which has been interpreted as reflecting a stimulus-specific aversive response or as increased orienting and attentional processing that varies with stimulus significance. Few studies, however, have examined cardiac changes during the viewing of high arousal pleasant films. To clarify this issue, the present study assessed evaluative, facial and autonomic reactions in both men and women during the viewing of highly arousing pleasant, as well as unpleasant, films. Results indicated a similar skin conductance increase and heart rate deceleration which were greater than those observed during the viewing of a neutral film. Compared to men, women rated both films as less pleasant and rated the unpleasant film as more arousing. The present findings suggest that sustained exposure to pleasant and unpleasant stimuli elicit similar cardiac orienting when stimuli are equated for subjective report of emotional arousal.

Music and emotion: Electrophysiological correlates of the processing of pleasant and unpleasant music

Human emotion and its electrophysiological correlates are still poorly understood. The present study examined whether the valence of perceived emotions would differentially influence EEG power spectra and heart rate (HR). Pleasant and unpleasant emotions were induced by consonant and dissonant music. Unpleasant (compared to pleasant) music evoked a significant decrease of HR, replicating the pattern of HR responses previously described for the processing of emotional pictures, sounds, and films. In the EEG, pleasant (contrasted to unpleasant) music was associated with an increase of frontal midline (Fm) theta power. This effect is taken to reflect emotional processing in close interaction with attentional functions. These findings show that Fm theta is modulated by emotion more strongly than previously believed.