Memorable Audiovisual Narratives Synchronize Sensory and Supramodal Neural Responses

Portable EEG in groups shows increased brain coupling to strong health messages

Health messages are core building blocks of public health efforts. Neuroscientific measures offer insights into how target audiences receive health messages. To move towards real-world applications, however, challenges regarding costs, lab restraints, and slow data acquisition need to be addressed. Using portable electroencephalography (EEG) and inter-subject correlation (ISC) analysis as measure of message strength, we ask whether these challenges can be met. Portable EEG was recorded while participants viewed strong and weak video health messages against risky alcohol use. Participants viewed the messages either individually or in a focus group-like setting with six participants simultaneously. For both viewing conditions, three correlated components were extracted. The topographies of these components showed a high spatial correlation with previous high-density EEG results. Moreover, ISC was strongly enhanced when viewing strong as compared to weak health messages in both the group and individual viewing conditions. The findings suggest that ISC analysis shows sensitivity to message strength, even in a group setting using low-density portable EEG. Measuring brain responses to messages in group settings is more efficient and scalable beyond the laboratory. Overall, these results support a translational perspective for the use of neuroscientific measures in health message development.

Contrastive learning of shared spatiotemporal EEG representations across individuals for naturalistic neuroscience

Neural representations induced by naturalistic stimuli offer insights into how humans respond to stimuli in daily life. Understanding neural mechanisms underlying naturalistic stimuli processing hinges on the precise identification and extraction of the shared neural patterns that are consistently present across individuals. Targeting the Electroencephalogram (EEG) technique, known for its rich spatial and temporal information, this study presents a framework for Contrastive Learning of Shared SpatioTemporal EEG Representations across individuals (CL-SSTER). CL-SSTER utilizes contrastive learning to maximize the similarity of EEG representations across individuals for identical stimuli, contrasting with those for varied stimuli. The network employs spatial and temporal convolutions to simultaneously learn the spatial and temporal patterns inherent in EEG. The versatility of CL-SSTER was demonstrated on three EEG datasets, including a synthetic dataset, a natural speech comprehension EEG dataset, and an emotional video watching EEG dataset. CL-SSTER attained the highest inter-subject correlation (ISC) values compared to the state-of-the-art ISC methods. The latent representations generated by CL-SSTER exhibited reliable spatiotemporal EEG patterns, which can be explained by properties of the naturalistic stimuli. CL-SSTER serves as an interpretable and scalable framework for the identification of inter-subject shared neural representations in naturalistic neuroscience.

Creating demand for unmet needs: Agile Storytelling

Introduction

The translational gap from the discovery of evidence-based solutions to their implementation in healthcare delivery organizations derives from an incorrect assumption that the need for change among executive, administrative, or clinical personnel is the same as the demand for change. For sickle cell disease (SCD), implementation of evidence-based guidelines is often delayed or obstructed due to lack of demand. This challenge allows for the persistence of resource limitations and care delivery models that do not meet the community's unique needs. Agile Storytelling is a process built on the scientific foundations of behavioral economics, complexity science, and network science to create local demand for the implementation of evidence-based solutions.

Methods

Agile Storytelling includes a design phase and a testing phase. The design phase converts the evidence-based solution into a minimally viable story of a hero, a villain, struggle, drama, and a resolution. The testing phase evaluates the effectiveness of the story via a series of storytelling sprints in the target local healthcare delivery organization. The efficacy of Agile Storytelling was tested in an iterative n-of-1 case study design.

Results

Agile Storytelling was used in a large, urban, healthcare system within the United States to facilitate implementation of national SCD best-practice guidelines. After repeated failures attempting to use national and local data regarding the high societal need to hire a SCD-specific social worker, an Agile change conductor using Agile Storytelling was able to create demand for the new position within a week. This decision has ultimately improved patient outcomes and led to the adoption of a specialized collaborative care team for SCD within the health network.

Discussion

Agile Storytelling can lead to structured, effective, and informed storytelling to create local demand within healthcare delivery organizations.

Client Perspectives of Case Stories in Internet-Delivered Cognitive Behavioral Therapy for Public Safety Personnel: Mixed Methods Study

BACKGROUND

Internet-delivered cognitive behavioral therapy (ICBT) is an effective and convenient means of offering cognitive behavioral therapy to the general population. To increase access to ICBT among Canadian public safety personnel (PSP)-a group that experiences elevated rates of mental health concerns and barriers to mental health care-a clinical research unit called PSPNET has tailored ICBT to PSP, primarily through offering case stories and PSP-specific examples within an ICBT program. PSPNET's first and most frequently used ICBT program, called the PSP Wellbeing Course, has been found to reduce symptoms of mental disorders (eg, anxiety, depression, and posttraumatic stress) among PSP. Little research, however, has investigated clients' perceptions of the case stories in this course.

OBJECTIVE

This study was designed to expand the literature on the use and evaluation of case stories in ICBT among PSP. Specifically, this study investigated (1) PSP's perceptions of the case stories using the theoretical model provided by Shaffer and Zikmund-Fisher and (2) PSP feedback on the case stories in the PSP Wellbeing Course.

METHODS

This study included 41 clients who completed the PSP Wellbeing Course. Of these, 27 clients completed a bespoke questionnaire called the Stories Questionnaire, 10 of whom also participated in a semistructured interview.

RESULTS

Findings show that perceptions of the case stories in the PSP Wellbeing Course were largely positive and that the case stories were generally successful in achieving the 5 purposes of case stories (ie, informing, comforting, modeling, engaging, and persuading) proposed by Shaffer and Zikmund-Fisher. Client feedback also identified 3 tangible areas for story improvement: characters, content, and delivery. Each area highlights the need for and potential benefits of story development. Not all PSP engaged with the case stories, though, so results must be interpreted with caution.

CONCLUSIONS

Overall, this study adds to the growing body of research supporting the use of case stories in internet-delivered interventions among PSP.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04127032; https://www.clinicaltrials.gov/ct2/show/NCT04127032.

Inter-subject gamma oscillation synchronization as a biomarker of abnormal processing of social interaction in ADHD

Children with attention-deficit hyperactivity disorder (ADHD) have difficulties in social interactions. Studying brain activity during social interactions is difficult with conventional artificial stimuli. This pioneering study examined the neural correlates of social perception in children with ADHD and matched controls using naturalistic stimuli. We presented 20 children with ADHD and 20 age-and-sex-matched controls with tailored movies featuring high- or low-level social interactions while recording electroencephalographic signals. Both groups exhibited synchronized gamma-band oscillations, but controls demonstrated greater inter-subject correlations. Additionally, the difference in inter-subject correlations between high- and low-interaction movies was significantly larger in controls compared to ADHD patients. Between 55 and 75 Hz comparing viewing high interaction movies with low interaction moves, controls had a significantly larger weighting in the right parietal lobe, while ADHD patients had a significantly smaller weighting in the left occipital lobe. These findings reveal distinct spatiotemporal neural signatures in social interaction processing among children with ADHD and controls using naturalistic stimuli. These neural markers offer potential for group differentiation and assessing intervention efficacy, advancing our understanding ADHD-related social interaction mechanisms.

Inter-subject correlation of electroencephalographic and behavioural responses reflects time-varying engagement with natural music

Musical engagement can be conceptualized through various activities, modes of listening and listener states. Recent research has reported that a state of focused engagement can be indexed by the inter-subject correlation (ISC) of audience responses to a shared naturalistic stimulus. While statistically significant ISC has been reported during music listening, we lack insight into the temporal dynamics of engagement over the course of musical works-such as those composed in the Western classical style-which involve the formulation of expectations that are realized or derailed at subsequent points of arrival. Here, we use the ISC of electroencephalographic (EEG) and continuous behavioural (CB) responses to investigate the time-varying dynamics of engagement with functional tonal music. From a sample of adult musicians who listened to a complete cello concerto movement, we found that ISC varied throughout the excerpt for both measures. In particular, significant EEG ISC was observed during periods of musical tension that built to climactic highpoints, while significant CB ISC corresponded more to declarative entrances and points of arrival. Moreover, we found that a control stimulus retaining envelope characteristics of the intact music, but little other temporal structure, also elicited significantly correlated EEG and CB responses, though to lesser extents than the original version. In sum, these findings shed light on the temporal dynamics of engagement during music listening and clarify specific aspects of musical engagement that may be indexed by each measure.

Bidirectional brain-body interactions during natural story listening

Narratives can synchronize neural and physiological signals between individuals, but the relationship between these signals, and the underlying mechanism, is unclear. We hypothesized a top-down effect of cognition on arousal and predicted that auditory narratives will drive not only brain signals but also peripheral physiological signals. We find that auditory narratives entrained gaze variation, saccade initiation, pupil size, and heart rate. This is consistent with a top-down effect of cognition on autonomic function. We also hypothesized a bottom-up effect, whereby autonomic physiology affects arousal. Controlled breathing affected pupil size, and heart rate was entrained by controlled saccades. Additionally, fluctuations in heart rate preceded fluctuations of pupil size and brain signals. Gaze variation, pupil size, and heart rate were all associated with anterior-central brain signals. Together, these results suggest bidirectional causal effects between peripheral autonomic function and central brain circuits involved in the control of arousal.

The convergence of naturalistic paradigms and cognitive neuroscience methods to investigate memory and its development

Studies that involve lab-based stimuli (e.g., words, pictures) are fundamental in the memory literature. At the same time, there is growing acknowledgment that memory processes assessed in the lab may not be analogous to how memory operates in the real world. Naturalistic paradigms can bridge this gap and over the decades a growing proportion of memory research has involved more naturalistic events. However, there is significant variation in the types of naturalistic studies used to study memory and its development, each with various advantages and limitations. Further, there are notable gaps in how often different types of naturalistic approaches have been combined with cognitive neuroscience methods (e.g., fMRI, EEG) to elucidate the neural processes and substrates involved in memory encoding and retrieval in the real world. Here we summarize and discuss what we identify as progressively more naturalistic methodologies used in the memory literature (movie, virtual reality, staged-events inside and outside of the lab, photo-taking, and naturally occurring event studies). Our goal is to describe each approach's benefits (e.g., naturalistic quality, feasibility), limitations (e.g., viability of neuroimaging method for event encoding versus event retrieval), and discuss possible future directions with each approach. We focus on child studies, when available, but also highlight past adult studies. Although there is a growing body of child memory research, naturalistic approaches combined with cognitive neuroscience methodologies in this domain remain sparse. Overall, this viewpoint article reviews how we can study memory through the lens of developmental cognitive neuroscience, while utilizing naturalistic and real-world events.

Reliability and generalizability of neural speech tracking in younger and older adults

Neural tracking of spoken speech is considered a potential clinical biomarker for speech-processing difficulties, but the reliability of neural speech tracking is unclear. Here, younger and older adults listened to stories in two sessions while electroencephalography was recorded to investigate the reliability and generalizability of neural speech tracking. Speech tracking amplitude was larger for older than younger adults, consistent with an age-related loss of inhibition. The reliability of neural speech tracking was moderate (ICC ∼0.5-0.75) and tended to be higher for older adults. However, reliability was lower for speech tracking than for neural responses to noise bursts (ICC >0.8), which we used as a benchmark for maximum reliability. Neural speech tracking generalized moderately across different stories (ICC ∼0.5-0.6), which appeared greatest for audiobook-like stories spoken by the same person. Hence, a variety of stories could possibly be used for clinical assessments. Overall, the current data are important for developing a biomarker of speech processing but suggest that further work is needed to increase the reliability to meet clinical standards.

The role of engagement and arousal in emotion regulation: an EEG study

Cognitive reappraisal and expressive suppression are well-studied strategies of emotion regulation (ER). However, the results on their physiological basis are controversial. While in some studies, ER was accompanied by the inhibition of the nervous system, others suggested that ER even might increase arousal and engagement. We calculated the inter-subject correlation (ISC) and indices of engagement, valence and arousal of EEG during suppression, reappraisal, or natural watching of neutral and negative videos. First, both suppression and reappraisal provoked a higher ISC in comparison with watching negative or neutral videos. We consider this as a marker of engagement to the task and feedback processing required for ER. Second, the engagement index was lower during ER compared to watching negative videos in central electrodes, whereas both strategies provoked a higher engagement in frontal electrodes. Third, the arousal index of EEG was higher during all negative conditions; therefore, regulation required a certain level of arousal. In summary, different EEG measures seem to be sensitive to different aspects of ER.

The Synergy Zone: Connecting the Mind, Brain, and Heart for the Ideal Classroom Learning Environment

This paper proposes a new perspective on implementing neuroeducation in the classroom. The pandemic exacerbated the mental health issues of faculty and students, creating a mental health crisis that impairs learning. It is important to get our students back in "the zone", both cognitively and emotionally, by creating an ideal learning environment for capturing our students and keeping them-the Synergy Zone. Research that examines the classroom environment often focuses on the foreground-instructors' organizational and instructional aspects and content. However, the emotional climate of the classroom affects student well-being. This emotional climate would ideally exhibit the brain states of engagement, attention, connection, and enjoyment by addressing the mind, brain, and heart. This ideal learning environment would be achieved by combining proposed practices derived from three areas of research: flow theory, brain synchronization, and positive emotion with heart engagement. Each of these enhances the desired brain states in a way that the whole is greater than the sum of the individual parts. I call this the Synergy Zone. A limitation of this proposed model is that implementation of some aspects may be challenging, and professional development resources might be needed. This essay presenting this perspective provides the relevant scientific research and the educational implications of implementation.

Intersubject similarity in neural representations underlies shared episodic memory content

Individuals generally form their unique memories from shared experiences, yet the neural representational mechanisms underlying this subjectiveness of memory are poorly understood. The current study addressed this important question from the cross-subject neural representational perspective, leveraging a large functional magnetic resonance imaging dataset (n = 415) of a face-name associative memory task. We found that individuals' memory abilities were predicted by their synchronization to the group-averaged, canonical trial-by-trial activation level and, to a lesser degree, by their similarity to the group-averaged representational patterns during encoding. More importantly, the memory content shared between pairs of participants could be predicted by their shared local neural activation pattern, particularly in the angular gyrus and ventromedial prefrontal cortex, even after controlling for differences in memory abilities. These results uncover neural representational mechanisms for individualized memory and underscore the constructive nature of episodic memory.

'The Mind' promotes brain synchronization: an ecological evaluation of brain synchronization in co-operative tasks

This paper presents an ecologically valid approach for using EEG hyperscanning methods to assess levels of interbrain synchrony (IBS) in teams during co-operative tasks. We employ a card-based task in an out-of-the-lab setting to evaluate levels of neural synchrony between team members completing a co-operative task. We also examine the interplay between the recorded synchronization levels and the collective performance of the team.Clinical Relevance- This study provides a simplistic and ecologically valid setup with potential to bring a better understanding of brain synchronization in clinical settings where co-operation would improve outcomes, such as home care facilities and memory clinics.

Physiological synchrony in electrodermal activity predicts decreased vigilant attention induced by sleep deprivation

Introduction

When multiple individuals are presented with narrative movie or audio clips, their electrodermal activity (EDA) and heart rate show significant similarities. Higher levels of such inter-subject physiological synchrony are related with higher levels of attention toward the narrative, as for instance expressed by more correctly answered questions about the narrative. We here investigate whether physiological synchrony in EDA and heart rate during watching of movie clips predicts performance on a subsequent vigilant attention task among participants exposed to a night of total sleep deprivation.

Methods

We recorded EDA and heart rate of 54 participants during a night of total sleep deprivation. Every hour from 22:00 to 07:00 participants watched a 10-min movie clip during which we computed inter-subject physiological synchrony. Afterwards, they answered questions about the movie and performed the psychomotor vigilance task (PVT) to capture attentional performance.

Results

We replicated findings that inter-subject correlations in EDA and heart rate predicted the number of correct answers on questions about the movie clips. Furthermore, we found that inter-subject correlations in EDA, but not in heart rate, predicted PVT performance. Individuals' mean EDA and heart rate also predicted their PVT performance. For EDA, inter-subject correlations explained more variance of PVT performance than individuals' mean EDA.

Discussion

Together, these findings confirm the association between physiological synchrony and attention. Physiological synchrony in EDA does not only capture the attentional processing during the time that it is determined, but also proves valuable for capturing more general changes in the attentional state of monitored individuals.

Higher levels of narrativity lead to similar patterns of posterior EEG activity across individuals

Introduction

The focus of cognitive and psychological approaches to narrative has not so much been on the elucidation of important aspects of narrative, but rather on using narratives as tools for the investigation of higher order cognitive processes elicited by narratives (e.g., understanding, empathy, etc.). In this study, we work toward a scalar model of narrativity, which can provide testable criteria for selecting and classifying communication forms in their level of narrativity. We investigated whether being exposed to videos with different levels of narrativity modulates shared neural responses, measured by inter-subject correlation, and engagement levels.

Methods

Thirty-two participants watched video advertisements with high-level and low-level of narrativity while their neural responses were measured through electroencephalogram. Additionally, participants' engagement levels were calculated based on the composite of their self-reported attention and immersion scores.

Results

Results demonstrated that both calculated inter-subject correlation and engagement scores for high-level video ads were significantly higher than those for low-level, suggesting that narrativity levels modulate inter-subject correlation and engagement.

Discussion

We believe that these findings are a step toward the elucidation of the viewers' way of processing and understanding a given communication artifact as a function of the narrative qualities expressed by the level of narrativity.

Audience immersion: validating attentional and physiological measures against self-report

When an audience member becomes immersed, their attention shifts towards the media and story, and they allocate cognitive resources to represent events and characters. Here, we investigate whether it is possible to measure immersion using continuous behavioural and physiological measures. Using television and film clips, we validated dual-task reaction times, heart rate, and skin conductance against self-reported narrative engagement. We find that reaction times to a secondary task were strongly positively correlated with self-reported immersion: slower reaction times were indicative of greater immersion, particularly emotional engagement. Synchrony in heart rate across participants was associated with self-reported attentional and emotional engagement with the story, although we found no such relationship with skin conductance. These results establish both dual-task reaction times and heart rate as candidate measures for the real-time, continuous, assessment of audience immersion.

The Temporal Dynamics of Brain-to-Brain Synchrony Between Students and Teachers Predict Learning Outcomes

Much of human learning happens through interaction with other people, but little is known about how this process is reflected in the brains of students and teachers. Here, we concurrently recorded electroencephalography (EEG) data from nine groups, each of which contained four students and a teacher. All participants were young adults from the northeast United States. Alpha-band (8-12 Hz) brain-to-brain synchrony between students predicted both immediate and delayed posttest performance. Further, brain-to-brain synchrony was higher in specific lecture segments associated with questions that students answered correctly. Brain-to-brain synchrony between students and teachers predicted learning outcomes at an approximately 300-ms lag in the students' brain activity relative to the teacher's brain activity, which is consistent with the time course of spoken-language comprehension. These findings provide key new evidence for the importance of collecting brain data simultaneously from groups of learners in ecologically valid settings.

Neural mechanisms of expert persuasion on willingness to pay for sugar

Introduction: Sugar consumption is associated with many negative health consequences. It is, therefore, important to understand what can effectively influence individuals to consume less sugar. We recently showed that a healthy eating call by a health expert can significantly decrease the willingness to pay (WTP) for sugar-containing food. Here, we investigate which aspects of neural responses to the same healthy eating call can predict the efficacy of expert persuasion.

Methods: Forty-five healthy participants performed two blocks of a bidding task, in which they had to bid on sugar-containing, sugar-free and non-edible products, while their electroencephalography (EEG) was recorded. In between the two blocks, they listened to a healthy eating call by a nutritionist emphasizing the risks of sugar consumption.

Results: We found that after listening to the healthy eating call, participants significantly decreased their WTP for sugar-containing products. Moreover, a higher intersubject correlation of EEG (a measure of engagement) during listening to the healthy eating call resulted in a larger decrease in WTP for sugar-containing food. Whether or not a participant’s valuation of a product was highly influenced by the healthy eating call could also be predicted by spatiotemporal patterns of EEG responses to the healthy eating call, using a machine learning classification model. Finally, the healthy eating call increased the amplitude of the P300 component of the visual event-related potential in response to sugar-containing food.

Disussion: Overall, our results shed light on the neural basis of expert persuasion and demonstrate that EEG is a powerful tool to design and assess health-related advertisements before they are released to the public.

Robustness of Physiological Synchrony in Wearable Electrodermal Activity and Heart Rate as a Measure of Attentional Engagement to Movie Clips

Individuals that pay attention to narrative stimuli show synchronized heart rate (HR) and electrodermal activity (EDA) responses. The degree to which this physiological synchrony occurs is related to attentional engagement. Factors that can influence attention, such as instructions, salience of the narrative stimulus and characteristics of the individual, affect physiological synchrony. The demonstrability of synchrony depends on the amount of data used in the analysis. We investigated how demonstrability of physiological synchrony varies with varying group size and stimulus duration. Thirty participants watched six 10 min movie clips while their HR and EDA were monitored using wearable sensors (Movisens EdaMove 4 and Wahoo Tickr, respectively). We calculated inter-subject correlations as a measure of synchrony. Group size and stimulus duration were varied by using data from subsets of the participants and movie clips in the analysis. We found that for HR, higher synchrony correlated significantly with the number of answers correct for questions about the movie, confirming that physiological synchrony is associated with attention. For both HR and EDA, with increasing amounts of data used, the percentage of participants with significant synchrony increased. Importantly, we found that it did not matter how the amount of data was increased. Increasing the group size or increasing the stimulus duration led to the same results. Initial comparisons with results from other studies suggest that our results do not only apply to our specific set of stimuli and participants. All in all, the current work can act as a guideline for future research, indicating the amount of data minimally needed for robust analysis of synchrony based on inter-subject correlations.

Developmental differences in the perception of naturalistic human movements

Introduction

It is widely believed that we are more attentive towards moving versus static stimuli. However, the neural correlates underlying the perception of human movements have not been extensively investigated in ecologically valid settings, nor has the developmental aspect of this phenomenon. Here, we set forth to investigate how human limb movements displayed in naturalistic videos influence the attentional engagement of children and young adults.

Methods

Thirty-nine healthy participants (4-26 years old) were presented with naturalistic videos featuring human goal-directed movements, while neural activity was recorded using electroencephalography (EEG). Video scenes were automatically annotated as containing arm, leg or no movement, using a machine learning algorithm. The viewers' attentional engagement was quantified by the intersubject correlation of EEG responses evoked by the videos.

Results

Our results demonstrate that scenes featuring limb movements, especially simultaneous arm and leg movements, elicit higher attentional engagement than scenes with no limb movement. Interestingly, this effect was found to diminish with age.

Discussion

Overall, our findings extend previous work on the perception of human motion by implementing naturalistic stimuli in the experimental design and extend the list of factors influencing the viewer's engagement exerted by naturalistic videos.

Altered Cerebral Processing of Videos in Children with Motor Dysfunction Suggests Broad Embodiment of Perceptual Cognitive Functions

Embodied cognition theory suggests that motor dysfunctions affect cognition. We examined this hypothesis by inspecting whether cerebral processing of movies, featuring both goal-directed movements and content without humans, differ between children with congenital motor dysfunction and healthy controls. Electroencephalography was recorded from 23 healthy children and 23 children with limited or absent arm movement due to either arthrogryposis multiplex congenita or obstetric brachial plexus palsy. Each individual patient exhibited divergent neural responses, disclosed by significantly lower inter-subject correlation (ISC) of brain activity, during the videos compared to the healthy children. We failed to observe associations between this finding and the motor-related content of the various video scenes, suggesting that differences between the patients and controls reflect modulation of perceptual-cognitive processing of videos by upper-limb motor dysfunctions not limited to the watching-mirroring of motor actions. Thus, perceptual-cognitive processes in the brain seem to be more robustly embodied than has previously been thought.

Strong displayed passion and preparedness of broadcaster in live streaming e-commerce increases consumers' neural engagement

Live streaming shopping, the streaming of real-time videos promoting products that consumers can purchase online, has recently been a booming area of e-commerce, especially during the COVID-19 pandemic. The success of live streaming e-commerce largely relies on the extent to which the broadcaster can get consumers engaged by the live stream. Thus, it is important to discover the antecedents of consumer engagement in such a context. Drawing on consumer engagement and neuroscience literature, this study used electroencephalography inter-subject correlation (EEG-ISC) to explore how broadcasters' entrepreneurial passion during live streaming videos influenced consumers' neural engagement as they watched the live streaming videos. We used the framework of displayed passion and preparedness from the entrepreneurial passion literature to predict consumer engagement. We found significant ISC for strong displayed passion, while preparedness had partially significant effects on the first, second, and summed components of ISC. The interaction effects of these two factors on the first and summed components of ISC were partially significant. Strong displayed passion and preparedness activated the left and right prefrontal regions of the consumers' brains. These findings indicate that broadcasters' displayed passion and preparedness can influence consumer engagement in live streaming e-commerce settings. Our findings suggest that a scientific approach could be used to improve a broadcaster's performance by testing ISC during rehearsals before live streaming.

Personality similarity predicts synchronous neural responses in fMRI and EEG data

Successful communication and cooperation among different members of society depends, in part, on a consistent understanding of the physical and social world. What drives this alignment in perspectives? We present evidence from two neuroimaging studies using functional magnetic resonance imaging (fMRI; N = 66 with 2145 dyadic comparisons) and electroencephalography (EEG; N = 225 with 25,200 dyadic comparisons) to show that: (1) the extent to which people’s neural responses are synchronized when viewing naturalistic stimuli is related to their personality profiles, and (2) that this effect is stronger than that of similarity in gender, ethnicity and political affiliation. The localization of the fMRI results in combination with the additional eye tracking analyses suggest that the relationship between personality similarity and neural synchrony likely reflects alignment in the interpretation of stimuli and not alignment in overt visual attention. Together, the findings suggest that similarity in psychological dispositions aligns people’s reality via shared interpretations of the external world.

Mapping and understanding of correlated electroencephalogram (EEG) responses to the newsvendor problem

Decision-making is one of the most critical activities of human beings. To better understand the underlying neurocognitive mechanism while making decisions under an economic context, we designed a decision-making paradigm based on the newsvendor problem (NP) with two scenarios: low-profit margins as the more challenging scenario and high-profit margins as the less difficult one. The EEG signals were acquired from healthy humans while subjects were performing the task. We adopted the Correlated Component Analysis (CorrCA) method to identify linear combinations of EEG channels that maximize the correlation across subjects ([Formula: see text]) or trials ([Formula: see text]). The inter-subject or inter-trial correlation values (ISC or ITC) of the first three components were estimated to investigate the modulation of the task difficulty on subjects' EEG signals and respective correlations. We also calculated the alpha- and beta-band power of the projection components obtained by the CorrCA to assess the brain responses across multiple task periods. Finally, the CorrCA forward models, which represent the scalp projections of the brain activities by the maximally correlated components, were further translated into source distributions of underlying cortical activity using the exact Low Resolution Electromagnetic Tomography Algorithm (eLORETA). Our results revealed strong and significant correlations in EEG signals among multiple subjects and trials during the more difficult decision-making task than the easier one. We also observed that the NP decision-making and feedback tasks desynchronized the normalized alpha and beta powers of the CorrCA components, reflecting the engagement state of subjects. Source localization results furthermore suggested several sources of neural activities during the NP decision-making process, including the dorsolateral prefrontal cortex, anterior PFC, orbitofrontal cortex, posterior cingulate cortex, and somatosensory association cortex.

Ear-EEG Measures of Auditory Attention to Continuous Speech

Auditory attention is an important cognitive function used to separate relevant from irrelevant auditory information. However, most findings on attentional selection have been obtained in highly controlled laboratory settings using bulky recording setups and unnaturalistic stimuli. Recent advances in electroencephalography (EEG) facilitate the measurement of brain activity outside the laboratory, and around-the-ear sensors such as the cEEGrid promise unobtrusive acquisition. In parallel, methods such as speech envelope tracking, intersubject correlations and spectral entropy measures emerged which allow us to study attentional effects in the neural processing of natural, continuous auditory scenes. In the current study, we investigated whether these three attentional measures can be reliably obtained when using around-the-ear EEG. To this end, we analyzed the cEEGrid data of 36 participants who attended to one of two simultaneously presented speech streams. Speech envelope tracking results confirmed a reliable identification of the attended speaker from cEEGrid data. The accuracies in identifying the attended speaker increased when fitting the classification model to the individual. Artifact correction of the cEEGrid data with artifact subspace reconstruction did not increase the classification accuracy. Intersubject correlations were higher for those participants attending to the same speech stream than for those attending to different speech streams, replicating previously obtained results with high-density cap-EEG. We also found that spectral entropy decreased over time, possibly reflecting the decrease in the listener's level of attention. Overall, these results support the idea of using ear-EEG measurements to unobtrusively monitor auditory attention to continuous speech. This knowledge may help to develop assistive devices that support listeners separating relevant from irrelevant information in complex auditory environments.

Teacher-student neural coupling during teaching and learning

Human communication is remarkably versatile, enabling teachers to share highly abstracted and novel information with their students. What neural processes enable such transfer of information across brains during naturalistic teaching and learning? Here, a teacher was scanned in functional magnetic resonance imaging while giving an oral lecture with slides on a scientific topic followed by a review lecture. Students were then scanned while watching either the intact Lecture and Review (N = 20) or a temporally scrambled version of the lecture (N = 20). Using intersubject correlation, we observed widespread Teacher-Student neural coupling spanning sensory cortex and language regions along the superior temporal sulcus as well as higher-level regions including posterior medial cortex (PMC), superior parietal lobule, and dorsolateral and dorsomedial prefrontal cortex. Teacher-student alignment in higher-level areas was not observed when learning was disrupted by temporally scrambling the lecture. Moreover, teacher-student coupling in PMC was significantly correlated with learning: the more closely the student's brain mirrored the teacher's brain, the more the student improved their learning score. Together, these results suggest that the alignment of neural responses between teacher and students may reflect effective communication of complex information across brains in classroom settings.

Neural Activity during Story Listening Is Synchronized across Individuals Despite Acoustic Masking

Older people with hearing problems often experience difficulties understanding speech in the presence of background sound. As a result, they may disengage in social situations, which has been associated with negative psychosocial health outcomes. Measuring listening (dis)engagement during challenging listening situations has received little attention thus far. We recruit young, normal-hearing human adults (both sexes) and investigate how speech intelligibility and engagement during naturalistic story listening is affected by the level of acoustic masking (12-talker babble) at different signal-to-noise ratios (SNRs). In Experiment 1, we observed that word-report scores were above 80% for all but the lowest SNR (-3 dB SNR) we tested, at which performance dropped to 54%. In Experiment 2, we calculated intersubject correlation (ISC) using EEG data to identify dynamic spatial patterns of shared neural activity evoked by the stories. ISC has been used as a neural measure of participants' engagement with naturalistic materials. Our results show that ISC was stable across all but the lowest SNRs, despite reduced speech intelligibility. Comparing ISC and intelligibility demonstrated that word-report performance declined more strongly with decreasing SNR compared to ISC. Our measure of neural engagement suggests that individuals remain engaged in story listening despite missing words because of background noise. Our work provides a potentially fruitful approach to investigate listener engagement with naturalistic, spoken stories that may be used to investigate (dis)engagement in older adults with hearing impairment.

Cognitive processing of a common stimulus synchronizes brains, hearts, and eyes

Neural, physiological, and behavioral signals synchronize between human subjects in a variety of settings. Multiple hypotheses have been proposed to explain this interpersonal synchrony, but there is no clarity under which conditions it arises, for which signals, or whether there is a common underlying mechanism. We hypothesized that cognitive processing of a shared stimulus is the source of synchrony between subjects, measured here as intersubject correlation (ISC). To test this, we presented informative videos to participants in an attentive and distracted condition and subsequently measured information recall. ISC was observed for electro-encephalography, gaze position, pupil size, and heart rate, but not respiration and head movements. The strength of correlation was co-modulated in the different signals, changed with attentional state, and predicted subsequent recall of information presented in the videos. There was robust within-subject coupling between brain, heart, and eyes, but not respiration or head movements. The results suggest that ISC is the result of effective cognitive processing, and thus emerges only for those signals that exhibit a robust brain-body connection. While physiological and behavioral fluctuations may be driven by multiple features of the stimulus, correlation with other individuals is co-modulated by the level of attentional engagement with the stimulus.

Behavioral Experience-Sampling Methods in Neuroimaging Studies With Movie and Narrative Stimuli

Movies and narratives are increasingly utilized as stimuli in functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), and electroencephalography (EEG) studies. Emotional reactions of subjects, what they pay attention to, what they memorize, and their cognitive interpretations are all examples of inner experiences that can differ between subjects during watching of movies and listening to narratives inside the scanner. Here, we review literature indicating that behavioral measures of inner experiences play an integral role in this new research paradigm via guiding neuroimaging analysis. We review behavioral methods that have been developed to sample inner experiences during watching of movies and listening to narratives. We also review approaches that allow for joint analyses of the behaviorally sampled inner experiences and neuroimaging data. We suggest that building neurophenomenological frameworks holds potential for solving the interrelationships between inner experiences and their neural underpinnings. Finally, we tentatively suggest that recent developments in machine learning approaches may pave way for inferring different classes of inner experiences directly from the neuroimaging data, thus potentially complementing the behavioral self-reports.

Unsupervised Clustering of Individuals Sharing Selective Attentional Focus Using Physiological Synchrony

Research on brain signals as indicators of a certain attentional state is moving from laboratory environments to everyday settings. Uncovering the attentional focus of individuals in such settings is challenging because there is usually limited information about real-world events, as well as a lack of data from the real-world context at hand that is correctly labeled with respect to individuals' attentional state. In most approaches, such data is needed to train attention monitoring models. We here investigate whether unsupervised clustering can be combined with physiological synchrony in the electroencephalogram (EEG), electrodermal activity (EDA), and heart rate to automatically identify groups of individuals sharing attentional focus without using knowledge of the sensory stimuli or attentional focus of any of the individuals. We used data from an experiment in which 26 participants listened to an audiobook interspersed with emotional sounds and beeps. Thirteen participants were instructed to focus on the narrative of the audiobook and 13 participants were instructed to focus on the interspersed emotional sounds and beeps. We used a broad range of commonly applied dimensionality reduction ordination techniques-further referred to as mappings-in combination with unsupervised clustering algorithms to identify the two groups of individuals sharing attentional focus based on physiological synchrony. Analyses were performed using the three modalities EEG, EDA, and heart rate separately, and using all possible combinations of these modalities. The best unimodal results were obtained when applying clustering algorithms on physiological synchrony data in EEG, yielding a maximum clustering accuracy of 85%. Even though the use of EDA or heart rate by itself did not lead to accuracies significantly higher than chance level, combining EEG with these measures in a multimodal approach generally resulted in higher classification accuracies than when using only EEG. Additionally, classification results of multimodal data were found to be more consistent across algorithms than unimodal data, making algorithm choice less important. Our finding that unsupervised classification into attentional groups is possible is important to support studies on attentional engagement in everyday settings.

Inter-subject Correlation While Listening to Minimalist Music: A Study of Electrophysiological and Behavioral Responses to Steve Reich's Piano Phase

Musical minimalism utilizes the temporal manipulation of restricted collections of rhythmic, melodic, and/or harmonic materials. One example, Steve Reich's Piano Phase, offers listeners readily audible formal structure with unpredictable events at the local level. For example, pattern recurrences may generate strong expectations which are violated by small temporal and pitch deviations. A hyper-detailed listening strategy prompted by these minute deviations stands in contrast to the type of listening engagement typically cultivated around functional tonal Western music. Recent research has suggested that the inter-subject correlation (ISC) of electroencephalographic (EEG) responses to natural audio-visual stimuli objectively indexes a state of "engagement," demonstrating the potential of this approach for analyzing music listening. But can ISCs capture engagement with minimalist music, which features less obvious expectation formation and has historically received a wide range of reactions? To approach this question, we collected EEG and continuous behavioral (CB) data while 30 adults listened to an excerpt from Steve Reich's Piano Phase, as well as three controlled manipulations and a popular-music remix of the work. Our analyses reveal that EEG and CB ISC are highest for the remix stimulus and lowest for our most repetitive manipulation, no statistical differences in overall EEG ISC between our most musically meaningful manipulations and Reich's original piece, and evidence that compositional features drove engagement in time-resolved ISC analyses. We also found that aesthetic evaluations corresponded well with overall EEG ISC. Finally we highlight co-occurrences between stimulus events and time-resolved EEG and CB ISC. We offer the CB paradigm as a useful analysis measure and note the value of minimalist compositions as a limit case for the neuroscientific study of music listening. Overall, our participants' neural, continuous behavioral, and question responses showed strong similarities that may help refine our understanding of the type of engagement indexed by ISC for musical stimuli.

Opportunities and Limitations of Mobile Neuroimaging Technologies in Educational Neuroscience

As the field of educational neuroscience continues to grow, questions have emerged regarding the ecological validity and applicability of this research to educational practice. Recent advances in mobile neuroimaging technologies have made it possible to conduct neuroscientific studies directly in naturalistic learning environments. We propose that embedding mobile neuroimaging research in a cycle (Matusz, Dikker, Huth, & Perrodin, 2019), involving lab-based, seminaturalistic, and fully naturalistic experiments, is well suited for addressing educational questions. With this review, we take a cautious approach, by discussing the valuable insights that can be gained from mobile neuroimaging technology, including electroencephalography and functional near-infrared spectroscopy, as well as the challenges posed by bringing neuroscientific methods into the classroom. Research paradigms used alongside mobile neuroimaging technology vary considerably. To illustrate this point, studies are discussed with increasingly naturalistic designs. We conclude with several ethical considerations that should be taken into account in this unique area of research.

Conscious processing of narrative stimuli synchronizes heart rate between individuals

Heart rate has natural fluctuations that are typically ascribed to autonomic function. Recent evidence suggests that conscious processing can affect the timing of the heartbeat. We hypothesized that heart rate is modulated by conscious processing and therefore dependent on attentional focus. To test this, we leverage the observation that neural processes synchronize between subjects by presenting an identical narrative stimulus. As predicted, we find significant inter-subject correlation of heart rate (ISC-HR) when subjects are presented with an auditory or audiovisual narrative. Consistent with our hypothesis, we find that ISC-HR is reduced when subjects are distracted from the narrative, and higher ISC-HR predicts better recall of the narrative. Finally, patients with disorders of consciousness have lower ISC-HR, as compared to healthy individuals. We conclude that heart rate fluctuations are partially driven by conscious processing, depend on attentional state, and may represent a simple metric to assess conscious state in unresponsive patients.

Neural signatures of attentional engagement during narratives and its consequences for event memory

As we comprehend narratives, our attentional engagement fluctuates over time. Despite theoretical conceptions of narrative engagement as emotion-laden attention, little empirical work has characterized the cognitive and neural processes that comprise subjective engagement in naturalistic contexts or its consequences for memory. Here, we relate fluctuations in narrative engagement to patterns of brain coactivation and test whether neural signatures of engagement predict subsequent memory. In behavioral studies, participants continuously rated how engaged they were as they watched a television episode or listened to a story. Self-reported engagement was synchronized across individuals and driven by the emotional content of the narratives. In functional MRI datasets collected as different individuals watched the same show or listened to the same story, engagement drove neural synchrony, such that default mode network activity was more synchronized across individuals during more engaging moments of the narratives. Furthermore, models based on time-varying functional brain connectivity predicted evolving states of engagement across participants and independent datasets. The functional connections that predicted engagement overlapped with a validated neuromarker of sustained attention and predicted recall of narrative events. Together, our findings characterize the neural signatures of attentional engagement in naturalistic contexts and elucidate relationships among narrative engagement, sustained attention, and event memory.

EEG-Based Intersubject Correlations Reflect Selective Attention in a Competing Speaker Scenario

Several solutions have been proposed to study the relationship between ongoing brain activity and natural sensory stimuli, such as running speech. Computing the intersubject correlation (ISC) has been proposed as one possible approach. Previous evidence suggests that ISCs between the participants' electroencephalogram (EEG) may be modulated by attention. The current study addressed this question in a competing-speaker paradigm, where participants (N = 41) had to attend to one of two concurrently presented speech streams. ISCs between participants' EEG were higher for participants attending to the same story compared to participants attending to different stories. Furthermore, we found that ISCs between individual and group data predicted whether an individual attended to the left or right speech stream. Interestingly, the magnitude of the shared neural response with others attending to the same story was related to the individual neural representation of the attended and ignored speech envelope. Overall, our findings indicate that ISC differences reflect the magnitude of selective attentional engagement to speech.

Cortical Responses to the Amplitude Envelopes of Sounds Change with Age

Many older listeners have difficulty understanding speech in noise, when cues to speech-sound identity are less redundant. The amplitude envelope of speech fluctuates dramatically over time, and features such as the rate of amplitude change at onsets (attack) and offsets (decay), signal critical information about the identity of speech sounds. Aging is also thought to be accompanied by increases in cortical excitability, which may differentially alter sensitivity to envelope dynamics. Here, we recorded electroencephalography in younger and older human adults (of both sexes) to investigate how aging affects neural synchronization to 4 Hz amplitude-modulated noises with different envelope shapes (ramped: slow attack and sharp decay; damped: sharp attack and slow decay). We observed that subcortical responses did not differ between age groups, whereas older compared with younger adults exhibited larger cortical responses to sound onsets, consistent with an increase in auditory cortical excitability. Neural activity in older adults synchronized more strongly to rapid-onset, slow-offset (damped) envelopes, was less sinusoidal, and was more peaked. Younger adults demonstrated the opposite pattern, showing stronger synchronization to slow-onset, rapid-offset (ramped) envelopes, as well as a more sinusoidal neural response shape. The current results suggest that age-related changes in the excitability of auditory cortex alter responses to envelope dynamics. This may be part of the reason why older adults experience difficulty understanding speech in noise.SIGNIFICANCE STATEMENT Many middle-aged and older adults report difficulty understanding speech when there is background noise, which can trigger social withdrawal and negative psychosocial health outcomes. The difficulty may be related to age-related changes in how the brain processes temporal sound features. We tested younger and older people on their sensitivity to different envelope shapes, using EEG. Our results demonstrate that aging is associated with heightened sensitivity to sounds with a sharp attack and gradual decay, and sharper neural responses that deviate from the sinusoidal features of the stimulus, perhaps reflecting increased excitability in the aged auditory cortex. Altered responses to temporal sound features may be part of the reason why older adults often experience difficulty understanding speech in social situations.

The default mode network: where the idiosyncratic self meets the shared social world

The default mode network (DMN) is classically considered an 'intrinsic' system, specializing in internally oriented cognitive processes such as daydreaming, reminiscing and future planning. In this Perspective, we suggest that the DMN is an active and dynamic 'sense-making' network that integrates incoming extrinsic information with prior intrinsic information to form rich, context-dependent models of situations as they unfold over time. We review studies that relied on naturalistic stimuli, such as stories and movies, to demonstrate how an individual's DMN neural responses are influenced both by external information accumulated as events unfold over time and by the individual's idiosyncratic past memories and knowledge. The integration of extrinsic and intrinsic information over long timescales provides a space for negotiating a shared neural code, which is necessary for establishing shared meaning, shared communication tools, shared narratives and, above all, shared communities and social networks.

Wisdom of crowds benefits perceptual decision making across difficulty levels

Decades of research on collective decision making has claimed that aggregated judgment of multiple individuals is more accurate than expert individual judgement. A longstanding problem in this regard has been to determine how decisions of individuals can be combined to form intelligent group decisions. Our study consisted of a random target detection task in natural scenes, where human subjects (18 subjects, 7 female) detected the presence or absence of a random target as indicated by the cue word displayed prior to stimulus display. Concurrently the neural activities (EEG signals) were recorded. A separate behavioural experiment was performed by different subjects (20 subjects, 11 female) on the same set of images to categorize the tasks according to their difficulty levels. We demonstrate that the weighted average of individual decision confidence/neural decision variables produces significantly better performance than the frequently used majority pooling algorithm. Further, the classification error rates from individual judgement were found to increase with increasing task difficulty. This error could be significantly reduced upon combining the individual decisions using group aggregation rules. Using statistical tests, we show that combining all available participants is unnecessary to achieve minimum classification error rate. We also try to explore if group aggregation benefits depend on the correlation between the individual judgements of the group and our results seem to suggest that reduced inter-subject correlation can improve collective decision making for a fixed difficulty level.

Morning brain: real-world neural evidence that high school class times matter

Researchers, parents and educators consistently observe a stark mismatch between biologically preferred and socially imposed sleep-wake hours in adolescents, fueling debate about high school start times. We contribute neural evidence to this debate with electroencephalogram data collected from high school students during their regular morning, mid-morning and afternoon classes. Overall, student alpha power was lower when class content was taught via videos than through lectures. Students' resting state alpha brain activity decreased as the day progressed, consistent with adolescents being least attentive early in the morning. During the lessons, students showed consistently worse performance and higher alpha power for early morning classes than for mid-morning classes, while afternoon quiz scores and alpha levels varied. Together, our findings demonstrate that both class activity and class time are reflected in adolescents' brain states in a real-world setting, and corroborate educational research suggesting that mid-morning may be the best time to learn.

Physiological Synchrony in EEG, Electrodermal Activity and Heart Rate Detects Attentionally Relevant Events in Time

Interpersonal physiological synchrony (PS), or the similarity of physiological signals between individuals over time, may be used to detect attentionally engaging moments in time. We here investigated whether PS in the electroencephalogram (EEG), electrodermal activity (EDA), heart rate and a multimodal metric signals the occurrence of attentionally relevant events in time in two groups of participants. Both groups were presented with the same auditory stimulus, but were instructed to attend either to the narrative of an audiobook (audiobook-attending: AA group) or to interspersed emotional sounds and beeps (stimulus-attending: SA group). We hypothesized that emotional sounds could be detected in both groups as they are expected to draw attention involuntarily, in a bottom-up fashion. Indeed, we found this to be the case for PS in EDA or the multimodal metric. Beeps, that are expected to be only relevant due to specific "top-down" attentional instructions, could indeed only be detected using PS among SA participants, for EDA, EEG and the multimodal metric. We further hypothesized that moments in the audiobook accompanied by high PS in either EEG, EDA, heart rate or the multimodal metric for AA participants would be rated as more engaging by an independent group of participants compared to moments corresponding to low PS. This hypothesis was not supported. Our results show that PS can support the detection of attentionally engaging events over time. Currently, the relation between PS and engagement is only established for well-defined, interspersed stimuli, whereas the relation between PS and a more abstract self-reported metric of engagement over time has not been established. As the relation between PS and engagement is dependent on event type and physiological measure, we suggest to choose a measure matching with the stimulus of interest. When the stimulus type is unknown, a multimodal metric is most robust.

Protocol for the Prognostication of Consciousness Recovery Following a Brain Injury

Individuals who have suffered a severe brain injury typically require extensive hospitalization in intensive care units (ICUs), where critical treatment decisions are made to maximize their likelihood of recovering consciousness and cognitive function. These treatment decisions can be difficult when the neurological assessment of the patient is limited by unreliable behavioral responses. Reliable objective and quantifiable markers are lacking and there is both (1) a poor understanding of the mechanisms underlying the brain's ability to reconstitute consciousness and cognition after an injury and (2) the absence of a reliable and clinically feasible method of tracking cognitive recovery in ICU survivors. Our goal is to develop and validate a clinically relevant EEG paradigm that can inform the prognosis of unresponsive, brain-injured patients in the ICU. This protocol describes a study to develop a point-of-care system intended to accurately predict outcomes of unresponsive, brain-injured patients in the ICU. We will recruit 200 continuously-sedated brain-injured patients across five ICUs. Between 24 h and 7 days post-ICU admission, high-density EEG will be recorded from behaviorally unresponsive patients before, during and after a brief cessation of pharmacological sedation. Once patients have reached the waking stage, they will be asked to complete an abridged Cambridge Brain Sciences battery, a web-based series of neurocognitive tests. The test series will be repeated every day during acute admission (ICU, ward), or as often as possible given the constraints of ICU and ward care. Following discharge, patients will continue to complete the same test series on weekly, and then monthly basis, for up to 12 months following injury. Functional outcomes will also be assessed up to 12 months post-injury. We anticipate our findings will lead to an increased ability to identify patients, as soon as possible after their brain injury, who are most likely to survive, and to make accurate predictions about their long-term cognitive and functional outcome. In addition to providing critically needed support for clinical decision-making, this study has the potential to transform our understanding of key functional EEG networks associated with consciousness and cognition.

Individualized assessment of residual cognition in patients with disorders of consciousness

Patients diagnosed with disorders of consciousness show minimal or inconsistent behavioural evidence of conscious awareness. However, using functional neuroimaging, recent research in clinical neuroscience has identified a subpopulation of these patients who reliably produce neural markers indicative of awareness. In this study, we recorded electroencephalograms during a response-free movie task to assess narrative processing in patients with disorders of consciousness. Thirteen patients diagnosed with a disorder of consciousness and 28 healthy controls participated in this study. We designed a movie-watching/listening paradigm involving two suspenseful movie clips, one audiovisual and one audio-only, and used electroencephalography to extract patterns of brain activity that were maximally correlated between subjects. These activity patterns served as electrophysiological indices of narrative processing, which were compared to the neural responses of patients during the same movies. Our analysis revealed two patterns of neural activity, one for each movie condition, that were significantly and reliably correlated between healthy participants. Of the twelve patients who watched the audiovisual movie, 25% produced a pattern of activity that was significantly correlated with the healthy group, while of the ten who listened to the audio narrative, 30% produced electrophysiological patterns similar to controls (one patient responded appropriately to both). The method presented here allows for rapid bedside assessment of higher-order cognitive processing in patients with disorders of consciousness. By leveraging the common neural response to movie stimuli, we were able to identify comparable patterns of brain activity in individual, behaviourally non-responsive patients, reflecting a capacity for narrative processing.

Movies and narratives as naturalistic stimuli in neuroimaging

Using movies and narratives as naturalistic stimuli in human neuroimaging studies has yielded significant advances in understanding of cognitive and emotional functions. The relevant literature was reviewed, with emphasis on how the use of naturalistic stimuli has helped advance scientific understanding of human memory, attention, language, emotions, and social cognition in ways that would have been difficult otherwise. These advances include discovering a cortical hierarchy of temporal receptive windows, which supports processing of dynamic information that accumulates over several time scales, such as immediate reactions vs. slowly emerging patterns in social interactions. Naturalistic stimuli have also helped elucidate how the hippocampus supports segmentation and memorization of events in day-to-day life and have afforded insights into attentional brain mechanisms underlying our ability to adopt specific perspectives during natural viewing. Further, neuroimaging studies with naturalistic stimuli have revealed the role of the default-mode network in narrative-processing and in social cognition. Finally, by robustly eliciting genuine emotions, these stimuli have helped elucidate the brain basis of both basic and social emotions apparently manifested as highly overlapping yet distinguishable patterns of brain activity.

Physiological synchrony in EEG, electrodermal activity and heart rate reflects shared selective auditory attention

OBJECTIVE

Concurrent changes in physiological signals across multiple listeners (physiological synchrony-PS), as caused by shared affective or cognitive processes, may be a suitable marker of selective attentional focus. We aimed to identify the selective attention of participants based on PS with individuals sharing attention with respect to different stimulus aspects.

APPROACH

We determined PS in electroencephalography (EEG), electrodermal activity (EDA) and electrocardiographic inter-beat interval (IBI) of participants who all heard the exact same audio track, but were instructed to either attend to the audiobook or to interspersed auditory events such as affective sounds and beeps that attending participants needed to keep track of.

MAIN RESULTS

PS in all three measures reflected the selective attentional focus of participants. In EEG and EDA, PS was higher for participants when linked to participants with the same attentional instructions than when linked to participants instructed to focus on different stimulus aspects, but in IBI this effect did not reach significance. Comparing PS between a participant and members from the same or the different attentional group allowed for the correct identification of the participant's attentional instruction in 96%, 73% and 73% of the cases, for EEG, EDA and IBI, respectively, all well above chance level. PS with respect to the attentional groups also predicted performance on post-audio questions about the groups' stimulus content.

SIGNIFICANCE

Our results show that selective attention of participants can be monitored using PS, not only in EEG, but also in EDA and IBI. These results are promising for real-world applications, where wearables measuring peripheral signals like EDA and IBI may be preferred over EEG sensors.

Olfaction Modulates Inter-Subject Correlation of Neural Responses

Odors can be powerful stimulants. It is well-established that odors provide strong cues for recall of locations, people and events. The effects of specific scents on other cognitive functions are less well-established. We hypothesized that scents with different odor qualities will have a different effect on attention. To assess attention, we used Inter-Subject Correlation of the EEG because this metric is strongly modulated by attentional engagement with natural audiovisual stimuli. We predicted that scents known to be "energizing" would increase Inter-Subject Correlation during watching of videos as compared to "calming" scents. In a first experiment, we confirmed this for eucalyptol and linalool while participants watched animated autobiographical narratives. The result was replicated in a second experiment, but did not generalize to limonene, also considered an "energizing" odorant. In a third, double-blind experiment, we tested a battery of scents including single molecules, as well as mixtures, as participants watched various short video clips. We found a varying effect of odor on Inter-Subject Correlation across the various scents. This study provides a basis for reliably and reproducibly assessing effects of odors on brain activity. Future research is needed to further explore the effect of scent-based up-modulation in engagement on learning and memory performance. Educators, product developers and fragrance brands might also benefit from such objective neurophysiological measures.

The effect of movie-watching on electroencephalographic responses to tactile stimulation

Movie-watching is becoming a popular acquisition method to increase compliance and enable neuroimaging data collection in challenging populations such as children, with potential to facilitate studying the somatosensory system. However, relatively little is known about the possible crossmodal (audiovisual) influence of movies on cortical somatosensory processing. In this study, we examined the impact of dynamic audiovisual movies on concurrent cortical somatosensory processing using electroencephalography (EEG). Forty healthy young adults (18-25 years) received passive tactile fingertip stimulation while watching an "entertaining" movie and a novel "low-demand" movie called 'Inscapes' compared to eyes-open rest. Watching a movie did not modulate properties of early or late somatosensory-evoked potentials (SEPs). Similarly, no crossmodal influence on somatosensory adaptation, denoted by a reduction in SEP amplitude with repetitive tactile stimulation, was found. The prominent oscillatory responses in the alpha and beta frequency bands following tactile stimulation differed as a function of viewing condition, with stronger alpha/beta event-related desynchronization (ERD) during movie-watching compared to rest. These findings highlight that movie-watching is a valid acquisition method during which SEPs can be measured in basic research and clinical studies, but that the attentional demands of movies need to be taken into account when performing oscillatory analyses.

Natural music evokes correlated EEG responses reflecting temporal structure and beat

The brain activity of multiple subjects has been shown to synchronize during salient moments of natural stimuli, suggesting that correlation of neural responses indexes a brain state operationally termed 'engagement'. While past electroencephalography (EEG) studies have considered both auditory and visual stimuli, the extent to which these results generalize to music-a temporally structured stimulus for which the brain has evolved specialized circuitry-is less understood. Here we investigated neural correlation during natural music listening by recording EEG responses from N=48 adult listeners as they heard real-world musical works, some of which were temporally disrupted through shuffling of short-term segments (measures), reversal, or randomization of phase spectra. We measured correlation between multiple neural responses (inter-subject correlation) and between neural responses and stimulus envelope fluctuations (stimulus-response correlation) in the time and frequency domains. Stimuli retaining basic musical features, such as rhythm and melody, elicited significantly higher behavioral ratings and neural correlation than did phase-scrambled controls. However, while unedited songs were self-reported as most pleasant, time-domain correlations were highest during measure-shuffled versions. Frequency-domain measures of correlation (coherence) peaked at frequencies related to the musical beat, although the magnitudes of these spectral peaks did not explain the observed temporal correlations. Our findings show that natural music evokes significant inter-subject and stimulus-response correlations, and suggest that the neural correlates of musical 'engagement' may be distinct from those of enjoyment.