Audience preferences are predicted by temporal reliability of neural processing

Inter-subject correlation of audience facial expressions predicts audience engagement during theatrical performances

During performances, audiences experience various emotional states, and these are reflected in their ongoing facial expressions. We investigated if audience engagement could be determined by measuring the inter-subject correlation (ISC) of non-invasively recorded audience facial expressions. We filmed the faces of multiple audience members at theatrical performances and determined the intensity of their different facial expressions throughout the performances. Neutral, happy, anger, and disgust expression ISCs accounted for up to 24% of the performance dramaturge's predictions of audience engagement. Expression synchrony was greater between individuals in close proximity, suggesting effects of emotional contagion or cognitive similarities between neighboring individuals, whereas expression synchrony was greatest between individuals who were younger, female, and with greater levels of empathy, showing that individual characteristics impact shared audience experiences. Together, our results show that facial expression synchronization could be used as a real-time non-invasive indicator of engagement in audiences larger than achieved using previous approaches.

Intersubject correlations in reward and mentalizing brain circuits separately predict persuasiveness of two types of ISIS video propaganda

The Islamist group ISIS has been particularly successful at recruiting Westerners as terrorists. A hypothesized explanation is their simultaneous use of two types of propaganda: Heroic narratives, emphasizing individual glory, alongside Social narratives, which emphasize oppression against Islamic communities. In the current study, functional MRI was used to measure brain responses to short ISIS propaganda videos distributed online. Participants were shown 4 Heroic and 4 Social videos categorized as such by another independent group of subjects. Persuasiveness was measured using post-scan predictions of recruitment effectiveness. Inter-subject correlation (ISC) was used to measure commonality of brain activity time courses across individuals. ISCs in ventral striatum predicted rated persuasiveness for Heroic videos, while ISCs in mentalizing and default networks, especially in dmPFC, predicted rated persuasiveness for Social videos. This work builds on past findings that engagement of the reward circuit and of mentalizing brain regions predicts preferences and persuasion. The observed dissociation as a function of stimulus type is novel, as is the finding that intersubject synchrony in ventral striatum predicts rated persuasiveness. These exploratory results identify possible neural mechanisms by which political extremists successfully recruit prospective members and specifically support the hypothesized distinction between Heroic and Social narratives for ISIS propaganda.

Inter-subject correlation of electroencephalographic and behavioural responses reflects time-varying engagement with natural music

Musical engagement can be conceptualized through various activities, modes of listening and listener states. Recent research has reported that a state of focused engagement can be indexed by the inter-subject correlation (ISC) of audience responses to a shared naturalistic stimulus. While statistically significant ISC has been reported during music listening, we lack insight into the temporal dynamics of engagement over the course of musical works-such as those composed in the Western classical style-which involve the formulation of expectations that are realized or derailed at subsequent points of arrival. Here, we use the ISC of electroencephalographic (EEG) and continuous behavioural (CB) responses to investigate the time-varying dynamics of engagement with functional tonal music. From a sample of adult musicians who listened to a complete cello concerto movement, we found that ISC varied throughout the excerpt for both measures. In particular, significant EEG ISC was observed during periods of musical tension that built to climactic highpoints, while significant CB ISC corresponded more to declarative entrances and points of arrival. Moreover, we found that a control stimulus retaining envelope characteristics of the intact music, but little other temporal structure, also elicited significantly correlated EEG and CB responses, though to lesser extents than the original version. In sum, these findings shed light on the temporal dynamics of engagement during music listening and clarify specific aspects of musical engagement that may be indexed by each measure.

Consistent movement of viewers' facial keypoints while watching emotionally evocative videos

Neuropsychological research aims to unravel how diverse individuals' brains exhibit similar functionality when exposed to the same stimuli. The evocation of consistent responses when different subjects watch the same emotionally evocative stimulus has been observed through modalities like fMRI, EEG, physiological signals and facial expressions. We refer to the quantification of these shared consistent signals across subjects at each time instant across the temporal dimension as Consistent Response Measurement (CRM). CRM is widely explored through fMRI, occasionally with EEG, physiological signals and facial expressions using metrics like Inter-Subject Correlation (ISC). However, fMRI tools are expensive and constrained, while EEG and physiological signals are prone to facial artifacts and environmental conditions (such as temperature, humidity, and health condition of subjects). In this research, facial expression videos are used as a cost-effective and flexible alternative for CRM, minimally affected by external conditions. By employing computer vision-based automated facial keypoint tracking, a new metric similar to ISC, called the Average t-statistic, is introduced. Unlike existing facial expression-based methodologies that measure CRM of secondary indicators like inferred emotions, keypoint, and ICA-based features, the Average t-statistic is closely associated with the direct measurement of consistent facial muscle movement using the Facial Action Coding System (FACS). This is evidenced in DISFA dataset where the time-series of Average t-statistic has a high correlation (R2 = 0.78) with a metric called AU consistency, which directly measures facial muscle movement through FACS coding of video frames. The simplicity of recording facial expressions with the automated Average t-statistic expands the applications of CRM such as measuring engagement in online learning, customer interactions, etc., and diagnosing outliers in healthcare conditions like stroke, autism, depression, etc. To promote further research, we have made the code repository publicly available.

Bidirectional brain-body interactions during natural story listening

Narratives can synchronize neural and physiological signals between individuals, but the relationship between these signals, and the underlying mechanism, is unclear. We hypothesized a top-down effect of cognition on arousal and predicted that auditory narratives will drive not only brain signals but also peripheral physiological signals. We find that auditory narratives entrained gaze variation, saccade initiation, pupil size, and heart rate. This is consistent with a top-down effect of cognition on autonomic function. We also hypothesized a bottom-up effect, whereby autonomic physiology affects arousal. Controlled breathing affected pupil size, and heart rate was entrained by controlled saccades. Additionally, fluctuations in heart rate preceded fluctuations of pupil size and brain signals. Gaze variation, pupil size, and heart rate were all associated with anterior-central brain signals. Together, these results suggest bidirectional causal effects between peripheral autonomic function and central brain circuits involved in the control of arousal.

Parental involvement affects parent-adolescents brain-to-brain synchrony when experiencing different emotions together: An EEG-based hyperscanning study

Parental involvement (PI) is a broad and multifaceted construct, which refers to the parental demonstration of interest in their child, caring, and warmth (Davis et al., 2021). Parental involvement affects parent-adolescent's relationship and communication. However, there was little research to examine the underlying neural mechanism. The present study aimed to explore how parental involvement is associated with the brain-to-brain synchronous activation between parent-adolescent dyads when sharing emotional experience together by using the electroencephalograph (EEG) hyperscanning. EEG was recorded simultaneously in a sample of 26 parent-adolescent dyads (Mparents'age=43.312, SD=5.468; Madolescents' age=12.077, SD=1.412) when completing the picture processing task. Phase locking values (PLVs) in beta band and gamma band were used to compare the differences in the parent-adolescent dyads' induced brain-to-brain synchrony between the high parental involvement group (HPI) and the low parental involvement group (LPI). Results showed that greater beta brain-to-brain synchrony was observed in the HPIs than in the LPIs when experiencing positive emotions together in the central region. However, there was no significant difference between the HPIs and the LPIs in the negative and neutral condition. Moreover, greater gamma brain-to-brain synchrony was observed when viewing negative emotional stimuli together than viewing positive emotional stimuli together in the LPIs in the central region. However, there was no significant difference between different emotional conditions in the HPIs. Findings of the present study provide neuroscientific evidence that parental involvement may strengthen parent-adolescent's emotional interaction and communication.

Identifying temporal correlations between natural single-shot videos and EEG signals

Objective.Electroencephalography (EEG) is a widely used technology for recording brain activity in brain-computer interface (BCI) research, where understanding the encoding-decoding relationship between stimuli and neural responses is a fundamental challenge. Recently, there is a growing interest in encoding-decoding natural stimuli in a single-trial setting, as opposed to traditional BCI literature where multi-trial presentations of synthetic stimuli are commonplace. While EEG responses to natural speech have been extensively studied, such stimulus-following EEG responses to natural video footage remain underexplored.Approach.We collect a new EEG dataset with subjects passively viewing a film clip and extract a few video features that have been found to be temporally correlated with EEG signals. However, our analysis reveals that these correlations are mainly driven by shot cuts in the video. To avoid the confounds related to shot cuts, we construct another EEG dataset with natural single-shot videos as stimuli and propose a new set of object-based features.Main results.We demonstrate that previous video features lack robustness in capturing the coupling with EEG signals in the absence of shot cuts, and that the proposed object-based features exhibit significantly higher correlations. Furthermore, we show that the correlations obtained with these proposed features are not dominantly driven by eye movements. Additionally, we quantitatively verify the superiority of the proposed features in a match-mismatch task. Finally, we evaluate to what extent these proposed features explain the variance in coherent stimulus responses across subjects.Significance.This work provides valuable insights into feature design for video-EEG analysis and paves the way for applications such as visual attention decoding.

Reliability and generalizability of neural speech tracking in younger and older adults

Neural tracking of spoken speech is considered a potential clinical biomarker for speech-processing difficulties, but the reliability of neural speech tracking is unclear. Here, younger and older adults listened to stories in two sessions while electroencephalography was recorded to investigate the reliability and generalizability of neural speech tracking. Speech tracking amplitude was larger for older than younger adults, consistent with an age-related loss of inhibition. The reliability of neural speech tracking was moderate (ICC ∼0.5-0.75) and tended to be higher for older adults. However, reliability was lower for speech tracking than for neural responses to noise bursts (ICC >0.8), which we used as a benchmark for maximum reliability. Neural speech tracking generalized moderately across different stories (ICC ∼0.5-0.6), which appeared greatest for audiobook-like stories spoken by the same person. Hence, a variety of stories could possibly be used for clinical assessments. Overall, the current data are important for developing a biomarker of speech processing but suggest that further work is needed to increase the reliability to meet clinical standards.

An evaluation of inter-brain EEG coupling methods in hyperscanning studies

EEG-based hyperscanning technology has been increasingly applied to analyze interpersonal interactions in social neuroscience in recent years. However, different methods are employed in various of studies without a complete investigation of the suitability of these methods. Our study aimed to systematically compare typical inter-brain EEG coupling methods, with simulated EEG data generated by real EEG data. In particular, two critical metrics of noise level and time delay were manipulated, and three different coupling models were tested. The results revealed that: (1) under certain conditions, various methods were leveraged by noise level and time delay, leading to different performances; (2) most algorithms achieved better experimental results and performance under high coupling degree; (3) with our simulation process, temporal and spectral models showed relatively good results, while data simulated with phase coupling model performed worse. This is the first systematic comparison of typical inter-brain EEG coupling methods, with simulated EEG data generated by real EEG data from different subjects. Existing methods mainly focused on intra-brain coupling. To our knowledge, there was only one previous study that compared five inter-brain EEG coupling methods (Burgess in Front Human Neurosci 7:881, 2013). However, the simulated data used in this study were generated time series with varied degrees of phase coupling without considering any EEG characteristics. For future research, appropriate methods need to be selected based on possible underlying mechanisms (temporal, spectral and phase coupling model hypothesis) of a specific study, as well as the expected coupling degree and conditions.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09911-1.

Searching for the Metaverse: Neuroscience of Physical and Digital Communities

What distinguishes real-world communities from their online counterparts? Social and cognitive neuroscience research on social networks and collective intentionality will be used in the article to answer this question. Physical communities are born in places. And places engage "we-mode" neurobiological and cognitive processes as behavioral synchrony, shared attention, deliberate attunement, interbrain synchronization, and so on, which create coherent social networks of very different individuals who are supported by a "wisdom of crowd." Digital technologies remove physical boundaries, giving people more freedom to choose their activities and groups. At the same time, however, the lack of physical co-presence of community members significantly reduces their possibility of activating "we-mode" cognitive processes and social motivation. Because of this, unlike physical communities that allow interaction between people from varied origins and stories, digital communities are always made up of people who have the same interests and knowledge (communities of practice). This new situation disrupts the "wisdom of crowd," making the community more radical and less accurate (polarization effect), allowing influential users to wield disproportionate influence over the group's beliefs, and producing inequalities in the distribution of social capital. However, a new emergent technology-the Metaverse-has the potential to reverse this trend. Several studies have revealed that virtual and augmented reality-the major technologies underlying the Metaverse-can engage the same neurobiological and cognitive "we-mode" processes as real-world environments. If the many flaws in this technology are fixed, it might encourage people to engage in more meaningful and constructive interactions in online communities.

Neuroselling: applying neuroscience to selling for a new business perspective. An analysis on teleshopping advertising

This paper presents an innovative research project that aims to study the emotional factors influencing decision-making elicited by infomercials, a powerful sales technique that uses emotional communication to engage viewers, capture attention, and build trust. Using cutting-edge consumer neuroscience techniques, this study focuses on the identification of the variables that most impact the Call-to-Action and Purchase Intention. Forty participants were selected and divided into two groups, with each group exposed to one of two infomercials (condition A = male seller; condition B = female seller). EEG signals were recorded as well as Eye-tracking data. After the viewing, participants completed a self-report questionnaire. Results show that seller characteristics such as Performance and Trustworthiness, as well as Neurophysiological variables such as Approach-Withdrawal Index, Willingness to Pay, Attention and Engagement, significantly impact the final Call-to-Action, Purchase Intention, and infomercial Likeability responses. Moreover, eye-tracking data revealed that the more time is spent observing crucial areas of the infomercial, the more it will increase our Willingness to Pay and our interest and willingness to approach the infomercial and product. These findings highlight the importance of considering both the Seller attributes and the consumers' Neurophysiological responses to understand and predict their behaviors in response to marketing stimuli since they all seem to play a crucial role in shaping consumers' attitudes and purchase intentions. Overall, the study is a significant pilot in the new field of neuroselling, shedding light on crucial emotional aspects of the seller/buyer relationship and providing valuable insights for researchers and marketers.

Music-emotion EEG coupling effects based on representational similarity

BACKGROUND

Music can evoke intense emotions and music emotion is a complex cognitive process. However, we know little about the cognitive mechanisms underlying these processes, and there are significant individual differences in the emotional responses to the same musical stimuli.

NEW METHOD

We used the inter-subject representational similarity analysis (IS-RSA) method to investigate the shared music emotion responses across multiple participants. In addition, we extended IS-RSA to estimate the group cross-frequency coupling effects of music emotion. Based on the cross-frequency coupling IS-RSA, we analyzed the differences in cross-frequency coupling patterns under different music emotions using MI. Comparison of existing methods: most current IS-RSA analyses focus on within-frequency band analysis. However, the cognitive processing of music emotion involves not only activation and brain network connections differences within frequency bands but also information communication between frequency bands.

RESULTS

The results of the within-frequency band IS-RSA analysis showed that the theta and gamma frequency bands play important roles in the inter-participant consistency of music emotion. The inter-frequency band IS-RSA analysis showed that the theta-beta coupling pattern exhibited stronger inter-participant consistency compared to the theta-gamma coupling pattern, and the theta-beta coupling had significant consistent representation across various music conditions. Through the significant regions of cross-frequency coupling representation similarity analysis, we performed phase-amplitude coupling analysis on FC4-C6 and FC4-Pz connections. For the theta-beta coupling pattern, we found that the MI of these two connections exhibited different coupling patterns under different music conditions, and they showed a significant decrease compared to the baseline period.

Single-trial visually evoked potentials predict both individual choice and market outcomes

A central assumption in the behavioral sciences is that choice behavior generalizes enough across individuals that measurements from a sampled group can predict the behavior of the population. Following from this assumption, the unit of behavioral sampling or measurement for most neuroimaging studies is the individual; however, cognitive neuroscience is increasingly acknowledging a dissociation between neural activity that predicts individual behavior and that which predicts the average or aggregate behavior of the population suggesting a greater importance of individual differences than is typically acknowledged. For instance, past work has demonstrated that some, but not all, of the neural activity observed during value-based decision-making is able to predict not just individual subjects' choices but also the success of products on large, online marketplaces-even when those two behavioral outcomes deviate from one another-suggesting that some neural component processes of decision-making generalize to aggregate market responses more readily across individuals than others do. While the bulk of such research has highlighted affect-related neural responses (i.e. in the nucleus accumbens) as a better predictor of group-level behavior than frontal cortical activity associated with the integration of more idiosyncratic choice components, more recent evidence has implicated responses in visual cortical regions as strong predictors of group preference. Taken together, these findings suggest a role of neural responses during early perception in reinforcing choice consistency across individuals and raise fundamental scientific questions about the role sensory systems in value-based decision-making processes. We use a multivariate pattern analysis approach to show that single-trial visually evoked electroencephalographic (EEG) activity can predict individual choice throughout the post-stimulus epoch; however, a nominally sparser set of activity predicts the aggregate behavior of the population. These findings support an account in which a subset of the neural activity underlying individual choice processes can scale to predict behavioral consistency across people, even when the choice behavior of the sample does not match the aggregate behavior of the population.

Intersubject similarity in neural representations underlies shared episodic memory content

Individuals generally form their unique memories from shared experiences, yet the neural representational mechanisms underlying this subjectiveness of memory are poorly understood. The current study addressed this important question from the cross-subject neural representational perspective, leveraging a large functional magnetic resonance imaging dataset (n = 415) of a face-name associative memory task. We found that individuals' memory abilities were predicted by their synchronization to the group-averaged, canonical trial-by-trial activation level and, to a lesser degree, by their similarity to the group-averaged representational patterns during encoding. More importantly, the memory content shared between pairs of participants could be predicted by their shared local neural activation pattern, particularly in the angular gyrus and ventromedial prefrontal cortex, even after controlling for differences in memory abilities. These results uncover neural representational mechanisms for individualized memory and underscore the constructive nature of episodic memory.

Inter-subject correlations of EEG reflect subjective arousal and acoustic features of music

Background

Research on music-induced emotion and brain activity is constantly expanding. Although studies using inter-subject correlation (ISC), a collectively shared brain activity analysis method, have been conducted, whether ISC during music listening represents the music preferences of a large population remains uncertain; additionally, it remains unclear which factors influence ISC during music listening. Therefore, here, we aimed to investigate whether the ISCs of electroencephalography (EEG) during music listening represent a preference for music reflecting engagement or interest of a large population in music.

Methods

First, we selected 21 pieces of music from the Billboard Japan Hot 100 chart of 2017, which served as an indicator of preference reflecting the engagement and interest of a large population. To ensure even representation, we chose one piece for every fifth song on the chart, spanning from highly popular music to less popular ones. Next, we recorded EEG signals while the subjects listened to the selected music, and they were asked to evaluate four aspects (preference, enjoyment, frequency of listening, and arousal) for each song. Subsequently, we conducted ISC analysis by utilizing the first three principal components of EEG, which were highly correlated across subjects and extracted through correlated component analysis (CorrCA). We then explored whether music with high preferences that reflected the engagement and interest of large population had high ISC values. Additionally, we employed cluster analysis on all 21 pieces of music, utilizing the first three principal components of EEG, to investigate the impact of emotions and musical characteristics on EEG ISC during music listening.

Results

A significant distinction was noted between the mean ISC values of the 10 higher-ranked pieces of music compared to the 10 lower-ranked pieces of music [t(542) = -1.97, p = 0.0025]. This finding suggests that ISC values may correspond preferences reflecting engagement or interest of a large population. Furthermore, we found that significant variations were observed in the first three principal component values among the three clusters identified through cluster analysis, along with significant differences in arousal levels. Moreover, the characteristics of the music (tonality and tempo) differed among the three clusters. This indicates that the principal components, which exhibit high correlation among subjects and were employed in calculating ISC values, represent both subjects' arousal levels and specific characteristics of the music.

Conclusion

Subjects' arousal values during music listening and music characteristics (tonality and tempo) affect ISC values, which represent the interest of a large population in music.

Measuring real-time cognitive engagement in remote audiences

Responses to arts and entertainment media offer a valuable window into human behaviour. Many individuals worldwide spend the vast majority of their leisure time engaging with video content at home. However, there are few ways to study engagement and attention in this natural home viewing context. We used motion-tracking of the head via a web-camera to measure real-time cognitive engagement in 132 individuals while they watched 30 min of streamed theatre content at home. Head movement was negatively associated with engagement across a constellation of measures. Individuals who moved less reported feeling more engaged and immersed, evaluated the performance as more engaging, and were more likely to express interest in watching further. Our results demonstrate the value of in-home remote motion tracking as a low-cost, scalable metric of cognitive engagement, which can be used to collect audience behaviour data in a natural setting.

Accurately predicting hit songs using neurophysiology and machine learning

Identifying hit songs is notoriously difficult. Traditionally, song elements have been measured from large databases to identify the lyrical aspects of hits. We took a different methodological approach, measuring neurophysiologic responses to a set of songs provided by a streaming music service that identified hits and flops. We compared several statistical approaches to examine the predictive accuracy of each technique. A linear statistical model using two neural measures identified hits with 69% accuracy. Then, we created a synthetic set data and applied ensemble machine learning to capture inherent non-linearities in neural data. This model classified hit songs with 97% accuracy. Applying machine learning to the neural response to 1st min of songs accurately classified hits 82% of the time showing that the brain rapidly identifies hit music. Our results demonstrate that applying machine learning to neural data can substantially increase classification accuracy for difficult to predict market outcomes.

DeePay: deep learning decodes EEG to predict consumer's willingness to pay for neuromarketing

There is an increasing demand within consumer-neuroscience (or neuromarketing) for objective neural measures to quantify consumers' subjective valuations and predict responses to marketing campaigns. However, the properties of EEG raise difficulties for these aims: small datasets, high dimensionality, elaborate manual feature extraction, intrinsic noise, and between-subject variations. We aimed to overcome these limitations by combining unique techniques of Deep Learning Networks (DLNs), while providing interpretable results for neuroscientific and decision-making insight. In this study, we developed a DLN to predict subjects' willingness to pay (WTP) based on their EEG data. In each trial, 213 subjects observed a product's image, from 72 possible products, and then reported their WTP for the product. The DLN employed EEG recordings from product observation to predict the corresponding reported WTP values. Our results showed 0.276 test root-mean-square-error and 75.09% test accuracy in predicting high vs. low WTP, surpassing other models and a manual feature extraction approach. Network visualizations provided the predictive frequencies of neural activity, their scalp distributions, and critical timepoints, shedding light on the neural mechanisms involved with evaluation. In conclusion, we show that DLNs may be the superior method to perform EEG-based predictions, to the benefit of decision-making researchers and marketing practitioners alike.

Higher levels of narrativity lead to similar patterns of posterior EEG activity across individuals

Introduction

The focus of cognitive and psychological approaches to narrative has not so much been on the elucidation of important aspects of narrative, but rather on using narratives as tools for the investigation of higher order cognitive processes elicited by narratives (e.g., understanding, empathy, etc.). In this study, we work toward a scalar model of narrativity, which can provide testable criteria for selecting and classifying communication forms in their level of narrativity. We investigated whether being exposed to videos with different levels of narrativity modulates shared neural responses, measured by inter-subject correlation, and engagement levels.

Methods

Thirty-two participants watched video advertisements with high-level and low-level of narrativity while their neural responses were measured through electroencephalogram. Additionally, participants' engagement levels were calculated based on the composite of their self-reported attention and immersion scores.

Results

Results demonstrated that both calculated inter-subject correlation and engagement scores for high-level video ads were significantly higher than those for low-level, suggesting that narrativity levels modulate inter-subject correlation and engagement.

Discussion

We believe that these findings are a step toward the elucidation of the viewers' way of processing and understanding a given communication artifact as a function of the narrative qualities expressed by the level of narrativity.

Integrating media content analysis, reception analysis, and media effects studies

Every day, the world of media is at our fingertips, whether it is watching movies, listening to the radio, or browsing online media. On average, people spend over 8 h per day consuming messages from the mass media, amounting to a total lifetime dose of more than 20 years in which conceptual content stimulates our brains. Effects from this flood of information range from short-term attention bursts (e.g., by breaking news features or viral 'memes') to life-long memories (e.g., of one's favorite childhood movie), and from micro-level impacts on an individual's memory, attitudes, and behaviors to macro-level effects on nations or generations. The modern study of media's influence on society dates back to the 1940s. This body of mass communication scholarship has largely asked, "what is media's effect on the individual?" Around the time of the cognitive revolution, media psychologists began to ask, "what cognitive processes are involved in media processing?" More recently, neuroimaging researchers started using real-life media as stimuli to examine perception and cognition under more natural conditions. Such research asks: "what can media tell us about brain function?" With some exceptions, these bodies of scholarship often talk past each other. An integration offers new insights into the neurocognitive mechanisms through which media affect single individuals and entire audiences. However, this endeavor faces the same challenges as all interdisciplinary approaches: Researchers with different backgrounds have different levels of expertise, goals, and foci. For instance, neuroimaging researchers label media stimuli as "naturalistic" although they are in many ways rather artificial. Similarly, media experts are typically unfamiliar with the brain. Neither media creators nor neuroscientifically oriented researchers approach media effects from a social scientific perspective, which is the domain of yet another species. In this article, we provide an overview of approaches and traditions to studying media, and we review the emerging literature that aims to connect these streams. We introduce an organizing scheme that connects the causal paths from media content → brain responses → media effects and discuss network control theory as a promising framework to integrate media content, reception, and effects analyses.

Audience immersion: validating attentional and physiological measures against self-report

When an audience member becomes immersed, their attention shifts towards the media and story, and they allocate cognitive resources to represent events and characters. Here, we investigate whether it is possible to measure immersion using continuous behavioural and physiological measures. Using television and film clips, we validated dual-task reaction times, heart rate, and skin conductance against self-reported narrative engagement. We find that reaction times to a secondary task were strongly positively correlated with self-reported immersion: slower reaction times were indicative of greater immersion, particularly emotional engagement. Synchrony in heart rate across participants was associated with self-reported attentional and emotional engagement with the story, although we found no such relationship with skin conductance. These results establish both dual-task reaction times and heart rate as candidate measures for the real-time, continuous, assessment of audience immersion.

Spatiotemporal consistency of neural responses to repeatedly presented video stimuli accounts for population preferences

Population preferences for video advertisements vary across short video clips. What underlies these differences? Repeatedly watching a video clip may produce a consistent spatiotemporal pattern of neural activity that is dependent on the individual and the stimulus. Moreover, such consistency may be associated with the degree of engagement and memory of individual viewers. Since the population preferences are associated with the engagement and memory of the individual viewers, the consistency observed in a smaller group of viewers can be a predictor of population preferences. To test the hypothesis, we measured the degree of inter-trial consistency in participants' electroencephalographic (EEG) responses to repeatedly presented television commercials. We observed consistency in the neural activity patterns across repetitive views and found that the similarity in the spatiotemporal patterns of neural responses while viewing popular television commercials predicts population preferences obtained from a large audience. Moreover, a regression model that used two datasets, including two separate groups of participants viewing different stimulus sets, showed good predictive performance in a leave-one-out cross-validation. These findings suggest that universal spatiotemporal patterns in EEG responses can account for population-level human behaviours.

Naturalistic viewing increases individual identifiability based on connectivity within functional brain networks

Naturalistic viewing (NV) is currently considered a promising paradigm for studying individual differences in functional brain organization. While whole brain functional connectivity (FC) under NV has been relatively well characterized, so far little work has been done on a network level. Here, we extend current knowledge by characterizing the influence of NV on FC in fourteen meta-analytically derived brain networks considering three different movie stimuli in comparison to resting-state (RS). We show that NV increases identifiability of individuals over RS based on functional connectivity in certain, but not all networks. Furthermore, movie stimuli including a narrative appear more distinct from RS. In addition, we assess individual variability in network FC by comparing within- and between-subject similarity during NV and RS. We show that NV can evoke individually distinct NFC patterns by increasing inter-subject variability while retaining within-subject similarity. Crucially, our results highlight that this effect is not observable across all networks, but rather dependent on the network-stimulus combination. Our results confirm that NV can improve the detection of individual differences over RS and underline the importance of selecting the appropriate combination of movie and cognitive network for the research question at hand.

Neural mechanisms of expert persuasion on willingness to pay for sugar

Introduction: Sugar consumption is associated with many negative health consequences. It is, therefore, important to understand what can effectively influence individuals to consume less sugar. We recently showed that a healthy eating call by a health expert can significantly decrease the willingness to pay (WTP) for sugar-containing food. Here, we investigate which aspects of neural responses to the same healthy eating call can predict the efficacy of expert persuasion.

Methods: Forty-five healthy participants performed two blocks of a bidding task, in which they had to bid on sugar-containing, sugar-free and non-edible products, while their electroencephalography (EEG) was recorded. In between the two blocks, they listened to a healthy eating call by a nutritionist emphasizing the risks of sugar consumption.

Results: We found that after listening to the healthy eating call, participants significantly decreased their WTP for sugar-containing products. Moreover, a higher intersubject correlation of EEG (a measure of engagement) during listening to the healthy eating call resulted in a larger decrease in WTP for sugar-containing food. Whether or not a participant’s valuation of a product was highly influenced by the healthy eating call could also be predicted by spatiotemporal patterns of EEG responses to the healthy eating call, using a machine learning classification model. Finally, the healthy eating call increased the amplitude of the P300 component of the visual event-related potential in response to sugar-containing food.

Disussion: Overall, our results shed light on the neural basis of expert persuasion and demonstrate that EEG is a powerful tool to design and assess health-related advertisements before they are released to the public.

Robustness of Physiological Synchrony in Wearable Electrodermal Activity and Heart Rate as a Measure of Attentional Engagement to Movie Clips

Individuals that pay attention to narrative stimuli show synchronized heart rate (HR) and electrodermal activity (EDA) responses. The degree to which this physiological synchrony occurs is related to attentional engagement. Factors that can influence attention, such as instructions, salience of the narrative stimulus and characteristics of the individual, affect physiological synchrony. The demonstrability of synchrony depends on the amount of data used in the analysis. We investigated how demonstrability of physiological synchrony varies with varying group size and stimulus duration. Thirty participants watched six 10 min movie clips while their HR and EDA were monitored using wearable sensors (Movisens EdaMove 4 and Wahoo Tickr, respectively). We calculated inter-subject correlations as a measure of synchrony. Group size and stimulus duration were varied by using data from subsets of the participants and movie clips in the analysis. We found that for HR, higher synchrony correlated significantly with the number of answers correct for questions about the movie, confirming that physiological synchrony is associated with attention. For both HR and EDA, with increasing amounts of data used, the percentage of participants with significant synchrony increased. Importantly, we found that it did not matter how the amount of data was increased. Increasing the group size or increasing the stimulus duration led to the same results. Initial comparisons with results from other studies suggest that our results do not only apply to our specific set of stimuli and participants. All in all, the current work can act as a guideline for future research, indicating the amount of data minimally needed for robust analysis of synchrony based on inter-subject correlations.

Neural bases of predictions during natural reading of known statements: An EEG and eye movements co-registration study

Predictions of incoming words performed during reading have an impact on how the reader moves their eyes and on the electrical brain potentials. Eye tracking (ET) experiments show that less predictable words are fixated for longer periods of times. Electroencephalography (EEG) experiments show that these words elicit a more negative potential around 400ms (N400) after the word onset when reading one word at a time (foveated reading). Nevertheless, there was no N400 potential during the foveated reading of previously known sentences (memory-encoded), which suggests that the prediction of words from memory-encoded sentences is based on different mechanisms than predictions performed on common sentences. Here, we performed an ET-EEG co-registration experiment where participants read common and memory-encoded sentences. Our results show that the N400 potential disappear when the reader recognises the sentence. Furthermore, time-frequency analyses show a larger alpha lateralisation and a beta power increase for memory-encoded sentences. This suggests a more distributed attention and an active maintenance of the cognitive set, in concordance to the predictive coding framework.

Persuader-receiver neural coupling underlies persuasive messaging and predicts persuasion outcome

Opportunities to persuade and be persuaded are ubiquitous. What interpersonal neural pathway in real-world settings determining successful information propagation in naturalistic two-person persuasion scenarios? Hereby, we extended prior research on a naturalistic dyadic persuasion paradigm (NDP) using dual-fNIRS protocol simultaneously measured the neural activity from persuader-receiver dyads while they engaged in a modified "Arctic Survival Task." Investigating whether neural coupling between persuaders and receivers underpinning of persuading and predict persuasion outcomes (i.e., receiver's compliance). Broadly, we indicated that the persuasive arguments increase neural coupling significantly compared to non-persuasive arguments in the left superior temporal gyrus-superior frontal gyrus and superior frontal gyrus-inferior frontal gyrus. G-causality indices further revealed the coupling directionality of information flows between the persuader and receiver. Critically, the neural coupling could be a better predictor of persuasion outcomes relative to traditional self-report measures. Eventually, temporal dynamics neural coupling incorporating video recording revealed neural coupling marked the micro-level processes in response to persuading messages and possibly reflecting the time that persuasion might occurs. The initial case of the arguments with targeted views is valuable as the first step in encouraging the receiver's compliance. Our investigation represented an innovative interpersonal approach toward comprehending the neuroscience and psychology underlying complex and true persuasion.

A multiscale modeling framework to study the interdependence of brain, behavior, and pandemic

A major constraint of the behavioral epidemiological models is the assumption that human behavior is static; however, it is highly dynamic, especially in uncertain circumstances during a pandemic. To incorporate the dynamicity of human nature in the existing epidemiological models, we propose a population-wide multi-time-scale theoretical framework that assimilates neuronal plasticity as the basis of altering human emotions and behavior. For that, variable connection weights between different brain regions and their firing frequencies are coupled with a compartmental susceptible-infected-recovered model to incorporate the intrinsic dynamicity in the contact transmission rate ( β ). As an illustration, a model of fear conditioning in conjunction with awareness campaigns is developed and simulated. Results indicate that in the presence of fear conditioning, there exists an optimum duration of daily broadcast time during which awareness campaigns are most effective in mitigating the pandemic. Further, global sensitivity analysis using the Morris method highlighted that the learning rate and firing frequency of the unconditioned circuit are crucial regulators in modulating the emergent pandemic waves. The present study makes a case for incorporating neuronal dynamics as a basis of behavioral immune response and has further implications in designing awareness campaigns.

Developmental differences in the perception of naturalistic human movements

Introduction

It is widely believed that we are more attentive towards moving versus static stimuli. However, the neural correlates underlying the perception of human movements have not been extensively investigated in ecologically valid settings, nor has the developmental aspect of this phenomenon. Here, we set forth to investigate how human limb movements displayed in naturalistic videos influence the attentional engagement of children and young adults.

Methods

Thirty-nine healthy participants (4-26 years old) were presented with naturalistic videos featuring human goal-directed movements, while neural activity was recorded using electroencephalography (EEG). Video scenes were automatically annotated as containing arm, leg or no movement, using a machine learning algorithm. The viewers' attentional engagement was quantified by the intersubject correlation of EEG responses evoked by the videos.

Results

Our results demonstrate that scenes featuring limb movements, especially simultaneous arm and leg movements, elicit higher attentional engagement than scenes with no limb movement. Interestingly, this effect was found to diminish with age.

Discussion

Overall, our findings extend previous work on the perception of human motion by implementing naturalistic stimuli in the experimental design and extend the list of factors influencing the viewer's engagement exerted by naturalistic videos.

Automatic facial coding predicts self-report of emotion, advertisement and brand effects elicited by video commercials

Introduction

Consumers' emotional responses are the prime target for marketing commercials. Facial expressions provide information about a person's emotional state and technological advances have enabled machines to automatically decode them.

Method

With automatic facial coding we investigated the relationships between facial movements (i.e., action unit activity) and self-report of commercials advertisement emotion, advertisement and brand effects. Therefore, we recorded and analyzed the facial responses of 219 participants while they watched a broad array of video commercials.

Results

Facial expressions significantly predicted self-report of emotion as well as advertisement and brand effects. Interestingly, facial expressions had incremental value beyond self-report of emotion in the prediction of advertisement and brand effects. Hence, automatic facial coding appears to be useful as a non-verbal quantification of advertisement effects beyond self-report.

Discussion

This is the first study to measure a broad spectrum of automatically scored facial responses to video commercials. Automatic facial coding is a promising non-invasive and non-verbal method to measure emotional responses in marketing.

Central EEG Beta/Alpha Ratio Predicts the Population-Wide Efficiency of Advertisements

Recent studies have demonstrated that the brain activity of a group of people can be used to forecast choices at the population level. In this study, we attempted to neuroforecast aggregate consumer behavior of Internet users. During our electroencephalography (EEG) and eye-tracking study, participants were exposed to 10 banners that were also used in the real digital marketing campaign. In the separate online study, we additionally collected self-reported preferences for the same banners. We explored the relationship between the EEG, eye-tracking, and behavioral indexes obtained in our studies and the banners' aggregate efficiency provided by the large food retailer based on the decisions of 291,301 Internet users. An EEG-based engagement index (central beta/alpha ratio) significantly correlated with the aggregate efficiency of banners. Furthermore, our multiple linear regression models showed that a combination of eye-tracking, EEG and behavioral measurements better explained the market-level efficiency of banner advertisements than each measurement alone. Overall, our results confirm that neural signals of a relatively small number of individuals can forecast aggregate behavior at the population level.

An Ensemble Model for Consumer Emotion Prediction Using EEG Signals for Neuromarketing Applications

Traditional advertising techniques seek to govern the consumer's opinion toward a product, which may not reflect their actual behavior at the time of purchase. It is probable that advertisers misjudge consumer behavior because predicted opinions do not always correspond to consumers' actual purchase behaviors. Neuromarketing is the new paradigm of understanding customer buyer behavior and decision making, as well as the prediction of their gestures for product utilization through an unconscious process. Existing methods do not focus on effective preprocessing and classification techniques of electroencephalogram (EEG) signals, so in this study, an effective method for preprocessing and classification of EEG signals is proposed. The proposed method involves effective preprocessing of EEG signals by removing noise and a synthetic minority oversampling technique (SMOTE) to deal with the class imbalance problem. The dataset employed in this study is a publicly available neuromarketing dataset. Automated features were extracted by using a long short-term memory network (LSTM) and then concatenated with handcrafted features like power spectral density (PSD) and discrete wavelet transform (DWT) to create a complete feature set. The classification was done by using the proposed hybrid classifier that optimizes the weights of two machine learning classifiers and one deep learning classifier and classifies the data between like and dislike. The machine learning classifiers include the support vector machine (SVM), random forest (RF), and deep learning classifier (DNN). The proposed hybrid model outperforms other classifiers like RF, SVM, and DNN and achieves an accuracy of 96.89%. In the proposed method, accuracy, sensitivity, specificity, precision, and F1 score were computed to evaluate and compare the proposed method with recent state-of-the-art methods.

Neural dynamics during emotional video engagement relate to anxiety

Inter-subject correlations (ISCs) of physiological data can reveal common stimulus-driven processing across subjects. ISC has been applied to passive video viewing in small samples to measure common engagement and emotional processing. Here, in a large sample study of healthy adults (N = 163) who watched an emotional film (The Lion Cage by Charlie Chaplin), we recorded electroencephalography (EEG) across participants and measured ISC in theta, alpha and beta frequency bands. Peak ISC on the emotionally engaging video was observed three-quarters into the film clip, during a time period which potentially elicited a positive emotion of relief. Peak ISC in all frequency bands was focused over centro-parietal electrodes localizing to superior parietal cortex. ISC in both alpha and beta frequencies had a significant inverse relationship with anxiety symptoms. Our study suggests that ISC measured during continuous non-event-locked passive viewing may serve as a useful marker for anxious mood.

Altered Cerebral Processing of Videos in Children with Motor Dysfunction Suggests Broad Embodiment of Perceptual Cognitive Functions

Embodied cognition theory suggests that motor dysfunctions affect cognition. We examined this hypothesis by inspecting whether cerebral processing of movies, featuring both goal-directed movements and content without humans, differ between children with congenital motor dysfunction and healthy controls. Electroencephalography was recorded from 23 healthy children and 23 children with limited or absent arm movement due to either arthrogryposis multiplex congenita or obstetric brachial plexus palsy. Each individual patient exhibited divergent neural responses, disclosed by significantly lower inter-subject correlation (ISC) of brain activity, during the videos compared to the healthy children. We failed to observe associations between this finding and the motor-related content of the various video scenes, suggesting that differences between the patients and controls reflect modulation of perceptual-cognitive processing of videos by upper-limb motor dysfunctions not limited to the watching-mirroring of motor actions. Thus, perceptual-cognitive processes in the brain seem to be more robustly embodied than has previously been thought.

Shared understanding and social connection: Integrating approaches from social psychology, social network analysis, and neuroscience

Meaningfully connecting with others is critical to the well-being of individuals. What phenomena contribute to and stem from social connection? In this paper, we integrate emerging work that uses neuroimaging and social network analysis with theories that explore the links between shared reality and social connection. We highlight recent work suggesting that the extent to which people have aligned mental processing and shared subjective construals to those around them-as shown by neural similarity-is associated with both objective and subjective social connection. On the other hand, idiosyncrasies are linked to difficulties with social connection. We conclude by suggesting how the links between shared understanding and social connection can be productively used as a framework to study psychosocial phenomena of interest.

Strong displayed passion and preparedness of broadcaster in live streaming e-commerce increases consumers' neural engagement

Live streaming shopping, the streaming of real-time videos promoting products that consumers can purchase online, has recently been a booming area of e-commerce, especially during the COVID-19 pandemic. The success of live streaming e-commerce largely relies on the extent to which the broadcaster can get consumers engaged by the live stream. Thus, it is important to discover the antecedents of consumer engagement in such a context. Drawing on consumer engagement and neuroscience literature, this study used electroencephalography inter-subject correlation (EEG-ISC) to explore how broadcasters' entrepreneurial passion during live streaming videos influenced consumers' neural engagement as they watched the live streaming videos. We used the framework of displayed passion and preparedness from the entrepreneurial passion literature to predict consumer engagement. We found significant ISC for strong displayed passion, while preparedness had partially significant effects on the first, second, and summed components of ISC. The interaction effects of these two factors on the first and summed components of ISC were partially significant. Strong displayed passion and preparedness activated the left and right prefrontal regions of the consumers' brains. These findings indicate that broadcasters' displayed passion and preparedness can influence consumer engagement in live streaming e-commerce settings. Our findings suggest that a scientific approach could be used to improve a broadcaster's performance by testing ISC during rehearsals before live streaming.

A functional magnetic resonance imaging examination of audiovisual observation of a point-light string quartet using intersubject correlation and physical feature analysis

We use functional Magnetic Resonance Imaging (fMRI) to explore synchronized neural responses between observers of audiovisual presentation of a string quartet performance during free viewing. Audio presentation was accompanied by visual presentation of the string quartet as stick figures observed from a static viewpoint. Brain data from 18 musical novices were obtained during audiovisual presentation of a 116 s performance of the allegro of String Quartet, No. 14 in D minor by Schubert played by the 'Quartetto di Cremona.' These data were analyzed using intersubject correlation (ISC). Results showed extensive ISC in auditory and visual areas as well as parietal cortex, frontal cortex and subcortical areas including the medial geniculate and basal ganglia (putamen). These results from a single fixed viewpoint of multiple musicians are greater than previous reports of ISC from unstructured group activity but are broadly consistent with related research that used ISC to explore listening to music or watching solo dance. A feature analysis examining the relationship between brain activity and physical features of the auditory and visual signals yielded findings of a large proportion of activity related to auditory and visual processing, particularly in the superior temporal gyrus (STG) as well as midbrain areas. Motor areas were also involved, potentially as a result of watching motion from the stick figure display of musicians in the string quartet. These results reveal involvement of areas such as the putamen in processing complex musical performance and highlight the potential of using brief naturalistic stimuli to localize distinct brain areas and elucidate potential mechanisms underlying multisensory integration.

Mapping and understanding of correlated electroencephalogram (EEG) responses to the newsvendor problem

Decision-making is one of the most critical activities of human beings. To better understand the underlying neurocognitive mechanism while making decisions under an economic context, we designed a decision-making paradigm based on the newsvendor problem (NP) with two scenarios: low-profit margins as the more challenging scenario and high-profit margins as the less difficult one. The EEG signals were acquired from healthy humans while subjects were performing the task. We adopted the Correlated Component Analysis (CorrCA) method to identify linear combinations of EEG channels that maximize the correlation across subjects ([Formula: see text]) or trials ([Formula: see text]). The inter-subject or inter-trial correlation values (ISC or ITC) of the first three components were estimated to investigate the modulation of the task difficulty on subjects' EEG signals and respective correlations. We also calculated the alpha- and beta-band power of the projection components obtained by the CorrCA to assess the brain responses across multiple task periods. Finally, the CorrCA forward models, which represent the scalp projections of the brain activities by the maximally correlated components, were further translated into source distributions of underlying cortical activity using the exact Low Resolution Electromagnetic Tomography Algorithm (eLORETA). Our results revealed strong and significant correlations in EEG signals among multiple subjects and trials during the more difficult decision-making task than the easier one. We also observed that the NP decision-making and feedback tasks desynchronized the normalized alpha and beta powers of the CorrCA components, reflecting the engagement state of subjects. Source localization results furthermore suggested several sources of neural activities during the NP decision-making process, including the dorsolateral prefrontal cortex, anterior PFC, orbitofrontal cortex, posterior cingulate cortex, and somatosensory association cortex.

Emotion Elicitation through Vibrotactile Stimulation as an Alternative for Deaf and Hard of Hearing People: An EEG Study

Despite technological and accessibility advances, the performing arts and their cultural offerings remain inaccessible to many people. By using vibrotactile stimulation as an alternative channel, we explored a different way to enhance emotional processes produced while watching audiovisual media and, thus, elicit a greater emotional reaction in hearing-impaired people. We recorded the brain activity of 35 participants with normal hearing and 8 participants with severe and total hearing loss. The results showed activation in the same areas both in participants with normal hearing while watching a video, and in hearing-impaired participants while watching the same video with synchronized soft vibrotactile stimulation in both hands, based on a proprietary stimulation glove. These brain areas (bilateral middle frontal orbitofrontal, bilateral superior frontal gyrus, and left cingulum) have been reported as emotional and attentional areas. We conclude that vibrotactile stimulation can elicit the appropriate cortex activation while watching audiovisual media.

fNIRS-Based Dynamic Functional Connectivity Reveals the Innate Musical Sensing Brain Networks in Preterm Infants

Humans have the ability to appreciate and create music. However, why and how humans have this distinctive ability to perceive music remains unclear. Additionally, the investigation of the innate perceiving skill in humans is compounded by the fact that we have been actively and passively exposed to auditory stimuli or have systematically learnt music after birth. Therefore, to explore the innate musical perceiving ability, infants with preterm birth may be the most suitable population. In this study, the auditory brain networks were explored using dynamic functional connectivity-based reliable component analysis (RCA) in preterm infants during music listening. The brain activation was captured by portable functional near-infrared spectroscopy (fNIRS) to simulate a natural environment for preterm infants. The components with the maximum inter-subject correlation were extracted. The generated spatial filters identified the shared spatial structural features of functional brain connectivity across subjects during listening to the common music, exhibiting a functional synchronization between the right temporal region and the frontal and motor cortex, and synchronization between the bilateral temporal regions. The specific pattern is responsible for the functions involving music comprehension, emotion generation, language processing, memory, and sensory. The fluctuation of the extracted components and the phase variation demonstrates the interactions between the extracted brain networks to encode musical information. These results are critically important for our understanding of the underlying mechanisms of the innate perceiving skills at early ages of human during naturalistic music listening.

Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges

In the past decade, decision neuroscience and neuroeconomics have developed many new insights in the study of decision making. This review provides an overarching update on how the field has advanced in this time period. Although our initial review a decade ago outlined several theoretical, conceptual, methodological, empirical, and practical challenges, there has only been limited progress in resolving these challenges. We summarize significant trends in decision neuroscience through the lens of the challenges outlined for the field and review examples where the field has had significant, direct, and applicable impacts across economics and psychology. First, we review progress on topics including reward learning, explore-exploit decisions, risk and ambiguity, intertemporal choice, and valuation. Next, we assess the impacts of emotion, social rewards, and social context on decision making. Then, we follow up with how individual differences impact choices and new exciting developments in the prediction and neuroforecasting of future decisions. Finally, we consider how trends in decision-neuroscience research reflect progress toward resolving past challenges, discuss new and exciting applications of recent research, and identify new challenges for the field. This article is categorized under: Psychology > Reasoning and Decision Making Psychology > Emotion and Motivation.

Resonance as a Design Strategy for AI and Social Robots

Resonance, a powerful and pervasive phenomenon, appears to play a major role in human interactions. This article investigates the relationship between the physical mechanism of resonance and the human experience of resonance, and considers possibilities for enhancing the experience of resonance within human-robot interactions. We first introduce resonance as a widespread cultural and scientific metaphor. Then, we review the nature of "sympathetic resonance" as a physical mechanism. Following this introduction, the remainder of the article is organized in two parts. In part one, we review the role of resonance (including synchronization and rhythmic entrainment) in human cognition and social interactions. Then, in part two, we review resonance-related phenomena in robotics and artificial intelligence (AI). These two reviews serve as ground for the introduction of a design strategy and combinatorial design space for shaping resonant interactions with robots and AI. We conclude by posing hypotheses and research questions for future empirical studies and discuss a range of ethical and aesthetic issues associated with resonance in human-robot interactions.

Naturalistic viewing conditions can increase task engagement and aesthetic preference but have only minimal impact on EEG quality

Free gaze and moving images are typically avoided in EEG experiments due to the expected generation of artifacts and noise. Yet for a growing number of research questions, loosening these rigorous restrictions would be beneficial. Among these is research on visual aesthetic experiences, which often involve open-ended exploration of highly variable stimuli. Here we systematically compare the effect of conservative vs. more liberal experimental settings on various measures of behavior, brain activity and physiology in an aesthetic rating task. Our primary aim was to assess EEG signal quality. 43 participants either maintained fixation or were allowed to gaze freely, and viewed either static images or dynamic (video) stimuli consisting of dance performances or nature scenes. A passive auditory background task (auditory steady-state response; ASSR) was added as a proxy measure for overall EEG recording quality. We recorded EEG, ECG and eyetracking data, and participants rated their aesthetic preference and state of boredom on each trial. Whereas both behavioral ratings and gaze behavior were affected by task and stimulus manipulations, EEG SNR was barely affected and generally robust across all conditions, despite only minimal preprocessing and no trial rejection. In particular, we show that using video stimuli does not necessarily result in lower EEG quality and can, on the contrary, significantly reduce eye movements while increasing both the participants' aesthetic response and general task engagement. We see these as encouraging results indicating that - at least in the lab - more liberal experimental conditions can be adopted without significant loss of signal quality.

How Does the Implicit Awareness of Consumers Influence the Effectiveness of Public Service Announcements? A Functional Near-Infrared Spectroscopy Study

A large number of scholars have conducted detailed studies on the effectiveness of commercial advertising by using neuroimaging methods, but only a few scholars have used this method to study the effectiveness of public service announcements (PSAs). To research the relationship between the effectiveness of PSAs and the audience’s implicit awareness, functional near-infrared spectroscopy (fNIRS) was employed to record the neural activity data of participants in this study. The results showed that there was a correlation between activation of dorsolateral prefrontal cortex (dlPFC) and the effectiveness of PSAs; The activation of the dlPFC could also be used as an indicator to represent the appeal of advertising content. The results means that neuroimaging tool can also be used to investigate the effectiveness of PSAs, not just commercial advertisements and a few PSAs study, and that neural activity can predict and improve the effectiveness of PSAs before they are released.

Neural Activity during Story Listening Is Synchronized across Individuals Despite Acoustic Masking

Older people with hearing problems often experience difficulties understanding speech in the presence of background sound. As a result, they may disengage in social situations, which has been associated with negative psychosocial health outcomes. Measuring listening (dis)engagement during challenging listening situations has received little attention thus far. We recruit young, normal-hearing human adults (both sexes) and investigate how speech intelligibility and engagement during naturalistic story listening is affected by the level of acoustic masking (12-talker babble) at different signal-to-noise ratios (SNRs). In Experiment 1, we observed that word-report scores were above 80% for all but the lowest SNR (-3 dB SNR) we tested, at which performance dropped to 54%. In Experiment 2, we calculated intersubject correlation (ISC) using EEG data to identify dynamic spatial patterns of shared neural activity evoked by the stories. ISC has been used as a neural measure of participants' engagement with naturalistic materials. Our results show that ISC was stable across all but the lowest SNRs, despite reduced speech intelligibility. Comparing ISC and intelligibility demonstrated that word-report performance declined more strongly with decreasing SNR compared to ISC. Our measure of neural engagement suggests that individuals remain engaged in story listening despite missing words because of background noise. Our work provides a potentially fruitful approach to investigate listener engagement with naturalistic, spoken stories that may be used to investigate (dis)engagement in older adults with hearing impairment.

Cognitive processing of a common stimulus synchronizes brains, hearts, and eyes

Neural, physiological, and behavioral signals synchronize between human subjects in a variety of settings. Multiple hypotheses have been proposed to explain this interpersonal synchrony, but there is no clarity under which conditions it arises, for which signals, or whether there is a common underlying mechanism. We hypothesized that cognitive processing of a shared stimulus is the source of synchrony between subjects, measured here as intersubject correlation (ISC). To test this, we presented informative videos to participants in an attentive and distracted condition and subsequently measured information recall. ISC was observed for electro-encephalography, gaze position, pupil size, and heart rate, but not respiration and head movements. The strength of correlation was co-modulated in the different signals, changed with attentional state, and predicted subsequent recall of information presented in the videos. There was robust within-subject coupling between brain, heart, and eyes, but not respiration or head movements. The results suggest that ISC is the result of effective cognitive processing, and thus emerges only for those signals that exhibit a robust brain-body connection. While physiological and behavioral fluctuations may be driven by multiple features of the stimulus, correlation with other individuals is co-modulated by the level of attentional engagement with the stimulus.

Unsupervised Clustering of Individuals Sharing Selective Attentional Focus Using Physiological Synchrony

Research on brain signals as indicators of a certain attentional state is moving from laboratory environments to everyday settings. Uncovering the attentional focus of individuals in such settings is challenging because there is usually limited information about real-world events, as well as a lack of data from the real-world context at hand that is correctly labeled with respect to individuals' attentional state. In most approaches, such data is needed to train attention monitoring models. We here investigate whether unsupervised clustering can be combined with physiological synchrony in the electroencephalogram (EEG), electrodermal activity (EDA), and heart rate to automatically identify groups of individuals sharing attentional focus without using knowledge of the sensory stimuli or attentional focus of any of the individuals. We used data from an experiment in which 26 participants listened to an audiobook interspersed with emotional sounds and beeps. Thirteen participants were instructed to focus on the narrative of the audiobook and 13 participants were instructed to focus on the interspersed emotional sounds and beeps. We used a broad range of commonly applied dimensionality reduction ordination techniques-further referred to as mappings-in combination with unsupervised clustering algorithms to identify the two groups of individuals sharing attentional focus based on physiological synchrony. Analyses were performed using the three modalities EEG, EDA, and heart rate separately, and using all possible combinations of these modalities. The best unimodal results were obtained when applying clustering algorithms on physiological synchrony data in EEG, yielding a maximum clustering accuracy of 85%. Even though the use of EDA or heart rate by itself did not lead to accuracies significantly higher than chance level, combining EEG with these measures in a multimodal approach generally resulted in higher classification accuracies than when using only EEG. Additionally, classification results of multimodal data were found to be more consistent across algorithms than unimodal data, making algorithm choice less important. Our finding that unsupervised classification into attentional groups is possible is important to support studies on attentional engagement in everyday settings.

Inter-subject Correlation While Listening to Minimalist Music: A Study of Electrophysiological and Behavioral Responses to Steve Reich's Piano Phase

Musical minimalism utilizes the temporal manipulation of restricted collections of rhythmic, melodic, and/or harmonic materials. One example, Steve Reich's Piano Phase, offers listeners readily audible formal structure with unpredictable events at the local level. For example, pattern recurrences may generate strong expectations which are violated by small temporal and pitch deviations. A hyper-detailed listening strategy prompted by these minute deviations stands in contrast to the type of listening engagement typically cultivated around functional tonal Western music. Recent research has suggested that the inter-subject correlation (ISC) of electroencephalographic (EEG) responses to natural audio-visual stimuli objectively indexes a state of “engagement,” demonstrating the potential of this approach for analyzing music listening. But can ISCs capture engagement with minimalist music, which features less obvious expectation formation and has historically received a wide range of reactions? To approach this question, we collected EEG and continuous behavioral (CB) data while 30 adults listened to an excerpt from Steve Reich's Piano Phase, as well as three controlled manipulations and a popular-music remix of the work. Our analyses reveal that EEG and CB ISC are highest for the remix stimulus and lowest for our most repetitive manipulation, no statistical differences in overall EEG ISC between our most musically meaningful manipulations and Reich's original piece, and evidence that compositional features drove engagement in time-resolved ISC analyses. We also found that aesthetic evaluations corresponded well with overall EEG ISC. Finally we highlight co-occurrences between stimulus events and time-resolved EEG and CB ISC. We offer the CB paradigm as a useful analysis measure and note the value of minimalist compositions as a limit case for the neuroscientific study of music listening. Overall, our participants' neural, continuous behavioral, and question responses showed strong similarities that may help refine our understanding of the type of engagement indexed by ISC for musical stimuli.