Emotions promote social interaction by synchronizing brain activity across individuals

Inter-subject correlation of audience facial expressions predicts audience engagement during theatrical performances

During performances, audiences experience various emotional states, and these are reflected in their ongoing facial expressions. We investigated if audience engagement could be determined by measuring the inter-subject correlation (ISC) of non-invasively recorded audience facial expressions. We filmed the faces of multiple audience members at theatrical performances and determined the intensity of their different facial expressions throughout the performances. Neutral, happy, anger, and disgust expression ISCs accounted for up to 24% of the performance dramaturge's predictions of audience engagement. Expression synchrony was greater between individuals in close proximity, suggesting effects of emotional contagion or cognitive similarities between neighboring individuals, whereas expression synchrony was greatest between individuals who were younger, female, and with greater levels of empathy, showing that individual characteristics impact shared audience experiences. Together, our results show that facial expression synchronization could be used as a real-time non-invasive indicator of engagement in audiences larger than achieved using previous approaches.

Intersubject correlations in reward and mentalizing brain circuits separately predict persuasiveness of two types of ISIS video propaganda

The Islamist group ISIS has been particularly successful at recruiting Westerners as terrorists. A hypothesized explanation is their simultaneous use of two types of propaganda: Heroic narratives, emphasizing individual glory, alongside Social narratives, which emphasize oppression against Islamic communities. In the current study, functional MRI was used to measure brain responses to short ISIS propaganda videos distributed online. Participants were shown 4 Heroic and 4 Social videos categorized as such by another independent group of subjects. Persuasiveness was measured using post-scan predictions of recruitment effectiveness. Inter-subject correlation (ISC) was used to measure commonality of brain activity time courses across individuals. ISCs in ventral striatum predicted rated persuasiveness for Heroic videos, while ISCs in mentalizing and default networks, especially in dmPFC, predicted rated persuasiveness for Social videos. This work builds on past findings that engagement of the reward circuit and of mentalizing brain regions predicts preferences and persuasion. The observed dissociation as a function of stimulus type is novel, as is the finding that intersubject synchrony in ventral striatum predicts rated persuasiveness. These exploratory results identify possible neural mechanisms by which political extremists successfully recruit prospective members and specifically support the hypothesized distinction between Heroic and Social narratives for ISIS propaganda.

Consistent movement of viewers' facial keypoints while watching emotionally evocative videos

Neuropsychological research aims to unravel how diverse individuals' brains exhibit similar functionality when exposed to the same stimuli. The evocation of consistent responses when different subjects watch the same emotionally evocative stimulus has been observed through modalities like fMRI, EEG, physiological signals and facial expressions. We refer to the quantification of these shared consistent signals across subjects at each time instant across the temporal dimension as Consistent Response Measurement (CRM). CRM is widely explored through fMRI, occasionally with EEG, physiological signals and facial expressions using metrics like Inter-Subject Correlation (ISC). However, fMRI tools are expensive and constrained, while EEG and physiological signals are prone to facial artifacts and environmental conditions (such as temperature, humidity, and health condition of subjects). In this research, facial expression videos are used as a cost-effective and flexible alternative for CRM, minimally affected by external conditions. By employing computer vision-based automated facial keypoint tracking, a new metric similar to ISC, called the Average t-statistic, is introduced. Unlike existing facial expression-based methodologies that measure CRM of secondary indicators like inferred emotions, keypoint, and ICA-based features, the Average t-statistic is closely associated with the direct measurement of consistent facial muscle movement using the Facial Action Coding System (FACS). This is evidenced in DISFA dataset where the time-series of Average t-statistic has a high correlation (R2 = 0.78) with a metric called AU consistency, which directly measures facial muscle movement through FACS coding of video frames. The simplicity of recording facial expressions with the automated Average t-statistic expands the applications of CRM such as measuring engagement in online learning, customer interactions, etc., and diagnosing outliers in healthcare conditions like stroke, autism, depression, etc. To promote further research, we have made the code repository publicly available.

Inter- and intra-subject similarity in network functional connectivity across a full narrative movie

Naturalistic paradigms, such as watching movies during functional magnetic resonance imaging (fMRI), are thought to prompt the emotional and cognitive processes typically elicited in real life situations. Therefore, naturalistic viewing (NV) holds great potential for studying individual differences. However, in how far NV elicits similarity within and between subjects on a network level, particularly depending on emotions portrayed in movies, is currently unknown. We used the studyforrest dataset to investigate the inter- and intra-subject similarity in network functional connectivity (NFC) of 14 meta-analytically defined networks across a full narrative, audio-visual movie split into 8 consecutive movie segments. We characterized the movie segments by valence and arousal portrayed within the sequences, before utilizing a linear mixed model to analyze which factors explain inter- and intra-subject similarity. Our results showed that the model best explaining inter-subject similarity comprised network, movie segment, valence and a movie segment by valence interaction. Intra-subject similarity was influenced significantly by the same factors and an additional three-way interaction between movie segment, valence and arousal. Overall, inter- and intra-subject similarity in NFC were sensitive to the ongoing narrative and emotions in the movie. Lowest similarity both within and between subjects was seen in the emotional regulation network and networks associated with long-term memory processing, which might be explained by specific features and content of the movie. We conclude that detailed characterization of movie features is crucial for NV research.

Studying the social mind: An updated summary of findings from the Vietnam Head Injury Study

Lesion mapping studies allow us to evaluate the potential causal contribution of specific brain areas to human cognition and complement other cognitive neuroscience methods, as several authors have recently pointed out. Here, we present an updated summary of the findings from the Vietnam Head Injury Study (VHIS) focusing on the studies conducted over the last decade, that examined the social mind and its intricate neural and cognitive underpinnings. The VHIS is a prospective, long-term follow-up study of Vietnam veterans with penetrating traumatic brain injury (pTBI) and healthy controls (HC). The scope of the work is to present the studies from the latest phases (3 and 4) of the VHIS, 70 studies since 2011, when the Raymont et al. paper was published (Raymont et al., 2011). These studies have contributed to our understanding of human social cognition, including political and religious beliefs, theory of mind, but also executive functions, intelligence, and personality. This work finally discusses the usefulness of lesion mapping as an approach to understanding the functions of the human brain from basic science and clinical perspectives.

Electrophysiological representations of multivariate human emotion experience

ABSTRACTDespite the fact that human daily emotions are co-occurring by nature, most neuroscience studies have primarily adopted a univariate approach to identify the neural representation of emotion (emotion experience within a single emotion category) without adequate consideration of the co-occurrence of different emotions (emotion experience across different emotion categories simultaneously). To investigate the neural representations of multivariate emotion experience, this study employed the inter-situation representational similarity analysis (RSA) method. Researchers used an EEG dataset of 78 participants who watched 28 video clips and rated their experience on eight emotion categories. The EEG-based electrophysiological representation was extracted as the power spectral density (PSD) feature per channel in the five frequency bands. The inter-situation RSA method revealed significant correlations between the multivariate emotion experience ratings and PSD features in the Alpha and Beta bands, primarily over the frontal and parietal-occipital brain regions. The study found the identified EEG representations to be reliable with sufficient situations and participants. Moreover, through a series of ablation analyses, the inter-situation RSA further demonstrated the stability and specificity of the EEG representations for multivariate emotion experience. These findings highlight the importance of adopting a multivariate perspective for a comprehensive understanding of the neural representation of human emotion experience.

Vicarious punishment of moral violations in naturalistic drama narratives predicts cortical synchronization

Punishment of moral norm violators is instrumental for human cooperation. Yet, social and affective neuroscience research has primarily focused on second- and third-party norm enforcement, neglecting the neural architecture underlying observed (vicarious) punishment of moral wrongdoers. We used naturalistic television drama as a sampling space for observing outcomes of morally-relevant behaviors to assess how individuals cognitively process dynamically evolving moral actions and their consequences. Drawing on Affective Disposition Theory, we derived hypotheses linking character morality with viewers' neural processing of characters' rewards and punishments. We used functional magnetic resonance imaging (fMRI) to examine neural responses of 28 female participants while free-viewing 15 short story summary video clips of episodes from a popular US television soap opera. Each summary included a complete narrative structure, fully crossing main character behaviors (moral/immoral) and the consequences (reward/punishment) characters faced for their actions. Narrative engagement was examined via intersubject correlation and representational similarity analysis. Highest cortical synchronization in 9 specifically selected regions previously implicated in processing moral information was observed when characters who act immorally are punished for their actions with participants' empathy as an important moderator. The results advance our understanding of the moral brain and the role of normative considerations and character outcomes in viewers' engagement with popular narratives.

Neural alignment during outgroup intervention predicts future change of affect towards outgroup

While social psychology studies have shown that paradoxical thinking intervention has a moderating effect on negative attitudes toward members from rival social groups (i.e. outgroup), the neural underpinnings of the intervention have not been studied. Here, we investigate this by examining neural alignment across individuals at different phases during the intervention regarding Covid-19 vaccine-supporters' attitudes against vaccine-opposers. We raise two questions: Whether neural alignment varies during the intervention, and whether it predicts a change in outgroup attitudes measured via a survey 2 days after the intervention and compared to baseline. We test the neural alignment using magnetoencephalography-recorded neural oscillations and multiset canonical correlation analysis. We find a build-up of neural alignment which emerges at the final phase of the paradoxical thinking intervention in the precuneus-a hub of mentalizing; there was no such effect in the control conditions. In parallel, we find a behavioral build-up of dissent to the interventional stimuli. These neural and behavioral patterns predict a prosocial future change in affect and actions toward the outgroup. Together, these findings reveal a new operational pattern of mentalizing on the outgroup, which can change the way individuals may feel and behave toward members of that outgroup.

Intersubject representational similarity analysis uncovers the impact of state anxiety on brain activation patterns in the human extrastriate cortex

The current study used functional magnetic resonance imaging (fMRI) and showed that state anxiety modulated extrastriate cortex activity in response to emotionally-charged visual images. State anxiety and neuroimaging data from 53 individuals were subjected to an intersubject representational similarity analysis (ISRSA), wherein the geometries between neural and behavioral data were compared. This analysis identified the extrastriate cortex (fusiform gyrus and area MT) to be the sole regions whose activity patterns covaried with state anxiety. Importantly, we show that this brain-behavior association is revealed when treating state anxiety data as a multidimensional response pattern, rather than a single composite score. This suggests that ISRSA using multivariate distances may be more sensitive in identifying the shared geometries between self-report questionnaires and brain imaging data. Overall, our findings demonstrate that a transient state of anxiety may influence how visual information - especially those relevant to the valence dimension - is processed in the extrastriate cortex.

Neural signatures of shared subjective affective engagement and disengagement during movie viewing

When watching a negative emotional movie, we differ from person to person in the ease with which we engage and the difficulty with which we disengage throughout a temporally evolving narrative. We investigated neural responses of emotional processing, by considering inter-individual synchronization in subjective emotional engagement and disengagement. The neural underpinnings of these shared responses are ideally studied in naturalistic scenarios like movie viewing, wherein individuals emotionally engage and disengage at their own time and pace throughout the course of a narrative. Despite the rich data that naturalistic designs can bring to the study, there is a challenge in determining time-resolved behavioral markers of subjective engagement and disengagement and their underlying neural responses. We used a within-subject cross-over design instructing 22 subjects to watch clips of either neutral or sad content while undergoing functional magnetic resonance imaging (fMRI). Participants watched the same movies a second time while continuously annotating the perceived emotional intensity, thus enabling the mapping of brain activity and emotional experience. Our analyses revealed that between-participant similarity in waxing (engagement) and waning (disengagement) of emotional intensity was directly related to the between-participant similarity in spatiotemporal patterns of brain activation during the movie(s). Similar patterns of engagement reflected common activation in the bilateral ventromedial prefrontal cortex, regions often involved in self-referenced evaluation and generation of negative emotions. Similar patterns of disengagement reflected common activation in central executive and default mode network regions often involved in top-down emotion regulation. Together this work helps to better understand cognitive and neural mechanisms underpinning engagement and disengagement from emotionally evocative narratives.

Emotion contagion and physiological synchrony: The more intimate relationships, the more contagion of positive emotions

The study aimed to explore how interpersonal closeness (friends vs. strangers) and emotion type (positive vs. negative) influenced emotion contagion and physiological synchrony between interacting partners. Twenty-eight friend dyads (n = 56) and 29 stranger dyads (n = 58) participated in an emotion contagion laboratory task. In each dyad, one participant, the 'sender', was randomly asked to watch a film clip (neutral, positive, or negative), while their partner, the 'observer' passively observed the sender's facial expressions. Participants' electrocardiograms (ECG) and facial electromyography (EMG) signals were recorded using the BIOPAC system. Results revealed that observing the sender's facial expressions led to the observer's spontaneous mimicry and emotional contagion, accompanied by enhanced physiological synchrony between interacting partners. In the positive emotion condition, the observers reported more positive emotions and displayed stronger zygomaticus major activity in friend dyads than in stranger dyads. Greater physiological synchrony (heart rate and heart rate variability) between interacting partners was also observed in friend dyads than in stranger dyads in the positive emotion condition. These results indicate that positive emotion contagion is more likely to occur between close partners than negative emotion contagion.

Parental involvement affects parent-adolescents brain-to-brain synchrony when experiencing different emotions together: An EEG-based hyperscanning study

Parental involvement (PI) is a broad and multifaceted construct, which refers to the parental demonstration of interest in their child, caring, and warmth (Davis et al., 2021). Parental involvement affects parent-adolescent's relationship and communication. However, there was little research to examine the underlying neural mechanism. The present study aimed to explore how parental involvement is associated with the brain-to-brain synchronous activation between parent-adolescent dyads when sharing emotional experience together by using the electroencephalograph (EEG) hyperscanning. EEG was recorded simultaneously in a sample of 26 parent-adolescent dyads (Mparents'age=43.312, SD=5.468; Madolescents' age=12.077, SD=1.412) when completing the picture processing task. Phase locking values (PLVs) in beta band and gamma band were used to compare the differences in the parent-adolescent dyads' induced brain-to-brain synchrony between the high parental involvement group (HPI) and the low parental involvement group (LPI). Results showed that greater beta brain-to-brain synchrony was observed in the HPIs than in the LPIs when experiencing positive emotions together in the central region. However, there was no significant difference between the HPIs and the LPIs in the negative and neutral condition. Moreover, greater gamma brain-to-brain synchrony was observed when viewing negative emotional stimuli together than viewing positive emotional stimuli together in the LPIs in the central region. However, there was no significant difference between different emotional conditions in the HPIs. Findings of the present study provide neuroscientific evidence that parental involvement may strengthen parent-adolescent's emotional interaction and communication.

An observation of a negative effect of social cohesion on creativity in musical improvisation

Although various social factors can significantly impact creative performance, it is still unclear how social cohesion (i.e., how close we feel to others) influences creativity. We therefore conducted two studies exploring the association between social cohesion and creativity within the domain of musical improvisation, a prime example of creative performance, which usually plays out in social contexts. The first study (n = 58 musical novices) showed that music-induced synchrony facilitates social cohesion. In our second study (n = 18 musical novices), we found that in two out of three experimental conditions, increased social cohesion is associated with less creative musical outcomes, as rated by nine expert musicians. In our subsequent analysis we related measures of social cohesion and creativity. This approach highlights how, within a musical setting, creativity unfolds in the context of social contingencies as social cohesion and related factors.

The role of external factors in affect-sharing and their neural bases

Affect-sharing, the ability to vicariously feel another person's emotions, is the primary component of empathy that is typically thought to rely on the observer's capacity to feel the emotions of others. However, external signals, such as the target's physical characteristics, have been demonstrated to influence affect-sharing in the neuroscientific literature that speaks to the underappreciated role of external factors in eliciting affect-sharing. We consider factors that influence affect-sharing, including physical cues, emotional cues, situational factors, and observer-target relationships, as well as the neural circuits involved in these processes. Our review reveals that, while neural network activation is primarily responsible for processing affect-sharing, external factors also co-activate a top-down cognitive processing network to modulate the conscious process of affect-sharing. From this knowledge, an integrative framework of external factor interactions with affect-sharing are explained in detail. Finally, we identify critical areas for future research in social and affective neuroscience, including research gaps and incorporation of ecologically valid paradigms.

How self-disclosure of negative experiences shapes prosociality?

People frequently share their negative experiences and feelings with others. Little is known, however, about the social outcomes of sharing negative experiences and the underlying neural mechanisms. We addressed this dearth of knowledge by leveraging functional near-infrared spectroscopy (fNIRS) hyperscanning: while dyad participants took turns to share their own (self-disclosure group) or a stranger's (non-disclosure group) negative and neutral experiences, their respective brain activity was recorded simultaneously by fNIRS. We observed that sharing negative (relative to neutral) experiences enhanced greater mutual prosociality, emotional empathy and interpersonal neural synchronization (INS) at the left superior frontal cortex in the self-disclosure group compared to the non-disclosure group. Importantly, mediation analyses further revealed that in the self-disclosure (but not non-disclosure) group, the increased emotional empathy and INS elicited by sharing negative experiences relative to sharing neutral experiences promoted the enhanced prosociality through increasing interpersonal liking. These results indicate that self-disclosure of negative experiences can promote prosocial behaviors via social dynamics (defined as social affective and cognitive factors, including empathy and liking) and shared neural responses. Our findings suggest that when people express negative sentiments, they incline to follow up with positive actions.

Emotional arousal pattern (EMAP): A new database for modeling momentary subjective and psychophysiological responding to affective stimuli

This article describes a new database (named "EMAP") of 145 individuals' reactions to emotion-provoking film clips. It includes electroencephalographic and peripheral physiological data as well as moment-by-moment ratings for emotional arousal in addition to overall and categorical ratings. The resulting variation in continuous ratings reflects inter-individual variability in emotional responding. To make use of the moment-by-moment data for ratings as well as neurophysiological activity, we used a machine learning approach. The results show that algorithms that are based on temporal information improve predictions compared to algorithms without a temporal component, both within and across participant modeling. Although predicting moment-by-moment changes in emotional experiences by analyzing neurophysiological activity was more difficult than using aggregated experience ratings, selecting a subset of predictors improved the prediction. This also showed that not only single features, for example, skin conductance, but a range of neurophysiological parameters explain variation in subjective fluctuations of subjective experience.

Social exclusion evokes different psychophysiological responses in individuals high on the psychopathy facets fearless dominance and self-centered impulsivity

Individuals with psychopathic personality traits are generally thought to have difficulties in processing and experiencing emotions. These difficulties could also translate to emotionally charged social situations such as social exclusion. Being socially excluded is often experienced as stressful and unpleasant, potentially even leading to selfish or aggressive behavior-both of which are linked to certain aspects of psychopathy. The current study investigated self-report and physiological responses to social exclusion in the cyberball paradigm in a carefully selected community sample of individuals either scoring high on primary (N = 24) or secondary psychopathy traits (N = 17). Across the sample, the cyberball paradigm decreased experiences of joy and approach motivation, increased subjective anger reports, and induced changes in heart rate. In contrast, individuals scoring high on secondary psychopathy traits (Self-Centered Impulsivity group) displayed stronger physiological reactivity during a habituation phase of prolonged social exclusion than individuals scoring high on primary psychopathy traits (Fearless Dominance group), indexed by changes in skin conductance level. Moreover, a potential mismatch between self-reported and physiological arousal seemed to be only observable in individuals with high secondary psychopathy traits. Overall, the current results suggest diverging patterns of emotional processing and regulation in a social exclusion situation when comparing well-functioning individuals with varying psychopathy traits. It seemed as if individuals high on primary psychopathy traits were insensitive to contextual social cues, while individuals high on secondary psychopathy traits were more affected by the potentially threatening social situation. Cautiously transferring the current findings to forensic samples, they support the idea of moving from a behavioral understanding of the psychopathy construct to a more clinical picture with distinct cognitive and emotional processing patterns in individuals high on either primary or secondary psychopathy traits.

Mini review: Challenges in EEG emotion recognition

Electroencephalography (EEG) stands as a pioneering tool at the intersection of neuroscience and technology, offering unprecedented insights into human emotions. Through this comprehensive review, we explore the challenges and opportunities associated with EEG-based emotion recognition. While recent literature suggests promising high accuracy rates, these claims necessitate critical scrutiny for their authenticity and applicability. The article highlights the significant challenges in generalizing findings from a multitude of EEG devices and data sources, as well as the difficulties in data collection. Furthermore, the disparity between controlled laboratory settings and genuine emotional experiences presents a paradox within the paradigm of emotion research. We advocate for a balanced approach, emphasizing the importance of critical evaluation, methodological standardization, and acknowledging the dynamism of emotions for a more holistic understanding of the human emotional landscape.

Individuals who see the good in the bad engage distinctive default network coordination during post-encoding rest

Focusing on the upside of negative events often promotes resilience. Yet, the underlying mechanisms that allow some people to spontaneously see the good in the bad remain unclear. The broaden-and-build theory of positive emotion has long suggested that positive affect, including positivity in the face of negative events, is linked to idiosyncratic thought patterns (i.e., atypical cognitive responses). Yet, evidence in support of this view has been limited, in part, due to difficulty in measuring idiosyncratic cognitive processes as they unfold. To overcome this barrier, we applied Inter-Subject Representational Similarity Analysis to test whether and how idiosyncratic neural responding supports positive reactions to negative experience. We found that idiosyncratic functional connectivity patterns in the brain's default network while resting after a negative experience predicts more positive descriptions of the event. This effect persisted when controlling for connectivity 1) before and during the negative experience, 2) before, during, and after a neutral experience, and 3) between other relevant brain regions (i.e., the limbic system). The relationship between idiosyncratic default network responding and positive affect was largely driven by functional connectivity patterns between the ventromedial prefrontal cortex and the rest of the default network and occurred relatively quickly during rest. We identified post-encoding rest as a key moment and the default network as a key brain system in which idiosyncratic responses correspond with seeing the good in the bad.

Processing of the same narrative stimuli elicits common functional connectivity dynamics between individuals

It has been suggested that conscious experience is linked to the richness of brain state repertories, which change in response to environmental and internal stimuli. High-level sensory stimulation has been shown to alter local brain activity and induce neural synchrony across participants. However, the dynamic interplay of cognitive processes underlying moment-to-moment information processing remains poorly understood. Using naturalistic movies as an ecological laboratory model of the real world, here we investigate how the processing of complex naturalistic stimuli alters the dynamics of brain network interactions and how these in turn support information processing. Participants underwent fMRI recordings during movie watching, scrambled movie watching, and resting. By measuring the phase-synchrony between different brain networks, we analyzed whole-brain connectivity patterns. Our finding revealed distinct connectivity patterns associated with each experimental condition. We found higher synchronization of brain patterns across participants during movie watching compared to rest and scrambled movie conditions. Furthermore, synchronization levels increased during the most engaging parts of the movie. The synchronization dynamics among participants were associated with suspense; scenes with higher levels of suspense induced greater synchronization. These results suggest that processing the same high-level information elicits common neural dynamics across individuals, and that whole-brain functional connectivity tracks variations in processed information and subjective experience.

From mother to child: How intergenerational transfer is reflected in similarity of corticolimbic brain structure and mental health

BACKGROUND

Intergenerational transfer effects include traits transmission from parent to child. While behaviorally well documented, studies on intergenerational transfer effects for brain structure or functioning are scarce, especially those examining relations of behavioral and neurobiological endophenotypes. This study aims to investigate behavioral and neural intergenerational transfer effects associated with the corticolimbic circuitry, relevant for socioemotional functioning and mental well-being.

METHODS

T1-neuroimaging and behavioral data was obtained from 72 participants (39 mother-child dyads/ 39 children; 7-13 years; 16 girls/ 33 mothers; 26-52 years). Gray matter volume (GMV) was extracted from corticolimbic regions (subcortical: amygdala, hippocampus, nucleus accumbens; neocortical: anterior cingulate, medial orbitofrontal areas). Mother-child similarity was quantified by correlation coefficients and comparisons to random adult-child pairs.

RESULTS

We identified significant corticolimbic mother-child similarity (r = 0.663) stronger for subcortical over neocortical regions. Mother-child similarity in mental well-being was significant (r = 0.409) and the degree of dyadic similarity in mental well-being was predicted by similarity in neocortical, but not subcortical GMV.

CONCLUSION

Intergenerational neuroimaging reveals significant mother-child transfer for corticolimbic GMV, most strongly for subcortical regions. However, variations in neocortical similarity predicted similarity in mother-child well-being. Ultimately, such techniques may enhance our knowledge of behavioral and neural familial transfer effects relevant for health and disease.

The physiological measurement and evaluation of empathy of video content

The COVID-19 pandemic has led to a surge in video content consumption, but measuring viewers' empathy towards the content has been limited to subjective evaluations or attached physiological apparatus. In this study, we introduced a novel non-contact physiological method for measuring empathy towards video content by assessing the synchronization of facial micromovements between the subject and object (i.e., person) within the media. We recorded facial micromovements and heart rate variability (HRV) remotely using a camera while 62 subjects watched one video each, designed and validated to elicit one of four two-dimensional emotions: pleasant-aroused, pleasant-relaxed, unpleasant-aroused, and unpleasant-relaxed. We also collected the subjects' self-assessed emotions and empathy using a questionnaire. The results confirmed that the stimuli effectively induced the intended arousal in the subjects, as evidenced by both self-reported emotions and HRV responses that suggested higher arousal was associated with stronger activity in the sympathetic nervous system. A closer examination of HRV indicators, such as SDNN and Total Power values, showed an amplification during the unpleasant state. We interpret this as the body's dynamic response to stressors, underlining the autonomic nervous system's proactive role in responding to such stimuli. In a broader context, our results emphasized that while subjects showcased augmented empathy during aroused conditions, the introduction of stressors, represented by unpleasant content, led to a dampening of this empathetic response. This findings demonstrate the potential of non-contact physiological methods for measuring empathy toward video content.

Probing neurodynamics of experienced emotions-a Hitchhiker's guide to film fMRI

Film functional magnetic resonance imaging (fMRI) has gained tremendous popularity in many areas of neuroscience. However, affective neuroscience remains somewhat behind in embracing this approach, even though films lend themselves to study how brain function gives rise to complex, dynamic and multivariate emotions. Here, we discuss the unique capabilities of film fMRI for emotion research, while providing a general guide of conducting such research. We first give a brief overview of emotion theories as these inform important design choices. Next, we discuss films as experimental paradigms for emotion elicitation and address the process of annotating them. We then situate film fMRI in the context of other fMRI approaches, and present an overview of results from extant studies so far with regard to advantages of film fMRI. We also give an overview of state-of-the-art analysis techniques including methods that probe neurodynamics. Finally, we convey limitations of using film fMRI to study emotion. In sum, this review offers a practitioners' guide to the emerging field of film fMRI and underscores how it can advance affective neuroscience.

A normative model of brain responses to social scenarios reflects the maturity of children and adolescents' social-emotional abilities

The rapid brain maturation in childhood and adolescence accompanies the development of socio-emotional functioning. However, it is unclear how the maturation of the neural activity drives the development of socio-emotional functioning and individual differences. This study aimed to reflect the age dependence of inter-individual differences in brain responses to socio-emotional scenarios and to develop naturalistic imaging indicators to assess the maturity of socio-emotional ability at the individual level. Using three independent naturalistic imaging datasets containing healthy participants (n = 111, 21 and 122), we found and validated that age-modulated inter-individual concordance of brain responses to socio-emotional movies in specific brain regions. The similarity of an individual's brain response to the average response of older participants was defined as response typicality, which predicted an individual's emotion regulation strategies in adolescence and theory of mind (ToM) in childhood. Its predictive power was not superseded by age, sex, cognitive performance or executive function. We further showed that the movie's valence and arousal ratings grounded the response typicality. The findings highlight that forming typical brain response patterns may be a neural phenotype underlying the maturation of socio-emotional ability. The proposed response typicality represents a neuroimaging approach to measure individuals' maturity of cognitive reappraisal and ToM.

Examining the consistency of continuous affect annotations and psychophysiological measures in response to emotional videos

Despite the growing necessity of understanding the dynamics of emotion by naturalistic stimuli, averaging time-locked responses seems insufficient to capture emotional experiences that change over time. Intersubject correlation (ISC) has been implemented to examine dynamic emotional experiences by quantifying the consistency of responses across individuals. While previous research has shown that enhanced psychophysiological ISC can capture dynamic emotional experiences in response to long-lasting videos that evoke dimensional emotions, it is not yet fully understood how psychophysiological consistency varies during videos that elicit distinct emotions, such as fear. In this study, we re-analyzed publicly available data consisting of continuous affect annotations and psychophysiological signals, namely heart rate (HR), electrodermal activity (EDA), electromyographic signals from zygomaticus major (EMG-z), and corrugator supercilii (EMG-c), in response to categorical emotional videos, namely amusing, boring, relaxing, and fearful. Results showed an overall increase in ISC in multiple measures during fearful videos, indicating that emotional experiences during fearful videos were reliably consistent across participants. The effect of amusing and boring videos on ISC revealed varying results depending on the measurements. In particular, larger ISC in valence rating, EDA, and EMG-z was found for amusing than boring videos, whereas larger ISC in HR and EMG-c was observed for boring than amusing movies. Lastly, decreased ISC for relaxing videos was observed across multiple measurements, showing inconsistent emotional experiences during relaxing videos. This study builds on previous research on physiological consistency during emotional experiences by examining how the consistency of continuous affect annotations and psychophysiological measures differs in response to videos that elicit distinct emotions.

Synchronized affect in shared experiences strengthens social connection

People structure their days to experience events with others. We gather to eat meals, watch TV, and attend concerts together. What constitutes a shared experience and how does it manifest in dyadic behavior? The present study investigates how shared experiences-measured through emotional, motoric, physiological, and cognitive alignment-promote social bonding. We recorded the facial expressions and electrodermal activity (EDA) of participants as they watched four episodes of a TV show for a total of 4 h with another participant. Participants displayed temporally synchronized and spatially aligned emotional facial expressions and the degree of synchronization predicted the self-reported social connection ratings between viewing partners. We observed a similar pattern of results for dyadic physiological synchrony measured via EDA and their cognitive impressions of the characters. All four of these factors, temporal synchrony of positive facial expressions, spatial alignment of expressions, EDA synchrony, and character impression similarity, contributed to a latent factor of a shared experience that predicted social connection. Our findings suggest that the development of interpersonal affiliations in shared experiences emerges from shared affective experiences comprising synchronous processes and demonstrate that these complex interpersonal processes can be studied in a holistic and multi-modal framework leveraging naturalistic experimental designs.

Attachment Reminders Trigger Widespread Synchrony across Multiple Brains

Infant stimuli elicit widespread neural and behavioral response in human adults, and such massive allocation of resources attests to the evolutionary significance of the primary attachment. Here, we examined whether attachment reminders also trigger cross-brain concordance and generate greater neural uniformity, as indicated by intersubject correlation. Human mothers were imaged twice in oxytocin/placebo administration design, and stimuli included four ecological videos of a standard unfamiliar mother and infant: two infant/mother alone (Alone) and two mother-infant dyadic contexts (Social). Theory-driven analysis measured cross-brain synchrony in preregistered nodes of the parental caregiving network (PCN), which integrates subcortical structures underpinning mammalian mothering with cortical areas implicated in simulation, mentalization, and emotion regulation, and data-driven analysis assessed brain-wide concordance using whole-brain parcellation. Results demonstrated widespread cross-brain synchrony in both the PCN and across the neuroaxis, from primary sensory/somatosensory areas, through insular-cingulate regions, to temporal and prefrontal cortices. The Social context yielded significantly more cross-brain concordance, with PCNs striatum, parahippocampal gyrus, superior temporal sulcus, ACC, and PFC displaying cross-brain synchrony only to mother-infant social cues. Moment-by-moment fluctuations in mother-infant social synchrony, ranging from episodes of low synchrony to tightly coordinated positive bouts, were tracked online by cross-brain concordance in the preregistered ACC. Findings indicate that social attachment stimuli, representing evolutionary-salient universal cues that require no verbal narrative, trigger substantial interbrain concordance and suggest that the mother-infant bond, an icon standing at the heart of human civilization, may function to glue brains into a unified experience and bind humans into social groups.SIGNIFICANCE STATEMENT Infant stimuli elicit widespread neural response in human adults, attesting to their evolutionary significance, but do they also trigger cross-brain concordance and induce neural uniformity among perceivers? We measured cross-brain synchrony to ecological mother-infant videos. We used theory-driven analysis, measuring cross-brain concordance in the parenting network, and data-driven analysis, assessing brain-wide concordance using whole-brain parcellation. Attachment cues triggered widespread cross-brain concordance in both the parenting network and across the neuroaxis. Moment-by-moment fluctuations in behavioral synchrony were tracked online by cross-brain variability in ACC. Attachment reminders bind humans' brains into a unitary experience and stimuli characterized by social synchrony enhance neural similarity among participants, describing one mechanism by which attachment bonds provide the neural template for the consolidation of social groups.

Music-emotion EEG coupling effects based on representational similarity

BACKGROUND

Music can evoke intense emotions and music emotion is a complex cognitive process. However, we know little about the cognitive mechanisms underlying these processes, and there are significant individual differences in the emotional responses to the same musical stimuli.

NEW METHOD

We used the inter-subject representational similarity analysis (IS-RSA) method to investigate the shared music emotion responses across multiple participants. In addition, we extended IS-RSA to estimate the group cross-frequency coupling effects of music emotion. Based on the cross-frequency coupling IS-RSA, we analyzed the differences in cross-frequency coupling patterns under different music emotions using MI. Comparison of existing methods: most current IS-RSA analyses focus on within-frequency band analysis. However, the cognitive processing of music emotion involves not only activation and brain network connections differences within frequency bands but also information communication between frequency bands.

RESULTS

The results of the within-frequency band IS-RSA analysis showed that the theta and gamma frequency bands play important roles in the inter-participant consistency of music emotion. The inter-frequency band IS-RSA analysis showed that the theta-beta coupling pattern exhibited stronger inter-participant consistency compared to the theta-gamma coupling pattern, and the theta-beta coupling had significant consistent representation across various music conditions. Through the significant regions of cross-frequency coupling representation similarity analysis, we performed phase-amplitude coupling analysis on FC4-C6 and FC4-Pz connections. For the theta-beta coupling pattern, we found that the MI of these two connections exhibited different coupling patterns under different music conditions, and they showed a significant decrease compared to the baseline period.

The Synergy Zone: Connecting the Mind, Brain, and Heart for the Ideal Classroom Learning Environment

This paper proposes a new perspective on implementing neuroeducation in the classroom. The pandemic exacerbated the mental health issues of faculty and students, creating a mental health crisis that impairs learning. It is important to get our students back in "the zone", both cognitively and emotionally, by creating an ideal learning environment for capturing our students and keeping them-the Synergy Zone. Research that examines the classroom environment often focuses on the foreground-instructors' organizational and instructional aspects and content. However, the emotional climate of the classroom affects student well-being. This emotional climate would ideally exhibit the brain states of engagement, attention, connection, and enjoyment by addressing the mind, brain, and heart. This ideal learning environment would be achieved by combining proposed practices derived from three areas of research: flow theory, brain synchronization, and positive emotion with heart engagement. Each of these enhances the desired brain states in a way that the whole is greater than the sum of the individual parts. I call this the Synergy Zone. A limitation of this proposed model is that implementation of some aspects may be challenging, and professional development resources might be needed. This essay presenting this perspective provides the relevant scientific research and the educational implications of implementation.

Narrative simulation of social experiences in naturalistic context - A neurocinematic approach

Narratives may be regarded as simulations of everyday social situations. They are key to studying the human mind in socio-culturally determined contexts as they allow anchoring to the common ground of embodied and environmentally-engaged cognition. Here we review recent findings from naturalistic neuroscience on neural functions in conditions that mimic lifelike situations. We will focus particularly on neurocinematics, a research field that applies mediated narratives as stimuli for neuroimaging experiments. During the last two decades, this paradigm has contributed to an accumulation of insights about the neural underpinnings of behavior and sense-making in various narratively contextualized situations particularly pertaining to socio-emotional encounters. One of the key questions in neurocinematics is, how do intersubjectively synchronized brain activations relate to subjective experiences? Another question we address is how to bring natural contexts into experimental studies. Seeking to respond to both questions, we suggest neurocinematic studies to examine three manifestations of the same phenomenon side-by-side: subjective experiences of narrative situations, unfolding of narrative stimulus structure, and neural processes that co-constitute the experience. This approach facilitates identifying experientially meaningful activity patterns in the brain and points out what they may mean in relation to shared and communicable contents. Via rich-featured and temporally contextualized narrative stimuli, neurocinematics attempts to contribute to emerging holistic theories of neural dynamics and connectomics explaining typical and atypical interindividual variability.

Intersubject similarity in neural representations underlies shared episodic memory content

Individuals generally form their unique memories from shared experiences, yet the neural representational mechanisms underlying this subjectiveness of memory are poorly understood. The current study addressed this important question from the cross-subject neural representational perspective, leveraging a large functional magnetic resonance imaging dataset (n = 415) of a face-name associative memory task. We found that individuals' memory abilities were predicted by their synchronization to the group-averaged, canonical trial-by-trial activation level and, to a lesser degree, by their similarity to the group-averaged representational patterns during encoding. More importantly, the memory content shared between pairs of participants could be predicted by their shared local neural activation pattern, particularly in the angular gyrus and ventromedial prefrontal cortex, even after controlling for differences in memory abilities. These results uncover neural representational mechanisms for individualized memory and underscore the constructive nature of episodic memory.

Inter-subject correlations of EEG reflect subjective arousal and acoustic features of music

Background

Research on music-induced emotion and brain activity is constantly expanding. Although studies using inter-subject correlation (ISC), a collectively shared brain activity analysis method, have been conducted, whether ISC during music listening represents the music preferences of a large population remains uncertain; additionally, it remains unclear which factors influence ISC during music listening. Therefore, here, we aimed to investigate whether the ISCs of electroencephalography (EEG) during music listening represent a preference for music reflecting engagement or interest of a large population in music.

Methods

First, we selected 21 pieces of music from the Billboard Japan Hot 100 chart of 2017, which served as an indicator of preference reflecting the engagement and interest of a large population. To ensure even representation, we chose one piece for every fifth song on the chart, spanning from highly popular music to less popular ones. Next, we recorded EEG signals while the subjects listened to the selected music, and they were asked to evaluate four aspects (preference, enjoyment, frequency of listening, and arousal) for each song. Subsequently, we conducted ISC analysis by utilizing the first three principal components of EEG, which were highly correlated across subjects and extracted through correlated component analysis (CorrCA). We then explored whether music with high preferences that reflected the engagement and interest of large population had high ISC values. Additionally, we employed cluster analysis on all 21 pieces of music, utilizing the first three principal components of EEG, to investigate the impact of emotions and musical characteristics on EEG ISC during music listening.

Results

A significant distinction was noted between the mean ISC values of the 10 higher-ranked pieces of music compared to the 10 lower-ranked pieces of music [t(542) = -1.97, p = 0.0025]. This finding suggests that ISC values may correspond preferences reflecting engagement or interest of a large population. Furthermore, we found that significant variations were observed in the first three principal component values among the three clusters identified through cluster analysis, along with significant differences in arousal levels. Moreover, the characteristics of the music (tonality and tempo) differed among the three clusters. This indicates that the principal components, which exhibit high correlation among subjects and were employed in calculating ISC values, represent both subjects' arousal levels and specific characteristics of the music.

Conclusion

Subjects' arousal values during music listening and music characteristics (tonality and tempo) affect ISC values, which represent the interest of a large population in music.

Family Cohesion Moderates the Relation between Parent-Child Neural Connectivity Pattern Similarity and Youth's Emotional Adjustment

Despite a recent surge in research examining parent-child neural similarity using fMRI, there remains a need for further investigation into how such similarity may play a role in children's emotional adjustment. Moreover, no prior studies explored the potential contextual factors that may moderate the link between parent-child neural similarity and children's developmental outcomes. In this study, 32 parent-youth dyads (parents: M age = 43.53 years, 72% female; children: M age = 11.69 years, 41% female) watched an emotion-evoking animated film while being scanned using fMRI. We first quantified how similarly emotion network interacts with other brain regions in responding to the emotion-evoking film between parents and their children. We then examined how such parent-child neural similarity is associated with children's emotional adjustment, with attention to the moderating role of family cohesion. Results revealed that higher parent-child similarity in functional connectivity pattern during movie viewing was associated with better emotional adjustment, including less negative affect, lower anxiety, and greater ego resilience in youth. Moreover, such associations were significant only among families with higher cohesion, but not among families with lower cohesion. The findings advance our understanding of the neural mechanisms underlying how children thrive by being in sync and attuned with their parents, and provide novel empirical evidence that the effects of parent-child concordance at the neural level on children's development are contextually dependent.SIGNIFICANCE STATEMENT What neural processes underlie the attunement between children and their parents that helps children thrive? Using a naturalistic movie-watching fMRI paradigm, we find that greater parent-child similarity in how emotion network interacts with other brain regions during movie viewing is associated with youth's better emotional adjustment including less negative affect, lower anxiety, and greater ego resilience. Interestingly, these associations are only significant among families with higher cohesion, but not among those with lower cohesion. Our findings provide novel evidence that parent-child shared neural processes to emotional situations can confer benefits to children, and underscore the importance of considering specific family contexts in which parent-child neural similarity may be beneficial or detrimental to children's development, highlighting a crucial direction for future research.

Neural synchronization as a function of engagement with the narrative

We can all agree that a good story engages us, however, agreeing which story is good is far more debatable. In this study, we explored whether engagement with a narrative synchronizes listeners' brain responses, by examining individual differences in engagement to the same story. To do so, we pre-registered and re-analyzed a previously collected dataset by Chang et al. (2021) of functional Magnetic Resonance Imaging (fMRI) scans of 25 participants who listened to a one-hour story and answered questionnaires. We assessed the degree of their overall engagement with the story and their engagement with the main characters. The questionnaires revealed individual differences in engagement with the story, as well as different valence towards specific characters. Neuroimaging data showed that the auditory cortex, the default mode network (DMN) and language regions were involved in processing the story. Increased engagement with the story was correlated with increased neural synchronization within regions in the DMN (especially the medial prefrontal cortex), as well as regions outside the DMN such as the dorso-lateral prefrontal cortex and the reward system. Interestingly, positively and negatively engaging characters elicited different patterns of neural synchronization. Finally, engagement increased functional connectivity within and between the DMN, the ventral attention network and the control network. Taken together, these findings suggest that engagement with a narrative synchronizes listeners' responses in regions involved in mentalizing, reward, working memory and attention. By examining individual differences in engagement, we revealed that these synchronization patterns are due to engagement, and not due to differences in the narrative's content.

The default network dominates neural responses to evolving movie stories

Neuroscientific studies exploring real-world dynamic perception often overlook the influence of continuous changes in narrative content. In our research, we utilize machine learning tools for natural language processing to examine the relationship between movie narratives and neural responses. By analyzing over 50,000 brain images of participants watching Forrest Gump from the studyforrest dataset, we find distinct brain states that capture unique semantic aspects of the unfolding story. The default network, associated with semantic information integration, is the most engaged during movie watching. Furthermore, we identify two mechanisms that underlie how the default network liaises with the amygdala and hippocampus. Our findings demonstrate effective approaches to understanding neural processes in everyday situations and their relation to conscious awareness.

Toward reframing brain-social dynamics: current assumptions and future challenges

Evolutionary analyses suggest that the human social brain and sociality appeared together. The two fundamental tools that accelerated the concurrent emergence of the social brain and sociality include learning and plasticity. The prevailing core idea is that the primate brain and the cortex in particular became reorganised over the course of evolution to facilitate dynamic adaptation to ongoing changes in physical and social environments. Encouraged by computational or survival demands or even by instinctual drives for living in social groups, the brain eventually learned how to learn from social experience via its massive plastic capacity. A fundamental framework for modeling these orchestrated dynamic responses is that social plasticity relies upon neuroplasticity. In the present article, we first provide a glimpse into the concepts of plasticity, experience, with emphasis on social experience. We then acknowledge and integrate the current theoretical concepts to highlight five key intertwined assumptions within social neuroscience that underlie empirical approaches for explaining the brain-social dynamics. We suggest that this epistemological view provides key insights into the ontology of current conceptual frameworks driving future research to successfully deal with new challenges and possible caveats in favour of the formulation of novel assumptions. In the light of contemporary societal challenges, such as global pandemics, natural disasters, violent conflict, and other human tragedies, discovering the mechanisms of social brain plasticity will provide new approaches to support adaptive brain plasticity and social resilience.

Predicting Moral Elevation Conveyed in Danmaku Comments Using EEGs

Moral elevation, the emotion that arises when individuals observe others' moral behaviors, plays an important role in determining moral behaviors in real life. While recent research has demonstrated the potential to decode basic emotions with brain signals, there has been limited exploration of affective computing for moral elevation, an emotion related to social cognition. To address this gap, we recorded electroencephalography (EEG) signals from 23 participants while they viewed videos that were expected to elicit moral elevation. More than 30,000 danmaku comments were extracted as a crowdsourcing tagging method to label moral elevation continuously at a 1-s temporal resolution. Then, by employing power spectra features and the least absolute shrinkage and selection operator regularized regression analyses, we achieved a promising prediction performance for moral elevation (prediction r = 0.44 ± 0.11). Our findings indicate that it is possible to decode moral elevation using EEG signals. Moreover, the small-sample neural data can predict the continuous moral elevation experience conveyed in danmaku comments from a large population.

EEG and peripheral markers of viewer ratings: a study of short films

Introduction

Cinema is an important part of modern culture, influencing millions of viewers. Research suggested many models for the prediction of film success, one of them being the use of neuroscientific tools. The aim of our study was to find physiological markers of viewer perception and correlate them to short film ratings given by our subjects. Short films are used as a test case for directors and screenwriters and can be created to raise funding for future projects; however, they have not been studied properly with physiological methods.

Methods

We recorded electroencephalography (18 sensors), facial electromyography (corrugator supercilii and zygomaticus major), photoplethysmography, and skin conductance in 21 participants while watching and evaluating 8 short films (4 dramas and 4 comedies). Also, we used machine learning (CatBoost, SVR) to predict the exact rating of each film (from 1 to 10), based on all physiological indicators. In addition, we classified each film as low or high rated by our subjects (with Logistic Regression, KNN, decision tree, CatBoost, and SVC).

Results

The results showed that ratings did not differ between genres. Corrugator supercilii activity ("frowning" muscle) was larger when watching dramas; whereas zygomaticus major ("smiling" muscle) activity was larger during the watching of comedies. Of all somatic and vegetative markers, only zygomaticus major activity, PNN50, SD1/SD2 (heart rate variability parameters) positively correlated to the film ratings. The EEG engagement indices, beta/(alpha+theta) and beta/alpha correlated positively with the film ratings in the majority of sensors. Arousal (beta_F3 + beta_F4)/(alpha_F3 + alpha_F4), and valence (alpha_F4/beta_F4) - (alpha_F3/beta_F3) indices also correlated positively to film ratings. When we attempted to predict exact ratings, MAPE was 0.55. As for the binary classification, logistic regression yielded the best values (area under the ROC curve = 0.62) than other methods (0.51-0.60).

Discussion

Overall, we revealed EEG and peripheral markers, which reflect viewer ratings and can predict them to a certain extent. In general, high film ratings can reflect a fusion of high arousal and different valence, positive valence being more important. These findings broaden our knowledge about the physiological basis of viewer perception and can be potentially used at the stage of film production.

Impact of external excitations on blinking enhanced synchronization in bistable vibrational energy harvesters

Vibrational energy harvesters are capable of converting low-frequency broad-band mechanical energy into electrical power and can be used in implantable medical devices and wireless sensors. With the use of such energy harvesters, it is feasible to generate continuous power that is more reliable and cost-effective. According to previous findings, the energy harvester can offer rich complex dynamics, one of which is obtaining the synchronization behavior, which is intriguing to achieve desirable power from energy harvesters. Therefore, we consider bistable energy harvesters with periodic and quasiperiodic excitations to investigate synchronization. Specifically, we introduce blinking into the coupling function to check whether it improves the synchronization. Interestingly, we discover that raising the normalized proportion of blinking can initiate synchronization behaviors even with lower optimal coupling strength than the absence of blinking in the coupling (i.e., continuous coupling). The existence of synchronization behaviors is confirmed by finding the largest Lyapunov exponents. In addition, the results show that the optimal coupling strength needed to achieve synchronization for quasiperiodic excitations is smaller than that for periodic excitations.

Lonely Individuals Process the World in Idiosyncratic Ways

Loneliness is detrimental to well-being and is often accompanied by self-reported feelings of not being understood by other people. What contributes to such feelings in lonely people? We used functional MRI of 66 first-year university students to unobtrusively measure the relative alignment of people's mental processing of naturalistic stimuli and tested whether lonely people actually process the world in idiosyncratic ways. We found evidence for such idiosyncrasy: Lonely individuals' neural responses were dissimilar to those of their peers, particularly in regions of the default-mode network in which similar responses have been associated with shared perspectives and subjective understanding. These relationships persisted when we controlled for demographic similarities, objective social isolation, and individuals' friendships with each other. Our findings raise the possibility that being surrounded by people who see the world differently from oneself, even if one is friends with them, may be a risk factor for loneliness.

Dynamic segregation and integration of brain functional networks associated with emotional arousal

The organization of brain functional networks dynamically changes with emotional stimuli, but its relationship to emotional behaviors is still unclear. In the DEAP dataset, we used the nested-spectral partition approach to identify the hierarchical segregation and integration of functional networks and investigated the dynamic transitions between connectivity states under different arousal conditions. The frontal and right posterior parietal regions were dominant for network integration whereas the bilateral temporal, left posterior parietal, and occipital regions were responsible for segregation and functional flexibility. High emotional arousal behavior was associated with stronger network integration and more stable state transitions. Crucially, the connectivity states of frontal, central, and right parietal regions were closely related to arousal ratings in individuals. Besides, we predicted the individual emotional performance based on functional connectivity activities. Our results demonstrate that brain connectivity states are closely associated with emotional behaviors and could be reliable and robust indicators for emotional arousal.

The Representational Similarity between Visual Perception and Recent Perceptual History

From moment to moment, the visual properties of objects in the world fluctuate because of external factors like ambient lighting, occlusion and eye movements, and internal (proximal) noise. Despite this variability in the incoming information, our perception is stable. Serial dependence, the behavioral attraction of current perceptual responses toward previously seen stimuli, may reveal a mechanism underlying stability: a spatiotemporally tuned operator that smooths over spurious fluctuations. The current study examined the neural underpinnings of serial dependence by recording the electroencephalographic (EEG) brain response of female and male human observers to prototypical objects (faces, cars, and houses) and morphs that mixed properties of two prototypes. Behavior was biased toward previously seen objects. Representational similarity analysis (RSA) revealed that responses evoked by visual objects contained information about the previous stimulus. The trace of previous representations in the response to the current object occurred immediately on object appearance, suggesting that serial dependence arises from a brain state or set that precedes processing of new input. However, the brain response to current visual objects was not representationally similar to the trace they leave on subsequent object representations. These results reveal that while past stimulus history influences current representations, this influence does not imply a shared neural code between the previous trial (memory) and the current trial (perception).SIGNIFICANCE STATEMENT The perception of visual objects is pulled toward instances of that object seen in the recent past. The neural underpinnings of this serial dependence remain to be fully investigated. The present study examined electroencephalographic (EEG) responses to faces, cars, and houses, and ambiguous between-category morphs. With representational similarity analysis (RSA), we showed (1) object-specific neural patterns that differentiate the three categories; (2) that the response to the current object contains information about the previous object, mirroring behavioral serial dependence; (3) that the object-specific neural pattern about the past was different from that in the current response, revealing that while past stimulus history influences current representations, this does not imply a shared neural code between the previous trial (memory) and the current trial (perception).

Unbalanced amygdala communication in major depressive disorder

BACKGROUND

Previous studies suggested an association between functional alteration of the amygdala and typical major depressive disorder (MDD) symptoms. Examining whether and how the interaction between the amygdala and regions/functional networks is altered in patients with MDD is important for understanding its neural basis.

METHODS

Resting-state functional magnetic resonance imaging data were recorded from 67 patients with MDD and 74 age- and sex-matched healthy controls (HCs). A framework for large-scale network analysis based on seed mappings of amygdala sub-regions, using a multi-connectivity-indicator strategy (cross-correlation, total interdependencies (TI), Granger causality (GC), and machine learning), was employed. Multiple indicators were compared between the two groups. The altered indicators were ranked in a supporting-vector machine-based procedure and associated with the Hamilton Rating Scale for Depression scores.

RESULTS

The amygdala connectivity with the default mode network and ventral attention network regions was enhanced and that with the somatomotor network, dorsal frontoparietal network, and putamen regions in patients with MDD was reduced. The machine learning analysis highlighted altered indicators that were most conducive to the classification between the two groups.

LIMITATIONS

Most patients with MDD received different pharmacological treatments. It is difficult to illustrate the medication state's effect on the alteration model because of its complex situation.

CONCLUSION

The results indicate an unbalanced interaction model between the amygdala and functional networks and regions essential for various emotional and cognitive functions. The model can help explain potential aberrancy in the neural mechanisms that underlie the functional impairments observed across various domains in patients with MDD.

Higher levels of narrativity lead to similar patterns of posterior EEG activity across individuals

Introduction

The focus of cognitive and psychological approaches to narrative has not so much been on the elucidation of important aspects of narrative, but rather on using narratives as tools for the investigation of higher order cognitive processes elicited by narratives (e.g., understanding, empathy, etc.). In this study, we work toward a scalar model of narrativity, which can provide testable criteria for selecting and classifying communication forms in their level of narrativity. We investigated whether being exposed to videos with different levels of narrativity modulates shared neural responses, measured by inter-subject correlation, and engagement levels.

Methods

Thirty-two participants watched video advertisements with high-level and low-level of narrativity while their neural responses were measured through electroencephalogram. Additionally, participants' engagement levels were calculated based on the composite of their self-reported attention and immersion scores.

Results

Results demonstrated that both calculated inter-subject correlation and engagement scores for high-level video ads were significantly higher than those for low-level, suggesting that narrativity levels modulate inter-subject correlation and engagement.

Discussion

We believe that these findings are a step toward the elucidation of the viewers' way of processing and understanding a given communication artifact as a function of the narrative qualities expressed by the level of narrativity.

Intention affects fairness processing: Evidence from behavior and representational similarity analysis of event-related potential signals

In an ultimatum game, the responder must decide between pursuing self-interest and insisting on fairness, and these choices are affected by the intentions of the proposer. However, the time course of this social decision-making process is unclear. Representational similarity analysis (RSA) is a useful technique for linking brain activity with rich behavioral data sets. In this study, electroencephalography (EEG) was used to measure the time course of neural responses to proposed allocation schemes with different intentions. Twenty-eight participants played an ultimatum game as responders. They had to choose between accepting and rejecting the fair or unfair money allocation schemes of proposers. The schemes were offered based on the proposer's selfish intention (monetary gain), altruistic intention (donation to charity), or ambiguous intention (unknown to the responder). We used a spatiotemporal RSA and inter-subject RSA (IS-RSA) to explore the connections between event-related potentials (ERPs) after offer presentation and intention presentation with four types of behavioral data (acceptance, response time, fairness ratings, and pleasantness ratings). The spatiotemporal RSA results revealed that only response time variation was linked with the difference in ERPs at 432-592 ms after offer presentation on the posterior parietal and prefrontal regions. Meanwhile, the IS-RSA results found a significant association between inter-individual differences in response time and differences in ERP activity at 596-812 ms after the presentation of ambiguous intention, particularly in the prefrontal region. This study expands the intention-based reciprocal model to the third-party context and demonstrates that brain activity can represent response time differences in social decision-making.

Every individual makes a difference: A trinity derived from linking individual brain morphometry, connectivity and mentalising ability

Mentalising ability, indexed as the ability to understand others' beliefs, feelings, intentions, thoughts and traits, is a pivotal and fundamental component of human social cognition. However, considering the multifaceted nature of mentalising ability, little research has focused on characterising individual differences in different mentalising components. And even less research has been devoted to investigating how the variance in the structural and functional patterns of the amygdala and hippocampus, two vital subcortical regions of the "social brain", are related to inter-individual variability in mentalising ability. Here, as a first step toward filling these gaps, we exploited inter-subject representational similarity analysis (IS-RSA) to assess relationships between amygdala and hippocampal morphometry (surface-based multivariate morphometry statistics, MMS), connectivity (resting-state functional connectivity, rs-FC) and mentalising ability (interactive mentalisation questionnaire [IMQ] scores) across the participants ( N = 24 $$ N=24 $$ ). In IS-RSA, we proposed a novel pipeline, that is, computing patching and pooling operations-based surface distance (CPP-SD), to obtain a decent representation for high-dimensional MMS data. On this basis, we found significant correlations (i.e., second-order isomorphisms) between these three distinct modalities, indicating that a trinity existed in idiosyncratic patterns of brain morphometry, connectivity and mentalising ability. Notably, a region-related mentalising specificity emerged from these associations: self-self and self-other mentalisation are more related to the hippocampus, while other-self mentalisation shows a closer link with the amygdala. Furthermore, by utilising the dyadic regression analysis, we observed significant interactions such that subject pairs with similar morphometry had even greater mentalising similarity if they were also similar in rs-FC. Altogether, we demonstrated the feasibility and illustrated the promise of using IS-RSA to study individual differences, deepening our understanding of how individual brains give rise to their mentalising abilities.

Intracranial study in humans: Neural spectral changes during watching comedy movie of Charlie Chaplin

Humor plays a prominent role in our lives. Thus, understanding the cognitive and neural mechanisms of humor is particularly important. Previous studies that investigated neural substrates of humor used functional MRI and to a lesser extent EEG. In the present study, we conducted intracranial recording in human patients, enabling us to obtain the signal with high temporal precision from within specific brain locations. Our analysis focused on the temporal lobe and the surrounding areas, the temporal lobe was most densely covered in our recording. Thirteen patients watched a fragment of a Charlie Chaplin movie. An independent group of healthy participants rated the same movie fragment, helping us to identify the most funny and the least funny frames of the movie. We compared neural activity occurring during the most funny and least funny frames across frequencies in the range of 1-170 Hz. The most funny compared to least funny parts of the movie were associated with activity modulation in the broadband high-gamma (70-170 Hz; mostly activation) and to a lesser extent gamma band (40-69Hz; activation) and low frequencies (1-12 Hz, delta, theta, alpha bands; mostly deactivation). With regard to regional specificity, we found three types of brain areas: (I) temporal pole, middle and inferior temporal gyrus (both anterior and posterior) in which there was both activation in the high-gamma/gamma bands and deactivation in low frequencies; (II) ventral part of the temporal lobe such as the fusiform gyrus, in which there was mostly deactivation the low frequencies; (III) posterior temporal cortex and its environment, such as the middle occipital and the temporo-parietal junction, in which there was activation in the high-gamma/gamma band. Overall, our results suggest that humor appreciation might be achieved by neural activity across the frequency spectrum.

Somatovisceral influences on emotional development

Frameworks of emotional development have tended to focus on how environmental factors shape children's emotion understanding. However, individual experiences of emotion represent a complex interplay between both external environmental inputs and internal somatovisceral signaling. Here, we discuss the importance of afferent signals and coordination between central and peripheral mechanisms in affective response processing. We propose that incorporating somatovisceral theories of emotions into frameworks of emotional development can inform how children understand emotions in themselves and others. We highlight promising directions for future research on emotional development incorporating this perspective, namely afferent cardiac processing and interoception, immune activation, physiological synchrony, and social touch.

Decoding brain basis of laughter and crying in natural scenes

Laughter and crying are universal signals of prosociality and distress, respectively. Here we investigated the functional brain basis of perceiving laughter and crying using naturalistic functional magnetic resonance imaging (fMRI) approach. We measured haemodynamic brain activity evoked by laughter and crying in three experiments with 100 subjects in each. The subjects i) viewed a 20-minute medley of short video clips, and ii) 30 minutes of a full-length feature film, and iii) listened to 15 minutes of a radio play that all contained bursts of laughter and crying. Intensity of laughing and crying in the videos and radio play was annotated by independent observes, and the resulting time series were used to predict hemodynamic activity to laughter and crying episodes. Multivariate pattern analysis (MVPA) was used to test for regional selectivity in laughter and crying evoked activations. Laughter induced widespread activity in ventral visual cortex and superior and middle temporal and motor cortices. Crying activated thalamus, cingulate cortex along the anterior-posterior axis, insula and orbitofrontal cortex. Both laughter and crying could be decoded accurately (66-77% depending on the experiment) from the BOLD signal, and the voxels contributing most significantly to classification were in superior temporal cortex. These results suggest that perceiving laughter and crying engage distinct neural networks, whose activity suppresses each other to manage appropriate behavioral responses to others' bonding and distress signals.

Functional organization of social perception in the human brain

Humans rapidly extract diverse and complex information from ongoing social interactions, but the perceptual and neural organization of the different aspects of social perception remains unresolved. We showed short movie clips with rich social content to 97 healthy participants while their haemodynamic brain activity was measured with fMRI. The clips were annotated moment-to-moment for a large set of social features and 45 of the features were evaluated reliably between annotators. Cluster analysis of the social features revealed that 13 dimensions were sufficient for describing the social perceptual space. Three different analysis methods were used to map the social perceptual processes in the human brain. Regression analysis mapped regional neural response profiles for different social dimensions. Multivariate pattern analysis then established the spatial specificity of the responses and intersubject correlation analysis connected social perceptual processing with neural synchronization. The results revealed a gradient in the processing of social information in the brain. Posterior temporal and occipital regions were broadly tuned to most social dimensions and the classifier revealed that these responses showed spatial specificity for social dimensions; in contrast Heschl gyri and parietal areas were also broadly associated with different social signals, yet the spatial patterns of responses did not differentiate social dimensions. Frontal and subcortical regions responded only to a limited number of social dimensions and the spatial response patterns did not differentiate social dimension. Altogether these results highlight the distributed nature of social processing in the brain.