Cognitive processing of a common stimulus synchronizes brains, hearts, and eyes

Auditory Challenges and Listening Effort in School-Age Children With Autism: Insights From Pupillary Dynamics During Speech-in-Noise Perception

PURPOSE

This study aimed to investigate challenges in speech-in-noise (SiN) processing faced by school-age children with autism spectrum conditions (ASCs) and their impact on listening effort.

METHOD

Participants, including 23 Mandarin-speaking children with ASCs and 19 age-matched neurotypical (NT) peers, underwent sentence recognition tests in both quiet and noisy conditions, with a speech-shaped steady-state noise masker presented at 0-dB signal-to-noise ratio in the noisy condition. Recognition accuracy rates and task-evoked pupil responses were compared to assess behavioral performance and listening effort during auditory tasks.

RESULTS

No main effect of group was found on accuracy rates. Instead, significant effects emerged for autistic trait scores, listening conditions, and their interaction, indicating that higher trait scores were associated with poorer performance in noise. Pupillometric data revealed significantly larger and earlier peak dilations, along with more varied pupillary dynamics in the ASC group relative to the NT group, especially under noisy conditions. Importantly, the ASC group's peak dilation in quiet mirrored that of the NT group in noise. However, the ASC group consistently exhibited reduced mean dilations than the NT group.

CONCLUSIONS

Pupillary responses suggest a different resource allocation pattern in ASCs: An initial sharper and larger dilation may signal an intense, narrowed resource allocation, likely linked to heightened arousal, engagement, and cognitive load, whereas a subsequent faster tail-off may indicate a greater decrease in resource availability and engagement, or a quicker release of arousal and cognitive load. The presence of noise further accentuates this pattern. This highlights the unique SiN processing challenges children with ASCs may face, underscoring the importance of a nuanced, individual-centric approach for interventions and support.

Inter-subject correlation of audience facial expressions predicts audience engagement during theatrical performances

During performances, audiences experience various emotional states, and these are reflected in their ongoing facial expressions. We investigated if audience engagement could be determined by measuring the inter-subject correlation (ISC) of non-invasively recorded audience facial expressions. We filmed the faces of multiple audience members at theatrical performances and determined the intensity of their different facial expressions throughout the performances. Neutral, happy, anger, and disgust expression ISCs accounted for up to 24% of the performance dramaturge's predictions of audience engagement. Expression synchrony was greater between individuals in close proximity, suggesting effects of emotional contagion or cognitive similarities between neighboring individuals, whereas expression synchrony was greatest between individuals who were younger, female, and with greater levels of empathy, showing that individual characteristics impact shared audience experiences. Together, our results show that facial expression synchronization could be used as a real-time non-invasive indicator of engagement in audiences larger than achieved using previous approaches.

Reliability for music-induced heart rate synchronization

Common inputs synchronize various biological systems, including human physical and cognitive processes. This mechanism potentially explains collective human emotions in theater as unintentional behavioral synchronization. However, the inter-subject correlation of physiological signals among individuals is small. Based on findings on the common-input synchronization of nonlinear systems, we hypothesized that individual differences in perceptual and cognitive systems reduce the reliability of physiological responses to aesthetic stimuli and, thus, disturb synchronization. We tested this by comparing the inter- and intra-subject Pearson's correlation coefficients and nonlinear phase synchronization, calculated using instantaneous heart rate data measured while appreciating music. The results demonstrated that inter-subject correlations were consistently lower than intra-subject correlations, regardless of participants' music preferences and daily moods. Further, music-induced heart rate synchronization depends on the reliability of physiological responses to musical pieces rather than mood or motivation. This study lays the foundation for future empirical research on collective emotions in theater.

How and Why People Synchronize: An Integrated Perspective

Academic AbstractInterpersonal synchrony, the alignment of behavior and/or physiology during interactions, is a pervasive phenomenon observed in diverse social contexts. Here we synthesize across contexts and behaviors to classify the different forms and functions of synchrony. We provide a concise framework for classifying the manifold forms of synchrony along six dimensions: periodicity, discreteness, spatial similarity, directionality, leader-follower dynamics, and observability. We also distill the various proposed functions of interpersonal synchrony into four interconnected functions: reducing complexity and improving understanding, accomplishing joint tasks, strengthening social connection, and influencing partners' behavior. These functions derive from first principles, emerge from each other, and are accomplished by some forms of synchrony more than others. Effective synchrony flexibly adapts to social goals and more synchrony is not always better. Our synthesis offers a shared framework and language for the field, allowing for better cross-context and cross-behavior comparisons, generating new hypotheses, and highlighting future research directions.

A practical guide to EEG hyperscanning in joint action research: from motivation to implementation

Developments in cognitive neuroscience have led to the emergence of hyperscanning, the simultaneous measurement of brain activity from multiple people. Hyperscanning is useful for investigating social cognition, including joint action, because of its ability to capture neural processes that occur within and between people as they coordinate actions toward a shared goal. Here, we provide a practical guide for researchers considering using hyperscanning to study joint action and seeking to avoid frequently raised concerns from hyperscanning skeptics. We focus specifically on Electroencephalography (EEG) hyperscanning, which is widely available and optimally suited for capturing fine-grained temporal dynamics of action coordination. Our guidelines cover questions that are likely to arise when planning a hyperscanning project, ranging from whether hyperscanning is appropriate for answering one's research questions to considerations for study design, dependent variable selection, data analysis and visualization. By following clear guidelines that facilitate careful consideration of the theoretical implications of research design choices and other methodological decisions, joint action researchers can mitigate interpretability issues and maximize the benefits of hyperscanning paradigms.

Bidirectional brain-body interactions during natural story listening

Narratives can synchronize neural and physiological signals between individuals, but the relationship between these signals, and the underlying mechanism, is unclear. We hypothesized a top-down effect of cognition on arousal and predicted that auditory narratives will drive not only brain signals but also peripheral physiological signals. We find that auditory narratives entrained gaze variation, saccade initiation, pupil size, and heart rate. This is consistent with a top-down effect of cognition on autonomic function. We also hypothesized a bottom-up effect, whereby autonomic physiology affects arousal. Controlled breathing affected pupil size, and heart rate was entrained by controlled saccades. Additionally, fluctuations in heart rate preceded fluctuations of pupil size and brain signals. Gaze variation, pupil size, and heart rate were all associated with anterior-central brain signals. Together, these results suggest bidirectional causal effects between peripheral autonomic function and central brain circuits involved in the control of arousal.

Onscreen presence of instructors in video lectures affects learners' neural synchrony and visual attention during multimedia learning

COVID-19 forced students to rely on online learning using multimedia tools, and multimedia learning continues to impact education beyond the pandemic. In this study, we combined behavioral, eye-tracking, and neuroimaging paradigms to identify multimedia learning processes and outcomes. College students viewed four video lectures including slides with either an onscreen human instructor, an animated instructor, or no onscreen instructor. Brain activity was recorded via fMRI, visual attention was recorded via eye-tracking, and learning outcome was assessed via post-tests. Onscreen presence of instructor, compared with no instructor presence, resulted in superior post-test performance, less visual attention on the slide, more synchronized eye movements during learning, and higher neural synchronization in cortical networks associated with socio-emotional processing and working memory. Individual variation in cognitive and socio-emotional abilities and intersubject neural synchronization revealed different levels of cognitive and socio-emotional processing in different learning conditions. The instructor-present condition evoked increased synchronization, likely reflecting extra processing demands in attentional control, working memory engagement, and socio-emotional processing. Although human instructors and animated instructors led to comparable learning outcomes, the effects were due to the dynamic interplay of information processing vs. attentional distraction. These findings reflect a benefit-cost trade-off where multimedia learning outcome is enhanced only when the cognitive benefits motivated by the social presence of onscreen instructor outweigh the cognitive costs brought about by concurrent attentional distraction unrelated to learning.

A dual-brain therapeutic approach using noninvasive brain stimulation based on two-person neuroscience: A perspective review

Our actions and decisions in everyday life are heavily influenced by social interactions, which are dynamic feedback loops involving actions, reactions, and internal cognitive processes between individual agents. Social interactions induce interpersonal synchrony, which occurs at different biobehavioral levels and comprises behavioral, physiological, and neurological activities. Hyperscanning-a neuroimaging technique that simultaneously measures the activity of multiple brain regions-has provided a powerful second-person neuroscience tool for investigating the phase alignment of neural processes during interactive social behavior. Neural synchronization, revealed by hyperscanning, is a phenomenon called inter-brain synchrony- a process that purportedly facilitates social interactions by prompting appropriate anticipation of and responses to each other's social behaviors during ongoing shared interactions. In this review, I explored the therapeutic dual-brain approach using noninvasive brain stimulation to target inter-brain synchrony based on second-person neuroscience to modulate social interaction. Artificially inducing synchrony between the brains is a potential adjunct technique to physiotherapy, psychotherapy, and pain treatment- which are strongly influenced by the social interaction between the therapist and patient. Dual-brain approaches to personalize stimulation parameters must consider temporal, spatial, and oscillatory factors. Multiple data fusion analysis, the assessment of inter-brain plasticity, a closed-loop system, and a brain-to-brain interface can support personalized stimulation.

The role of engagement and arousal in emotion regulation: an EEG study

Cognitive reappraisal and expressive suppression are well-studied strategies of emotion regulation (ER). However, the results on their physiological basis are controversial. While in some studies, ER was accompanied by the inhibition of the nervous system, others suggested that ER even might increase arousal and engagement. We calculated the inter-subject correlation (ISC) and indices of engagement, valence and arousal of EEG during suppression, reappraisal, or natural watching of neutral and negative videos. First, both suppression and reappraisal provoked a higher ISC in comparison with watching negative or neutral videos. We consider this as a marker of engagement to the task and feedback processing required for ER. Second, the engagement index was lower during ER compared to watching negative videos in central electrodes, whereas both strategies provoked a higher engagement in frontal electrodes. Third, the arousal index of EEG was higher during all negative conditions; therefore, regulation required a certain level of arousal. In summary, different EEG measures seem to be sensitive to different aspects of ER.

Do we all synch alike? Brain-body-environment interactions in ASD

Autism Spectrum Disorder (ASD) is characterized by rigidity of routines and restricted interests, and atypical social communication and interaction. Recent evidence for altered synchronization of neuro-oscillatory brain activity with regularities in the environment and of altered peripheral nervous system function in ASD present promising novel directions for studying pathophysiology and its relationship to ASD clinical phenotype. Human cognition and action are significantly influenced by physiological rhythmic processes that are generated by both the central nervous system (CNS) and the autonomic nervous system (ANS). Normally, perception occurs in a dynamic context, where brain oscillations and autonomic signals synchronize with external events to optimally receive temporally predictable rhythmic information, leading to improved performance. The recent findings on the time-sensitive coupling between the brain and the periphery in effective perception and successful social interactions in typically developed highlight studying the interactions within the brain-body-environment triad as a critical direction in the study of ASD. Here we offer a novel perspective of autism as a case where the temporal dynamics of brain-body-environment coupling is impaired. We present evidence from the literature to support the idea that in autism the nervous system fails to operate in an adaptive manner to synchronize with temporally predictable events in the environment to optimize perception and behavior. This framework could potentially lead to novel biomarkers of hallmark deficits in ASD such as cognitive rigidity and altered social interaction.

Physiological synchrony and shared flow state in Javanese gamelan: positively associated while improvising, but not for traditional performance

The experience of shared flow refers to the optimal balance between challenge and ability for a given task, resulting from interpersonal action in a group situation. The performance of Javanese gamelan is an ideal setting to investigate shared flow, due to the requirement that all performers on varying instrumental parts work harmoniously, allowing for shared flow and its native equivalent, ngeli. To minimise the disruption of flow, while still measuring it continuously, one way to assess a person's state is by measuring physiological responses of the sympathetic (i.e., fight-or-flight) system, namely heart rate and skin conductance. Flow has been related to physiological signatures, and shared actions in music-making have been related to synchronised physiology. However, to our knowledge, no study yet has directly investigated the links between shared physiology and shared flow. Therefore, this study aimed to assess the associations between flow states, physiological synchrony, and Javanese gamelan playing. Subsequently, we tested for differences between advanced and beginner groups playing traditional gamelan pieces and improvising. Firstly, a factor analysis revealed a two-factor solution of Awareness and Absorption for self-reported shared flow. Next, using inter-subject correlation to assess synchrony and circular shuffling to infer significance, we found a greater proportion of significance in traditional playing compared to improvised playing for the experienced group, and the opposite for the beginner group. Lastly, linear mixed models revealed largely positive associations between synchronised physiology and shared flow during improvised playing, and negative associations during traditional playing, regardless of experience levels. This study demonstrates methodological possibilities for the quantitative study of shared flow in music-making contexts, and potential differences in shared flow experience in improvised and traditional, or prescribed, playing.

Physiological synchrony in electrodermal activity predicts decreased vigilant attention induced by sleep deprivation

Introduction

When multiple individuals are presented with narrative movie or audio clips, their electrodermal activity (EDA) and heart rate show significant similarities. Higher levels of such inter-subject physiological synchrony are related with higher levels of attention toward the narrative, as for instance expressed by more correctly answered questions about the narrative. We here investigate whether physiological synchrony in EDA and heart rate during watching of movie clips predicts performance on a subsequent vigilant attention task among participants exposed to a night of total sleep deprivation.

Methods

We recorded EDA and heart rate of 54 participants during a night of total sleep deprivation. Every hour from 22:00 to 07:00 participants watched a 10-min movie clip during which we computed inter-subject physiological synchrony. Afterwards, they answered questions about the movie and performed the psychomotor vigilance task (PVT) to capture attentional performance.

Results

We replicated findings that inter-subject correlations in EDA and heart rate predicted the number of correct answers on questions about the movie clips. Furthermore, we found that inter-subject correlations in EDA, but not in heart rate, predicted PVT performance. Individuals' mean EDA and heart rate also predicted their PVT performance. For EDA, inter-subject correlations explained more variance of PVT performance than individuals' mean EDA.

Discussion

Together, these findings confirm the association between physiological synchrony and attention. Physiological synchrony in EDA does not only capture the attentional processing during the time that it is determined, but also proves valuable for capturing more general changes in the attentional state of monitored individuals.

Functional MRI responses to naturalistic stimuli are increasingly typical across early childhood

While findings show that throughout development, there are child- and age-specific patterns of brain functioning, there is also evidence for significantly greater inter-individual response variability in young children relative to adults. It is currently unclear whether this increase in functional "typicality" (i.e., inter-individual similarity) is a developmental process that occurs across early childhood, and what changes in BOLD response may be driving changes in typicality. We collected fMRI data from 81 typically developing 4-8-year-old children during passive viewing of age-appropriate television clips and asked whether there is increasing typicality of brain response across this age range. We found that the "increasing typicality" hypothesis was supported across many regions engaged by passive viewing. Post hoc analyses showed that in a priori ROIs related to language and face processing, the strength of the group-average shared component of activity increased with age, with no concomitant decline in residual signal or change in spatial extent or variability. Together, this suggests that increasing inter-individual similarity of functional responses to audiovisual stimuli is an important feature of early childhood functional brain development.

EEG and peripheral markers of viewer ratings: a study of short films

Introduction

Cinema is an important part of modern culture, influencing millions of viewers. Research suggested many models for the prediction of film success, one of them being the use of neuroscientific tools. The aim of our study was to find physiological markers of viewer perception and correlate them to short film ratings given by our subjects. Short films are used as a test case for directors and screenwriters and can be created to raise funding for future projects; however, they have not been studied properly with physiological methods.

Methods

We recorded electroencephalography (18 sensors), facial electromyography (corrugator supercilii and zygomaticus major), photoplethysmography, and skin conductance in 21 participants while watching and evaluating 8 short films (4 dramas and 4 comedies). Also, we used machine learning (CatBoost, SVR) to predict the exact rating of each film (from 1 to 10), based on all physiological indicators. In addition, we classified each film as low or high rated by our subjects (with Logistic Regression, KNN, decision tree, CatBoost, and SVC).

Results

The results showed that ratings did not differ between genres. Corrugator supercilii activity ("frowning" muscle) was larger when watching dramas; whereas zygomaticus major ("smiling" muscle) activity was larger during the watching of comedies. Of all somatic and vegetative markers, only zygomaticus major activity, PNN50, SD1/SD2 (heart rate variability parameters) positively correlated to the film ratings. The EEG engagement indices, beta/(alpha+theta) and beta/alpha correlated positively with the film ratings in the majority of sensors. Arousal (beta_F3 + beta_F4)/(alpha_F3 + alpha_F4), and valence (alpha_F4/beta_F4) - (alpha_F3/beta_F3) indices also correlated positively to film ratings. When we attempted to predict exact ratings, MAPE was 0.55. As for the binary classification, logistic regression yielded the best values (area under the ROC curve = 0.62) than other methods (0.51-0.60).

Discussion

Overall, we revealed EEG and peripheral markers, which reflect viewer ratings and can predict them to a certain extent. In general, high film ratings can reflect a fusion of high arousal and different valence, positive valence being more important. These findings broaden our knowledge about the physiological basis of viewer perception and can be potentially used at the stage of film production.

Semantic novelty modulates neural responses to visual change across the human brain

Our continuous visual experience in daily life is dominated by change. Previous research has focused on visual change due to stimulus motion, eye movements or unfolding events, but not their combined impact across the brain, or their interactions with semantic novelty. We investigate the neural responses to these sources of novelty during film viewing. We analyzed intracranial recordings in humans across 6328 electrodes from 23 individuals. Responses associated with saccades and film cuts were dominant across the entire brain. Film cuts at semantic event boundaries were particularly effective in the temporal and medial temporal lobe. Saccades to visual targets with high visual novelty were also associated with strong neural responses. Specific locations in higher-order association areas showed selectivity to either high or low-novelty saccades. We conclude that neural activity associated with film cuts and eye movements is widespread across the brain and is modulated by semantic novelty.

Audience immersion: validating attentional and physiological measures against self-report

When an audience member becomes immersed, their attention shifts towards the media and story, and they allocate cognitive resources to represent events and characters. Here, we investigate whether it is possible to measure immersion using continuous behavioural and physiological measures. Using television and film clips, we validated dual-task reaction times, heart rate, and skin conductance against self-reported narrative engagement. We find that reaction times to a secondary task were strongly positively correlated with self-reported immersion: slower reaction times were indicative of greater immersion, particularly emotional engagement. Synchrony in heart rate across participants was associated with self-reported attentional and emotional engagement with the story, although we found no such relationship with skin conductance. These results establish both dual-task reaction times and heart rate as candidate measures for the real-time, continuous, assessment of audience immersion.

Neural mechanisms of expert persuasion on willingness to pay for sugar

Introduction: Sugar consumption is associated with many negative health consequences. It is, therefore, important to understand what can effectively influence individuals to consume less sugar. We recently showed that a healthy eating call by a health expert can significantly decrease the willingness to pay (WTP) for sugar-containing food. Here, we investigate which aspects of neural responses to the same healthy eating call can predict the efficacy of expert persuasion.

Methods: Forty-five healthy participants performed two blocks of a bidding task, in which they had to bid on sugar-containing, sugar-free and non-edible products, while their electroencephalography (EEG) was recorded. In between the two blocks, they listened to a healthy eating call by a nutritionist emphasizing the risks of sugar consumption.

Results: We found that after listening to the healthy eating call, participants significantly decreased their WTP for sugar-containing products. Moreover, a higher intersubject correlation of EEG (a measure of engagement) during listening to the healthy eating call resulted in a larger decrease in WTP for sugar-containing food. Whether or not a participant’s valuation of a product was highly influenced by the healthy eating call could also be predicted by spatiotemporal patterns of EEG responses to the healthy eating call, using a machine learning classification model. Finally, the healthy eating call increased the amplitude of the P300 component of the visual event-related potential in response to sugar-containing food.

Disussion: Overall, our results shed light on the neural basis of expert persuasion and demonstrate that EEG is a powerful tool to design and assess health-related advertisements before they are released to the public.

Robustness of Physiological Synchrony in Wearable Electrodermal Activity and Heart Rate as a Measure of Attentional Engagement to Movie Clips

Individuals that pay attention to narrative stimuli show synchronized heart rate (HR) and electrodermal activity (EDA) responses. The degree to which this physiological synchrony occurs is related to attentional engagement. Factors that can influence attention, such as instructions, salience of the narrative stimulus and characteristics of the individual, affect physiological synchrony. The demonstrability of synchrony depends on the amount of data used in the analysis. We investigated how demonstrability of physiological synchrony varies with varying group size and stimulus duration. Thirty participants watched six 10 min movie clips while their HR and EDA were monitored using wearable sensors (Movisens EdaMove 4 and Wahoo Tickr, respectively). We calculated inter-subject correlations as a measure of synchrony. Group size and stimulus duration were varied by using data from subsets of the participants and movie clips in the analysis. We found that for HR, higher synchrony correlated significantly with the number of answers correct for questions about the movie, confirming that physiological synchrony is associated with attention. For both HR and EDA, with increasing amounts of data used, the percentage of participants with significant synchrony increased. Importantly, we found that it did not matter how the amount of data was increased. Increasing the group size or increasing the stimulus duration led to the same results. Initial comparisons with results from other studies suggest that our results do not only apply to our specific set of stimuli and participants. All in all, the current work can act as a guideline for future research, indicating the amount of data minimally needed for robust analysis of synchrony based on inter-subject correlations.

Developmental differences in the perception of naturalistic human movements

Introduction

It is widely believed that we are more attentive towards moving versus static stimuli. However, the neural correlates underlying the perception of human movements have not been extensively investigated in ecologically valid settings, nor has the developmental aspect of this phenomenon. Here, we set forth to investigate how human limb movements displayed in naturalistic videos influence the attentional engagement of children and young adults.

Methods

Thirty-nine healthy participants (4-26 years old) were presented with naturalistic videos featuring human goal-directed movements, while neural activity was recorded using electroencephalography (EEG). Video scenes were automatically annotated as containing arm, leg or no movement, using a machine learning algorithm. The viewers' attentional engagement was quantified by the intersubject correlation of EEG responses evoked by the videos.

Results

Our results demonstrate that scenes featuring limb movements, especially simultaneous arm and leg movements, elicit higher attentional engagement than scenes with no limb movement. Interestingly, this effect was found to diminish with age.

Discussion

Overall, our findings extend previous work on the perception of human motion by implementing naturalistic stimuli in the experimental design and extend the list of factors influencing the viewer's engagement exerted by naturalistic videos.

Bringing social interaction at the core of organizational neuroscience

Organizations are composed of individuals working together for achieving specific goals, and interpersonal dynamics do exert a strong influence on workplace behaviour. Nevertheless, the dual and multiple perspective of interactions has been scarcely considered by Organizational Neuroscience (ON), the emerging field of study that aims at incorporating findings from cognitive and brain sciences into the investigation of organizational behaviour. This perspective article aims to highlight the potential benefits of adopting experimental settings involving two or more participants (the so-called "second person" approach) for studying the neural bases of organizational behaviour. Specifically, we stress the idea that moving beyond the individual perspective and capturing the dynamical relationships occurring within dyads or groups (e.g., leaders and followers, salespersons and clients, teams) might bring novel insights into the rising field of ON. In addition, designing research paradigms that reliably recreate real work and life situations might increase the generalizability and ecological validity of its results. We start with a brief overview of the current state of ON research and we continue by describing the second-person approach to social neuroscience. In the last paragraph, we try and outline how this approach could be extended to ON. To this end, we focus on leadership, group processes and emotional contagion as potential targets of interpersonal ON research.

Brain-heart interactions in the neurobiology of consciousness

Recent experimental evidence on patients with disorders of consciousness revealed that observing brain-heart interactions helps to detect residual consciousness, even in patients with absence of behavioral signs of consciousness. Those findings support hypotheses suggesting that visceral activity is involved in the neurobiology of consciousness, and sum to the existing evidence in healthy participants in which the neural responses to heartbeats reveal perceptual and self-consciousness. More evidence obtained through mathematical modeling of physiological dynamics revealed that emotion processing is prompted by an initial modulation from ascending vagal inputs to the brain, followed by sustained bidirectional brain-heart interactions. Those findings support long-lasting hypotheses on the causal role of bodily activity in emotions, feelings, and potentially consciousness. In this paper, the theoretical landscape on the potential role of heartbeats in cognition and consciousness is reviewed, as well as the experimental evidence supporting these hypotheses. I advocate for methodological developments on the estimation of brain-heart interactions to uncover the role of cardiac inputs in the origin, levels, and contents of consciousness. The ongoing evidence depicts interactions further than the cortical responses evoked by each heartbeat, suggesting the potential presence of non-linear, complex, and bidirectional communication between brain and heartbeat dynamics. Further developments on methodologies to analyze brain-heart interactions may contribute to a better understanding of the physiological dynamics involved in homeostatic-allostatic control, cognitive functions, and consciousness.