The phi complex as a neuromarker of human social coordination

Unifying Large- and Small-Scale Theories of Coordination

Coordination is a ubiquitous feature of all living things. It occurs by virtue of informational coupling among component parts and processes and can be quite specific (as when cells in the brain resonate to signals in the environment) or nonspecific (as when simple diffusion creates a source–sink dynamic for gene networks). Existing theoretical models of coordination—from bacteria to brains to social groups—typically focus on systems with very large numbers of elements (N→∞) or systems with only a few elements coupled together (typically N = 2). Though sharing a common inspiration in Nature’s propensity to generate dynamic patterns, both approaches have proceeded largely independent of each other. Ideally, one would like a theory that applies to phenomena observed on all scales. Recent experimental research by Mengsen Zhang and colleagues on intermediate-sized ensembles (in between the few and the many) proves to be the key to uniting large- and small-scale theories of coordination. Disorder–order transitions, multistability, order–order phase transitions, and especially metastability are shown to figure prominently on multiple levels of description, suggestive of a basic Coordination Dynamics that operates on all scales. This unified coordination dynamics turns out to be a marriage of two well-known models of large- and small-scale coordination: the former based on statistical mechanics (Kuramoto) and the latter based on the concepts of Synergetics and nonlinear dynamics (extended Haken–Kelso–Bunz or HKB). We show that models of the many and the few, previously quite unconnected, are thereby unified in a single formulation. The research has led to novel topological methods to handle the higher-dimensional dynamics of coordination in complex systems and has implications not only for understanding coordination but also for the design of (biorhythm inspired) computers.

The Interpersonal Neurobiology of Intersubjectivity

In 1975, Colwyn Trevarthen first presented his groundbreaking explorations into the early origins of human intersubjectivity. His influential model dictates that, during intimate and playful spontaneous face-to-face protoconversations, the emotions of both the 2-3-month-old infant and mother are nonverbally communicated, perceived, mutually regulated, and intersubjectively shared. This primordial basic interpersonal interaction is expressed in synchronized rhythmic-turn-taking transactions that promote the intercoordination and awareness of positive brain states in both. In this work, I offer an interpersonal neurobiological model of Trevarthen's intersubjective protoconversations as rapid, reciprocal, bidirectional visual-facial, auditory-prosodic, and tactile-gestural right brain-to-right brain implicit nonverbal communications between the psychobiologically attuned mother and the developing infant. These co-constructed positive emotional interactions facilitate the experience-dependent maturation of the infant's right brain, which is in an early critical period of growth. I then address the central role of interpersonal synchrony in intersubjectivity, expressed in a mutual alignment or coupling between the minds and bodies of the mother and infant in face-to-face protoconversations, as well as how these right brain-to-right brain emotional transmissions generate bioenergetic positively charged interbrain synchrony within the dyad. Following this, I offer recent brain laterality research on the essential functions of the right temporoparietal junction, a central node of the social brain, in face-to-face nonverbal communications. In the next section, I describe the ongoing development of the protoconversation over the 1st year and beyond, and the co-creation of a fundamental energy-dependent, growth-promoting social emotional matrix that facilitates the emergence of the highly adaptive human functions of mutual play and mutual love. In the final section, I discuss the clinical applications of this interpersonal neurobiological model of intersubjectivity, which has a long history in the psychotherapy literature. Toward that end, I offer very recent paradigm-shifting hyperscanning research that simultaneously measures both the patient and therapist during a psychotherapeutic interaction. Using the Trevarthen's two-person intersubjective model, this research demonstrates changes in both brains of the therapeutic dyad and the critical role of nonverbal communications in an emotionally-focused psychotherapy session. These studies specifically document interbrain synchronization between the right temporoparietal junction of the patient and the right temporoparietal junction of the clinician, a right brain-to-right brain nonverbal communication system in the co-constructed therapeutic alliance. Lastly, I discuss the relationship between the affect communicating functions of the intersubjective motivational system and the affect regulating functions of the attachment motivational system.

Association of visual motor processing and social cognition in schizophrenia

Patients with schizophrenia have difficulties in social cognitive domains including emotion recognition and mentalization, and in sensorimotor processing and learning. The relationship between social cognitive deficits and sensorimotor function in patients with schizophrenia remains largely unexplored. With the hypothesis that impaired visual motor processing may decelerate information processing and subsequently affects various domains of social cognition, we examined the association of nonverbal emotion recognition, mentalization, and visual motor processing in schizophrenia. The study examined mentalization using the verbal subset of the Chinese version of Theory of Mind (CToM) Task, an equivalent task of the Faux Pas Test; emotion recognition using the Diagnostic Analysis of Nonverbal Accuracy 2-Taiwan version (DANVA-2-TW), and visual motor processing using a joystick tracking task controlled for basic motor function in 34 individuals with chronic schizophrenia in the community and 42 healthy controls. Patients with schizophrenia had significantly worse performance than healthy controls in social cognition, including facial, prosodic emotion recognition, and mentalization. Visual motor processing was also significantly worse in patients with schizophrenia. Only in patients with schizophrenia, both emotion recognition (mainly in prosodic modality, happy, and sad emotions) and mentalization were positively associated with their learning capacity of visual motor processing. These findings suggest a prospective role of sensorimotor function in their social cognitive deficits. Despite that the underlying neural mechanism needs further research, our findings may provide a new direction for restoration of social cognitive function in schizophrenia by enhancing visual motor processing ability.

Genomic Intelligence as Über Bio-Cybersecurity: The Gödel Sentence in Immuno-Cognitive Systems

This paper gives formal foundations and evidence from gene science in the post Barbara McClintock era that the Gödel Sentence, far from being an esoteric construction in mathematical logic, is ubiquitous in genomic intelligence that evolved with multi-cellular life. Conditions uniquely found in the Adaptive Immune System (AIS) and Mirror Neuron System (MNS), termed the genomic immuno-cognitive system, coincide with three building blocks in computation theory of Gödel, Turing and Post (G-T-P). (i) Biotic elements have unique digital identifiers with gene codes executing 3D self-assembly for morphology and regulation of the organism using the recursive operation of Self-Ref (Self-Reference) with the other being a self-referential projection of self. (ii) A parallel offline simulation meta/mirror environment in 1–1 relation to online machine executions of self-codes gives G-T-P Self-Rep (Self-Representation). (iii) This permits a digital biotic entity to self-report that it is under attack by a biotic malware or non-self antigen in the format of the Gödel sentence, resulting in the “smarts” for contextual novelty production. The proposed unitary G-T-P recursive machinery in AIS and in MNS for social cognition yields a new explanation that the Interferon Gamma factor, known for friend-foe identification in AIS, is also integral to social behaviors. New G-T-P bio-informatics of AIS and novel anti-body production is given with interesting testable implications for COVID-19 pathology.

Neural substrates for sharing intention in action during face-to-face imitation

Face-to-face imitation is a unique social interaction wherein a shared action is executed based on the feedback of the partner. Imitation by the partner is the feedback to the imitatee's action, resulting in sharing actions. The neural mechanisms of the shared representation of action during face-to-face imitation, the core of inter-subjectivity, are not well-known. Here, based on the predictive coding account, we hypothesized that the pair-specific forward internal model is the shared representation of action which is represented by the inter-individual synchronization of some portion of the mirror neuron system. Hyperscanning functional magnetic resonance imaging was conducted during face-to-face interaction in 16 pairs of participants who completed an immediate imitation task of facial expressions. Paired participants were alternately assigned to either an imitator or an imitatee who was prompted to express a happy, sad, or non-emotional face. While neural activation elicited by imitating and being imitated were distinct with little overlap, on-line imitative interaction enhanced inter-brain synchronization in the right inferior parietal lobule that correlated with the similarity in facial movement kinematic profile. This finding indicates a critical role of the right inferior parietal lobule in sharing representation of action as a pair-specific forward internal model through imitative interaction.

Neural correlates underpinning synchronized movement of double scull rowers

We investigated the neural correlates underpinning synchronized movement in rowers using a neural index for social interaction termed the phi complex. Phi 1 and phi 2 indicate the enhancement and reduction of mirror neuron activation, respectively. We hypothesized that in a leader-follower relation, followers would exhibit a larger phi 2 power than leaders due to enhanced mirror neuron activation by the followers to accurately mimic their partner's movements. We also expected that brain activation underlying social interaction would be enhanced during synchronization. Although phi 2 was not modulated by role (leader vs. follower) or condition (usual-pair vs. unusual-pair), the statistical analysis suggested the relationship between the magnitude of phi 2 and empathetic ability in followers in the usual-pair condition. Given that the activation of the mirror neuron system underlies empathic ability, it is plausible that the participants used the mirror neuron system to follow the movement of a usual partner. In other words, the leader in the synchronization did not need to use the mirror neuron system, which was consistent with the result of a larger phi 1 for leading than following the movement. These results suggest that the neural correlates of empathy may be used to synchronize with partners as the follower.

HyPyP: a Hyperscanning Python Pipeline for inter-brain connectivity analysis

The bulk of social neuroscience takes a 'stimulus-brain' approach, typically comparing brain responses to different types of social stimuli, but most of the time in the absence of direct social interaction. Over the last two decades, a growing number of researchers have adopted a 'brain-to-brain' approach, exploring similarities between brain patterns across participants as a novel way to gain insight into the social brain. This methodological shift has facilitated the introduction of naturalistic social stimuli into the study design (e.g. movies) and, crucially, has spurred the development of new tools to directly study social interaction, both in controlled experimental settings and in more ecologically valid environments. Specifically, 'hyperscanning' setups, which allow the simultaneous recording of brain activity from two or more individuals during social tasks, has gained popularity in recent years. However, currently, there is no agreed-upon approach to carry out such 'inter-brain connectivity analysis', resulting in a scattered landscape of analysis techniques. To accommodate a growing demand to standardize analysis approaches in this fast-growing research field, we have developed Hyperscanning Python Pipeline, a comprehensive and easy open-source software package that allows (social) neuroscientists to carry-out and to interpret inter-brain connectivity analyses.

The motor-related brain activity that supports joint action: A review

Recent years have seen a rapid increase in research investigating the motor-related brain activity that supports joint action. This research has employed a variety of joint action tasks and an array of neuroimaging techniques, including fMRI, fNIRS, EEG, and TMS. In this review, we provide an overview of this research to delineate what is known about the motor-related brain activity that contributes to joint action and to highlight key questions for future research. Taken together, the surveyed research supports three major conclusions. First, the mere presence of a joint action context is sufficient to modulate motor activity elicited by observing others' actions. Second, joint action is supported by dissociable motor activity associated with a person's own actions, their partner's actions, and the joint action, and by between-brain coupling of motor-related oscillatory activity. Third, the structure of a joint action modulates the motor activity involved: Unique motor activity is associated with performing joint actions comprised of complementary actions and with holding the roles of leader and follower within a joint action. We conclude the review by highlighting overarching themes and key questions for future research.

A Review of Hyperscanning and Its Use in Virtual Environments

Hyperscanning is a technique which simultaneously records the neural activity of two or more people. This is done using one of several neuroimaging methods, such as electroencephalography (EEG), functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The use of hyperscanning has seen a dramatic rise in recent years to monitor social interactions between two or more people. Similarly, there has been an increase in the use of virtual reality (VR) for collaboration, and an increase in the frequency of social interactions being carried out in virtual environments (VE). In light of this, it is important to understand how interactions function within VEs, and how they can be enhanced to improve their quality in a VE. In this paper, we present some of the work that has been undertaken in the field of social neuroscience, with a special emphasis on hyperscanning. We also cover the literature detailing the work that has been carried out in the human–computer interaction domain that addresses remote collaboration. Finally, we present a way forward where these two research domains can be combined to explore how monitoring the neural activity of a group of participants in VE could enhance collaboration among them.

Human attachments shape interbrain synchrony toward efficient performance of social goals

The human brain has undergone massive expansion across primate evolution through life amidst multi-layered social attachments; within families, among friends, and between clan members and this enabled humans to coordinate their brains with those of others toward the execution of complex social goals.  We examined how human attachments facilitate efficient, resource-sensitive performance of social goals by balancing neural and behavioral synchrony. Using hyperscanning EEG, we collected neural data from male-female pairs in three groups (N=158, 79 pairs); long-term couples, best friends, and unfamiliar group members, during two ecologically-valid naturalistic tasks; motor coordination and empathy giving.  Across groups and tasks, neural synchrony was supported by behavior coordination and orchestrated multiple neural rhythms.  In the goal-directed motor task, interbrain synchrony implicated beta and gamma rhythms localized to sensorimotor areas. Couples showed the highest neural synchrony combined with greatest behavioral synchrony and such brain-behavior linkage resulted in speedy performance, conserving energy in the long run.  The socially-oriented empathy task triggered neural synchrony in widely-distributed sensorimotor and bilateral temporal regions, integrated alpha, beta, and gamma rhythms, and implicated brain-behavior complementarity; couples displayed the highest behavioral synchrony combined with lowest neural synchrony toward greatest felt support while strangers exhibited the opposite pattern.  Findings suggest that human attachments provide a familiar backdrop of temporal regularities, required for the brain's allostatic function, and interbrain and behavioral synchrony are sculpted by familiarity and closeness toward resource-sensitive performance of survival-related social goals, toiled by two.

Moving in unison after perceptual interruption

Humans interact in groups through various perception and action channels. The continuity of interaction despite a transient loss of perceptual contact often exists and contributes to goal achievement. Here, we study the dynamics of this continuity, in two experiments involving groups of participants ([Formula: see text]) synchronizing their movements in space and in time. We show that behavioural unison can be maintained after perceptual contact has been lost, for about 7s. Agent similarity and spatial configuration in the group modulated synchronization performance, differently so when perceptual interaction was present or when it was memorized. Modelling these data through a network of oscillators enabled us to clarify the double origin of this memory effect, of individual and social nature. These results shed new light into why humans continue to move in unison after perceptual interruption, and are consequential for a wide variety of applications at work, in art and in sport.

Walking and talking independently predict interpersonal impressions

When walking alongside someone, you may feel that your legs move in synchrony with theirs. Recent studies have shown that walk-in-synch behaviour observed in natural settings occurs at a rate significantly greater than would be expected by chance, and that the amount of this synchrony is related to interpersonal impressions. However, in such natural settings, the existence of verbal conversations between paired walkers should affect the interpersonal impressions and the effect is not distinguished from the effect of walk-in-synch on the impressions so far. In the current study, we used the analysis of conversation and path analysis to discriminate these two effects (i.e., the effects of synchronization of walking and conversation on interpersonal impressions). Analysis of conversation during the walk revealed that the amount of utterance overlap and the number of turn-takings between two walkers as well as the synchronization of steps predicted their positive interpersonal impression, while synchronization of steps and these two conversational indices were not correlated with each other. We propose that interpersonal synchronization of body movements, such as synchronization of steps itself in paired walking, plays a role in fostering the development of interpersonal relationships.

Brain-to-Brain Neural Synchrony During Social Interactions: A Systematic Review on Hyperscanning Studies

The aim of this study was to conduct a comprehensive review on hyperscanning research (measuring brain activity simultaneously from more than two people interacting) using an explicit systematic method, the preferred reporting items for systematic reviews and meta-analyses (PRISMA). Data were searched from IEEE Xplore, PubMed, Engineering Village, Web of Science and Scopus databases. Inclusion criteria were journal articles written in English from 2000 to 19 June 2019. A total of 126 empirical studies were screened out to address three specific questions regarding the neuroimaging method, the application domain, and the experiment paradigm. Results showed that the most used neuroimaging method with hyperscanning was magnetoencephalography/electroencephalography (MEG/EEG; 47%), and the least used neuroimaging method was hyper-transcranial Alternating Current Stimulation (tACS) (1%). Applications in cognition accounted for almost half the studies (48%), while educational applications accounted for less than 5% of the studies. Applications in decision-making tasks were the second most common (26%), shortly followed by applications in motor synchronization (23%). The findings from this systematic review that were based on documented, transparent and reproducible searches should help build cumulative knowledge and guide future research regarding inter-brain neural synchrony during social interactions, that is, hyperscanning research.

Coordination Dynamics: A Foundation for Understanding Social Behavior

Humans' interactions with each other or with socially competent machines exhibit lawful coordination patterns at multiple levels of description. According to Coordination Dynamics, such laws specify the flow of coordination states produced by functional synergies of elements (e.g., cells, body parts, brain areas, people…) that are temporarily organized as single, coherent units. These coordinative structures or synergies may be mathematically characterized as informationally coupled self-organizing dynamical systems (Coordination Dynamics). In this paper, we start from a simple foundation, an elemental model system for social interactions, whose behavior has been captured in the Haken-Kelso-Bunz (HKB) model. We follow a tried and tested scientific method that tightly interweaves experimental neurobehavioral studies and mathematical models. We use this method to further develop a body of empirical research that advances the theory toward more generalized forms. In concordance with this interdisciplinary spirit, the present paper is written both as an overview of relevant advances and as an introduction to its mathematical underpinnings. We demonstrate HKB's evolution in the context of social coordination along several directions, with its applicability growing to increasingly complex scenarios. In particular, we show that accommodating for symmetry breaking in intrinsic dynamics and coupling, multiscale generalization and adaptation are principal evolutions. We conclude that a general framework for social coordination dynamics is on the horizon, in which models support experiments with hypothesis generation and mechanistic insights.

Brain-to-Brain Coupling in the Gamma-Band as a Marker of Shared Intentionality

Cooperation and competition are two ways of social interaction keys to life in society. Recent EEG-based hyperscanning studies reveal that cooperative and competitive interactions induce an increase in interbrain coupling. However, whether this interbrain coupling effect is just a reflection of inter-subject motor coordination or can also signal the type of social interaction is unknown. Here, we show that behavioral coordination and social interaction type can be distinguished according to the frequency of oscillation in which the brains are coupled. We use EEG to simultaneously measure the brain activity of pairs of subjects, while they were performing a visual cue-target task in a cooperative and competitive manner. Behavioral responses were quasi-simultaneous between subject pairs for both competitive and cooperative conditions, with faster average response times for the competitive condition. Concerning brain activity, we found increased interbrain coupling in theta band (3–7 Hz) during cooperation and competition, with stronger coupling during competitive interactions. This increase of interbrain theta coupling correlated with a decrease in reaction times of the dyads. Interestingly, we also found an increase in brain-to-brain coupling in gamma band (38–42 Hz) only during cooperative interactions. Unlike the theta coupling effect, the gamma interbrain coupling did not correlate with dyads’ reaction times. Taken together, these results suggest that theta interbrain coupling could be linked to motor coordination processes common to cooperative and competitive interactions, while gamma brain-to-brain coupling emerges as an electrophysiological marker of shared intentionality during cooperative interactions.

Shall We Play the Same? Pedagogical Perspectives on Infants’ and Children’s Imitation of Musical Gestures

Imitation, both gestural and vocal, has been acknowledged to be at the origin of human communication (Donald, 1991). Music is often considered to be the first means of communication of emotion via both vocal and gestural synchronization (Malloch, 1999; Malloch and Trevarthen, 2009). Instrumental music is part of the human heritage for more than 35,000 years before our era (Aimé et al., 2020). However, very little is known about the acquisition of gestures that produce sounds (i.e., musical gestures) and their role in the development of music and musicality. In the present paper, we propose that studying early synchronous imitation of musical gestures is essential both for investigating the development of the early action–perception system and for outlining early music interventions during infancy. We designed double musical objects which can be used in preschool music education for prompting synchronic imitation of musical gestures between adult and child, and between dyads of infants. We conclude by proposing a novel pedagogical perspective in music education for the early years which links the privileged orientation of infants and children towards sound discoveries with the development of perception-action coupling via imitation of musical gestures.

Behavioral and EEG Measures Show no Amplifying Effects of Shared Attention on Attention or Memory

Shared attention experiments examine the potential differences in function or behavior when stimuli are experienced alone or in the presence of others, and when simultaneous attention of the participants to the same stimulus or set is involved. Previous work has found enhanced reactions to emotional stimuli in social situations, yet these changes might represent enhanced communicative or motivational purposes. This study examines whether viewing emotional stimuli in the presence of another person influences attention to or memory for the stimulus. Participants passively viewed emotionally-valenced stimuli while completing another task (counting flowers). Each participant performed this task both alone and in a shared attention condition (simultaneously with another person in the same room) while EEG signals were measured. Recognition of the emotional pictures was later measured. A significant shared attention behavioral effect was found in the attention task but not in the recognition task. Compared to event-related potential responses for neutral pictures, we found higher P3b response for task relevant stimuli (flowers), and higher Late Positive Potential (LPP) responses for emotional stimuli. However, no main effect was found for shared attention between presence conditions. To conclude, shared attention may therefore have a more limited effect on cognitive processes than previously suggested.

Physiological and Behavioral Synchrony Predict Group Cohesion and Performance

Interpersonal synchrony contributes to social functioning in dyads, but it remains unknown how synchrony shapes group experiences and performance. To this end, we designed a novel group drumming task in which participants matched their drumming to either predictable or unpredictable tempos. Fifty-one three-person groups were randomly assigned to one of two conditions: synchronized or asynchronized drumming. Outcome measures included electrocardiograms and self-reports of group cohesion and synchrony. The drumming task elicited an increase in physiological synchrony between group members (specifically their hearts' interbeat intervals). We also found that physiological synchronization and behavioral synchronization predicted individuals' experience of group cohesion. Physiological synchrony also predicted performance in a subsequent group task that involved freely drumming together. The findings suggest that the behavioral and physiological consequences of synchronization contribute to the formation of group bonds and coordination. They also confirm that insights from translational social neuroscience can inform our knowledge of the development of cohesive and efficacious groups.

The Human Dynamic Clamp Reveals the Fronto-Parietal Network Linking Real-Time Social Coordination and Cognition

How does the brain allow us to interact with others? Social neuroscience has already provided some answers to these questions but has tended to treat high-level, cognitive interpretations of social behavior separately from the sensorimotor mechanisms upon which they rely. The goal here is to identify the underlying neural processes and mechanisms linking sensorimotor coordination and intention attribution. We combine the human dynamic clamp, a novel paradigm for studyingrealistic social behavior, with high-resolution electroencephalography. The collection of humanness and intention attribution reports, kinematics, and neural data affords an opportunity to relate brain activity to the ongoing social behavior. Behavioral results demonstrate that sensorimotor coordination influences the judgments of cooperativeness and humanness. Analysis of brain dynamics reveals two distinct networks related to the integration of visuo-motor information from self and other which overlap over the right parietal region. Furthermore, judgment of humanness and cooperation of others modulate the functional connectivity between this right parietal hub and the prefrontal cortex. These results reveal how distributed neural dynamics integrates information from "low-level" sensorimotor mechanisms and "high-level" social cognition to support the realistic social behaviors that play out in real time during interactive scenarios.

The Presence of Another Person Influences Oscillatory Cortical Dynamics During Dual Brain EEG Recording

Humans are innately social creatures and the social environment strongly influences brain development. As such, the human brain is primed for and sensitive to social information even in the absence of explicit task or instruction. In this study, we examined the influence of different levels of interpersonal proximity on resting state brain activity and its association with social cognition. We measured EEG in pairs of 13 typically developing (TD) adults seated in separate rooms, in the same room back-to-back, and in the same room facing each other. Interpersonal proximity modulated broadband EEG power from 4-55 Hz and individual differences in self-reported social cognition modulated these effects in the beta and gamma frequency bands. These findings provide novel insight into the influence of social environment on brain activity and its association with social cognition through dual-brain EEG recording and demonstrate the importance of using interactive methods to study the human brain.

Using second-person neuroscience to elucidate the mechanisms of social interaction

Although a large proportion of our lives are spent participating in social interactions, the investigation of the neural mechanisms supporting these interactions has largely been restricted to situations of social observation - that is, situations in which an individual observes a social stimulus without opportunity for interaction. In recent years, efforts have been made to develop a truly social, or 'second-person', neuroscientific approach to these investigations in which neural processes are examined within the context of a real-time reciprocal social interaction. These developments have helped to elucidate the behavioural and neural mechanisms of social interactions; however, further theoretical and methodological innovations are still needed. Findings to date suggest that the neural mechanisms supporting social interaction differ from those involved in social observation and highlight a role of the so-called 'mentalizing network' as important in this distinction. Taking social interaction seriously may also be particularly important for the advancement of the neuroscientific study of different psychiatric conditions.

Hyperscanning: A Valid Method to Study Neural Inter-brain Underpinnings of Social Interaction

Social interactions are a crucial part of human life. Understanding the neural underpinnings of social interactions is a challenging task that the hyperscanning method has been trying to tackle over the last two decades. Here, we review the existing literature and evaluate the current state of the hyperscanning method. We review the type of methods (fMRI, M/EEG, and fNIRS) that are used to measure brain activity from more than one participant simultaneously and weigh their pros and cons for hyperscanning. Further, we discuss different types of analyses that are used to estimate brain networks and synchronization. Lastly, we present results of hyperscanning studies in the context of different cognitive functions and their relations to social interactions. All in all, we aim to comprehensively present methods, analyses, and results from the last 20 years of hyperscanning research.

Midline frontal and occipito-temporal activity during error monitoring in dyadic motor interactions

Discrepancies between sensory predictions and action outcome are at the base of error coding. However, these phenomena have mainly been studied focussing on individual performance. Here, we explored EEG responses to motor prediction errors during a human-avatar interaction and show that Theta/Alpha activity of the frontal error-monitoring system works in phase with activity of the occipito-temporal node of the action observation network. Our motor interaction paradigm required healthy individuals to synchronize their reach-to-grasp movements with those of a virtual partner in conditions that did (Interactive) or did not require (Cued) movement prediction and adaptation to the partner's actions. Crucially, in 30% of the trials the virtual partner suddenly and unpredictably changed its movement trajectory thereby violating the human participant's expectation. These changes elicited error-related neuromarkers (ERN/Pe - Theta/Alpha modulations) over fronto-central electrodes during the Interactive condition. Source localization and connectivity analyses showed that the frontal Theta/Alpha activity induced by violations of the expected interactive movements was in phase with occipito-temporal Theta/Alpha activity. These results expand current knowledge about the neural correlates of on-line interpersonal motor interactions linking the frontal error-monitoring system to visual, body motion-related, responses.

Toward a cooperation index based on EEG-workload causality: preliminary findings on aerospace-like tasks

According to Human-System Integration analyses, cooperation between humans is one of the most relevant factors in many of today's human activities: do not take it into account in models of working environments is highly farfetched. Although the Human Factor aspects have obtained much benefit from the use of neurophysiological signals to estimate human-machine interaction, very few are the indications about neurophysiological analysis of human cooperation deviating from typical laboratory tasks. Among these, some evidence showed that there is a relationship between the mental workload experienced by the subjects cooperating and some characteristics of the brain network obtained through multi-subjects connectivity analysis. Accordingly, this work aimed to identify common dynamics in time series that describe the EEG-based mental workload of cooperating subjects and to exploit this information to create an index of cooperation. In order to answer the question whether a causality between the workload values of the two subjects can be in some way discerned and related to the cooperation required by the task, Granger's causality test has been performed. This method was applied to two different tasks simulating features of the aerospace domain. The results showed that the causality test was statistically significant for the most collaborating couple. In addition, causality values are modulated by the presence of real couples compared to fake couples. The extension of the experimental sample could open up the possibility for the development of an objective and neurophysiological signals-based cooperation index.

Dissecting social interaction: dual-fMRI reveals patterns of interpersonal brain-behavior relationships that dissociate among dimensions of social exchange

During social interactions, each individual’s actions are simultaneously a consequence of and an antecedent to their interaction partner’s behavior. Capturing online the brain processes underlying such mutual dependency requires simultaneous measurements of all interactants’ brains during real-world exchange (‘hyperscanning’). This demands a precise characterization of the type of interaction under investigation, however, and analytical techniques capable of capturing interpersonal dependencies. We adapted an interactive task capable of dissociating between two dimensions of interdependent social exchange: goal structure (cooperation vs competition) and interaction structure [concurrent (CN) vs turn-based]. Performing dual-functional magnetic resonance imaging hyperscanning on pairs of individuals interacting on this task, and modeling brain responses in both interactants as systematic reactions to their partner’s behavior, we investigated interpersonal brain-behavior dependencies (iBBDs) during each dimension. This revealed patterns of iBBDs that differentiated among exchanges; in players supporting the actions of another, greater brain responses to the co-player’s actions were expressed in regions implicated in social cognition, such as the medial prefrontal cortex, precuneus and temporal cortices. Stronger iBBD during CN competitive exchanges was observed in brain systems involved in movement planning and updating, however, such as the supplementary motor area. This demonstrates the potential for hyperscanning to elucidate neural processes underlying different forms of social exchange.

Spectral dissociation of lateralized brain rhythms

Using high resolution spectral methods to uncover neuromarkers of social, cognitive and behavioral function, we have found that hemi-lateralized pairs of oscillations such as left and right occipital alpha or left and right rolandic mu dissociate spectrally. That is, they show a shifted frequency distribution, with one member of the pair peaking at a slightly lower frequency than the other. Resorting to the analysis of EEG spatio-spectral patterns, we provide examples of dissociations in the 10Hz frequency band. Occasionally, hemi-lateralized pairs blend into medial aggregates, probably when functional interactions lead to strongly coherent dynamics through frequency-locking or metastability. Our observations support the hypothesis that homologous pairs of neuromarkers have characteristically distinct intrinsic frequencies and coordinate their oscillations into synchronous ensembles only transiently. This property could play a role in the balance of integration and segregation in the brain: spectral separation of the oscillations from homologous cortical areas allows them to function independently under certain circumstances, all the while preserving a potential for stronger interactions supported by structural and functional symmetries. Spectral dissociation (and its methodological corollary: spectral analysis with high frequency resolution) may be harnessed to better track the individual power of each member of a hemi-lateralized pair and their respective time-course, leading to enhanced internal validity and reproducibility of research on neural oscillations. Resulting insights may shed light on the functional interaction between homologous cortices in studies of attention (alpha), e.g. during perceptual and social interaction tasks, and in studies of somatomotor processing (mu), e.g. in bimanual coordination and neuroprosthetics.

Leader-employee emotional "interpersonal tuning".An EEG coherence study

Empirical management research has focused more on the investigation of important interpersonal factors that could be beneficial for a company's well-being, including emotional and empathic engagement between managers and employees. The capacity to understand and mirror others' feelings could result in a mutual adaptation that generates interpersonal tuning (IT). In the present study, we measured IT by applying a hyperscanning approach with simultaneous recording of electroencephalographic (EEG) signals from two participants interacting together. Eleven leaders and 11 employees were recruited and asked to role-play an employee performance review with a rating (R) or no rating (NR) condition. In the NR condition leaders describe by words the employee's proficiency, while in R they provided a quantitative rating. The NR condition emerged as a more engaging situation in terms of empathic responses and mirroring. This difference was detectable from the localization of neurophysiological effects over the frontopolar and frontal brain areas, and the higher synchronization of EEG delta frequency coherence. Behavioral results also revealed an increase of self-perceived emotional tuning, agreement on content, and interpersonal cooperation in the NR condition compared to R condition. These effects were present in both leaders and employees and have several implications for social and company well-being.

More than Meets the Mind's Eye? Preliminary Observations Hint at Heterogeneous Alpha Neuromarkers for Visual Attention

With their salient power distribution and privileged timescale for cognition and behavior, brainwaves within the 10 Hz band are special in human waking electroencephalography (EEG). From the inception of electroencephalographic technology, the contribution of alpha rhythm to attention is well-known: Its amplitude increases when visual attention wanes or visual input is removed. However, alpha is not alone in the 10 Hz frequency band. A number of other 10 Hz neuromarkers have function and topography clearly distinct from alpha. In small pilot studies, an activity that we named xi was found over left centroparietal scalp regions when subjects held their attention to spatially peripheral locations while maintaining their gaze centrally ("looking from the corner of the eyes"). I outline several potential functions for xi as a putative neuromarker of covert attention distinct from alpha. I review methodological aids to test and validate their functional role. They emphasize high spectral resolution, sufficient spatial resolution to provide topographical separation, and an acute attention to dynamics that caters to neuromarkers' transiency.

Simultaneous EEG Acquisition System for Multiple Users: Development and Related Issues

Social interaction is one of humans' most important activities and many efforts have been made to understand the phenomenon. Recently, some investigators have attempted to apply advanced brain signal acquisition systems that allow dynamic brain activities to be measured simultaneously during social interactions. Most studies to date have investigated dyadic interactions, although multilateral interactions are more common in reality. However, it is believed that most studies have focused on such interactions because of methodological limitations, in that it is very difficult to design a well-controlled experiment for multiple users at a reasonable cost. Accordingly, there are few simultaneous acquisition systems for multiple users. In this study, we propose a design framework for an acquisition system that measures EEG data simultaneously in an environment with 10 or more people. Our proposed framework allowed us to acquire EEG data at up to 1 kHz frequency from up to 20 people simultaneously. Details of our acquisition system are described from hardware and software perspectives. In addition, various related issues that arose in the system's development-such as synchronization techniques, system loads, electrodes, and applications-are discussed. In addition, simultaneous visual ERP experiments were conducted with a group of nine people to validate the EEG acquisition framework proposed. We found that our framework worked reasonably well with respect to less than 4 ms delay and average loss rates of 1%. It is expected that this system can be used in various hyperscanning studies, such as those on crowd psychology, large-scale human interactions, and collaborative brain-computer interface, among others.

Musical Role Asymmetries in Piano Duet Performance Influence Alpha-Band Neural Oscillation and Behavioral Synchronization

Recent work in interpersonal coordination has revealed that neural oscillations, occurring spontaneously in the human brain, are modulated during the sensory, motor, and cognitive processes involved in interpersonal interactions. In particular, alpha-band (8-12 Hz) activity, linked to attention in general, is related to coordination dynamics and empathy traits. Researchers have also identified an association between each individual's attentiveness to their co-actor and the relative similarity in the co-actors' roles, influencing their behavioral synchronization patterns. We employed music ensemble performance to evaluate patterns of behavioral and neural activity when roles between co-performers are systematically varied with complete counterbalancing. Specifically, we designed a piano duet task, with three types of co-actor dissimilarity, or asymmetry: (1) musical role (starting vs. joining), (2) musical task similarity (similar vs. dissimilar melodic parts), and (3) performer animacy (human-to-human vs. human-to-non-adaptive computer). We examined how the experience of these asymmetries in four initial musical phrases, alternatingly played by the co-performers, influenced the pianists' performance of a subsequent unison phrase. Electroencephalography was recorded simultaneously from both performers while playing keyboards. We evaluated note-onset timing and alpha modulation around the unison phrase. We also investigated whether each individual's self-reported empathy was related to behavioral and neural activity. Our findings revealed closer behavioral synchronization when pianists played with a human vs. computer partner, likely because the computer was non-adaptive. When performers played with a human partner, or a joining performer played with a computer partner, having a similar vs. dissimilar musical part did not have a significant effect on their alpha modulation immediately prior to unison. However, when starting performers played with a computer partner with a dissimilar vs. similar part there was significantly greater alpha synchronization. In other words, starting players attended less to the computer partner playing a similar accompaniment, operating in a solo-like mode. Moreover, this alpha difference based on melodic similarity was related to a difference in note-onset adaptivity, which was in turn correlated with performer trait empathy. Collectively our results extend previous findings by showing that musical ensemble performance gives rise to a socialized context whose lasting effects encompass attentiveness, perceptual-motor coordination, and empathy.

Hyperscanning of Interactive Juggling: Expertise Influence on Source Level Functional Connectivity

Hyperscanning studies, wherein brain activity is recorded from multiple participants simultaneously, offer an opportunity to investigate interpersonal dynamics during interactive tasks at the neurophysiological level. In this study, we employed a dyadic juggling paradigm and electroencephalography (EEG) hyperscanning to evaluate functional connectivity between EEG sources within and between jugglers’ brains during individual and interactive juggling. We applied graph theoretical measures to identify significant differences in functional connectivity between the individual and interactive juggling conditions. Connectivity was measured in multiple juggler pairs with various skill levels where dyads were either skill-level matched or skill-level unmatched. We observed that global efficiency was reduced during paired juggling for less skilled jugglers and increased for more skilled jugglers. When jugglers were skill-level matched, additional reductions were found in the mean clustering coefficient and small-world topology during interactive juggling. A significant difference in hemispheric brain lateralization was detected between skill-level matched and skill-level unmatched jugglers during interactive juggling: matched jugglers had an increased right hemisphere lateralization while unmatched jugglers had an increased left hemisphere lateralization. These results reveal multiple differences in functional brain networks during individual and interactive juggling and suggest that similarities and disparities in individual skills can impact inter-brain dynamics in the performance and learning of motor tasks.

Connecting empirical phenomena and theoretical models of biological coordination across scales

Coordination in living systems-from cells to people-must be understood at multiple levels of description. Analyses and modelling of empirically observed patterns of biological coordination often focus either on ensemble-level statistics in large-scale systems with many components, or on detailed dynamics in small-scale systems with few components. The two approaches have proceeded largely independent of each other. To bridge this gap between levels and scales, we have recently conducted a human experiment of mid-scale social coordination specifically designed to reveal coordination at multiple levels (ensemble, subgroups and dyads) simultaneously. Based on this experiment, the present work shows that, surprisingly, a single system of equations captures key observations at all relevant levels. It also connects empirically validated models of large- and small-scale biological coordination-the Kuramoto and extended Haken-Kelso-Bunz (HKB) models-and the hallmark phenomena that each is known to capture. For example, it exhibits both multistability and metastability observed in small-scale empirical research (via the second-order coupling and symmetry breaking in extended HKB) and the growth of biological complexity as a function of scale (via the scalability of the Kuramoto model). Only by incorporating both of these features simultaneously can we reproduce the essential coordination behaviour observed in our experiment.

Using second-person neuroscience to elucidate the mechanisms of social interaction

Although a large proportion of our lives are spent participating in social interactions, the investigation of the neural mechanisms supporting these interactions has largely been restricted to situations of social observation - that is, situations in which an individual observes a social stimulus without opportunity for interaction. In recent years, efforts have been made to develop a truly social, or 'second-person', neuroscientific approach to these investigations in which neural processes are examined within the context of a real-time reciprocal social interaction. These developments have helped to elucidate the behavioural and neural mechanisms of social interactions; however, further theoretical and methodological innovations are still needed. Findings to date suggest that the neural mechanisms supporting social interaction differ from those involved in social observation and highlight a role of the so-called 'mentalizing network' as important in this distinction. Taking social interaction seriously may also be particularly important for the advancement of the neuroscientific study of different psychiatric conditions.

Combining Phase Advancement and Period Correction Explains Rushing during Joint Rhythmic Activities

When people engage in rhythmic joint actions, from simple clapping games to elaborate joint music making, they tend to increase their tempo unconsciously. Despite the rich literature on rhythmic performance in humans, the mechanisms underlying joint rushing are still unknown. We propose that joint rushing arises from the concurrent activity of two separate mechanisms. The phase advance mechanism was first proposed in research on synchronously flashing fireflies and chorusing insects. When this mechanism is combined with a human-specific period correction mechanism, the shortened periods of individual intervals are translated into a tempo increase. In three experiments, we investigated whether joint rushing can be reliably observed in a joint synchronization-continuation drumming task. Furthermore, we asked whether perceptual similarities produced by the actions of different individuals modulate the joint rushing effect. The results showed that joint rushing is a robust phenomenon occurring in groups of different sizes. Joint rushing was more pronounced when the action effects produced by different individuals were perceptually similar, supporting the assumption that a phase advance mechanism contributed to rushing. Further control conditions ruled out the alternative hypothesis that rushing during rhythmic interactions can be explained by social facilitation or action mirroring effects.

Correlated Neural Activity across the Brains of Socially Interacting Bats

Social interactions occur between multiple individuals, but what is the detailed relationship between the neural dynamics across their brains? To address this question across timescales and levels of neural activity, we used wireless electrophysiology to simultaneously record from pairs of bats engaged in a wide range of natural social interactions. We found that neural activity was remarkably correlated between their brains over timescales from seconds to hours. The correlation depended on a shared social environment and was most prominent in high frequency local field potentials (>30 Hz), followed by local spiking activity. Furthermore, the degree of neural correlation covaried with the extent of social interactions, and an increase in correlation preceded their initiation. These results show that inter-brain correlation is an inherent feature of natural social interactions, reveal the domain of neural activity where it is most prominent, and provide a foundation for studying its functional role in social behaviors.

Framing a trust game as a power game greatly affects interbrain synchronicity between trustor and trustee

We used dual electroencephalography (EEG) to measure brain activity simultaneously in pairs of trustors and trustees playing a 15-round trust game framed as a "trust game" versus a "power game". Four major findings resulted: first, earnings in each round were higher in the trust than in the power game. Second, in the trust game, reaction time for strategic deliberations was significantly longer for the trustee than the trustor. In the power game, however, the trustee took longer to think about how much money to repay, whereas the trustor took longer to think about how much money to invest. Third, prediction accuracy for the amount exchanged was higher in the trust game than in the power game. Fourth, interbrain synchronicity gauged with the phase-locking value of alpha bands in the brain - especially the frontal and central regions - was higher in the power game than in the trust game. We infer that this last finding reflects elevated mutual strategic deliberation in the power game. These behavioral and neuroscience-based findings give a better understanding of the framing effects of a trust game on the strategic deliberations of both trustor and trustee seeking to attain wealth.

Holistic cognitive and neural processes: a fNIRS-hyperscanning study on interpersonal sensorimotor synchronization

Interpersonal sensorimotor synchronization (interpersonal SMS) is the foundation of complex human social interaction. Previous studies primarily focused on the individual cognitive processes of interpersonal SMS. However, all individuals compose an entire interaction system with emerged holistic properties during interpersonal SMS. Therefore, we proposed the `holistic cognitive and neural processes' of interpersonal SMS and defined quantitative measurements that included Holistic Correction Gain (HCG), Holistic Timekeeper Variance (HTV) and Holistic Motor Variance (HMV) based on linear error correction model and inter-brain couplings obtained by hyperscanning technique. We performed a joint-tapping experiment including bidirectional and unidirectional conditions using functional near-infrared spectroscopy (fNIRS) hyperscanning to evaluate effects of these holistic processes on synchronization performance. We found that the dyads' performance highly correlated with the integrated effect of holistic cognitive processes in both conditions. Each holistic cognitive process played different roles in interpersonal SMS. HCG was critical to maintain synchronization. HTV related to mentalizing others' behavior. Holistic neural process, the inter-brain coupling of right prefrontal cortex (PFC), was significantly different between bidirectional and unidirectional conditions, which suggested the existence of neural markers at holistic level in interpersonal SMS.

Interactive Brain Activity: Review and Progress on EEG-Based Hyperscanning in Social Interactions

When individuals interact with others, perceived information is transmitted among their brains. The EEG-based hyperscanning technique, which provides an approach to explore dynamic brain activities between two or more interactive individuals and their underlying neural mechanisms, has been applied to study different aspects of social interactions since 2010. Recently there has been an increase in research on EEG-based hyperscanning of social interactions. This paper summarizes the application of EEG-based hyperscanning on the dynamic brain activities during social interactions according to the experimental designs and contents, discusses the possibility of applying inter-brain synchrony to social communication systems and analyzes the contributions and the limitations of these investigations. Furthermore, this paper sheds light on some new challenges to future EEG-based hyperscanning studies and the emerging field of EEG-based hyperscanning for pursuing the broader research field of social interactions.

Alpha Waves as a Neuromarker of Autism Spectrum Disorder: The Challenge of Reproducibility and Heterogeneity

Background: There is no consensus in the literature concerning the presence of abnormal alpha wave profiles in patients with autism spectrum disorder (ASD). This may be due to phenotypic heterogeneity among patients as well as the limited sample sizes utilized. Here we present our results of alpha wave profile analysis based on a sample larger than most of those in the field, performed using a robust processing pipeline. Methods: We compared the alpha waves profiles at rest in children with ASD to those of age-, sex-, and IQ-matched control individuals. We used linear regression and non-parametric normative models using age as covariate forparsing the clinical heterogeneity. We explored the correlation between EEG profiles and the patient's brain volumes, obtained from structural MRI. We automatized the detection of the alpha peak and visually quality controled our MRI measurements. We assessed the robustness of our results by running the EEG preprocessing with two different versions of Matlab as well as Python. Results: A simple linear regression between peak power or frequency of the alpha waves and the status or age of the participants did not allow to identify any statistically significant relationship. The non-parametric normative model (which took account the non-linear effect of age on the alpha profiles) suggested that participants with ASD displayed more variability than control participants for both frequency and amplitude of the alpha peak (p < 0.05). Independent of the status of the individual, we also observed weak associations (uncorrected p < 0.05) between the alpha frequency, and the volumes of several cortical and subcortical structures (in particular the striatum), but which did not survive correction for multiple testing and changed between analysis pelines. Discussions: Our study did not find evidence for abnormal alpha wave profiles in ASD. We propose, however, an analysis pipeline to perform standardized and automatized EEG analyses on large cohorts. These should help the community to address the challenge of clinical heterogeneity of ASD and to tackle the problems of reproducibility.

Single (1:1) vs. double (1:2) metronomes for the spontaneous entrainment and stabilisation of human rhythmic movements

Rhythmic movements produced by humans become spontaneously entrained to auditory rhythms in the environment. Evidence suggests that synchronisation to external auditory rhythms can contribute to the stabilisation of movements in time and space, opening new perspectives for motor training and rehabilitation. Here we compared the effects of single (1:1) and double (1:2) metronomes (i.e., one or two stimulations per preferred movement cycle) on spontaneous movement entrainment and stabilisation. We examined the spontaneous entrainment of self-paced hand-held pendulum swinging when single or double metronomes were presented either at the participant's preferred tempo or slightly slower or faster (± 10%). The results showed that participants' movements spontaneously entrained to auditory rhythms, and that the strength of this entrainment was the same for single and double metronomes. However, double metronomes decreased movement tempo stability, whereas single metronomes increased movement tempo stability compared to a control condition without a stimulus. These effects preferentially occurred for metronomes presented at participants' preferred movement tempi and especially for participants whose movements were intrinsically more variable. Participants' movement amplitude stability was also modulated in such a way that the stability of participants who were intrinsically less stable increased, whereas the stability of intrinsically more stable participants decreased with auditory rhythms, an effect that was stronger with double than single metronomes. Moreover, movement stabilisation in time and space were positively correlated, suggesting that tempo and amplitude stabilisation depend on similar processes and may be complementary. These findings provide new insight into the processes underlying auditory-motor entrainment and how auditory rhythms can be used to improve movement stability in time and space.

Concurrent mapping of brain activation from multiple subjects during social interaction by hyperscanning: a mini-review

Social interaction plays an essential role in acquiring knowledge and developing our own personalities in our daily life. Meanwhile, functional magnetic resonance imaging (fMRI)-, electroencephalograph (EEG)-, and functional near inferred spectroscopy (fNIRS)-hyperscanning, enables us to concurrently map brain activation from two or more participants who are engaged in social interaction simultaneously. In this review, we first highlight the recent technologies advances and the most significant findings towards social interaction by using the hyperscanning method. In addition, we also illustrate several well-designed hyperscanning tasks that have been extensively adopted for the study of social interaction. Basically, hyperscanning contains six categories of experimental paradigms that can track the interactive neural process of interest. Furthermore, it contains two main elucidated neural systems which are involved in social interaction, including the mirror neuron system (MNS) and mentalizing system (MS). Finally, future research directions and clinical implications that are associated with hyperscanning are also highlighted and discussed.