fNIRS Responses in Professional Violinists While Playing Duets: Evidence for Distinct Leader and Follower Roles at the Brain Level

Lights on music cognition: A systematic and critical review of fNIRS applications and future perspectives

Research investigating the neural processes related to music perception and production constitutes a well-established field within the cognitive neurosciences. While most neuroimaging tools have limitations in studying the complexity of musical experiences, functional Near-Infrared Spectroscopy (fNIRS) represents a promising, relatively new tool for studying music processes in both laboratory and ecological settings, which is also suitable for both typical and pathological populations across development. Here we systematically review fNIRS studies on music cognition, highlighting prospects and potentialities. We also include an overview of fNIRS basic theory, together with a brief comparison to characteristics of other neuroimaging tools. Fifty-nine studies meeting inclusion criteria (i.e., using fNIRS with music as the primary stimulus) are presented across five thematic sections. Critical discussion of methodology leads us to propose guidelines of good practices aiming for robust signal analyses and reproducibility. A continuously updated world map is proposed, including basic information from studies meeting the inclusion criteria. It provides an organized, accessible, and updatable reference database, which could serve as a catalyst for future collaborations within the community. In conclusion, fNIRS shows potential for investigating cognitive processes in music, particularly in ecological contexts and with special populations, aligning with current research priorities in music cognition.

Inter-Brain Synchrony Pattern Investigation on Triadic Board Game Play-Based Social Interaction: An fNIRS Study

Recent advances in functional neuroimaging techniques, including methodologies such as fNIRS, have enabled the evaluation of inter-brain synchrony (IBS) induced by interpersonal interactions. However, the social interactions assumed in existing dyadic hyperscanning studies do not sufficiently emulate polyadic social interactions in the real world. Therefore, we devised an experimental paradigm that incorporates the Korean folk board game "Yut-nori" to reproduce social interactions that emulate social activities in the real world. We recruited 72 participants aged 25.2 ± 3.9 years (mean ± standard deviation) and divided them into 24 triads to play Yut-nori, following the standard or modified rules. The participants either competed against an opponent (standard rule) or cooperated with an opponent (modified rule) to achieve a goal efficiently. Three different fNIRS devices were employed to record cortical hemodynamic activations in the prefrontal cortex both individually and simultaneously. Wavelet transform coherence (WTC) analyses were performed to assess prefrontal IBS within a frequency range of 0.05-0.2 Hz. Consequently, we observed that cooperative interactions increased prefrontal IBS across overall frequency bands of interest. In addition, we also found that different purposes for cooperation generated different spectral characteristics of IBS depending on the frequency bands. Moreover, IBS in the frontopolar cortex (FPC) reflected the influence of verbal interactions. The findings of our study suggest that future hyperscanning studies should consider polyadic social interactions to reveal the properties of IBS in real-world interactions.

Music as a window into real-world communication

Communication has been studied extensively in the context of speech and language. While speech is tremendously effective at transferring ideas between people, music is another communicative mode that has a unique power to bring people together and transmit a rich tapestry of emotions, through joint music-making and listening in a variety of everyday contexts. Research has begun to examine the behavioral and neural correlates of the joint action required for successful musical interactions, but it has yet to fully account for the rich, dynamic, multimodal nature of musical communication. We review the current literature in this area and propose that naturalistic musical paradigms will open up new ways to study communication more broadly.

The neuroscience of music – towards ecological validity

Studies in the neuroscience of music gained momentum in the 1990s as an integrated part of the well-controlled experimental research tradition. However, during the past two decades, these studies have moved toward more naturalistic, ecologically valid paradigms. Here, I introduce this move in three frameworks: (i) sound stimulation and empirical paradigms, (ii) study participants, and (iii) methods and contexts of data acquisition. I wish to provide a narrative historical overview of the development of the field and, in parallel, to stimulate innovative thinking to further advance the ecological validity of the studies without overlooking experimental rigor.

Cortico-cerebellar audio-motor regions coordinate self and other in musical joint action

Joint music performance requires flexible sensorimotor coordination between self and other. Cognitive and sensory parameters of joint action-such as shared knowledge or temporal (a)synchrony-influence this coordination by shifting the balance between self-other segregation and integration. To investigate the neural bases of these parameters and their interaction during joint action, we asked pianists to play on an MR-compatible piano, in duet with a partner outside of the scanner room. Motor knowledge of the partner's musical part and the temporal compatibility of the partner's action feedback were manipulated. First, we found stronger activity and functional connectivity within cortico-cerebellar audio-motor networks when pianists had practiced their partner's part before. This indicates that they simulated and anticipated the auditory feedback of the partner by virtue of an internal model. Second, we observed stronger cerebellar activity and reduced behavioral adaptation when pianists encountered subtle asynchronies between these model-based anticipations and the perceived sensory outcome of (familiar) partner actions, indicating a shift towards self-other segregation. These combined findings demonstrate that cortico-cerebellar audio-motor networks link motor knowledge and other-produced sounds depending on cognitive and sensory factors of the joint performance, and play a crucial role in balancing self-other integration and segregation.

Shining a light on cultural neuroscience: Recommendations on the use of fNIRS to study how sociocultural contexts shape the brain

Functional near-infrared spectroscopy (fNIRS) is a portable neuroimaging technique that may serve as a methodological tool for studying how sociocultural contexts can shape the human brain and impact cognition and behavior. The use of fNIRS in community-based research may (a) advance theoretical knowledge in psychology and neuroscience, particularly regarding underrepresented ethnic-racial communities; (b) increase diversity in samples; and (c) provide neurobiological evidence of sociocultural factors supporting human development. The review aims to introduce the use of fNIRS, including its practicalities and limitations, to new adopters inquiring how sociocultural inputs affect the brain. The review begins with an introduction to cultural neuroscience, and a review on the use of fNIRS follows. Next, benefits and guidelines to the design of fNIRS research in naturalistic environments (in the community or in the field) using a cultural lens are discussed. Strengths-based and community-based approaches in cultural neuroscience are recommended throughout. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Alpha oscillations related to self-other integration and distinction during live orchestral performance: A naturalistic case study

Ensemble music performance requires musicians to achieve precise interpersonal coordination while maintaining autonomous control over their own actions. To do so, musicians dynamically shift between integrating other performers' actions into their own action plans and maintaining a distinction between their own and others' actions. Research in laboratory settings has shown that this dynamic process of self-other integration and distinction is indexed by sensorimotor alpha oscillations. The purpose of the current descriptive case study was to examine oscillations related to self-other integration and distinction in a naturalistic performance context. We measured alpha activity from four violinists during a concert hall performance of a 60-musician orchestra. We selected a musical piece from the orchestra's repertoire and, before analyzing alpha activity, performed a score analysis to divide the piece into sections that were expected to strongly promote self-other integration and distinction. In line with previous laboratory findings, performers showed suppressed and enhanced alpha activity during musical sections that promoted self-other integration and distinction, respectively. The current study thus provides preliminary evidence that findings from carefully controlled laboratory experiments generalize to complex real-world performance. Its findings also suggest directions for future research and potential applications of interest to musicians, music educators, and music therapists.

Follow the sound of my violin: Granger causality reflects information flow in sound

Recent research into how musicians coordinate their expressive timing, phrasing, articulation, dynamics, and other stylistic characteristics during performances has highlighted the role of predictive processes, as musicians must anticipate how their partners will play in order to be together. Several studies have used information flow techniques such as Granger causality to show that upcoming movements of a musician can be predicted from immediate past movements of fellow musicians. Although musicians must move to play their instruments, a major goal of music making is to create a joint interpretation through the sounds they produce. Yet, information flow techniques have not been applied previously to examine the role that fellow musicians' sound output plays in these predictive processes and whether this changes as they learn to play together. In the present experiment, we asked professional violinists to play along with recordings of two folk pieces, each eight times in succession, and compared the amplitude envelopes of their performances with those of the recordings using Granger causality to measure information flow and cross-correlation to measure similarity and synchronization. In line with our hypotheses, our measure of information flow was higher from the recordings to the performances than vice versa, and decreased as the violinists became more familiar with the recordings over trials. This decline in information flow is consistent with a gradual shift from relying on auditory cues to predict the recording to relying on an internally-based (learned) model built through repetition. There was also evidence that violinists became more synchronized with the recordings over trials. These results shed light on the planning and learning processes involved in the aligning of expressive intentions in group music performance and lay the groundwork for the application of Granger causality to investigate information flow through sound in more complex musical interactions.

Endogenous sources of interbrain synchrony in duetting pianists

When people interact with each other, their brains synchronize. However, it remains unclear whether interbrain synchrony (IBS) is functionally relevant for social interaction or stems from exposure of individual brains to identical sensorimotor information. To disentangle these views, the current dual-EEG study investigated amplitude-based IBS in pianists jointly performing duets containing a silent pause followed by a tempo change. First, we manipulated the similarity of the anticipated tempo change and measured IBS during the pause, hence, capturing the alignment of purely endogenous, temporal plans without sound or movement. Notably, right posterior gamma IBS was higher when partners planned similar tempi, it predicted whether partners' tempi matched after the pause, and it was modulated only in real, not in surrogate pairs. Second, we manipulated the familiarity with the partner's actions and measured IBS during joint performance with sound. Although sensorimotor information was similar across conditions, gamma IBS was higher when partners were unfamiliar with each other's part and had to attend more closely to the sound of the performance. These combined findings demonstrate that IBS is not merely an epiphenomenon of shared sensorimotor information but can also hinge on endogenous, cognitive processes crucial for behavioral synchrony and successful social interaction.

Magnetoencephalography Hyperscanning Evidence of Differing Cognitive Strategies Due to Social Role During Auditory Communication

Auditory communication is an essential form of human social interaction. However, the intra-brain cortical-oscillatory drivers of auditory communication exchange remain relatively unexplored. We used improvisational music performance to simulate and capture the creativity and turn-taking dynamics of natural auditory communication. Using magnetoencephalography (MEG) hyperscanning in musicians, we targeted brain activity during periods of music communication imagery, and separately analyzed theta (5–7 Hz), alpha (8–13 Hz), and beta (15–29 Hz) source-level activity using a within-subjects, two-factor approach which considered the assigned social role of the subject (leader or follower) and whether communication responses were improvisational (yes or no). Theta activity related to improvisational communication and social role significantly interacted in the left isthmus cingulate cortex. Social role was furthermore differentiated by pronounced occipital alpha and beta amplitude increases suggestive of working memory retention engagement in Followers but not Leaders. The results offer compelling evidence for both musical and social neuroscience that the cognitive strategies, and correspondingly the memory and attention-associated oscillatory brain activities of interlocutors during communication differs according to their social role/hierarchy, thereby indicating that social role/hierarchy needs to be controlled for in social neuroscience research.

Cortical activation during cooperative joint actions and competition in children with and without an autism spectrum condition (ASC): an fNIRS study

Children with an Autism Spectrum Condition (ASC) have social communication and perceptuomotor difficulties that affect their ability to engage in dyadic play. In this study, we compared spatio-temporal errors and fNIRS-related cortical activation between children with and without an ASC during a Lincoln Log dyadic game requiring them to play leader or follower roles, move in synchrony or while taking turns, and move cooperatively or competitively with an adult partner. Children with an ASC had greater motor, planning, and spatial errors and took longer to complete the building tasks compared to typically developing (TD) children. Children with an ASC had lower superior temporal sulcus (STS) activation during Turn-take and Compete, and greater Inferior Parietal Lobe (IPL) activation during Lead and Turn-take compared to TD children. As dyadic play demands increased, TD children showed greater STS activation during Turn-take (vs. Synchrony) and Compete (vs. Cooperate) whereas children with an ASC showed greater IPL activation during Lead and Compete (vs. Cooperate). Our findings suggest that children with an ASC rely on self-generated action plans (i.e., increased IPL activation) more than relying on their partner’s action cues (i.e., reduced STS activation) when engaging in dyadic play including joint actions and competition.

Observation and motor imagery balance tasks evaluation: An fNIRS feasibility study

In this study, we aimed at exploring the feasibility of functional near-infrared spectroscopy (fNIRS) for studying the observation and/or motor imagination of various postural tasks. Thirteen healthy adult subjects followed five trials of static and dynamic standing balance tasks, throughout three different experimental setups of action observation (AO), a combination of action observation and motor imagery (AO+MI), and motor imagery (MI). During static and dynamic standing tasks, both the AO+MI and MI experiments revealed that many channels in prefrontal or motor regions are significantly activated while the AO experiment showed almost no significant increase in activations in most of the channels. The contrast between static and dynamic standing tasks showed that with more demanding balance tasks, relative higher activation patterns were observed, particularly during AO and in AO+MI experiments in the frontopolar area. Moreover, the AO+MI experiment revealed a significant difference in premotor and supplementary motor cortices that are related to balance control. Furthermore, it has been observed that the AO+MI experiment induced relatively higher activation patterns in comparison to AO or MI alone. Remarkably, the results of this work match its counterpart from previous functional magnetic resonance imaging studies. Therefore, they may pave the way for using the fNIRS as a diagnostic tool for evaluating the performance of the non-physical balance training during the rehabilitation period of temporally immobilized patients.

Neural Effects of Physical Activity and Movement Interventions in Individuals With Developmental Disabilities-A Systematic Review

Individuals with developmental disabilities present with perceptuo-motor, social communication, and cognitive impairments that often relate to underlying atypical brain structure and functioning. Physical activity/movement interventions improve behavioral performance of individuals with and without developmental disabilities. Majority of the evidence on potential neural mechanisms explaining the impact of physical activity/movement interventions is based on studies in individuals with typical development; there is a dearth of systematic reviews synthesizing the neural effects of physical activity/movement interventions in individuals with developmental disabilities. In this systematic review, we have gathered evidence on the neural effects of physical activity/movement interventions from 32 papers reporting substantial neural effects and behavioral improvements in individuals with developmental disabilities. Chronic intervention effects (multiple sessions) were greater than acute intervention effects (single session). Specifically, using electroencephalogram, functional magnetic resonance imaging, diffusion tensor imaging, and functional near-infrared spectroscopy, studies found physical activity/movement intervention-related changes in neural activity, indicating normalization of cortical arousal in individuals with attention-deficit /hyperactivity disorder (ADHD), increased social brain connectivity in individuals with autism spectrum disorder (ASD), and more efficient executive functioning processes in individuals with a wide range of other developmental disabilities. Despite promising results, more research is clearly needed in this area with larger sample sizes, using standardized neuroimaging tools/variables, and across multiple diagnoses to further explore the neural mechanisms underlying physical activity/movement interventions and to replicate findings from the present review.

Inter-Brain Synchrony Levels According to Task Execution Modes and Difficulty Levels: an fNIRS/GSR Study

Hyperscanning is a brain imaging technique that measures brain synchrony caused by social interactions. Recent research on hyperscanning has revealed substantial inter-brain synchrony (IBS), but little is known about the link between IBS and mental workload. To study this link, we conducted an experiment consisting of button-pressing tasks of three different difficulty levels for the cooperation and competition modes with 56 participants aged 23.7±3.8 years (mean±standard deviation). We attempted to observe IBS using functional near-infrared spectroscopy (fNIRS) and galvanic skin response (GSR) to assess the activities of the human autonomic nervous system. We found that the IBS levels increased in a frequency band of 0.075-0.15 Hz, which was unrelated to the task repetition frequency in the cooperation mode according to the task difficulty level. Significant relative inter-brain synchrony (RIBS) increases were observed in three and 10 channels out of 15 for the hard tasks compared to the normal and easy tasks, respectively. We observed that the average GSR values increased with increasing task difficulty levels for the competition mode only. Thus, our results suggest that the IBS revealed by fNIRS and GSR is not related to the hemodynamic changes induced by mental workload, simple behavioral synchrony such as button-pressing timing, or autonomic nervous system activity. IBS is thus explicitly caused by social interactions such as cooperation.

Inter-Brain Synchronization During Sandplay Therapy: Individual Analyses

Interactions between the client (Cl) and therapist (Th) evolve therapeutic relationships in psychotherapy. An interpersonal link or therapeutic space is implicitly developed, wherein certain important elements are expressed and shared. However, neural basis of psychotherapy, especially of non-verbal modalities, have scarcely been explored. Therefore, we examined the neural backgrounds of such therapeutic alliances during sandplay, a powerful art/play therapy technique. Real-time and simultaneous measurement of hemodynamics was conducted in the prefrontal cortex (PFC) of Cl-Th pairs participating in sandplay and subsequent interview sessions through multichannel near-infrared spectroscopy. As sandplay is highly individualized, and no two sessions and products (sandtrays) are the same, we expected variation in interactive patterns in the Cl–Th pairs. Nevertheless, we observed a statistically significant correlation between the spatio-temporal patterns in signals produced by the homologous regions of the brains. During the sandplay condition, significant correlations were obtained in the lateral PFC and frontopolar (FP) regions in the real Cl-Th pairs. Furthermore, a significant correlation was observed in the FP region for the interview condition. The correlations found in our study were explained as a “remote” synchronization (i.e., unconnected peripheral oscillators synchronizing through a hub maintaining free desynchronized dynamics) between two subjects in a pair, possibly representing the neural foundation of empathy, which arises commonly in sandplay therapy (ST).

Temporal Coordination in Piano Duet Networked Music Performance (NMP): Interactions Between Acoustic Transmission Latency and Musical Role Asymmetries

Today’s audio, visual, and internet technologies allow people to interact despite physical distances, for casual conversation, group workouts, or musical performance. Musical ensemble performance is unique because interaction integrity critically depends on the timing between each performer’s actions and when their acoustic outcomes arrive. Acoustic transmission latency (ATL) between players is substantially longer for networked music performance (NMP) compared to traditional in-person spaces where musicians can easily adapt. Previous work has shown that longer ATLs slow the average tempo in ensemble performance, and that asymmetric co-actor roles and empathy-related traits affect coordination patterns in joint action. Thus, we are interested in how musicians collectively adapt to a given latency and how such adaptation patterns vary with their task-related and person-related asymmetries. Here, we examined how two pianists performed duets while hearing each other’s auditory outcomes with an ATL of 10, 20, or 40 ms. To test the hypotheses regarding task-related asymmetries, we designed duets such that pianists had: (1) a starting or joining role and (2) a similar or dissimilar musical part compared to their co-performer, with respect to pitch range and melodic contour. Results replicated previous clapping-duet findings showing that longer ATLs are associated with greater temporal asynchrony between partners and increased average tempo slowing. While co-performer asynchronies were not affected by performer role or part similarity, at the longer ATLs starting performers displayed slower tempos and smaller tempo variability than joining performers. This asymmetry of stability vs. flexibility between starters and joiners may sustain coordination, consistent with recent joint action findings. Our data also suggest that relative independence in musical parts may mitigate ATL-related challenges. Additionally, there may be a relationship between co-performer differences in empathy-related personality traits such as locus of control and coordination during performance under the influence of ATL. Incorporating the emergent coordinative dynamics between performers could help further innovation of music technologies and composition techniques for NMP.

Extreme precision in rhythmic interaction is enabled by role-optimized sensorimotor coupling: analysis and modelling of West African drum ensemble music

Human social interactions often involve carefully synchronized behaviours. Musical performance in particular features precise timing and depends on the differentiation and coordination of musical/social roles. Here, we study the influence of musical/social roles, individual musicians and different ensembles on rhythmic synchronization in Malian drum ensemble music, which features synchronization accuracy near the limits of human performance. We analysed 72 recordings of the same piece performed by four trios, in which two drummers in each trio systematically switched roles (lead versus accompaniment). Musical role, rather than individual or group differences, is the main factor influencing synchronization accuracy. Using linear causal modelling, we found a consistent pattern of bi-directional couplings between players, in which the direction and strength of rhythmic adaptation is asymmetrically distributed across musical roles. This differs from notions of musical leadership, which assume that ensemble synchronization relies predominantly on a single dominant personality and/or musical role. We then ran simulations that varied the direction and strength of sensorimotor coupling and found that the coupling pattern used by the Malian musicians affords nearly optimal synchronization. More broadly, our study showcases the importance of ecologically valid and culturally diverse studies of human behaviour. This article is part of the theme issue 'Synchrony and rhythm interaction: from the brain to behavioural ecology'.

Interactive brains, social minds: Neural and physiological mechanisms of interpersonal action coordination

It is now widely accepted that inter-brain synchronization is an important and inevitable mechanism of interpersonal action coordination and social interaction behavior. This review of the current literature focuses first on the forward model for interpersonal action coordination and functional system theory for biological systems, two broadly similar concepts for adaptive system behavior. Further, we review interacting-brain and/or hyper-brain dynamics studies, to show the interplay between intra- and inter-brain connectivity resulting in hyper-brain network structure and network topology dynamics, and consider the functioning of interacting brains as a superordinate system. The concept of a superordinate system, or superorganism, is then evaluated with respect to neuronal and physiological systems group dynamics, which show further accompanying mechanisms of interpersonal interaction. We note that fundamental problems need to be resolved to better understand the neural mechanisms of interpersonal action coordination.

Affiliative bonding between teachers and students through interpersonal synchronisation in brain activity

Human beings organise socially. Theories have posited that interpersonal neural synchronisation might underlie the creation of affiliative bonds. Previous studies tested this hypothesis mainly during a social interaction, making it difficult to determine whether the identified synchronisation is associated with affiliative bonding or with social interaction. This study addressed this issue by focusing on the teacher–student relationship in the resting state both before and after a teaching period. Brain activity was simultaneously measured in both individuals using functional near-infrared spectroscopy. The results showed a significant increase in brain synchronisation at the right sensorimotor cortex between the teacher and student in the resting state after, but not before, the teaching period. Moreover, the synchronisation increased only after a turn-taking mode of teaching but not after a lecturing or video mode of teaching. A chain mediation analysis showed that brain synchronisation during teaching partially mediated the relationship between the brain synchronisation increase in the resting state and strength of the affiliative bond. Finally, both role assignment and social interaction were found to be required for affiliative bonding. Together, these results support the hypothesis that interpersonal synchronisation in brain activity underlies affiliative bonding and that social interaction mechanically mediates the bonding process.

Affective Neurofeedback Under Naturalistic Conditions: A Mini-Review of Current Achievements and Open Challenges

Affective neurofeedback training allows for the self-regulation of the putative circuits of emotion regulation. This approach has recently been studied as a possible additional treatment for psychiatric disorders, presenting positive effects in symptoms and behaviors. After neurofeedback training, a critical aspect is the transference of the learned self-regulation strategies to outside the laboratory and how to continue reinforcing these strategies in non-controlled environments. In this mini-review, we discuss the current achievements of affective neurofeedback under naturalistic setups. For this, we first provide a brief overview of the state-of-the-art for affective neurofeedback protocols. We then discuss virtual reality as a transitional step toward the final goal of "in-the-wild" protocols and current advances using mobile neurotechnology. Finally, we provide a discussion of open challenges for affective neurofeedback protocols in-the-wild, including topics such as convenience and reliability, environmental effects in attention and workload, among others.

Psychotherapy in the Framework of Embodied Cognition-Does Interpersonal Synchrony Influence Therapy Success?

Mental health problems remain among the main generators of costs within and beyond the health care system. Psychotherapy, the tool of choice in their treatment, is qualified by social interaction, and cooperation within the therapist-patient-dyad. Research into the factors influencing therapy success to date is neither exhaustive nor conclusive. Among many others, the quality of the relationship between therapist and patient stands out regardless of the followed psychotherapy school. Emerging research points to a connection between interpersonal synchronization within the sessions and therapy outcome. Consequently, it can be considered significant for the shaping of this relationship. The framework of Embodied Cognition assumes bodily and neuronal correlates of thinking. Therefore, the present paper reviews investigations on interpersonal, non-verbal synchrony in two domains: firstly, studies on interpersonal synchrony in psychotherapy are reviewed (synchronization of movement). Secondly, findings on neurological correlates of interpersonal synchrony (assessed with EEG, fMRI, fNIRS) are summarized in a narrative manner. In addition, the question is asked whether interpersonal synchrony can be achieved voluntarily on an individual level. It is concluded that there might be mechanisms which could give more insights into therapy success, but as of yet remain uninvestigated. Further, the framework of embodied cognition applies more to the current body of evidence than classical cognitivist views. Nevertheless, deeper research into interpersonal physical and neurological processes utilizing the framework of Embodied Cognition emerges as a possible route of investigation on the road to lower drop-out rates, improved and quality-controlled therapeutic interventions, thereby significantly reducing healthcare costs.

The motor-related brain activity that supports joint action: A review

Recent years have seen a rapid increase in research investigating the motor-related brain activity that supports joint action. This research has employed a variety of joint action tasks and an array of neuroimaging techniques, including fMRI, fNIRS, EEG, and TMS. In this review, we provide an overview of this research to delineate what is known about the motor-related brain activity that contributes to joint action and to highlight key questions for future research. Taken together, the surveyed research supports three major conclusions. First, the mere presence of a joint action context is sufficient to modulate motor activity elicited by observing others' actions. Second, joint action is supported by dissociable motor activity associated with a person's own actions, their partner's actions, and the joint action, and by between-brain coupling of motor-related oscillatory activity. Third, the structure of a joint action modulates the motor activity involved: Unique motor activity is associated with performing joint actions comprised of complementary actions and with holding the roles of leader and follower within a joint action. We conclude the review by highlighting overarching themes and key questions for future research.

Capturing Human Interaction in the Virtual Age: A Perspective on the Future of fNIRS Hyperscanning

Advances in video conferencing capabilities combined with dramatic socio-dynamic shifts brought about by COVID-19, have redefined the ways in which humans interact in modern society. From business meetings to medical exams, or from classroom instruction to yoga class, virtual interfacing has permeated nearly every aspect of our daily lives. A seemingly endless stream of technological advances combined with our newfound reliance on virtual interfacing makes it likely that humans will continue to use this modern form of social interaction into the future. However, emergent evidence suggests that virtual interfacing may not be equivalent to face-to-face interactions. Ultimately, too little is currently understood about the mechanisms that underlie human interactions over the virtual divide, including how these mechanisms differ from traditional face-to-face interaction. Here, we propose functional near-infrared spectroscopy (fNIRS) hyperscanning—simultaneous measurement of two or more brains—as an optimal approach to quantify potential neurocognitive differences between virtual and in-person interactions. We argue that increased focus on this understudied domain will help elucidate the reasons why virtual conferencing doesn't always stack up to in-person meetings and will also serve to spur new technologies designed to improve the virtual interaction experience. On the basis of existing fNIRS hyperscanning literature, we highlight the current gaps in research regarding virtual interactions. Furthermore, we provide insight into current hurdles regarding fNIRS hyperscanning hardware and methodology that should be addressed in order to shed light on this newly critical element of everyday life.

Brain-to-Brain Neural Synchrony During Social Interactions: A Systematic Review on Hyperscanning Studies

The aim of this study was to conduct a comprehensive review on hyperscanning research (measuring brain activity simultaneously from more than two people interacting) using an explicit systematic method, the preferred reporting items for systematic reviews and meta-analyses (PRISMA). Data were searched from IEEE Xplore, PubMed, Engineering Village, Web of Science and Scopus databases. Inclusion criteria were journal articles written in English from 2000 to 19 June 2019. A total of 126 empirical studies were screened out to address three specific questions regarding the neuroimaging method, the application domain, and the experiment paradigm. Results showed that the most used neuroimaging method with hyperscanning was magnetoencephalography/electroencephalography (MEG/EEG; 47%), and the least used neuroimaging method was hyper-transcranial Alternating Current Stimulation (tACS) (1%). Applications in cognition accounted for almost half the studies (48%), while educational applications accounted for less than 5% of the studies. Applications in decision-making tasks were the second most common (26%), shortly followed by applications in motor synchronization (23%). The findings from this systematic review that were based on documented, transparent and reproducible searches should help build cumulative knowledge and guide future research regarding inter-brain neural synchrony during social interactions, that is, hyperscanning research.

Raising the bar: Can dual scanning improve our understanding of joint action?

Two-person neuroscience (2 ​PN) is a recently introduced conceptual and methodological framework used to investigate the neural basis of human social interaction from simultaneous neuroimaging of two or more subjects (hyperscanning). In this study, we adopted a 2 ​PN approach and a multiple-brain connectivity model to investigate the neural basis of a form of cooperation called joint action. We hypothesized different intra-brain and inter-brain connectivity patterns when comparing the interpersonal properties of joint action with non-interpersonal conditions, with a focus on co-representation, a core ability at the basis of cooperation. 32 subjects were enrolled in dual-EEG recordings during a computerized joint action task including three conditions: one in which the dyad jointly acted to pursue a common goal (joint), one in which each subject interacted with the PC (PC), and one in which each subject performed the task individually (Solo). A combination of multiple-brain connectivity estimation and specific indices derived from graph theory allowed to compare interpersonal with non-interpersonal conditions in four different frequency bands. Our results indicate that all the indices were modulated by the interaction, and returned a significantly stronger integration of multiple-subject networks in the joint vs. PC and Solo conditions. A subsequent classification analysis showed that features based on multiple-brain indices led to a better discrimination between social and non-social conditions with respect to single-subject indices. Taken together, our results suggest that multiple-brain connectivity can provide a deeper insight into the understanding of the neural basis of cooperation in humans.

Optimization of wavelet coherence analysis as a measure of neural synchrony during hyperscanning using functional near-infrared spectroscopy

Significance: The expanding field of human social interaction is enabled by functional near-infrared spectroscopy (fNIRS) that acquires hemodynamic signals during live two-person interactions. These advances call for development of methods to quantify interactive processes. Aim: Wavelet coherence analysis has been applied to cross-brain neural coupling. However, fNIRS-specific computations have not been explored. This investigation determines the effects of global mean removal, wavelet equation, and choice of oxyhemoglobin versus deoxyhemoglobin signals. Approach: We compare signals with a known coherence with acquired signals to determine optimal computational approaches. The known coherence was calculated using three visual stimulation sequences of a contrast-reversing checkerboard convolved with the canonical hemodynamic response function. This standard was compared with acquired human fNIRS responses within visual cortex using the same sequences. Results: Observed coherence was consistent with known coherence with highest correlations within the wavelength range between 10 and 20 s. Removal of the global mean improved the correlation irrespective of the specific equation for wavelet coherence, and the oxyhemoglobin signal was associated with a marginal correlation advantage. Conclusions: These findings provide both methodological and computational guidance that enhances the validity and interpretability of wavelet coherence analysis for fNIRS signals acquired during live social interactions.

Neural Efficiency of Human-Robotic Feedback Modalities Under Stress Differs With Gender

Sensory feedback, which can be presented in different modalities - single and combined, aids task performance in human-robotic interaction (HRI). However, combining feedback modalities does not always lead to optimal performance. Indeed, it is not known how feedback modalities affect operator performance under stress. Furthermore, there is limited information on how feedback affects neural processes differently for males and females and under stress. This is a critical gap in the literature, particularly in the domain of surgical robotics, where surgeons are under challenging socio-technical environments that burden them physiologically. In the present study, we posited operator performance as the summation of task performance and neurophysiological cost of maintaining that performance. In a within-subject design, we used functional near-infrared spectroscopy to assess cerebral activations of 12 participants who underwent a 3D manipulation task within a virtual environment with concurrent feedback (visual and visual + haptic) in the presence and absence of a cognitive stressor. Cognitive stress was induced with the serial-7 subtraction test. We found that while task performance was higher with visual than visual + haptic feedback, it degraded under stress. The two feedback modalities were found to be associated with varying neural activities and neural efficiencies, and these were stress- and gender-dependent. Our findings engender further investigation into effectiveness of feedback modalities on males and females under stressful conditions in HRI.

Empathic perspective taking promotes interpersonal coordination through music

Coordinated behavior promotes collaboration among humans. To shed light upon this relationship, we investigated whether and how interpersonal coordination is promoted by empathic perspective taking (EPT). In a joint music-making task, pairs of participants rotated electronic music-boxes, producing two streams of musical sounds that were meant to be played synchronously. Participants – who were not musically trained – were assigned to high and low EPT groups based on pre-experimental assessments using a standardized personality questionnaire. Results indicated that high EPT pairs were generally more accurate in synchronizing their actions. When instructed to lead the interaction, high and low EPT leaders were equally cooperative with followers, making their performance tempo more regular, presumably in order to increase their predictability and help followers to synchronize. Crucially, however, high EPT followers were better able to use this information to predict leaders’ behavior and thus improve interpersonal synchronization. Thus, empathic perspective taking promotes interpersonal coordination by enhancing accuracy in predicting others’ behavior while leaving the aptitude for cooperation unaltered. We argue that such predictive capacity relies on a sensorimotor mechanism responsible for simulating others’ actions in an anticipatory manner, leading to behavioral advantages that may impact social cognition on a broad scale.