Capturing Human Interaction in the Virtual Age: A Perspective on the Future of fNIRS Hyperscanning

Hyperscanning literature after two decades of neuroscientific research: A scientometric review

Hyperscanning, a neuroimaging approach introduced in 2002 for simultaneously recording the brain activity of multiple participants, has significantly contributed to our understanding of social interactions. Nevertheless, the existing literature requires systematic organization to advance our knowledge. This study, after two decades of hyperscanning research, aims to identify the primary thematic domains and the most influential documents in the field. We conducted a scientometric analysis to examine co-citation patterns quantitatively, using a sample of 548 documents retrieved from Scopus and their 32,022 cited references. Our analysis revealed ten major thematic domains in hyperscanning research, with the most impactful document authored by Czeszumski and colleagues in 2020. Notably, while hyperscanning was initially developed for functional magnetic resonance imaging (fMRI), our findings indicate a substantial influence of research conducted using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). The introduction of fNIRS and advancements in EEG methods have enabled the implementation of more ecologically valid experiments for investigating social interactions. The study also highlights the need for more research that combines multi-brain neural stimulation with neuroimaging techniques to understand the causal role played by interpersonal neural synchrony in social interactions.

From physical to digital: A theoretical-methodological primer on designing hyperscanning investigations to explore remote exchanges

As individuals increasingly engage in social interactions through digital mediums, understanding the neuroscientific underpinnings of such exchanges becomes a critical challenge and a valuable opportunity. In line with a second-person neuroscience approach, understanding the forms of interpersonal syntonisation that occur during digital interactions is pivotal for grasping the mechanisms underlying successful collaboration in virtual spaces. The hyperscanning paradigm, involving the simultaneous monitoring of the brains and bodies of multiple interacting individuals, seems to be a powerful tool for unravelling the neural correlates of interpersonal syntonisation in social exchanges. We posit that such approach can now open new windows on interacting brains' responses even to digitally-conveyed social cues, offering insights into how social information is processed in the absence of traditional face-to-face settings. Yet, such paradigm shift raises challenging methodological questions, which should be answered properly to conduct significant and informative hyperscanning investigations. Here, we provide an introduction to core methodological issues dedicated to novices approaching the design of hyperscanning investigations of remote exchanges in natural settings, focusing on the selection of neuroscientific devices, synchronization of data streams, and data analysis approaches. Finally, a methodological checklist for devising robust hyperscanning studies on digital interactions is presented.

Using interbrain synchrony to study teamwork: A systematic review and meta-analysis

It has been proposed that interbrain synchrony (IBS) may help to elucidate the neural mechanisms underpinning teamwork. As hyperscanning studies have provided abundant findings on IBS in team environments, the current review aims to synthesize the findings of hyperscanning studies in a way that is relevant to the teamwork research. A systematic review was conducted. Included studies were classified according to the IPO (i.e. input, process, output) model of teamwork. Three multi-level meta-analyses were performed to quantify the associations between IBS and the three IPO variables. The methodology followed PRISMA guidelines and the protocol was pre-registered (https://osf.io/7h8sa/). Of the 229 studies, 41 were included, representing 1326 teams. The three meta-analyses found statistically significant positive effects, indicating a positive association between IBS and the three IPO teamwork variables. This study provides evidence that IBS is a relevant measure of the teamwork process and argues for the continued use of IBS to study teamwork.

Neural responses to gender-based microaggressions in academic medicine

Gender-based microaggressions have been associated with persistent disparities between women and men in academia. Little is known about the neural mechanisms underlying those often subtle and unintentional yet detrimental behaviors. Here, we assessed the neural responses to gender-based microaggressions in 28 early career faculty in medicine (N = 16 female, N = 12 male sex) using fMRI. Participants watched 33 videos of situations demonstrating gender-based microaggressions and control situations in academic medicine. Video topics had been previously identified through real-life anecdotes about microaggression from women faculty and were scripted and reenacted using professional actors. Primary voxel-wise analyses comparing group differences in activation elucidated a significant group by condition interaction in a right-lateralized cluster across the frontal (inferior and middle frontal gyri, frontal pole, precentral gyrus, postcentral gyrus) and parietal lobes (supramarginal gyrus, angular gyrus). Whereas women faculty exhibited reduced activation in these regions during the microaggression relative to the control condition, the opposite was true for men. Posthoc analyses showed that these patterns were significantly associated with the degree to which participants reported feeling judged for their gender in academic medicine. Lastly, secondary exploratory ROI analyses showed significant between-group differences in the right dorsolateral prefrontal cortex and inferior frontal gyrus. Women activated these two regions less in the microaggression condition compared to the control condition, whereas men did not. These findings indicate that the observation of gender-based microaggressions results in a specific pattern of neural reactivity in women early career faculty.

Quantification of inter-brain coupling: A review of current methods used in haemodynamic and electrophysiological hyperscanning studies

Hyperscanning is a form of neuroimaging experiment where the brains of two or more participants are imaged simultaneously whilst they interact. Within the domain of social neuroscience, hyperscanning is increasingly used to measure inter-brain coupling (IBC) and explore how brain responses change in tandem during social interaction. In addition to cognitive research, some have suggested that quantification of the interplay between interacting participants can be used as a biomarker for a variety of cognitive mechanisms aswell as to investigate mental health and developmental conditions including schizophrenia, social anxiety and autism. However, many different methods have been used to quantify brain coupling and this can lead to questions about comparability across studies and reduce research reproducibility. Here, we review methods for quantifying IBC, and suggest some ways moving forward. Following the PRISMA guidelines, we reviewed 215 hyperscanning studies, across four different brain imaging modalities: functional near-infrared spectroscopy (fNIRS), functional magnetic resonance (fMRI), electroencephalography (EEG) and magnetoencephalography (MEG). Overall, the review identified a total of 27 different methods used to compute IBC. The most common hyperscanning modality is fNIRS, used by 119 studies, 89 of which adopted wavelet coherence. Based on the results of this literature survey, we first report summary statistics of the hyperscanning field, followed by a brief overview of each signal that is obtained from each neuroimaging modality used in hyperscanning. We then discuss the rationale, assumptions and suitability of each method to different modalities which can be used to investigate IBC. Finally, we discuss issues surrounding the interpretation of each method.

Virtual (Zoom) Interactions Alter Conversational Behavior and Interbrain Coherence

A growing number of social interactions are taking place virtually on videoconferencing platforms. Here, we explore potential effects of virtual interactions on observed behavior, subjective experience, and neural "single-brain" and "interbrain" activity via functional near-infrared spectroscopy neuroimaging. We scanned a total of 36 human dyads (72 participants, 36 males, 36 females) who engaged in three naturalistic tasks (i.e., problem-solving, creative-innovation, socio-emotional task) in either an in-person or virtual (Zoom) condition. We also coded cooperative behavior from audio recordings. We observed reduced conversational turn-taking behavior during the virtual condition. Given that conversational turn-taking was associated with other metrics of positive social interaction (e.g., subjective cooperation and task performance), this measure may be an indicator of prosocial interaction. In addition, we observed altered patterns of averaged and dynamic interbrain coherence in virtual interactions. Interbrain coherence patterns that were characteristic of the virtual condition were associated with reduced conversational turn-taking. These insights can inform the design and engineering of the next generation of videoconferencing technology.SIGNIFICANCE STATEMENT Videoconferencing has become an integral part of our lives. Whether this technology impacts behavior and neurobiology is not well understood. We explored potential effects of virtual interaction on social behavior, brain activity, and interbrain coupling. We found that virtual interactions were characterized by patterns of interbrain coupling that were negatively implicated in cooperation. Our findings are consistent with the perspective that videoconferencing technology adversely affects individuals and dyads during social interaction. As virtual interactions become even more necessary, improving the design of videoconferencing technology will be crucial for supporting effective communication.

Machine Learning Distinguishes Familiar from Unfamiliar Pairs of Subjects Performing an Eye Contact Task: A Systemic Physiology Augmented Functional Near-Infrared Spectroscopy Hyperscanning Study

BACKGROUND

Eye contact is an important aspect of human communication and social interactions. Changes in brain and systemic physiological activity associated with interactions between humans can be measured with systemic physiology augmented functional near-infrared spectroscopy (SPA-fNIRS) hyperscanning, enabling inter-brain and inter-body synchronisation to be determined. In a previous study, we found that pairs of subjects that are socially connected show higher brain and body synchrony.

AIM

To enable a deeper understanding, our aim was to build and automatically detect the best set of features to distinguish between two different groups (familiar and unfamiliar pairs).

MATERIAL AND METHODS

We defined several features based on the Spearman correlation and wavelet transform coherence (WTC) of biosignals measured on 23 pairs of subjects (13 familiar and 10 unfamiliar pairs) during eye contact for 10 min. Additional custom features that identify the maximum brain-to-body coupling instants between pairs were generated.

RESULTS

After testing on combinations of different feature extraction methods, four subsets of features with the strongest discrimination power were taken into account to train a decision tree (DT) machine learning (ML) algorithm. We have obtained 95.65% classification accuracy using a leave-one-out cross-validation. The coupling features which represent the two maximum mean values resulting from the sum of 7 time-dependent WTC signals (oxyhaemoglobin concentration of the right prefrontal region, total haemoglobin concentration of the left and right prefrontal region, heart rate, electrodermal activity on the left and right wrist, and skin temperature on the right wrist) played an essential role in the classification accuracy.

CONCLUSION

Training the DT-ML algorithm with combined brain and systemic physiology data provided higher accuracy than training it only with brain or systemic data alone. The results demonstrate the power of the SPA-fNIRS hyperscanning approach and the potential in applying ML to investigate the strength of social bonds in a wide range of social interaction contexts.

Dynamic inter-brain synchrony in real-life inter-personal cooperation: A functional near-infrared spectroscopy hyperscanning study

How two brains communicate with each other during social interaction is highly dynamic and complex. Multi-person (i.e., hyperscanning) studies to date have focused on analyzing the entire time series of brain signals to reveal an overall pattern of inter-brain synchrony (IBS). However, this approach does not account for the dynamic nature of social interaction. In the present study, we propose a data-driven approach based on sliding windows and k-mean clustering to capture the dynamic modulation of IBS patterns during interactive cooperation tasks. We used a portable functional near-infrared spectroscopy (fNIRS) system to measure brain hemodynamic response between interacting partners (20 dyads) engaged in a creative design task and a 3D model building task. Results indicated that inter-personal communication during naturalistic cooperation generally presented with a series of dynamic IBS states along the tasks. Compared to the model building task, the creative design task appeared to involve more complex and active IBS between multiple regions in specific dynamic IBS states. In summary, the proposed approach stands as a promising tool to distill complex inter-brain dynamics associated with social interaction into a set of representative brain states with more fine-grained temporal resolution. This approach holds promise for advancing our current understanding of the dynamic nature of neurocognitive processes underlying social interaction.

Shedding light on neuroscience: Two decades of functional near-infrared spectroscopy applications and advances from a bibliometric perspective

Functional near-infrared spectroscopy (fNIRS) is a noninvasive optical brain-imaging technique that detects changes in hemoglobin concentration in the cerebral cortex. fNIRS devices are safe, silent, portable, robust against motion artifacts, and have good temporal resolution. fNIRS is reliable and trustworthy, as well as an alternative and a complement to other brain-imaging modalities, such as electroencephalography or functional magnetic resonance imaging. Given these advantages, fNIRS has become a well-established tool for neuroscience research, used not only for healthy cortical activity but also as a biomarker during clinical assessment in individuals with schizophrenia, major depressive disorder, bipolar disease, epilepsy, Alzheimer's disease, vascular dementia, and cancer screening. Owing to its wide applicability, studies on fNIRS have increased exponentially over the last two decades. In this study, scientific publications indexed in the Web of Science databases were collected and a bibliometric-type methodology was developed. For this purpose, a comprehensive science mapping analysis, including top-ranked authors, journals, institutions, countries, and co-occurring keywords network, was conducted. From a total of 2310 eligible documents, 6028 authors and 531 journals published fNIRS-related papers, Fallgatter published the highest number of articles and was the most cited author. University of Tübingen in Germany has produced the most trending papers since 2000. USA was the most prolific country with the most active institutions, followed by China, Japan, Germany, and South Korea. The results also revealed global trends in emerging areas of research, such as neurodevelopment, aging, and cognitive and emotional assessment.