Optimization of wavelet coherence analysis as a measure of neural synchrony during hyperscanning using functional near-infrared spectroscopy

Lights on music cognition: A systematic and critical review of fNIRS applications and future perspectives

Research investigating the neural processes related to music perception and production constitutes a well-established field within the cognitive neurosciences. While most neuroimaging tools have limitations in studying the complexity of musical experiences, functional Near-Infrared Spectroscopy (fNIRS) represents a promising, relatively new tool for studying music processes in both laboratory and ecological settings, which is also suitable for both typical and pathological populations across development. Here we systematically review fNIRS studies on music cognition, highlighting prospects and potentialities. We also include an overview of fNIRS basic theory, together with a brief comparison to characteristics of other neuroimaging tools. Fifty-nine studies meeting inclusion criteria (i.e., using fNIRS with music as the primary stimulus) are presented across five thematic sections. Critical discussion of methodology leads us to propose guidelines of good practices aiming for robust signal analyses and reproducibility. A continuously updated world map is proposed, including basic information from studies meeting the inclusion criteria. It provides an organized, accessible, and updatable reference database, which could serve as a catalyst for future collaborations within the community. In conclusion, fNIRS shows potential for investigating cognitive processes in music, particularly in ecological contexts and with special populations, aligning with current research priorities in music cognition.

Inter-brain synchrony during mother-infant interactive parenting in 3-4-month-old infants with and without an elevated likelihood of autism spectrum disorder

Maternal bonding for mammalian infants is critical for their survival. Additionally, it is important for human infants' development into social creatures. However, despite the ample neurobiological evidence of attachment for the mother's brain, the interplay of this system in infants is poorly understood. We aimed to identify the neural substrates of synchrony in mothers and infants under three interactive conditions and compare the differences between groups with (n = 16) and without (n = 71) an elevated likelihood of autism spectrum disorder by examining the inter-brain synchrony between mothers and their 3-4-month-old infants. Mother-infant hyperscanning with functional near-infrared spectroscopy was performed during breastfeeding and while each of the mother and experimenter was holding the infants. The results showed almost no group differences, with both groups demonstrating the strongest inter-brain coupling for breastfeeding. The cerebral foci underlying these couplings differed between mothers and infants: the ventral prefrontal cortex, focusing on the right orbitofrontal cortex, in the mother and the left temporoparietal junction in the infant were chiefly involved in connecting the two brains. Furthermore, these synchronizations revealed many significant correlations with behavioral measures, including subsequent language development. The maternal reward-motivational system and the infant's elementary mentalization system seem to underlie mother-infant coupling during breastfeeding.

Quantification of inter-brain coupling: A review of current methods used in haemodynamic and electrophysiological hyperscanning studies

Hyperscanning is a form of neuroimaging experiment where the brains of two or more participants are imaged simultaneously whilst they interact. Within the domain of social neuroscience, hyperscanning is increasingly used to measure inter-brain coupling (IBC) and explore how brain responses change in tandem during social interaction. In addition to cognitive research, some have suggested that quantification of the interplay between interacting participants can be used as a biomarker for a variety of cognitive mechanisms aswell as to investigate mental health and developmental conditions including schizophrenia, social anxiety and autism. However, many different methods have been used to quantify brain coupling and this can lead to questions about comparability across studies and reduce research reproducibility. Here, we review methods for quantifying IBC, and suggest some ways moving forward. Following the PRISMA guidelines, we reviewed 215 hyperscanning studies, across four different brain imaging modalities: functional near-infrared spectroscopy (fNIRS), functional magnetic resonance (fMRI), electroencephalography (EEG) and magnetoencephalography (MEG). Overall, the review identified a total of 27 different methods used to compute IBC. The most common hyperscanning modality is fNIRS, used by 119 studies, 89 of which adopted wavelet coherence. Based on the results of this literature survey, we first report summary statistics of the hyperscanning field, followed by a brief overview of each signal that is obtained from each neuroimaging modality used in hyperscanning. We then discuss the rationale, assumptions and suitability of each method to different modalities which can be used to investigate IBC. Finally, we discuss issues surrounding the interpretation of each method.

Brightening the Study of Listening Effort with Functional Near-Infrared Spectroscopy: A Scoping Review

Listening effort is a long-standing area of interest in auditory cognitive neuroscience. Prior research has used multiple techniques to shed light on the neurophysiological mechanisms underlying listening during challenging conditions. Functional near-infrared spectroscopy (fNIRS) is growing in popularity as a tool for cognitive neuroscience research, and its recent advances offer many potential advantages over other neuroimaging modalities for research related to listening effort. This review introduces the basic science of fNIRS and its uses for auditory cognitive neuroscience. We also discuss its application in recently published studies on listening effort and consider future opportunities for studying effortful listening with fNIRS. After reading this article, the learner will know how fNIRS works and summarize its uses for listening effort research. The learner will also be able to apply this knowledge toward generation of future research in this area.

Deep-learning informed Kalman filtering for priori-free and real-time hemodynamics extraction in functional near-infrared spectroscopy

Separation of the physiological interferences and the neural hemodynamics has been a vitally important task in the realistic implementation of functional near-infrared spectroscopy (fNIRS). Although many efforts have been devoted, the established solutions to this issue additionally rely on priori information on the interferences and activation responses, such as time-frequency characteristics and spatial patterns, etc., also hindering the realization of real-time. To tackle the adversity, we herein propose a novel priori-free scheme for real-time physiological interference suppression. This method combines the robustness of deep-leaning-based interference characterization and adaptivity of Kalman filtering: a long short-term memory (LSTM) network is trained with the time-courses of the absorption perturbation baseline for interferences profiling, and successively, a Kalman filtering process is applied with reference to the noise prediction for real-time activation extraction. The proposed method is validated using both simulated dynamic data and in-vivo experiments, showing the comprehensively improved performance and promisingly appended superiority achieved in the purely data-driven way.

Speaker-listener neural coupling reveals a right-lateralized mechanism for non-native speech-in-noise comprehension

While the increasingly globalized world has brought more and more demands for non-native language communication, the prevalence of background noise in everyday life poses a great challenge to non-native speech comprehension. The present study employed an interbrain approach based on functional near-infrared spectroscopy (fNIRS) to explore how people adapt to comprehend non-native speech information in noise. A group of Korean participants who acquired Chinese as their non-native language was invited to listen to Chinese narratives at 4 noise levels (no noise, 2 dB, -6 dB, and - 9 dB). These narratives were real-life stories spoken by native Chinese speakers. Processing of the non-native speech was associated with significant fNIRS-based listener-speaker neural couplings mainly over the right hemisphere at both the listener's and the speaker's sides. More importantly, the neural couplings from the listener's right superior temporal gyrus, the right middle temporal gyrus, as well as the right postcentral gyrus were found to be positively correlated with their individual comprehension performance at the strongest noise level (-9 dB). These results provide interbrain evidence in support of the right-lateralized mechanism for non-native speech processing and suggest that both an auditory-based and a sensorimotor-based mechanism contributed to the non-native speech-in-noise comprehension.

Lawrence K, Tachtsidis I, Tong Y, Torricelli A, Urner T, Wabnitz H, Wolf M, Wolf U, Xu S, Yang C, Yodh AG, Yücel MA, Zhou W. Optical imaging and spectroscopy for the study of the human brain: status report

This report is the second part of a comprehensive two-part series aimed at reviewing an extensive and diverse toolkit of novel methods to explore brain health and function. While the first report focused on neurophotonic tools mostly applicable to animal studies, here, we highlight optical spectroscopy and imaging methods relevant to noninvasive human brain studies. We outline current state-of-the-art technologies and software advances, explore the most recent impact of these technologies on neuroscience and clinical applications, identify the areas where innovation is needed, and provide an outlook for the future directions.

Reproducible Inter-Personal Brain Coupling Measurements in Hyperscanning Settings With functional Near Infra-Red Spectroscopy

Despite a huge advancement in neuroimaging techniques and growing importance of inter-personal brain research, few studies assess the most appropriate computational methods to measure brain-brain coupling. Here, we focus on the signal processing methods to detect brain-coupling in dyads. From a public dataset of functional Near Infra-Red Spectroscopy signals (N=24 dyads), we derived a synthetic control condition by randomization, we investigated the effectiveness of four most used signal similarity metrics: Cross Correlation, Mutual Information, Wavelet Coherence and Dynamic Time Warping. We also accounted for temporal variations between signals by allowing for misalignments up to a maximum lag. Starting from the observed effect sizes, computed in terms of Cohen's d, the power analysis indicated that a high sample size ([Formula: see text]) would be required to detect significant brain-coupling. We therefore discuss the need for specialized statistical approaches and propose bootstrap as an alternative method to avoid over-penalizing the results. In our settings, and based on bootstrap analyses, Cross Correlation and Dynamic Time Warping outperform Mutual Information and Wavelet Coherence for all considered maximum lags, with reproducible results. These results highlight the need to set specific guidelines as the high degree of customization of the signal processing procedures prevents the comparability between studies, their reproducibility and, ultimately, undermines the possibility of extracting new knowledge.

Cross-validated Adaboost Classification of Emotion Regulation Strategies Identified by Spectral Coherence in Resting-State

In the present study, quantitative relations between Cognitive Emotion Regulation strategies (CERs) and EEG synchronization levels have been investigated for the first time. For this purpose, spectral coherence (COH), phase locking value and mutual information have been applied to short segments of 62-channel resting state eyes-opened EEG data collected from healthy adults who use contrasting emotion regulation strategies (frequently and rarely use of rumination&distraction, frequently and rarely use of suppression&reappraisal). In tests, the individuals are grouped depending on their self-responses to both emotion regulation questionnaire (ERQ) and cognitive ERQ. Experimental data are downloaded from publicly available data-base, LEMON. Regarding EEG electrode pairs that placed on right and left cortical regions, inter-hemispheric dependency measures are computed for non-overlapped short segments of 2 sec at 2 min duration trials. In addition to full-band EEG analysis, dependency metrics are also obtained for both alpha and beta sub-bands. The contrasting groups are discriminated from each other with respect to the corresponding features using cross-validated adaboost classifiers. High classification accuracies (CA) of 99.44% and 98.33% have been obtained through instant classification driven by full-band COH estimations. Considering regional features that provide the high CA, CERs are found to be highly relevant with associative memory functions and cognition. The new findings may indicate the close relation between neuroplasticity and cognitive skills.

An Approach to Neuroimaging Interpersonal Interactions in Mental Health Interventions

BACKGROUND

Conventional paradigms in clinical neuroscience tend to be constrained in terms of ecological validity, raising several challenges to studying the mechanisms mediating treatments and outcomes in clinical settings. Addressing these issues requires real-world neuroimaging techniques that are capable of continuously collecting data during free-flowing interpersonal interactions and that allow for experimental designs that are representative of the clinical situations in which they occur.

METHODS

In this work, we developed a paradigm that fractionates the major components of human-to-human verbal interactions occurring in clinical situations and used functional near-infrared spectroscopy to assess the brain systems underlying clinician-client discourse (N = 30).

RESULTS

Cross-brain neural coupling between people was significantly greater during clinical interactions compared with everyday life verbal communication, particularly between the prefrontal cortex (e.g., inferior frontal gyrus) and inferior parietal lobule (e.g., supramarginal gyrus). The clinical tasks revealed extensive increases in activity across the prefrontal cortex, especially in the rostral prefrontal cortex (area 10), during periods in which participants were required to silently reason about the dysfunctional cognitions of the other person.

CONCLUSIONS

This work demonstrates a novel experimental approach to investigating the neural underpinnings of interpersonal interactions that typically occur in clinical settings, and its findings support the idea that particular prefrontal systems might be critical to cultivating mental health.

Effects of Hemodynamic Differences on the Assessment of Inter-Brain Synchrony Between Adults and Infants

The simultaneous recording of brain activity in two or more people, termed hyperscanning, is an emerging field of research investigating the neural basis of social interaction. Hyperscanning studies of adult–infant dyads (e.g., parent and infant) have great potential to provide insights into how social functions develop. In particular, taking advantage of functional near-infrared spectroscopy (fNIRS) for its spatial resolution and invulnerability to motion artifacts, adult–infant fNIRS may play a major role in this field. However, there remains a problem in analyzing hyperscanning data between adult and young populations. Namely, there are intrinsic differences in hemodynamic time latencies depending on age, and the peak latency of the hemodynamic response function (HRF) is longer in younger populations. Despite this fact, the effects of such differences on quantified synchrony have not yet been examined. Consequently, the present study investigated the influence of intrinsic hemodynamic differences on wavelet coherence for assessing brain synchrony, and further examined the statistical removal of these effects through simulation experiments. First, we assumed a social signal model, where one counterpart of the dyad (e.g., infant) sends a social signal to the other (e.g., parent), which eventually results in simultaneous brain activation. Based on this model, simulated fNIRS activation sequences were synthesized by convolving boxcar event sequences with HRFs. We set two conditions for the event: synchronized and asynchronized event conditions. We also modeled the HRFs of adults and infants by referring to previous studies. After preprocessing with additional statistical processing, we calculated the wavelet coherence for each synthesized fNIRS activation sequence pair. The simulation results showed that the wavelet coherence in the synchronized event condition was attenuated for the combination of different HRFs. We also confirmed that prewhitening via an autoregressive filter could recover the attenuation of wavelet coherence in the 0.03–0.1 Hz frequency band, which was regarded as being associated with synchronous neural activity. Our results showed that variability in hemodynamics affected the analysis of inter-brain synchrony, and that the application of prewhitening is critical for such evaluations between adult and young populations.

Future Applications of Real-World Neuroimaging to Clinical Psychology

Clinical neuroimaging has largely been limited to examining the neurophysiological outcomes of treatments for psychiatric conditions rather than the neurocognitive mechanisms by which these outcomes are brought about as a function of clinical strategies, and the cognitive neuroscientific research aiming to investigate these mechanisms in nonclinical and clinical populations has been ecologically challenged by the extent to which tasks represent and generalize to intervention strategies. However, recent technological and methodological advancements to neuroimaging techniques such as functional near-infrared spectroscopy and functional near-infrared spectroscopy-based hyperscanning provide novel opportunities to investigate the mechanisms of change in more naturalistic and interactive settings, representing a unique prospect for improving our understanding of the intra- and interbrain systems supporting the recogitation of dysfunctional cognitive operations.

Functional Brain Network Analysis of Knowledge Transfer While Engineering Problem-Solving

As a complex cognitive activity, knowledge transfer is mostly correlated to cognitive processes such as working memory, behavior control, and decision-making in the human brain while engineering problem-solving. It is crucial to explain how the alteration of the functional brain network occurs and how to express it, which causes the alteration of the cognitive structure of knowledge transfer. However, the neurophysiological mechanisms of knowledge transfer are rarely considered in existing studies. Thus, this study proposed functional connectivity (FC) to describe and evaluate the dynamic brain network of knowledge transfer while engineering problem-solving. In this study, we adopted the modified Wisconsin Card-Sorting Test (M-WCST) reported in the literature. The neural activation of the prefrontal cortex was continuously recorded for 31 participants using functional near-infrared spectroscopy (fNIRS). Concretely, we discussed the prior cognitive level, knowledge transfer distance, and transfer performance impacting the wavelet amplitude and wavelet phase coherence. The paired t-test results showed that the prior cognitive level and transfer distance significantly impact FC. The Pearson correlation coefficient showed that both wavelet amplitude and phase coherence are significantly correlated to the cognitive function of the prefrontal cortex. Therefore, brain FC is an available method to evaluate cognitive structure alteration in knowledge transfer. We also discussed why the dorsolateral prefrontal cortex (DLPFC) and occipital face area (OFA) distinguish themselves from the other brain areas in the M-WCST experiment. As an exploratory study in NeuroManagement, these findings may provide neurophysiological evidence about the functional brain network of knowledge transfer while engineering problem-solving.

Dynamic inter-brain synchrony in real-life inter-personal cooperation: A functional near-infrared spectroscopy hyperscanning study

How two brains communicate with each other during social interaction is highly dynamic and complex. Multi-person (i.e., hyperscanning) studies to date have focused on analyzing the entire time series of brain signals to reveal an overall pattern of inter-brain synchrony (IBS). However, this approach does not account for the dynamic nature of social interaction. In the present study, we propose a data-driven approach based on sliding windows and k-mean clustering to capture the dynamic modulation of IBS patterns during interactive cooperation tasks. We used a portable functional near-infrared spectroscopy (fNIRS) system to measure brain hemodynamic response between interacting partners (20 dyads) engaged in a creative design task and a 3D model building task. Results indicated that inter-personal communication during naturalistic cooperation generally presented with a series of dynamic IBS states along the tasks. Compared to the model building task, the creative design task appeared to involve more complex and active IBS between multiple regions in specific dynamic IBS states. In summary, the proposed approach stands as a promising tool to distill complex inter-brain dynamics associated with social interaction into a set of representative brain states with more fine-grained temporal resolution. This approach holds promise for advancing our current understanding of the dynamic nature of neurocognitive processes underlying social interaction.

Interpersonal Agreement and Disagreement During Face-to-Face Dialogue: An fNIRS Investigation

Although the neural systems that underlie spoken language are well-known, how they adapt to evolving social cues during natural conversations remains an unanswered question. In this work we investigate the neural correlates of face-to-face conversations between two individuals using functional near infrared spectroscopy (fNIRS) and acoustical analyses of concurrent audio recordings. Nineteen pairs of healthy adults engaged in live discussions on two controversial topics where their opinions were either in agreement or disagreement. Participants were matched according to their a priori opinions on these topics as assessed by questionnaire. Acoustic measures of the recorded speech including the fundamental frequency range, median fundamental frequency, syllable rate, and acoustic energy were elevated during disagreement relative to agreement. Consistent with both the a priori opinion ratings and the acoustic findings, neural activity associated with long-range functional networks, rather than the canonical language areas, was also differentiated by the two conditions. Specifically, the frontoparietal system including bilateral dorsolateral prefrontal cortex, left supramarginal gyrus, angular gyrus, and superior temporal gyrus showed increased activity while talking during disagreement. In contrast, talking during agreement was characterized by increased activity in a social and attention network including right supramarginal gyrus, bilateral frontal eye-fields, and left frontopolar regions. Further, these social and visual attention networks were more synchronous across brains during agreement than disagreement. Rather than localized modulation of the canonical language system, these findings are most consistent with a model of distributed and adaptive language-related processes including cross-brain neural coupling that serves dynamic verbal exchanges.

Capturing Human Interaction in the Virtual Age: A Perspective on the Future of fNIRS Hyperscanning

Advances in video conferencing capabilities combined with dramatic socio-dynamic shifts brought about by COVID-19, have redefined the ways in which humans interact in modern society. From business meetings to medical exams, or from classroom instruction to yoga class, virtual interfacing has permeated nearly every aspect of our daily lives. A seemingly endless stream of technological advances combined with our newfound reliance on virtual interfacing makes it likely that humans will continue to use this modern form of social interaction into the future. However, emergent evidence suggests that virtual interfacing may not be equivalent to face-to-face interactions. Ultimately, too little is currently understood about the mechanisms that underlie human interactions over the virtual divide, including how these mechanisms differ from traditional face-to-face interaction. Here, we propose functional near-infrared spectroscopy (fNIRS) hyperscanning—simultaneous measurement of two or more brains—as an optimal approach to quantify potential neurocognitive differences between virtual and in-person interactions. We argue that increased focus on this understudied domain will help elucidate the reasons why virtual conferencing doesn't always stack up to in-person meetings and will also serve to spur new technologies designed to improve the virtual interaction experience. On the basis of existing fNIRS hyperscanning literature, we highlight the current gaps in research regarding virtual interactions. Furthermore, we provide insight into current hurdles regarding fNIRS hyperscanning hardware and methodology that should be addressed in order to shed light on this newly critical element of everyday life.

Joint Attention During Live Person-to-Person Contact Activates rTPJ, Including a Sub-Component Associated With Spontaneous Eye-to-Eye Contact

Eye-to-eye contact is a spontaneous behavior between interacting partners that occurs naturally during social interactions. However, individuals differ with respect to eye gaze behaviors such as frequency of eye-to-eye contacts, and these variations may reflect underlying differences in social behavior in the population. While the use of eye signaling to indicate a shared object of attention in joint attention tasks has been well-studied, the effects of the natural variation in establishing eye contact during joint attention have not been isolated. Here, we investigate this question using a novel two-person joint attention task. Participants were not instructed regarding the use of eye contacts; thus all mutual eye contact events between interacting partners that occurred during the joint attention task were spontaneous and varied with respect to frequency. We predicted that joint attention systems would be modulated by differences in the social behavior across participant pairs, which could be measured by the frequency of eye contact behavior. We used functional near-infrared spectroscopy (fNIRS) hyperscanning and eye-tracking to measure the neural signals associated with joint attention in interacting dyads and to record the number of eye contact events between them. Participants engaged in a social joint attention task in which real partners used eye gaze to direct each other's attention to specific targets. Findings were compared to a non-social joint attention task in which an LED cue directed both partners' attention to the same target. The social joint attention condition showed greater activity in right temporoparietal junction than the non-social condition, replicating prior joint attention results. Eye-contact frequency modulated the joint attention activity, revealing bilateral activity in social and high level visual areas associated with partners who made more eye contact. Additionally, when the number of mutual eye contact events was used to classify each pair as either "high eye contact" or "low eye contact" dyads, cross-brain coherence analysis revealed greater coherence between high eye contact dyads than low eye contact dyads in these same areas. Together, findings suggest that variation in social behavior as measured by eye contact modulates activity in a subunit of the network associated with joint attention.