Co-localization of theta-band activity and hemodynamic responses during face perception: simultaneous electroencephalography and functional near-infrared spectroscopy recordings

A comparative analysis of face and object perception in 2D laboratory and virtual reality settings: insights from induced oscillatory responses

In psychophysiological research, the use of Virtual Reality (VR) for stimulus presentation allows for the investigation of how perceptual processing adapts to varying degrees of realism. Previous time-domain studies have shown that perceptual processing involves modality-specific neural mechanisms, as evidenced by distinct stimulus-locked components. Analyzing induced oscillations across different frequency bands can provide further insights into neural processes that are not strictly phase-locked to stimulus onset. This study uses a simple perceptual paradigm presenting images of faces and cars on both a standard 2D monitor and in an immersive VR environment. To investigate potential modality-dependent differences in attention, cognitive load, and task-related post-movement processing, the induced alpha, theta and beta band responses are compared between the two modalities. No evidence was found for differences in stimulus-dependent attention or task-related post-movement processing between the 2D conditions and the realistic virtual conditions in electrode space, as posterior alpha suppression and re-synchronization of centro-parietal beta did not differ between conditions. However, source analysis revealed differences in the attention networks engaged during 2D and 3D perception. Midfrontal theta was significantly stronger in laboratory conditions, indicating higher cognitive load than in the VR environment. Exploratory analysis of posterior theta showed stronger responses in VR, possibly reflecting the processing of depth information provided only by the 3D material. In addition, the theta response seems to be generated by distinct neuronal sources under realistic virtual conditions indicating enhanced involvement of semantic information processing and social cognition.

Deep Multimodal Representations and Classification of First-Episode Psychosis via Live Face Processing

Schizophrenia is a severe psychiatric disorder associated with a wide range of cognitive and neurophysiological dysfunctions and long-term social difficulties. In this paper, we test the hypothesis that integration of multiple simultaneous acquisitions of neuroimaging, behavioral, and clinical information will be better for prediction of early psychosis than unimodal recordings. We propose a novel framework to investigate the neural underpinnings of the early psychosis symptoms (that can develop into Schizophrenia with age) using multimodal acquisitions of neural and behavioral recordings including functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), and facial features. Our data acquisition paradigm is based on live face-to-face interaction in order to study the neural correlates of social cognition in first-episode psychosis (FEP). We propose a novel deep representation learning framework, Neural-PRISM, for learning joint multimodal compressed representations combining neural as well as behavioral recordings. These learned representations are subsequently used to describe, classify, and predict the severity of early psychosis in patients, as measured by the Positive and Negative Syndrome Scale (PANSS) and Global Assessment of Functioning (GAF) scores. We found that incorporating joint multimodal representations from fNIRS and EEG along with behavioral recordings enhances classification between typical controls and FEP individuals. Additionally, our results suggest that geometric and topological features such as curvatures and path signatures of the embedded trajectories of brain activity enable detection of discriminatory neural characteristics in early psychosis.

Neural dynamics during emotional video engagement relate to anxiety

Inter-subject correlations (ISCs) of physiological data can reveal common stimulus-driven processing across subjects. ISC has been applied to passive video viewing in small samples to measure common engagement and emotional processing. Here, in a large sample study of healthy adults (N = 163) who watched an emotional film (The Lion Cage by Charlie Chaplin), we recorded electroencephalography (EEG) across participants and measured ISC in theta, alpha and beta frequency bands. Peak ISC on the emotionally engaging video was observed three-quarters into the film clip, during a time period which potentially elicited a positive emotion of relief. Peak ISC in all frequency bands was focused over centro-parietal electrodes localizing to superior parietal cortex. ISC in both alpha and beta frequencies had a significant inverse relationship with anxiety symptoms. Our study suggests that ISC measured during continuous non-event-locked passive viewing may serve as a useful marker for anxious mood.

Lawrence K, Tachtsidis I, Tong Y, Torricelli A, Urner T, Wabnitz H, Wolf M, Wolf U, Xu S, Yang C, Yodh AG, Yücel MA, Zhou W. Optical imaging and spectroscopy for the study of the human brain: status report

This report is the second part of a comprehensive two-part series aimed at reviewing an extensive and diverse toolkit of novel methods to explore brain health and function. While the first report focused on neurophotonic tools mostly applicable to animal studies, here, we highlight optical spectroscopy and imaging methods relevant to noninvasive human brain studies. We outline current state-of-the-art technologies and software advances, explore the most recent impact of these technologies on neuroscience and clinical applications, identify the areas where innovation is needed, and provide an outlook for the future directions.

Construction of a fiber-optically connected MEG hyperscanning system for recording brain activity during real-time communication

Communication is one of the most important abilities in human society, which makes clarification of brain functions that underlie communication of great importance to cognitive neuroscience. To investigate the rapidly changing cortical-level brain activity underlying communication, a hyperscanning system with both high temporal and spatial resolution is extremely desirable. The modality of magnetoencephalography (MEG) would be ideal, but MEG hyperscanning systems suitable for communication studies remain rare. Here, we report the establishment of an MEG hyperscanning system that is optimized for natural, real-time, face-to-face communication between two adults in sitting positions. Two MEG systems, which are installed 500m away from each other, were directly connected with fiber optic cables. The number of intermediate devices was minimized, enabling transmission of trigger and auditory signals with almost no delay (1.95-3.90 μs and 3 ms, respectively). Additionally, video signals were transmitted at the lowest latency ever reported (60-100 ms). We furthermore verified the function of an auditory delay line to synchronize the audio with the video signals. This system is thus optimized for natural face-to-face communication, and additionally, music-based communication which requires higher temporal accuracy is also possible via audio-only transmission. Owing to the high temporal and spatial resolution of MEG, our system offers a unique advantage over existing hyperscanning modalities of EEG, fNIRS, or fMRI. It provides novel neuroscientific methodology to investigate communication and other forms of social interaction, and could potentially aid in the development of novel medications or interventions for communication disorders.

Racial Ingroup Bias and Efficiency Consideration Influence Distributive Decisions: A Dynamic Analysis of Time Domain and Time Frequency

Although previous studies have demonstrated that identity had effect on justice norms and behavioral decisions, the neural mechanism of that effect remains unclear. In this study, the subjects made their distributive decisions on the trade-off between equity and efficiency among Chinese and foreign children and their scalp potentials were recorded. Behavioral results showed that efficiency consideration played an important part in the distribution task. Meanwhile, participants gave preferential treatment to same-race children. Relative to the distribution within ingroup children, the distribution involving outgroup children induced higher N170 amplitude. The distribution involving outgroup children also elicited weakened P300 amplitude and enhanced delta response than the distribution within ingroup children when subjects are facing the conflict between equality and efficiency. In other words, ingroup bias affected the neural process of the trade-off between equality and efficiency. The combination of time-domain and time-frequency analyses provided spatiotemporal and spectral results for a better understanding of racial ingroup favoritism on distributive justice.

Event-related EEG oscillatory responses elicited by dynamic facial expression

BACKGROUND

Recognition of facial expressions (FEs) plays a crucial role in social interactions. Most studies on FE recognition use static (image) stimuli, even though real-life FEs are dynamic. FE processing is complex and multifaceted, and its neural correlates remain unclear. Transitioning from static to dynamic FE stimuli might help disentangle the neural oscillatory mechanisms underlying face processing and recognition of emotion expression. To our knowledge, we here present the first time-frequency exploration of oscillatory brain mechanisms underlying the processing of dynamic FEs.

RESULTS

Videos of joyful, fearful, and neutral dynamic facial expressions were presented to 18 included healthy young adults. We analyzed event-related activity in electroencephalography (EEG) data, focusing on the delta, theta, and alpha-band oscillations. Since the videos involved a transition from neutral to emotional expressions (onset around 500 ms), we identified time windows that might correspond to face perception initially (time window 1; first TW), and emotion expression recognition subsequently (around 1000 ms; second TW). First TW showed increased power and phase-locking values for all frequency bands. In the first TW, power and phase-locking values were higher in the delta and theta bands for emotional FEs as compared to neutral FEs, thus potentially serving as a marker for emotion recognition in dynamic face processing.

CONCLUSIONS

Our time-frequency exploration revealed consistent oscillatory responses to complex, dynamic, ecologically meaningful FE stimuli. We conclude that while dynamic FE processing involves complex network dynamics, dynamic FEs were successfully used to reveal temporally separate oscillation responses related to face processing and subsequently emotion expression recognition.

Head-Down Tilt Position, but Not the Duration of Bed Rest Affects Resting State Electrocortical Activity

Adverse cognitive and behavioral conditions and psychiatric disorders are considered a critical and unmitigated risk during future long-duration space missions (LDSM). Monitoring and mitigating crew health and performance risks during these missions will require tools and technologies that allow to reliably assess cognitive performance and mental well-being. Electroencephalography (EEG) has the potential to meet the technical requirements for the non-invasive and objective monitoring of neurobehavioral conditions during LDSM. Weightlessness is associated with fluid and brain shifts, and these effects could potentially challenge the interpretation of resting state EEG recordings. Head-down tilt bed rest (HDBR) provides a unique spaceflight analog to study these effects on Earth. Here, we present data from two long-duration HDBR experiments, which were used to systematically investigate the time course of resting state electrocortical activity during prolonged HDBR. EEG spectral power significantly reduced within the delta, theta, alpha, and beta frequency bands. Likewise, EEG source localization revealed significantly lower activity in a broad range of centroparietal and occipital areas within the alpha and beta frequency domains. These changes were observed shortly after the onset of HDBR, did not change throughout HDBR, and returned to baseline after the cessation of bed rest. EEG resting state functional connectivity was not affected by HDBR. The results provide evidence for a postural effect on resting state brain activity that persists throughout long-duration HDBR, indicating that immobilization and inactivity per se do not affect resting state electrocortical activity during HDBR. Our findings raise an important issue on the validity of EEG to identify the time course of changes in brain function during prolonged HBDR, and highlight the importance to maintain a consistent body posture during all testing sessions, including data collections at baseline and recovery.

Comparison of Human Social Brain Activity During Eye-Contact With Another Human and a Humanoid Robot

Robot design to simulate interpersonal social interaction is an active area of research with applications in therapy and companionship. Neural responses to eye-to-eye contact in humans have recently been employed to determine the neural systems that are active during social interactions. Whether eye-contact with a social robot engages the same neural system remains to be seen. Here, we employ a similar approach to compare human-human and human-robot social interactions. We assume that if human-human and human-robot eye-contact elicit similar neural activity in the human, then the perceptual and cognitive processing is also the same for human and robot. That is, the robot is processed similar to the human. However, if neural effects are different, then perceptual and cognitive processing is assumed to be different. In this study neural activity was compared for human-to-human and human-to-robot conditions using near infrared spectroscopy for neural imaging, and a robot (Maki) with eyes that blink and move right and left. Eye-contact was confirmed by eye-tracking for both conditions. Increased neural activity was observed in human social systems including the right temporal parietal junction and the dorsolateral prefrontal cortex during human-human eye contact but not human-robot eye-contact. This suggests that the type of human-robot eye-contact used here is not sufficient to engage the right temporoparietal junction in the human. This study establishes a foundation for future research into human-robot eye-contact to determine how elements of robot design and behavior impact human social processing within this type of interaction and may offer a method for capturing difficult to quantify components of human-robot interaction, such as social engagement.