Classifying different emotional states by means of EEG-based functional connectivity patterns

The promoting effect of the absence of second-party's punishment power on third-party punishment in maintaining social fairness norms: An EEG hyper-scanning study

Third-party punishment (TPP) plays an irreplaceable role in maintaining social fairness. Punishment power is a significant area of study within economic games. However, the impact of whether or not the second-party possesses punishment power on TPP remains unexplored. The present study utilizes the high temporal resolution of EEG and time-frequency analysis, intra-barin functional connectivity analysis, inter-brain synchronization (IBS) analysis, and granger causality analysis(GCA) to comprehensively explore the neural mechanism of TPP from the perspective of third-party individual's decision-making and IBS in the real-time social interaction. Time-frequency results found that, the absence of the punishment power activated more theta-band and alpha-band power compare to when second-party has punishment power. When second-party has no punishment power, functional connection results observed stronger functional connectivity in theta band for medium unfair offers between rTPJ and PFC. Dual-brain analysis revealed that when the second-party has no punishment power, there is a significantly higher IBS in the alpha band between the frontal and frontal-central lobes of the second-party and the parietal and parietal occipital lobes of the third-party. GCA results further showed that the direction of IBS from third-party to second-party was significantly stronger than from second-party to third-party. This study demonstrates that the absence of the second-party's punishment power promote TPP, and similar cognitive process of thinking on how to maintain social fairness enhances IBS. The current study emphasizes the influence of punishment power on TPP, broadens the research perspective and contributes crucial insights into maintain social fairness.

Classification of mindfulness experiences from gamma-band effective connectivity: Application of machine-learning algorithms on resting, breathing, and body scan

BACKGROUND AND OBJECTIVE

Practicing mindfulness is a mental process toward interoceptive awareness, achieving stress reduction and emotion regulation through brain-function alteration. Literature has shown that electroencephalography (EEG)-derived connectivity possesses the potential to differentiate brain functions between mindfulness naïve and mindfulness experienced, where such quantitative differentiation could benefit telediagnosis for mental health. However, there is no prior guidance in model selection targeting on the mindfulness-experience prediction. Here we hypothesized that the EEG effective connectivity could reach a good prediction performance in mindfulness experiences with brain interpretability.

METHODS

We aimed at probing direct Directed Transfer Function (dDTF) to classify the participants' history of mindfulness-based stress reduction (MBSR), and aimed at optimizing the prediction accuracy by comparing multiple machine learning (ML) algorithms. Targeting the gamma-band effective connectivity, we evaluated the EEG-based prediction of the mindfulness experiences across 7 machine learning (ML) algorithms and 3 sessions (i.e., resting, focus-breathing, and body-scan).

RESULTS

The support vector machine and naïve Bayes classifiers exhibited significant accuracies above the chance level across all three sessions, and the decision tree algorithm reached the highest prediction accuracy of 91.7 % with the resting state, compared to the classification accuracies with the other two mindful states. We further conducted the analysis on essential EEG channels to preserve the classification accuracy, revealing that preserving just four channels (F7, F8, T7, and P7) out of 19 yielded the accuracy of 83.3 %. Delving into the contribution of connectivity features, specific connectivity features predominantly located in the frontal lobe contributed more to classifier construction, which aligned well with the existing mindfulness literature.

CONCLUSION

In the present study, we initiated a milestone of developing an EEG-based classifier to detect a person's mindfulness experience objectively. The prediction accuracy of the decision tree was optimal to differentiate the mindfulness experiences using the local resting-state EEG data. The suggested algorithm and key channels on the mindfulness-experience prediction may provide guidance for predicting mindfulness experiences using the EEG-based classification embedded in future wearable neurofeedback systems or plausible digital therapeutics.

Effective Emotion Recognition by Learning Discriminative Graph Topologies in EEG Brain Networks

Multichannel electroencephalogram (EEG) is an array signal that represents brain neural networks and can be applied to characterize information propagation patterns for different emotional states. To reveal these inherent spatial graph features and increase the stability of emotion recognition, we propose an effective emotion recognition model that performs multicategory emotion recognition with multiple emotion-related spatial network topology patterns (MESNPs) by learning discriminative graph topologies in EEG brain networks. To evaluate the performance of our proposed MESNP model, we conducted single-subject and multisubject four-class classification experiments on two public datasets, MAHNOB-HCI and DEAP. Compared with existing feature extraction methods, the MESNP model significantly enhances the multiclass emotional classification performance in the single-subject and multisubject conditions. To evaluate the online version of the proposed MESNP model, we designed an online emotion monitoring system. We recruited 14 participants to conduct the online emotion decoding experiments. The average online experimental accuracy of the 14 participants was 84.56%, indicating that our model can be applied in affective brain-computer interface (aBCI) systems. The offline and online experimental results demonstrate that the proposed MESNP model effectively captures discriminative graph topology patterns and significantly improves emotion classification performance. Moreover, the proposed MESNP model provides a new scheme for extracting features from strongly coupled array signals.

EEG based functional connectivity in resting and emotional states may identify major depressive disorder using machine learning

OBJECTIVE

Disrupted brain network connectivity underlies major depressive disorder (MDD). Altered EEG based Functional connectivity (FC) with Emotional stimuli in major depressive disorder (MDD) in addition to resting state FC may help in improving the diagnostic accuracy of machine learning classification models. We explored the potential of EEG-based FC during resting state and emotional processing, for diagnosing MDD using machine learning approach.

METHODS

EEG was recorded during resting state and while watching emotionally contagious happy and sad videos in 24 drug-naïve MDD patients and 25 healthy controls. FC was quantified using the Phase Lag Index. Three Random Forest classifier models were constructed to classify MDD patients and healthy controls, Model-I incorporating FC features from the resting state and Model-II and Model-III incorporating FC features while watching happy and sad videos respectively.

RESULTS

Important features distinguishing MDD and healthy controls were from all frequency bands and represent functional connectivity between fronto-temporal, fronto-parietal and fronto occipital regions. The cross-validation accuracies for Model-I, Model-II and Model-III were 92.3%, 94.9% and 89.7% and test accuracies were 60%, 80% and 70% respectively. Incorporating emotionally contagious videos improved the classification accuracies.

CONCLUSION

Findings support EEG FC patterns during resting state and emotional processing along with machine learning can be used to diagnose MDD. Future research should focus on replicating and validating these results.

SIGNIFICANCE

EEG FC pattern combined with machine learning may be used for assisting in diagnosing MDD.

The paradox of social avoidance and the yearning for understanding: Elevated interbrain synchrony among socially avoidant individuals during expression of negative emotions

Social avoidance refers to the tendency to be alone and non-participating to social interactions, which is considered to hamper health interpersonal relationship. However, the neural underpinnings of social and emotional interactions among social avoidant individuals have not been fully studied. In the present study, we used EEG hyperscanning technology to investigate the brain activity and its synchronization of 25 socially avoidant dyads and 28 comparison dyads during an emotional communication task. The emotional communication task consisted of the emotional processing stage and emotional interaction stage. Event-related potentials (ERPs) of the senders during the emotional processing stage and the interbrain synchrony (IBS) of the dyads during the emotional interaction stage were analyzed. Results showed that (1) socially avoidant group showed higher beta, theta and gamma IBS in the negative condition than in the positive and neutral condition; (2) in positive condition, the N1 and LPP amplitudes during the emotional processing stage of socially avoidant individuals were negatively correlated with the IBS within dyads during the emotional communication stage. The findings suggest that the dysfunctional emotional interaction of social avoidant individuals may be attributed to the negative impact of emotional stimuli processing during emotional communication.

Moving toward reality: Electrocortical reactivity to naturalistic multimodal emotional videos

While previous research has investigated the effects of emotional videos on peripheral physiological measures and conscious experience, this study extends the research to include electrocortical measures, specifically the steady-state visual-evoked potential (ssVEP). A carefully curated set of 45 videos, designed to represent a wide range of emotional and neutral content, were presented with a flickering border. The videos featured a continuous single-shot perspective, natural soundtrack, and excluded elements associated with professional films, to enhance realism. The results demonstrate a consistent reduction in ssVEP amplitude during emotional videos which strongly correlates with the rated emotional intensity of the clips. This suggests that narrative audiovisual stimuli have the potential to track dynamic emotional processing in the cortex, providing new avenues for research in affective neuroscience. The findings highlight the potential of using realistic video stimuli to investigate how the human brain processes emotional events in a paradigm that increases ecological validity. Future studies can further develop this paradigm by expanding the video set, targeting specific cortical networks, and manipulating narrative predictability. Overall, this study establishes a foundation for investigating emotional perception using realistic video stimuli and has the potential to expand our understanding of real-world emotional processing in the human brain.

EEG-based emotion recognition systems; comprehensive study

Emotion recognition technology through EEG signal analysis is currently a fundamental concept in artificial intelligence. This recognition has major practical implications in emotional health care, human-computer interaction, and so on. This paper provides a comprehensive study of different methods for extracting electroencephalography (EEG) features for emotion recognition from four different perspectives, including time domain features, frequency domain features, time-frequency features, and nonlinear features. We summarize the current pattern recognition methods adopted in most related works, and with the rapid development of deep learning (DL) attracting the attention of researchers in this field, we pay more attention to deep learning-based studies and analyse the characteristics, advantages, disadvantages, and applicable scenarios. Finally, the current challenges and future development directions in this field were summarized. This paper can help novice researchers in this field gain a systematic understanding of the current status of emotion recognition research based on EEG signals and provide ideas for subsequent related research.

Identification of Optimal and Most Significant Event Related Brain Functional Network

Advancements in network science have facilitated the study of brain communication networks. Existing techniques for identifying event-related brain functional networks (BFNs) often result in fully connected networks. However, determining the optimal and most significant network representation for event-related BFNs is crucial for understanding complex brain networks. The presence of both false and genuine connections in the fully connected network requires network thresholding to eliminate false connections. However, a generalized framework for thresholding in network neuroscience is currently lacking. To address this, we propose four novel methods that leverage network properties, energy, and efficiency to select a generalized threshold level. This threshold serves as the basis for identifying the optimal and most significant event-related BFN. We validate our methods on an openly available emotion dataset and demonstrate their effectiveness in identifying multiple events. Our proposed approach can serve as a versatile thresholding technique to represent the fully connected network as an event-related BFN.

Decoding emotion with phase-amplitude fusion features of EEG functional connectivity network

Decoding emotional neural representations from the electroencephalographic (EEG)-based functional connectivity network (FCN) is of great scientific importance for uncovering emotional cognition mechanisms and developing harmonious human-computer interactions. However, existing methods mainly rely on phase-based FCN measures (e.g., phase locking value [PLV]) to capture dynamic interactions between brain oscillations in emotional states, which fail to reflect the energy fluctuation of cortical oscillations over time. In this study, we initially examined the efficacy of amplitude-based functional networks (e.g., amplitude envelope correlation [AEC]) in representing emotional states. Subsequently, we proposed an efficient phase-amplitude fusion framework (PAF) to fuse PLV and AEC and used common spatial pattern (CSP) to extract fused spatial topological features from PAF for multi-class emotion recognition. We conducted extensive experiments on the DEAP and MAHNOB-HCI datasets. The results showed that: (1) AEC-derived discriminative spatial network topological features possess the ability to characterize emotional states, and the differential network patterns of AEC reflect dynamic interactions in brain regions associated with emotional cognition. (2) The proposed fusion features outperformed other state-of-the-art methods in terms of classification accuracy for both datasets. Moreover, the spatial filter learned from PAF is separable and interpretable, enabling a description of affective activation patterns from both phase and amplitude perspectives.

The effects of electroencephalography functional connectivity during emotional recognition among patients with major depressive disorder and healthy controls

BACKGROUND

The brain of major depressive disorder (MDD) is associated with altered functional connectivity (FC) compared to that of healthy individuals when processing positive and negative visual stimuli. Building upon alterations in brain connectivity, some researchers have employed electroencephalography (EEG) to study FC in MDD, aiming to enhance both diagnosis and treatment; however, the results have been inconsistent and the studies involving FC during emotional recognition are limited. This study aims to 1) investigate the effects of MDD on EEG patterns during visual emotional processing, 2) explore the therapeutic effects of antidepressant treatment on brain FC within the first week, and assess whether these effects can be predictive of treatment outcomes four weeks later, and 3) study baseline FC parameter biomarkers that can be used to predict treatment responsiveness in MDD patients.

METHODS

This clinical observational study recruited 38 healthy controls (HC) and 48 MDD patients. Patients underwent an EEG exam while looking at validated images of happy and sad faces at week 0 and 1. MDD patients were categorized into treatment responders and non-responders after 4 weeks of treatment. We conducted the FC analysis (node strength (NS), global efficiency (GE), and cluster coefficient (CC)) on HC and MDD patients using graph theoretical analysis. Multivariable linear regression was used to evaluate the influence of MDD on FC compared to HC, while controlling for confounding variables including age, gender, and academic degrees.

RESULTS

At week 0 and week 1, MDD patients revealed to have significant reductions in FC parameters (NS, GE and CC) compared to HC. When comparing MDD patients at week 1 post-antidepressant treatment and pre-treatment, no significant differences in FC changes were observed. Multivariable regression revealed a significant negative effect on FC of MDD. Compared to the treatment non-responsive group, the responsive group revealed a significantly higher FC in delta band frequency at baseline.

CONCLUSIONS

MDD patient group showed impaired FC during visual emotion-processing and we observed baseline FC parameters to be associated with treatment response at week 4. While signs of FC changes were observed in the brain after a week of treatment, it is possible that one week may still be insufficient to demonstrate significant alterations in the brain. Our results suggest the potential utilization of EEG-based FC as an indicative measure for predicting treatment response and monitoring treatment progress in MDD patients.

The shorter a man is, the more he defends fairness: relative height disadvantage promoting third-party punishment-evidence from inter-brain synchronization

Third-party punishment occurs in interpersonal interactions to sustain social norms, and is strongly influenced by the characteristics of the interacting individuals. During social interactions, height is the striking physical appearance features first observed, height disadvantage may critically influence men's behavior and mental health. Herein, we explored the influence of height disadvantage on third-party punishment through time-frequency analysis and electroencephalography hyperscanning. Two participants were randomly designated as the recipient and third party after height comparison and instructed to complete third-party punishment task. Compared with when the third party's height is higher than the recipient's height, when the third party's height is lower, the punishment rate and transfer amount were significantly higher. Only for highly unfair offers, the theta power was significantly greater when the third party's height was lower. The inter-brain synchronization between the recipient and the third party was significantly stronger when the third party's height was lower. Compared with the fair and medium unfair offers, the inter-brain synchronization was strongest for highly unfair offers. Our findings indicate that the height disadvantage-induced anger and reputation concern promote third-party punishment and inter-brain synchronization. This study enriches research perspective and expands the application of the theory of Napoleon complex.

Brain functional connectivity in children with a mild traumatic brain injury: A scoping review

INTRODUCTION

The occurrence of mild traumatic brain injury(mTBI) is estimated at 0,2-0,3% cases annually. Following a mTBI, some children experience persistent symptoms, and functional connectivity(FC) changes may be implicated. However, characteristics of FC have not been widely described in this population. This scoping review aimed to identify and understand the impacts of mTBI on EEG-measured FC in children, provide an overview of the available literature, detail analysis techniques, and describe gaps in the research.

METHODS

PubMed, Web of Science, Medline, Embase, ProQuest and CINAHL were searched up to June 25, 2023, with the terms child, mTBI, EEG, FC, and their synonyms. Ten studies were identified.

RESULTS

Five studies reported significant differences between the mTBI group and controls. In addition to group differences, six studies reported significant variation over time. Brain Network Analysis(BNA), utilized in seven studies, was the primary FC analysis recorded. Two of the five studies that reported significant differences following mTBI utilized the BNA. The other three applied alternative analysis methods.

DISCUSSION

FC assessment based on EEG can identify some differences in children with mTBI. BNA was more useful in following changes over time. Further research is suggested, considering the limited age range and number of retrieved studies.

Evaluation of interpretability for deep learning algorithms in EEG emotion recognition: A case study in autism

Current models on Explainable Artificial Intelligence (XAI) have shown a lack of reliability when evaluating feature-relevance for deep neural biomarker classifiers. The inclusion of reliable saliency-maps for obtaining trustworthy and interpretable neural activity is still insufficiently mature for practical applications. These limitations impede the development of clinical applications of Deep Learning. To address, these limitations we propose the RemOve-And-Retrain (ROAR) algorithm which supports the recovery of highly relevant features from any pre-trained deep neural network. In this study we evaluated the ROAR methodology and algorithm for the Face Emotion Recognition (FER) task, which is clinically applicable in the study of Autism Spectrum Disorder (ASD). We trained a Convolutional Neural Network (CNN) from electroencephalography (EEG) signals and assessed the relevance of FER-elicited EEG features from individuals diagnosed with and without ASD. Specifically, we compared the ROAR reliability from well-known relevance maps such as Layer-Wise Relevance Propagation, PatternNet, Pattern-Attribution, and Smooth-Grad Squared. This study is the first to bridge previous neuroscience and ASD research findings to feature-relevance calculation for EEG-based emotion recognition with CNN in typically-development (TD) and in ASD individuals.

Emotion detection from EEG using transfer learning

In this study, we employed transfer learning to overcome the challenge of limited data availability in EEG-based emotion detection. The base model used in this study was Resnet50. Additionally, we employed a novel feature combination in EEG-based emotion detection. The input to the model was in the form of an image matrix, which comprised Mean Phase Coherence (MPC) and Magnitude Squared Coherence (MSC) in the upper-triangular and lower-triangular matrices, respectively. We further improved the technique by incorporating features obtained from the Differential Entropy (DE) into the diagonal. The dataset used in this study, SEED EEG (62 channel EEG), comprises three classes (Positive, Neutral, and Negative). We calculated both subject-independent and subject-dependent accuracy. The subject-dependent accuracy was obtained using a 10-fold cross-validation method and was 93.1%, while the subject-independent classification was performed by employing the leave-one-subject-out (LOSO) strategy. The accuracy obtained in subject-independent classification was 71.6%. Both of these accuracies are at least twice better than the chance accuracy of classifying 3 classes. The study found the use of MSC and MPC in EEG-based emotion detection promising for emotion classification. The future scope of this work includes the use of data augmentation techniques, enhanced classifiers, and better features for emotion classification.

Moral transgression modulates fairness considerations in the ultimatum game: Evidence from ERP and EEG data

People tend to dislike and punish unfair behaviors in social interactions, and this disposition may be moderated by the characteristics of their interaction partner. We used a modified ultimatum game (UG) to investigate players' responses to fair and unfair offers from proposers described as having performed either a moral transgression or a neutral behavior, and recorded an electroencephalogram. The participants' behavior in the UG suggests that people quickly demand more fairness from proposers who have committed moral transgressions rather than neutral behavior. Event-related potentials (ERPs) revealed a significant effect of offer type and of proposer type on P300 activity. The prestimulus α-oscillation power in the neutral behavior condition was significantly lower than that in the moral transgression condition. The post-stimulus β-event-related synchronization (β-ERS) was larger for the moral transgression condition than the neutral behavior condition in response to the least fair offers, and larger for neutral behavior than the moral transgression condition in response to the fairest offers. In summary, β-ERS was influenced by both proposer type and offer type, which revealed different neural responses to the offer from either a morally transgressive or a neutral behavior proposer.

Relationships between feeding behaviors and emotions: an electroencephalogram (EEG) frequency analysis study

Feeding behaviors may be easily affected by emotions, both being based on brain activity; however, the relationships between them have not been explicitly defined. In this study, we investigated how emotional environments modulate subjective feelings, brain activity, and feeding behaviors. Electroencephalogram (EEG) recordings were obtained from healthy participants in conditions of virtual comfortable space (CS) and uncomfortable space (UCS) while eating chocolate, and the times required for eating it were measured. We found that the more participants tended to feel comfortable under the CS, the more it took time to eat in the UCS. However, the EEG emergence patterns in the two virtual spaces varied across the individuals. Upon focusing on the theta and low-beta bands, the strength of the mental condition and eating times were found to be guided by these frequency bands. The results determined that the theta and low-beta bands are likely important and relevant waves for feeding behaviors under emotional circumstances, following alterations in mental conditions.

Stylized faces enhance ERP features used for the detection of emotional responses

For their ease of accessibility and low cost, current Brain-Computer Interfaces (BCI) used to detect subjective emotional and affective states rely largely on electroencephalographic (EEG) signals. Public datasets are available for researchers to design models for affect detection from EEG. However, few designs focus on optimally exploiting the nature of the stimulus elicitation to improve accuracy. The RSVP protocol is used in this experiment to present human faces of emotion to 28 participants while EEG was measured. We found that artificially enhanced human faces with exaggerated, cartoonish visual features significantly improve some commonly used neural correlates of emotion as measured by event-related potentials (ERPs). These images elicit an enhanced N170 component, well known to relate to the facial visual encoding process. Our findings suggest that the study of emotion elicitation could exploit consistent, high detail, AI generated stimuli transformations to study the characteristics of electrical brain activity related to visual affective stimuli. Furthermore, this specific result might be useful in the context of affective BCI design, where a higher accuracy in affect decoding from EEG can improve the experience of a user.

Sleep patterns correlates with the efficacy of tDCS on post-stroke patients with prolonged disorders of consciousness

BACKGROUND

The subclassification of prolonged disorders of consciousness (DoC) based on sleep patterns is important for the evaluation and treatment of the disease. This study evaluates the correlation between polysomnographic patterns and the efficacy of transcranial direct current stimulation (tDCS) in patients with prolonged DoC due to stroke.

METHODS

In total, 33 patients in the vegetative state (VS) with sleep cycles or without sleep cycles were randomly assigned to either active or sham tDCS groups. Polysomnography was used to monitor sleep changes before and after intervention. Additionally, clinical scale scores and electroencephalogram (EEG) analysis were performed before and after intervention to evaluate the efficacy of tDCS on the patients subclassified according to their sleep patterns.

RESULTS

The results suggest that tDCS improved the sleep structure, significantly prolonged total sleep time (TST) (95%CI: 14.387-283.527, P = 0.013) and NREM sleep stage 2 (95%CI: 3.157-246.165, P = 0.040) of the VS patients with sleep cycles. It also significantly enhanced brain function of patients with sleep cycles, which were reflected by the increased clinical scores (95%CI: 0.340-3.440, P < 0.001), the EEG powers and functional connectivity in the brain and the 6-month prognosis. Moreover, the changes in NREM sleep stage 2 had a significant positive correlation with each index of the β band.

CONCLUSION

This study reveals the importance of sleep patterns in the prognosis and treatment of prolonged DoC and provides new evidence for the efficacy of tDCS in post-stroke patients with VS patients subclassified by sleep pattern. Trial registration URL: https://www.

CLINICALTRIALS

gov . Unique identifier: NCT03809936. Registered 18 January 2019.

Diagnosis of Epilepsy with Functional Connectivity in EEG after a Suspected First Seizure

Epilepsy is regarded as a structural and functional network disorder, affecting around 50 million people worldwide. A correct disease diagnosis can lead to quicker medical action, preventing adverse effects. This paper reports the design of a classifier for epilepsy diagnosis in patients after a first ictal episode, using electroencephalogram (EEG) recordings. The dataset consists of resting-state EEG from 629 patients, of which 504 were retained for the study. The patient's cohort exists out of 291 patients with epilepsy and 213 patients with other pathologies. The data were split into two sets: 80% training set and 20% test set. The extracted features from EEG included functional connectivity measures, graph measures, band powers and brain asymmetry ratios. Feature reduction was performed, and the models were trained using Machine Learning (ML) techniques. The models' evaluation was performed with the area under the receiver operating characteristic curve (AUC). When focusing specifically on focal lesional epileptic patients, better results were obtained. This classification task was optimized using a 5-fold cross-validation, where SVM using PCA for feature reduction achieved an AUC of 0.730 ± 0.030. In the test set, the same model achieved 0.649 of AUC. The verified decrease is justified by the considerable diversity of pathologies in the cohort. An analysis of the selected features across tested models shows that functional connectivity and its graph measures have the most considerable predictive power, along with full-spectrum frequency-based features. To conclude, the proposed algorithms, with some refinement, can be of added value for doctors diagnosing epilepsy from EEG recordings after a suspected first seizure.

Wearable electroencephalography and multi-modal mental state classification: A systematic literature review

BACKGROUND

Wearable multi-modal time-series classification applications outperform their best uni-modal counterparts and hold great promise. A modality that directly measures electrical correlates from the brain is electroencephalography. Due to varying noise sources, different key brain regions, key frequency bands, and signal characteristics like non-stationarity, techniques for data pre-processing and classification algorithms are task-dependent.

METHOD

Here, a systematic literature review on mental state classification for wearable electroencephalography is presented. Four search terms in different combinations were used for an in-title search. The search was executed on the 29th of June 2022, across Google Scholar, PubMed, IEEEXplore, and ScienceDirect. 76 most relevant publications were set into context as the current state-of-the-art in mental state time-series classification.

RESULTS

Pre-processing techniques, features, and time-series classification models were analyzed. Across publications, a window length of one second was mainly chosen for classification and spectral features were utilized the most. The achieved performance per time-series classification model is analyzed, finding linear discriminant analysis, decision trees, and k-nearest neighbors models outperform support-vector machines by a factor of up to 1.5. A historical analysis depicts future trends while under-reported aspects relevant to practical applications are discussed.

CONCLUSIONS

Five main conclusions are given, covering utilization of available area for electrode placement on the head, most often or scarcely utilized features and time-series classification model architectures, baseline reporting practices, as well as explainability and interpretability of Deep Learning. The importance of a 'test battery' assessing the influence of data pre-processing and multi-modality on time-series classification performance is emphasized.

Classification of brain states that predicts future performance in visual tasks based on co-integration analysis of EEG data

Electroencephalogram (EEG) is a popular tool for studying brain activity. Numerous statistical techniques exist to enhance understanding of the complex dynamics underlying the EEG recordings. Inferring the functional network connectivity between EEG channels is of interest, and non-parametric inference methods are typically applied. We propose a fully parametric model-based approach via cointegration analysis. It not only estimates the network but also provides further insight through cointegration vectors, which characterize equilibrium states, and the corresponding loadings, which describe the mechanism of how the EEG dynamics is drawn to the equilibrium. We outline the estimation procedure in the context of EEG data, which faces specific challenges compared with the common econometric problems, for which cointegration analysis was originally conceived. In particular, the dimension is higher, typically around 64; there is usually access to repeated trials; and the data are artificially linearly dependent through the normalization done in EEG recordings. Finally, we illustrate the method on EEG data from a visual task experiment and show how brain states identified via cointegration analysis can be utilized in further investigations of determinants playing roles in sensory identifications.

Assessment of EEG-based functional connectivity in response to haptic delay

Haptic technologies enable users to physically interact with remote or virtual environments by applying force, vibration, or motion via haptic interfaces. However, the delivery of timely haptic feedback remains a challenge due to the stringent computation and communication requirements associated with haptic data transfer. Haptic delay disrupts the realism of the user experience and interferes with the quality of interaction. Research efforts have been devoted to studying the neural correlates of delayed sensory stimulation to better understand and thus mitigate the impact of delay. However, little is known about the functional neural networks that process haptic delay. This paper investigates the underlying neural networks associated with processing haptic delay in passive and active haptic interactions. Nineteen participants completed a visuo-haptic task using a computer screen and a haptic device while electroencephalography (EEG) data were being recorded. A combined approach based on phase locking value (PLV) functional connectivity and graph theory was used. To assay the effects of haptic delay on functional connectivity, we evaluate a global connectivity property through the small-worldness index and a local connectivity property through the nodal strength index. Results suggest that the brain exhibits significantly different network characteristics when a haptic delay is introduced. Haptic delay caused an increased manifestation of the small-worldness index in the delta and theta bands as well as an increased nodal strength index in the middle central region. Inter-regional connectivity analysis showed that the middle central region was significantly connected to the parietal and occipital regions as a result of haptic delay. These results are expected to indicate the detection of conflicting visuo-haptic information at the middle central region and their respective resolution and integration at the parietal and occipital regions.

Emotion recognition using effective connectivity and pre-trained convolutional neural networks in EEG signals

Convolutional Neural Networks (CNN) have recently made considerable advances in the field of biomedical signal processing. These methodologies can assist in emotion recognition for affective brain computer interface. In this paper, a novel emotion recognition system based on the effective connectivity and the fine-tuned CNNs from multichannel Electroencephalogram (EEG) signal is presented. After preprocessing EEG signals, the relationships among 32 channels of EEG in the form of effective brain connectivity analysis which represents information flow between regions are computed by direct Directed Transfer Function (dDTF) method which yields a 32*32 image. Then, these constructed images from EEG signals for each subject were fed as input to four versions of pre-trained CNN models, AlexNet, ResNet-50, Inception-v3 and VGG-19 and the parameters of these models are fine-tuned, independently. The proposed deep learning architectures automatically learn patterns in the constructed image of the EEG signals in frequency bands. The efficiency of the proposed approach is evaluated on MAHNOB-HCI and DEAP databases. The experiments for classifying five emotional states show that the ResNet-50 applied on dDTF images in alpha band achieves best results due to specific architecture which captures the brain connectivity, efficiently. The accuracy and F1-score values for MAHNOB-HCI were obtained 99.41, 99.42 and for DEAP databases, 98.17, and 98.23. Newly proposed model is capable of effectively analyzing the brain function using information flow from multichannel EEG signals using effective connectivity measure of dDTF and ResNet-50.

Relations between family cohesion and adolescent-parent's neural synchrony in response to emotional stimulations

BACKGROUND

The interaction between parent and adolescent is more challenging than in other age periods. Family cohesion seriously impacts parent-adolescent emotional interactions. However, the underlying neural mechanism has not been fully examined. This study examined the differences in the neural synchrony in response to emotional film clips between high and low family cohesion adolescent-parent dyads by using the electroencephalograph (EEG) hyperscanning.

RESULTS

Simultaneously electroencephalograph (EEG) was recorded while 15 low family cohesion parent-adolescent dyads (LFCs)and 14 high family cohesion parent-adolescent dyads (HFCs)received different emotional induction when viewing film clips. Interbrain phase-locking-value (PLV) in gamma band was used to calculate parent-adolescent dyads' interbrain synchrony. Results showed that higher gamma interbrain synchrony was observed in the HFCs than the LFCs in the positive conditions. However, there was no significant difference between the HFCs and LFCs in other conditions. Also, the HFCs had significantly higher gamma interbrain synchrony in the positive conditions than in the negative conditions.

CONCLUSION

Interbrain synchrony may represent an underlying neural mechanism of the parent-adolescent emotional bonding, which is the core of family cohesion.

Classifying oscillatory brain activity associated with Indian Rasas using network metrics

Neural signatures for the western classification of emotions have been widely discussed in the literature. The ancient Indian treatise on performing arts known as Natyashastra categorizes emotions into nine classes, known as Rasas. Rasa-as opposed to a pure emotion-is defined as a superposition of certain transitory, dominant, and temperamental emotional states. Although Rasas have been widely discussed in the text, dedicated brain imaging studies have not been conducted in their research. Our study examines the neural oscillations, recorded through electroencephalography (EEG) imaging, that are elicited while experiencing emotional states corresponding to Rasas. We identify differences among them using network-based functional connectivity metrics in five different frequency bands. Further, Random Forest models are trained on the extracted network features, and we present our findings based on classifier predictions. We observe slow (delta) and fast brain waves (beta and gamma) exhibited the maximum discriminating features between Rasas, whereas alpha and theta bands showed fewer distinguishable pairs. Out of nine Rasas, Sringaram (love), Bibhatsam (odious), and Bhayanakam (terror) were distinguishable from other Rasas the most across frequency bands. On the scale of most network metrics, Raudram (rage) and Sringaram are on the extremes, which also resulted in their good classification accuracy of 95%. This is reminiscent of the circumplex model where anger and contentment/happiness are on extremes on the pleasant scale. Interestingly, our results are consistent with the previous studies which highlight the significant role of higher frequency oscillations in the classification of emotions, in contrast to the alpha band that has shows non-significant differences across emotions. This research contributes to one of the first attempts to investigate the neural correlates of Rasas. Therefore, the results of this study can potentially guide the explorations into the entrainment of brain oscillations between performers and viewers, which can further lead to better performances and viewer experience.

Assessment of emotional states in EEG signals using multi-frequency power spectrum and functional connectivity patterns

In this work, an attempt has been made to characterize arousal and valence emotional states using Electroencephalogram (EEG) signals and Phase lag index (PLI) based functional connectivity features. For this, EEG signals are considered from a publicly available DEAP database. Signals are decomposed into four frequency bands, namely theta (θ, 4-7 Hz), alpha (a, 8-12 Hz), beta (ß, 13-30 Hz), and gamma (γ, 30-45 Hz). Two features, namely relative PSD and PLI, are calculated from each band of signals with Welch's periodogram. Four classifiers, namely Random Forest (RF), Support Vector Machines (SVM), Linear Discriminant Analysis (LDA), and K-Nearest Neighbor (KNN), are employed to discriminate the emotional states. Results show that the proposed approach can differentiate emotional states using EEG signals. It is observed that there is strong functional connectivity in Fp1-02 and Fp2-Pz in all emotional states for different frequency bands. SVM classifier yields the highest classification performance for arousal, and RF yields the highest performance for valence in the y band. The combination of all features performs the best for the valence dimension. Thus, the proposed approach could be extended for classifying various emotional states in clinical settings. Clinical Relevance- This establishes PLI based approach for improved classification (fl = 74.77% for Arousal fl = 74.94 for valence) of emotional states.

Electroencephalograph-Based Emotion Recognition Using Brain Connectivity Feature and Domain Adaptive Residual Convolution Model

In electroencephalograph (EEG) emotion recognition research, obtaining high-level emotional features with more discriminative information has become the key to improving the classification performance. This study proposes a new end-to-end emotion recognition method based on brain connectivity (BC) features and domain adaptive residual convolutional network (short for BC-DA-RCNN), which could effectively extract the spatial connectivity information related to the emotional state of the human brain and introduce domain adaptation to achieve accurate emotion recognition within and across the subject’s EEG signals. The BC information is represented by the global brain network connectivity matrix. The DA-RCNN is used to extract high-level emotional features between different dimensions of EEG signals, reduce the domain offset between different subjects, and strengthen the common features between different subjects. The experimental results on the large public DEAP data set show that the accuracy of the subject-dependent and subject-independent binary emotion classification in valence reaches 95.15 and 88.28%, respectively, which outperforms all the benchmark methods. The proposed method is proven to have lower complexity, better generalization ability, and domain robustness that help to lay a solid foundation for the development of high-performance affective brain-computer interface applications.

Human emotion recognition from EEG-based brain–computer interface using machine learning: a comprehensive review

Affective computing, a subcategory of artificial intelligence, detects, processes, interprets, and mimics human emotions. Thanks to the continued advancement of portable non-invasive human sensor technologies, like brain–computer interfaces (BCI), emotion recognition has piqued the interest of academics from a variety of domains. Facial expressions, speech, behavior (gesture/posture), and physiological signals can all be used to identify human emotions. However, the first three may be ineffectual because people may hide their true emotions consciously or unconsciously (so-called social masking). Physiological signals can provide more accurate and objective emotion recognition. Electroencephalogram (EEG) signals respond in real time and are more sensitive to changes in affective states than peripheral neurophysiological signals. Thus, EEG signals can reveal important features of emotional states. Recently, several EEG-based BCI emotion recognition techniques have been developed. In addition, rapid advances in machine and deep learning have enabled machines or computers to understand, recognize, and analyze emotions. This study reviews emotion recognition methods that rely on multi-channel EEG signal-based BCIs and provides an overview of what has been accomplished in this area. It also provides an overview of the datasets and methods used to elicit emotional states. According to the usual emotional recognition pathway, we review various EEG feature extraction, feature selection/reduction, machine learning methods (e.g., k-nearest neighbor), support vector machine, decision tree, artificial neural network, random forest, and naive Bayes) and deep learning methods (e.g., convolutional and recurrent neural networks with long short term memory). In addition, EEG rhythms that are strongly linked to emotions as well as the relationship between distinct brain areas and emotions are discussed. We also discuss several human emotion recognition studies, published between 2015 and 2021, that use EEG data and compare different machine and deep learning algorithms. Finally, this review suggests several challenges and future research directions in the recognition and classification of human emotional states using EEG.

Enhancing Emotion Recognition Using Region-Specific Electroencephalogram Data and Dynamic Functional Connectivity

Recognizing the emotional states of humans through EEG signals are of great significance to the progress of human-computer interaction. The present study aimed to perform automatic recognition of music-evoked emotions through region-specific information and dynamic functional connectivity of EEG signals and a deep learning neural network. EEG signals of 15 healthy volunteers were collected when different emotions (high-valence-arousal vs. low-valence-arousal) were induced by a musical experimental paradigm. Then a sequential backward selection algorithm combining with deep neural network called Xception was proposed to evaluate the effect of different channel combinations on emotion recognition. In addition, we also assessed whether dynamic functional network of frontal cortex, constructed through different trial number, may affect the performance of emotion cognition. Results showed that the binary classification accuracy based on all 30 channels was 70.19%, the accuracy based on all channels located in the frontal region was 71.05%, and the accuracy based on the best channel combination in the frontal region was 76.84%. In addition, we found that the classification performance increased as longer temporal functional network of frontal cortex was constructed as input features. In sum, emotions induced by different musical stimuli can be recognized by our proposed approach though region-specific EEG signals and time-varying functional network of frontal cortex. Our findings could provide a new perspective for the development of EEG-based emotional recognition systems and advance our understanding of the neural mechanism underlying emotion processing.

Mood-Driven Colorization of Virtual Indoor Scenes

One of the challenging tasks in virtual scene design for Virtual Reality (VR) is causing it to invoke a particular mood in viewers. The subjective nature of moods brings uncertainty to the purpose. We propose a novel approach to automatic adjustment of the colors of textures for objects in a virtual indoor scene, enabling it to match a target mood. A dataset of 25,000 images, including building/home interiors, was used to train a classifier with the features extracted via deep learning. It contributes to an optimization process that colorizes virtual scenes automatically according to the target mood. Our approach was tested on four different indoor scenes, and we conducted a user study demonstrating its efficacy through statistical analysis with the focus on the impact of the scenes experienced with a VR headset.

Fusion of EEG-Based Activation, Spatial, and Connection Patterns for Fear Emotion Recognition

At present, emotion recognition based on electroencephalograms (EEGs) has attracted much more attention. Current studies of affective brain-computer interfaces (BCIs) focus on the recognition of happiness and sadness using brain activation patterns. Fear recognition involving brain activities in different spatial distributions and different brain functional networks has been scarcely investigated. In this study, we propose a multifeature fusion method combining energy activation, spatial distribution, and brain functional connection network (BFCN) features for fear emotion recognition. The affective brain pattern was identified by not only the power activation features of differential entropy (DE) but also the spatial distribution features of the common spatial pattern (CSP) and the EEG phase synchronization features of phase lock value (PLV). A total of 15 healthy subjects took part in the experiment, and the average accuracy rate was 85.00% ± 8.13%. The experimental results showed that the fear emotions of subjects were fully stimulated and effectively identified. The proposed fusion method on fear recognition was thus validated and is of great significance to the development of effective emotional BCI systems.

Unsupervised classification reveals consistency and degeneracy in neural network patterns of emotion

In the present study, we used an unsupervised classification algorithm to reveal both consistency and degeneracy in neural network connectivity during anger and anxiety. Degeneracy refers to the ability of different biological pathways to produce the same outcomes. Previous research is suggestive of degeneracy in emotion, but little research has explicitly examined whether degenerate functional connectivity patterns exist for emotion categories such as anger and anxiety. Twenty-four subjects underwent functional magnetic resonance imaging (fMRI) while listening to unpleasant music and self-generating experiences of anger and anxiety. A data-driven model building algorithm with unsupervised classification (subgrouping Group Iterative Multiple Model Estimation) identified patterns of connectivity among 11 intrinsic networks that were associated with anger vs anxiety. As predicted, degenerate functional connectivity patterns existed within these overarching consistent patterns. Degenerate patterns were not attributable to differences in emotional experience or other individual-level factors. These findings are consistent with the constructionist account that emotions emerge from flexible functional neuronal assemblies and that emotion categories such as anger and anxiety each describe populations of highly variable instances.

Representation Learning for EEG-Based Biometrics Using Hilbert–Huang Transform

A promising approach to overcome the various shortcomings of password systems is the use of biometric authentication, in particular the use of electroencephalogram (EEG) data. In this paper, we propose a subject-independent learning method for EEG-based biometrics using Hilbert spectrograms of the data. The proposed neural network architecture treats the spectrogram as a collection of one-dimensional series and applies one-dimensional dilated convolutions over them, and a multi-similarity loss was used as the loss function for subject-independent learning. The architecture was tested on the publicly available PhysioNet EEG Motor Movement/Imagery Dataset (PEEGMIMDB) with a 14.63% Equal Error Rate (EER) achieved. The proposed approach’s main advantages are subject independence and suitability for interpretation via created spectrograms and the integrated gradients method.

Brain functional and effective connectivity based on electroencephalography recordings: A review

Functional connectivity and effective connectivity of the human brain, representing statistical dependence and directed information flow between cortical regions, significantly contribute to the study of the intrinsic brain network and its functional mechanism. Many recent studies on electroencephalography (EEG) have been focusing on modeling and estimating brain connectivity due to increasing evidence that it can help better understand various brain neurological conditions. However, there is a lack of a comprehensive updated review on studies of EEG-based brain connectivity, particularly on visualization options and associated machine learning applications, aiming to translate those techniques into useful clinical tools. This article reviews EEG-based functional and effective connectivity studies undertaken over the last few years, in terms of estimation, visualization, and applications associated with machine learning classifiers. Methods are explored and discussed from various dimensions, such as either linear or nonlinear, parametric or nonparametric, time-based, and frequency-based or time-frequency-based. Then it is followed by a novel review of brain connectivity visualization methods, grouped by Heat Map, data statistics, and Head Map, aiming to explore the variation of connectivity across different brain regions. Finally, the current challenges of related research and a roadmap for future related research are presented.

Investigating EEG-based functional connectivity patterns for multimodal emotion recognition

Objective. Previous studies on emotion recognition from electroencephalography (EEG) mainly rely on single-channel-based feature extraction methods, which ignore the functional connectivity between brain regions. Hence, in this paper, we propose a novel emotion-relevant critical subnetwork selection algorithm and investigate three EEG functional connectivity network features: strength, clustering coefficient, and eigenvector centrality. Approach. After constructing the brain networks by the correlations between pairs of EEG signals, we calculated critical subnetworks through the average of brain network matrices with the same emotion label to eliminate the weak associations. Then, three network features were conveyed to a multimodal emotion recognition model using deep canonical correlation analysis along with eye movement features. The discrimination ability of the EEG connectivity features in emotion recognition is evaluated on three public datasets: SEED, SEED-V, and DEAP. Main results. The experimental results reveal that the strength feature outperforms the state-of-the-art features based on single-channel analysis. The classification accuracies of multimodal emotion recognition are 95.08 ± 6.42 % on the SEED dataset, 84.51 ± 5.11 % on the SEED-V dataset, and 85.34 ± 2.90 % and 86.61 ± 3.76 % for arousal and valence on the DEAP dataset, respectively, which all achieved the best performance. In addition, the brain networks constructed with 18 channels achieve comparable performance with that of the 62-channel network and enable easier setups in real scenarios. Significance. The EEG functional connectivity networks combined with emotion-relevant critical subnetworks selection algorithm we proposed is a successful exploration to excavate the information between channels.

Objective Recognition of Tinnitus Location Using Electroencephalography Connectivity Features

Purpose: Tinnitus is a common but obscure auditory disease to be studied. This study will determine whether the connectivity features in electroencephalography (EEG) signals can be used as the biomarkers for an efficient and fast diagnosis method for chronic tinnitus.

Methods: In this study, the resting-state EEG signals of tinnitus patients with different tinnitus locations were recorded. Four connectivity features [including the Phase-locking value (PLV), Phase lag index (PLI), Pearson correlation coefficient (PCC), and Transfer entropy (TE)] and two time-frequency domain features in the EEG signals were extracted, and four machine learning algorithms, included two support vector machine models (SVM), a multi-layer perception network (MLP) and a convolutional neural network (CNN), were used based on the selected features to classify different possible tinnitus sources.

Results: Classification accuracy was highest when the SVM algorithm or the MLP algorithm was applied to the PCC feature sets, achieving final average classification accuracies of 99.42 or 99.1%, respectively. And based on the PLV feature, the classification result was also particularly good. And MLP ran the fastest, with an average computing time of only 4.2 s, which was more suitable than other methods when a real-time diagnosis was required.

Conclusion: Connectivity features of the resting-state EEG signals could characterize the differentiation of tinnitus location. The connectivity features (PCC and PLV) were more suitable as the biomarkers for the objective diagnosing of tinnitus. And the results were helpful for clinicians in the initial diagnosis of tinnitus.

Minimum spanning tree based graph neural network for emotion classification using EEG

Emotion classification based on neurophysiology signals has been a challenging issue in the literature. Recent neuroscience findings suggest that brain network structure underlying the different emotions provides a window in understanding human affection. In this paper, we propose a novel method to capture the distinct minimum spanning tree (MST) topology underpinning the different emotions. Specifically, we propose a hierarchical aggregation-based graph neural network to investigate the MST structure in emotion recognition. Extensive experiments on the public available DEAP dataset demonstrate the superior performance of the model in emotion classification as compared to existing methods. In addition, the results show that the theta, lower beta and gamma frequency band network information are more sensitive to emotions, suggesting a multi-frequency interaction in emotion processing.

Tracking of Mental Workload with a Mobile EEG Sensor

The aim of the present investigation was to assess if a mobile electroencephalography (EEG) setup can be used to track mental workload, which is an important aspect of learning performance and motivation and may thus represent a valuable source of information in the evaluation of cognitive training approaches. Twenty five healthy subjects performed a three-level N-back test using a fully mobile setup including tablet-based presentation of the task and EEG data collection with a self-mounted mobile EEG device at two assessment time points. A two-fold analysis approach was chosen including a standard analysis of variance and an artificial neural network to distinguish the levels of cognitive load. Our findings indicate that the setup is feasible for detecting changes in cognitive load, as reflected by alterations across lobes in different frequency bands. In particular, we observed a decrease of occipital alpha and an increase in frontal, parietal and occipital theta with increasing cognitive load. The most distinct levels of cognitive load could be discriminated by the integrated machine learning models with an accuracy of 86%.

EEG-Based Emotion Recognition by Convolutional Neural Network with Multi-Scale Kernels

Besides facial or gesture-based emotion recognition, Electroencephalogram (EEG) data have been drawing attention thanks to their capability in countering the effect of deceptive external expressions of humans, like faces or speeches. Emotion recognition based on EEG signals heavily relies on the features and their delineation, which requires the selection of feature categories converted from the raw signals and types of expressions that could display the intrinsic properties of an individual signal or a group of them. Moreover, the correlation or interaction among channels and frequency bands also contain crucial information for emotional state prediction, and it is commonly disregarded in conventional approaches. Therefore, in our method, the correlation between 32 channels and frequency bands were put into use to enhance the emotion prediction performance. The extracted features chosen from the time domain were arranged into feature-homogeneous matrices, with their positions following the corresponding electrodes placed on the scalp. Based on this 3D representation of EEG signals, the model must have the ability to learn the local and global patterns that describe the short and long-range relations of EEG channels, along with the embedded features. To deal with this problem, we proposed the 2D CNN with different kernel-size of convolutional layers assembled into a convolution block, combining features that were distributed in small and large regions. Ten-fold cross validation was conducted on the DEAP dataset to prove the effectiveness of our approach. We achieved the average accuracies of 98.27% and 98.36% for arousal and valence binary classification, respectively.

Time-Frequency Representation and Convolutional Neural Network-Based Emotion Recognition

Emotions composed of cognizant logical reactions toward various situations. Such mental responses stem from physiological, cognitive, and behavioral changes. Electroencephalogram (EEG) signals provide a noninvasive and nonradioactive solution for emotion identification. Accurate and automatic classification of emotions can boost the development of human-computer interface. This article proposes automatic extraction and classification of features through the use of different convolutional neural networks (CNNs). At first, the proposed method converts the filtered EEG signals into an image using a time-frequency representation. Smoothed pseudo-Wigner-Ville distribution is used to transform time-domain EEG signals into images. These images are fed to pretrained AlexNet, ResNet50, and VGG16 along with configurable CNN. The performance of four CNNs is evaluated by measuring the accuracy, precision, Mathew's correlation coefficient, F1-score, and false-positive rate. The results obtained by evaluating four CNNs show that configurable CNN requires very less learning parameters with better accuracy. Accuracy scores of 90.98%, 91.91%, 92.71%, and 93.01% obtained by AlexNet, ResNet50, VGG16, and configurable CNN show that the proposed method is best among other existing methods.

EEG Coherence Metrics for Vigilance: Sensitivity to Workload, Time-on-Task, and Individual Differences

The vigilance decrement in performance is a significant operational issue in various applied settings. Psychophysiological methods for diagnostic monitoring of vigilance have focused on power spectral density measures from the electroencephalogram (EEG). This article addresses the diagnosticity of an alternative set of EEG measures, coherence between different electrode sites. Coherence metrics may index the functional connectivity between brain regions that supports sustained attention. Coherence was calculated for seven pre-defined brain networks. Workload and time-on-task factors primarily influenced alpha and theta coherence in anterior, central, and inter-hemispheric networks. Individual differences in coherence in inter-hemispheric, left intro-hemispheric and posterior networks correlated with performance. These findings demonstrate the potential applied utility of coherence metrics, although several methodological limitations and challenges must be overcome.

A novel beamformer-based imaging of phase-amplitude coupling (BIPAC) unveiling the inter-regional connectivity of emotional prosody processing in women with primary dysmenorrhea

Objective. Neural communication or the interactions of brain regions play a key role in the formation of functional neural networks. A type of neural communication can be measured in the form of phase-amplitude coupling (PAC), which is the coupling between the phase of low-frequency oscillations and the amplitude of high-frequency oscillations. This paper presents a beamformer-based imaging method, beamformer-based imaging of PAC (BIPAC), to quantify the strength of PAC between a seed region and other brain regions.Approach. A dipole is used to model the ensemble of neural activity within a group of nearby neurons and represents a mixture of multiple source components of cortical activity. From ensemble activity at each brain location, the source component with the strongest coupling to the seed activity is extracted, while unrelated components are suppressed to enhance the sensitivity of coupled-source estimation.Main results. In evaluations using simulation data sets, BIPAC proved advantageous with regard to estimation accuracy in source localization, orientation, and coupling strength. BIPAC was also applied to the analysis of magnetoencephalographic signals recorded from women with primary dysmenorrhea in an implicit emotional prosody experiment. In response to negative emotional prosody, auditory areas revealed strong PAC with the ventral auditory stream and occipitoparietal areas in the theta-gamma and alpha-gamma bands, which may respectively indicate the recruitment of auditory sensory memory and attention reorientation. Moreover, patients with more severe pain experience appeared to have stronger coupling between auditory areas and temporoparietal regions.Significance. Our findings indicate that the implicit processing of emotional prosody is altered by menstrual pain experience. The proposed BIPAC is feasible and applicable to imaging inter-regional connectivity based on cross-frequency coupling estimates. The experimental results also demonstrate that BIPAC is capable of revealing autonomous brain processing and neurodynamics, which are more subtle than active and attended task-driven processing.