Brain-Based Indices for User System Symbiosis

EEG-TNet: An End-To-End Brain Computer Interface Framework for Mental Workload Estimation

The mental workload (MWL) of different occupational groups' workers is the main and direct factor of unsafe behavior, which may cause serious accidents. One of the new and useful technologies to estimate MWL is the Brain computer interface (BCI) based on EEG signals, which is regarded as the gold standard of cognitive status. However, estimation systems involving handcrafted EEG features are time-consuming and unsuitable to apply in real-time. The purpose of this study was to propose an end-to-end BCI framework for MWL estimation. First, a new automated data preprocessing method was proposed to remove the artifact without human interference. Then a new neural network structure named EEG-TNet was designed to extract both the temporal and frequency information from the original EEG. Furthermore, two types of experiments and ablation studies were performed to prove the effectiveness of this model. In the subject-dependent experiment, the estimation accuracy of dual-task estimation (No task vs. TASK) and triple-task estimation (Lo vs. Mi vs. Hi) reached 99.82 and 99.21%, respectively. In contrast, the accuracy of different tasks reached 82.78 and 66.83% in subject-independent experiments. Additionally, the ablation studies proved that preprocessing method and network structure had significant contributions to estimation MWL. The proposed method is convenient without any human intervention and outperforms other related studies, which becomes an effective way to reduce human factor risks.

Objective Assessment of Human Workload in Physical Human-robot Cooperation Using Brain Monitoring

The notions of safe and compliant interaction are not sufficient to ensure effective physical human-robot cooperation. To obtain an optimal compliant behavior (e.g., variable impedance/admittance control), assessment techniques are required to measure the effectiveness of the interaction in terms of perceived workload by users. This study investigates electroencephalography (EEG) monitoring as an objective measure to classify workload in cooperative manipulation with compliance. An experimental study is conducted including two types of manipulation (gross and fine) with two admittance levels (low- and high-damping). Performance and self-reported measures indicate that a proper admittance level that enhances perceived workload is task-dependent. This information is used to form a binary classification problem (low- and high-workload) with spectral power density and coherence as the features extracted from EEG data. Using a subject-independent feature selection approach, a subject-dependent Linear Discriminant Analysis (LDA) is used for classification. An average classification rate of 81% is achieved that indicates the reliability of the proposed approach for assessing human workload in interaction with varying compliance across the gross and fine manipulation. Furthermore, to validate our proposed objective measure of workload, we have conducted a second experiment composed of both fine and gross motor tasks. Compared to interaction with a constant admittance, a lower EEG-based workload is observed with an open-loop variable admittance controller. This observation is in agreement with the subjective workload score (NASA-TLX).

Effects of Aging Stereotype Threat on Working Self-Concepts: An Event-Related Potentials Approach

Although the influence of stereotype threat (ST) on working self-concepts has been highlighted in recent years, its neural underpinnings are unclear. Notably, the aging ST, which largely influences older adults' cognitive ability, mental and physical health, did not receive much attention. In order to investigate these issues, electroencephalogram (EEG) data were obtained from older adults during a modified Stroop task using neutral words, positive and negative self-concept words in aging ST vs. neutral control conditions. Results showed longer reaction times (RTs) for identifying colors of words under the aging ST compared to the neutral condition. More importantly, the negative self-concept elicited more positive late P300 amplitudes and enhanced theta band activities compared to the positive self-concept or neutral words under the aging ST condition, whereas no difference was found between these self-concepts and neutral words in the control condition. Furthermore, the aging ST induced smaller theta band synchronization and enhanced alpha band synchronization compared to the control condition. Moreover, we also observed valence differences in self-concepts where the negative self-concept words reduced early P150/N170 complex relative to neutral words. These findings suggest that priming ST could activate negative self-concepts as current working self-concept, and that this influence occurred during a late neural time course.

User Experience May be Producing Greater Heart Rate Variability than Motor Imagery Related Control Tasks during the User-System Adaptation in Brain-Computer Interfaces

Brain-computer interface (BCI) is technology that is developing fast, but it remains inaccurate, unreliable and slow due to the difficulty to obtain precise information from the brain. Consequently, the involvement of other biosignals to decode the user control tasks has risen in importance. A traditional way to operate a BCI system is via motor imagery (MI) tasks. As imaginary movements activate similar cortical structures and vegetative mechanisms as a voluntary movement does, heart rate variability (HRV) has been proposed as a parameter to improve the detection of MI related control tasks. However, HR is very susceptible to body needs and environmental demands, and as BCI systems require high levels of attention, perceptual processing and mental workload, it is important to assess the practical effectiveness of HRV. The present study aimed to determine if brain and heart electrical signals (HRV) are modulated by MI activity used to control a BCI system, or if HRV is modulated by the user perceptions and responses that result from the operation of a BCI system (i.e., user experience). For this purpose, a database of 11 participants who were exposed to eight different situations was used. The sensory-cognitive load (intake and rejection tasks) was controlled in those situations. Two electrophysiological signals were utilized: electroencephalography and electrocardiography. From those biosignals, event-related (de-)synchronization maps and event-related HR changes were respectively estimated. The maps and the HR changes were cross-correlated in order to verify if both biosignals were modulated due to MI activity. The results suggest that HR varies according to the experience undergone by the user in a BCI working environment, and not because of the MI activity used to operate the system.

Gaze-independent ERP-BCIs: augmenting performance through location-congruent bimodal stimuli

Gaze-independent event-related potential (ERP) based brain-computer interfaces (BCIs) yield relatively low BCI performance and traditionally employ unimodal stimuli. Bimodal ERP-BCIs may increase BCI performance due to multisensory integration or summation in the brain. An additional advantage of bimodal BCIs may be that the user can choose which modality or modalities to attend to. We studied bimodal, visual-tactile, gaze-independent BCIs and investigated whether or not ERP components’ tAUCs and subsequent classification accuracies are increased for (1) bimodal vs. unimodal stimuli; (2) location-congruent vs. location-incongruent bimodal stimuli; and (3) attending to both modalities vs. to either one modality. We observed an enhanced bimodal (compared to unimodal) P300 tAUC, which appeared to be positively affected by location-congruency (p = 0.056) and resulted in higher classification accuracies. Attending either to one or to both modalities of the bimodal location-congruent stimuli resulted in differences between ERP components, but not in classification performance. We conclude that location-congruent bimodal stimuli improve ERP-BCIs, and offer the user the possibility to switch the attended modality without losing performance.

EEG-based workload estimation across affective contexts

Workload estimation from electroencephalographic signals (EEG) offers a highly sensitive tool to adapt the human–computer interaction to the user state. To create systems that reliably work in the complexity of the real world, a robustness against contextual changes (e.g., mood), has to be achieved. To study the resilience of state-of-the-art EEG-based workload classification against stress we devise a novel experimental protocol, in which we manipulated the affective context (stressful/non-stressful) while the participant solved a task with two workload levels. We recorded self-ratings, behavior, and physiology from 24 participants to validate the protocol. We test the capability of different, subject-specific workload classifiers using either frequency-domain, time-domain, or both feature varieties to generalize across contexts. We show that the classifiers are able to transfer between affective contexts, though performance suffers independent of the used feature domain. However, cross-context training is a simple and powerful remedy allowing the extraction of features in all studied feature varieties that are more resilient to task-unrelated variations in signal characteristics. Especially for frequency-domain features, across-context training is leading to a performance comparable to within-context training and testing. We discuss the significance of the result for neurophysiology-based workload detection in particular and for the construction of reliable passive brain–computer interfaces in general.