Biocybernetic system evaluates indices of operator engagement in automated task

REViVe: Development of a reactive environmental vigilance in-vehicle system to mitigate drowsiness-induced inattention during automated driving

With monotonous or conditionally automated driving conditions that may lead to the degradation of driver vigilance and increase the risk of drowsy driving, it is crucial to implement advanced systems that assist drivers in returning to a state of optimal driving readiness. While these systems have shown significant effects in reducing the risks related to drowsy driving, most warning systems heavily rely on auditory and visual sensory channels. These modalities are susceptible to "alarm fatigue" due to frequent and annoying alarms, which may lead drivers to ignore or deactivate the systems entirely, thus rendering them less suitable for preemptive cautionary warnings. To address these limitations, a Reactive Environmental Vigilance in-Vehicle (REViVe) system was developed to counteract driver drowsiness by utilizing alternative sensory modalities. A total of 35 drivers were divided into three condition groups: olfactory, climate, and control. To evaluate the effectiveness of the system, five dependent measurements were analyzed: time taken for PERCLOS to return to baseline and engagement index to measure salient effect; time interval between drowsiness events and peripheral detection task score difference to measure sustained arousal effect; and satisfaction rating to measure driver acceptability. Both the olfactory and climate REViVe systems showed potential as effective preemptive warnings compared to control. With REViVe, drowsy drivers quickly returned to an awake state and sustained vigilance significantly longer than control, while driver satisfaction was positive. Thus, the REViVe system provides a balanced solution for alert functionality and driving experience, suggesting a novel approach to designing preemptive warning systems.

Pumping up your predictive power for cognitive state detection with the proper GAINS

Detecting cognitive states and impairments through EEG signals is crucial for applications in aviation and medicine and has broad applications in the field of human-machine interaction. However, existing methods often fail to capture the fine-grained neural dynamics of critical brain processes due to limited temporal resolution and inadequate signal decomposition techniques. To address this, we introduce the Spectral Intensity Stability (SIS) algorithm, a novel technique that analyzes the stability and competition of dominant brain frequency oscillations across granular timescales (≈4 ms). Unlike traditional spectral methods, SIS captures rapid neural transitions and hierarchical frequency dynamics, enabling more accurate characterization of task-specific cognitive processes. Our study focuses on EEG data from pilots performing multitasking simulations under hypoxic and non-hypoxic conditions, a high-stakes scenario where cognitive performance is crucial. We divided this multitasking scenario into specific cognitive states, such as task precursor, interruption, execution, and recovery. Our algorithm SIS achieved a 29.8% improvement in cognitive state classification compared to conventional methods, demonstrating superior accuracy in distinguishing both task states and hypoxic impairments. This work is novel because it bridges gaps left by traditional methods by revealing the role of hierarchical spectral dynamics in maintaining cognitive performance. Through the Granular Analysis Informing Neural Stability (GAINS) framework, we reveal how neuronal groups self-organize across fine-grained time scales, providing new understanding of task-switching, neural communication, and criticality. The findings highlight the potential for developing real-time cognitive monitoring systems to enhance safety and performance in environments where cognitive impairments can have serious consequences. Future research should extend these insights by incorporating transient behaviors and spatial dynamics to achieve a more comprehensive framework for characterizing cognitive states.

Enhancing shooting performance and cognitive engagement in virtual reality environments through brief meditation training

Mindfulness meditation training has been associated with improved cognitive and sport performance, but the mechanisms linking mindfulness, cognitive engagement, and performance remain unclear, especially in simultaneous assessments during sports tasks. This study explored whether brief mindfulness meditation (BMM) training impacts shooting performance and cognitive engagement. We hypothesized that: (1) the meditation group would show improved shooting performance compared to the control group, and (2) daily 15-min mindfulness meditation would enhance cognitive engagement, reflected by brain activity. Sixty participants were randomly assigned to either an 18-day mindfulness meditation group or a control group. A virtual reality (VR) shooting task assessed performance before and after the intervention, while portable EEG devices recorded brain activity. The meditation group improved shooting scores by 6.56 points (p = 0.036) and showed a higher Beta/(Alpha + Theta) ratio-a marker of cognitive engagement reflecting greater focus and alertness versus relaxation-in left frontal regions (AF3, AF7, Fp1), but not right regions (AF4, AF8, Fp2). These findings suggest that BMM can improve both motor precision and mental focus, making it a valuable tool for athletes and professionals in high-precision fields such as surgery and aviation. Integrating short BMM sessions into training routines may help optimize focus and performance.

Neurophysiological Markers of Design-Induced Cognitive Changes: A Feasibility Study with Consumer-Grade Mobile EEG

BACKGROUND

Evidence-based design aims to create healthy environments grounded in scientific data, yet the influence of spatial qualities on cognitive processes remains underexplored. Advances in neuroscience offer promising tools to address this gap while meeting both scientific and practical demands. Consumer-grade mobile EEG devices are increasingly used; however, their lack of transparency complicates output interpretation. Well-established EEG indicators from cognitive neuroscience may offer a more accessible and interpretable alternative.

METHODS

This feasibility study explored the sensitivity of five established EEG power band ratios to cognitive shifts in response to subtle environmental design experiences. Twenty participants completed two crossover sessions in an office-like setting with nature-inspired versus urban-inspired design elements. Each session included controlled phases of focused on-screen cognitive task and off-screen breaks.

RESULTS

Factorial analyses revealed no significant interaction effects of cognitive state and environmental exposure on EEG outcomes. Nonetheless, frontal (θ/β) and frontocentral (β/[α + θ]) ratios showed distinct patterns across cognitive states, with more pronounced contrasts in the nature-inspired compared to the urban-inspired design conditions. Conversely, occipital ([θ + α]/β), (θ/α), and (β/α) ratios remained consistent across exposures. Data triangulation with autonomic nervous system responses and performance metrics supported these observations.

CONCLUSIONS

The findings suggest that EEG power band ratios can capture brain-environment interactions. However, limitations of consumer-grade EEG devices challenge both scientific rigour and practical application. Refining methodological reliability could improve interpretability, supporting more transparent and robust data-driven design decisions.

Eye-Based Recognition of User Traits and States-A Systematic State-of-the-Art Review

Eye-tracking technology provides high-resolution information about a user's visual behavior and interests. Combined with advances in machine learning, it has become possible to recognize user traits and states using eye-tracking data. Despite increasing research interest, a comprehensive systematic review of eye-based recognition approaches has been lacking. This study aimed to fill this gap by systematically reviewing and synthesizing the existing literature on the machine-learning-based recognition of user traits and states using eye-tracking data following PRISMA 2020 guidelines. The inclusion criteria focused on studies that applied eye-tracking data to recognize user traits and states with machine learning or deep learning approaches. Searches were performed in the ACM Digital Library and IEEE Xplore and the found studies were assessed for the risk of bias using standard methodological criteria. The data synthesis included a conceptual framework that covered the task, context, technology and data processing, and recognition targets. A total of 90 studies were included that encompassed a variety of tasks (e.g., visual, driving, learning) and contexts (e.g., computer screen, simulator, wild). The recognition targets included cognitive and affective states (e.g., emotions, cognitive workload) and user traits (e.g., personality, working memory). A set of various machine learning techniques, such as Support Vector Machines (SVMs), Random Forests, and deep learning models were applied to recognize user states and traits. This review identified state-of-the-art approaches and gaps, which highlighted the need for building up best practices, larger-scale datasets, and diversifying tasks and contexts. Future research should focus on improving the ecological validity, multi-modal approaches for robust user modeling, and developing gaze-adaptive systems.

EEG-Based Engagement Monitoring in Cognitive Games

Cognitive decline and dementia prevention are global priorities, with cognitive rehabilitation games showing potential to delay their onset or progression. However, these games require sufficient user engagement to be effective. Assessing the engagement through questionnaires is challenging for the individuals suffering from cognitive decline due to age or dementia. This study aims to explore the relationship between game difficulty levels, three EEG engagement indices (β/(θ + α), β/α, 1/α), and the self-reported flow state scale score during video gameplay, and to develop an accurate machine learning algorithm for the classification of user states into high- and low-engagement. Twenty-seven participants (nine older adults) played a stunt plane video game while their EEG signals were recorded using EPOCX. They also completed the flow state scale for occupational tasks questionnaire after the easy, optimal, and hard levels of gameplay. Self-reported engagement scores significantly varied across the difficulty levels (p = 0.027), with the optimal level yielding the highest scores. Combining the three EEG indices achieved the best performance, with F1 scores of 89% (within-subject) and 81% (cross-subject). Engagement classification F1 scores were 90% for young adults and 85% for older adults. The findings provide preliminary data that supports using EEG data for engagement analysis in adults and older adults.

A neuroadaptive interface shows intentional control alters the experience of time

A reliable experience of time is critical for perception and action in the present, for accurately remembering our past, and for successfully planning a future. Theories of time perception commonly assume a central mechanism keeps time by providing a relatively independent, internal clock. Recent work, however, shows imaginary self-movements alter subjective time, suggesting a critical role for action in temporal cognition. To test the hypothesis that time perception derives from the relationship between action and perception, we designed a neuroadaptive interface operating on imaginary movement to visualize movement through virtual reality. EEG activity was classified online as reflecting accelerating movement or static imagery, which was then used in providing feedback for adapting the velocity of optical flow presented in a star field to enable neuroadaptive control. Two cybernetic experiments were conducted to determine how neuroadaptivity in the relation between action and perception affected temporal perception in the verbal time estimation task. In particularly, we contrasted neuroadaptive feedback (e.g. imagined running > visual acceleration) with non-adaptive (imagined standing > visual acceleration) and pseudoadaptive (sham) feedback conditions. Movement imagery biased estimated duration while intentional control increased judgements of the passage of time. We conclude that perception and imaginary action co-determine temporal cognition. Furthermore, the relationship between perception and action-our evaluation of perceived movement as intentionally produced-alters the subjective experience of time. Finally, we discuss the potential for our novel, neuroadaptive methodology as an investigative tool for temporal disturbances observed in psychopathology.

Neuroergonomic Attention Assessment in Safety-Critical Tasks: EEG Indices and Subjective Metrics Validation in a Novel Task-Embedded Reaction Time Paradigm

Background/Objectives: This study addresses the gap in methodological guidelines for neuroergonomic attention assessment in safety-critical tasks, focusing on validating EEG indices, including the engagement index (EI) and beta/alpha ratio, alongside subjective ratings. Methods: A novel task-embedded reaction time paradigm was developed to evaluate the sensitivity of these metrics to dynamic attentional demands in a more naturalistic multitasking context. By manipulating attention levels through varying secondary tasks in the NASA MATB-II task while maintaining a consistent primary reaction-time task, this study successfully demonstrated the effectiveness of the paradigm. Results: Results indicate that both the beta/alpha ratio and EI are sensitive to changes in attentional demands, with beta/alpha being more responsive to dynamic variations in attention, and EI reflecting more the overall effort required to sustain performance, especially in conditions where maintaining attention is challenging. Conclusions: The potential for predicting the attention lapses through integration of performance metrics, EEG measures, and subjective assessments was demonstrated, providing a more nuanced understanding of dynamic fluctuations of attention in multitasking scenarios, mimicking those in real-world safety-critical tasks. These findings provide a foundation for advancing methods to monitor attention fluctuations accurately and mitigate risks in critical scenarios, such as train-driving or automated vehicle operation, where maintaining a high attention level is crucial.

Enhancing learning experiences: EEG-based passive BCI system adapts learning speed to cognitive load in real-time, with motivation as catalyst

Computer-based learning has gained popularity in recent years, providing learners greater flexibility and freedom. However, these learning environments do not consider the learner's mental state in real-time, resulting in less optimized learning experiences. This research aimed to explore the effect on the learning experience of a novel EEG-based Brain-Computer Interface (BCI) that adjusts the speed of information presentation in real-time during a learning task according to the learner's cognitive load. We also explored how motivation moderated these effects. In accordance with three experimental groups (non-adaptive, adaptive, and adaptive with motivation), participants performed a calibration task (n-back), followed by a memory-based learning task concerning astrological constellations. Learning gains were assessed based on performance on the learning task. Self-perceived mental workload, cognitive absorption and satisfaction were assessed using a post-test questionnaire. Between-group analyses using Mann-Whitney tests suggested that combining BCI and motivational factors led to more significant learning gains and an improved learning experience. No significant difference existed between the BCI without motivational factor and regular non-adaptive interface for overall learning gains, self-perceived mental workload, and cognitive absorption. However, participants who undertook the experiment with an imposed learning pace reported higher overall satisfaction with their learning experience and a higher level of temporal stress. Our findings suggest BCI's potential applicability and feasibility in improving memorization-based learning experiences. Further work should seek to optimize the BCI adaptive index and explore generalizability to other learning contexts.

From E-Commerce to the Metaverse: A Neuroscientific Analysis of Digital Consumer Behavior

The growing interest in consumer behavior in the digital environment is leading scholars and companies to focus on consumer behavior and choices on digital platforms, such as the metaverse. On this immersive digital shopping platform, consumer neuroscience provides an optimal opportunity to explore consumers' emotions and cognitions. In this study, neuroscience techniques (EEG, SC, BVP) were used to compare emotional and cognitive aspects of shopping between metaverse and traditional e-commerce platforms. Participants were asked to purchase the same product once on a metaverse platform (Second Life, SL) and once via an e-commerce website (EC). After each task, questionnaires were administered to measure perceived enjoyment, informativeness, ease of use, cognitive effort, and flow. Statistical analyses were conducted to examine differences between SL and EC at the neurophysiological and self-report levels, as well as between different stages of the purchase process. The results show that SL elicits greater cognitive engagement than EC, but it is also more mentally demanding, with a higher workload and more memorization, and fails to elicit a strong positive emotional response, leading to a poorer shopping experience. These findings provide insights not only for digital-related consumer research but also for companies to improve their metaverse shopping experience. Before investing in the platform or creating a digital retail space, companies should thoroughly analyze it, focusing on how to enhance users' cognition and emotions, ultimately promoting a better consumer experience. Despite its limitations, this pilot study sheds light on the emotional and cognitive aspects of metaverse shopping and suggests potential for further research with a consumer neuroscience approach in the metaverse field.

Physiological measures of operators' mental state in supervisory process control tasks: a scoping review

Physiological measures are often used to assess the mental state of human operators in supervisory process control tasks. However, the diversity of research approaches creates a heterogeneous landscape of empirical evidence. To map existing evidence and provide guidance to researchers and practitioners, this paper systematically reviews 109 empirical studies that report relationships between peripheral nervous system measures and mental state dimensions (e.g. mental workload, mental fatigue, stress, and vigilance) of interest. Ocular and electrocardiac measures were the most prominent measures across application fields. Most studies sought to validate such measures for reliable assessments of cognitive task demands and time on task, with measures of pupil size receiving the most empirical support. In comparison, less research examined the utility of physiological measures in predicting human task performance. This approach is discussed as an opportunity to focus on operators' individual response to cognitive task demands and to advance the state of research.

Brain waves and landscape settings: emotional responses to attractiveness

Neuro-architecture is a specific branch of architecture that studies how the physical environment can change our mental processes and influence our behaviors. One of the main purposes of this field is to use changes in brain activities as a measure to quantify attractiveness of the landscapes. In this study, we investigated how changes in elements of attractiveness influence ones' emotional perception and present the related pattern of changes in brain activities. Therefore, we implied five elements of attractiveness including mystery, visual openness, landscape or greenness, walkability, and social interaction using the Delphi method. Then, we made changes in each element separately to make the landscape more attractive and assessed their effects on a group of young adults. We used the self-assessment manikin questionnaire to measure the participants' emotional perception while the participants' brain activities were recorded using a 32-channel EEG while exposed to the landscape images. The results showed that changes in attractive elements of the landscape could significantly improve ones' emotional perception of the landscape. In addition, these changes are perceived by changing the oscillatory pattern of brain activities. We hope these findings could shed a light to use of neural markers in measurement of place attractiveness.

CAEVR: Biosignals-Driven Context-Aware Empathy in Virtual Reality

There is little research on how Virtual Reality (VR) applications can identify and respond meaningfully to users' emotional changes. In this paper, we investigate the impact of Context-Aware Empathic VR (CAEVR) on the emotional and cognitive aspects of user experience in VR. We developed a real-time emotion prediction model using electroencephalography (EEG), electrodermal activity (EDA), and heart rate variability (HRV) and used this in personalized and generalized models for emotion recognition. We then explored the application of this model in a context-aware empathic (CAE) virtual agent and an emotion-adaptive (EA) VR environment. We found a significant increase in positive emotions, cognitive load, and empathy toward the CAE agent, suggesting the potential of CAEVR environments to refine user-agent interactions. We identify lessons learned from this study and directions for future work.

A Subject-Specific Attention Index Based on the Weighted Spectral Power

As an essential cognitive function, attention has been widely studied and various indices based on EEG have been proposed for its convenience and easy availability for real-time attention monitoring. Although existing indices based on spectral power of empirical frequency bands are able to describe the attentional state in some way, the reliability still needs to be improved. This paper proposed a subject-specific attention index based on the weighted spectral power. Unlike traditional indices, the ranges of frequency bands are not empirical but obtained from subject-specific change patterns of spectral power of electroencephalograph (EEG) to overcome the great inter-subject variance. In addition, the contribution of each frequency component in the frequency band is considered different. Specifically, the ratio of power spectral density (PSD) function in attentional and inattentional state is utilized to calculate the weight to enhance the effectiveness of the proposed index. The proposed subject-specific attention index based on the weighted spectral power is evaluated on two open datasets including EEG data of a total of 44 subjects. The results of the proposed index are compared with 3 traditional attention indices using various statistical analysis methods including significance tests and distribution variance measurements. According to the experimental results, the proposed index can describe the attentional state more accurately. The proposed index respectively achieves accuracies of 86.21% and 70.00% at the 1% significance level in both the t-test and Wilcoxon rank-sum test for two datasets, which obtains improvements of 41.38% and 20.00% compared to the best result of the traditional indices. These results indicate that the proposed index provides an efficient way to measure attentional state.

EEG as a potential ground truth for the assessment of cognitive state in software development activities: A multimodal imaging study

Cognitive human error and recent cognitive taxonomy on human error causes of software defects support the intuitive idea that, for instance, mental overload, attention slips, and working memory overload are important human causes for software bugs. In this paper, we approach the EEG as a reliable surrogate to MRI-based reference of the programmer's cognitive state to be used in situations where heavy imaging techniques are infeasible. The idea is to use EEG biomarkers to validate other less intrusive physiological measures, that can be easily recorded by wearable devices and useful in the assessment of the developer's cognitive state during software development tasks. Herein, our EEG study, with the support of fMRI, presents an extensive and systematic analysis by inspecting metrics and extracting relevant information about the most robust features, best EEG channels and the best hemodynamic time delay in the context of software development tasks. From the EEG-fMRI similarity analysis performed, we found significant correlations between a subset of EEG features and the Insula region of the brain, which has been reported as a region highly related to high cognitive tasks, such as software development tasks. We concluded that despite a clear inter-subject variability of the best EEG features and hemodynamic time delay used, the most robust and predominant EEG features, across all the subjects, are related to the Hjorth parameter Activity and Total Power features, from the EEG channels F4, FC4 and C4, and considering in most of the cases a hemodynamic time delay of 4 seconds used on the hemodynamic response function. These findings should be taken into account in future EEG-fMRI studies in the context of software debugging.

Real-time estimation of EEG-based engagement in different tasks

Objective.Recent trends in brain-computer interface (BCI) research concern the passive monitoring of brain activity, which aim to monitor a wide variety of cognitive states. Engagement is such a cognitive state, which is of interest in contexts such as learning, entertainment or rehabilitation. This study proposes a novel approach for real-time estimation of engagement during different tasks using electroencephalography (EEG).Approach.Twenty-three healthy subjects participated in the BCI experiment. A modified version of the d2 test was used to elicit engagement. Within-subject classification models which discriminate between engaging and resting states were trained based on EEG recorded during a d2 test based paradigm. The EEG was recorded using eight electrodes and the classification model was based on filter-bank common spatial patterns and a linear discriminant analysis. The classification models were evaluated in cross-task applications, namely when playing Tetris at different speeds (i.e. slow, medium, fast) and when watching two videos (i.e. advertisement and landscape video). Additionally, subjects' perceived engagement was quantified using a questionnaire.Main results.The models achieved a classification accuracy of 90% on average when tested on an independent d2 test paradigm recording. Subjects' perceived and estimated engagement were found to be greater during the advertisement compared to the landscape video (p= 0.025 andp<0.001, respectively); greater during medium and fast compared to slow Tetris speed (p<0.001, respectively); not different between medium and fast Tetris speeds. Additionally, a common linear relationship was observed for perceived and estimated engagement (rrm= 0.44,p<0.001). Finally, theta and alpha band powers were investigated, which respectively increased and decreased during more engaging states.Significance.This study proposes a task-specific EEG engagement estimation model with cross-task capabilities, offering a framework for real-world applications.

Viewer Engagement in Response to Mixed and Uniform Emotional Content in Marketing Videos-An Electroencephalographic Study

This study presents the results of an experiment designed to investigate whether marketing videos containing mixed emotional content can sustain consumers interest longer compared to videos conveying a consistent emotional message. During the experiment, thirteen participants, wearing EEG (electroencephalographic) caps, were exposed to eight marketing videos with diverse emotional tones. Participant engagement was measured with an engagement index, a metric derived from the power of brain activity recorded over the frontal and parietal cortex and computed within three distinct frequency bands: theta (4-8 Hz), alpha (8-13 Hz), and beta (13-30 Hz). The outcomes indicated a statistically significant influence of emotional content type (mixed vs. consistent) on the duration of user engagement. Videos containing a mixed emotional message were notably more effective in sustaining user engagement, whereas the engagement level for videos with a consistent emotional message declined over time. The principal inference drawn from the study is that advertising materials conveying a consistent emotional message should be notably briefer than those featuring a mixed emotional message to achieve an equivalent level of message effectiveness, measured through engagement duration.

Facing emotions: real-time fMRI-based neurofeedback using dynamic emotional faces to modulate amygdala activity

Introduction

Maladaptive functioning of the amygdala has been associated with impaired emotion regulation in affective disorders. Recent advances in real-time fMRI neurofeedback have successfully demonstrated the modulation of amygdala activity in healthy and psychiatric populations. In contrast to an abstract feedback representation applied in standard neurofeedback designs, we proposed a novel neurofeedback paradigm using naturalistic stimuli like human emotional faces as the feedback display where change in the facial expression intensity (from neutral to happy or from fearful to neutral) was coupled with the participant's ongoing bilateral amygdala activity.

Methods

The feasibility of this experimental approach was tested on 64 healthy participants who completed a single training session with four neurofeedback runs. Participants were assigned to one of the four experimental groups (n = 16 per group), i.e., happy-up, happy-down, fear-up, fear-down. Depending on the group assignment, they were either instructed to "try to make the face happier" by upregulating (happy-up) or downregulating (happy-down) the amygdala or to "try to make the face less fearful" by upregulating (fear-up) or downregulating (fear-down) the amygdala feedback signal.

Results

Linear mixed effect analyses revealed significant amygdala activity changes in the fear condition, specifically in the fear-down group with significant amygdala downregulation in the last two neurofeedback runs as compared to the first run. The happy-up and happy-down groups did not show significant amygdala activity changes over four runs. We did not observe significant improvement in the questionnaire scores and subsequent behavior. Furthermore, task-dependent effective connectivity changes between the amygdala, fusiform face area (FFA), and the medial orbitofrontal cortex (mOFC) were examined using dynamic causal modeling. The effective connectivity between FFA and the amygdala was significantly increased in the happy-up group (facilitatory effect) and decreased in the fear-down group. Notably, the amygdala was downregulated through an inhibitory mechanism mediated by mOFC during the first training run.

Discussion

In this feasibility study, we intended to address key neurofeedback processes like naturalistic facial stimuli, participant engagement in the task, bidirectional regulation, task congruence, and their influence on learning success. It demonstrated that such a versatile emotional face feedback paradigm can be tailored to target biased emotion processing in affective disorders.

The role of engagement and arousal in emotion regulation: an EEG study

Cognitive reappraisal and expressive suppression are well-studied strategies of emotion regulation (ER). However, the results on their physiological basis are controversial. While in some studies, ER was accompanied by the inhibition of the nervous system, others suggested that ER even might increase arousal and engagement. We calculated the inter-subject correlation (ISC) and indices of engagement, valence and arousal of EEG during suppression, reappraisal, or natural watching of neutral and negative videos. First, both suppression and reappraisal provoked a higher ISC in comparison with watching negative or neutral videos. We consider this as a marker of engagement to the task and feedback processing required for ER. Second, the engagement index was lower during ER compared to watching negative videos in central electrodes, whereas both strategies provoked a higher engagement in frontal electrodes. Third, the arousal index of EEG was higher during all negative conditions; therefore, regulation required a certain level of arousal. In summary, different EEG measures seem to be sensitive to different aspects of ER.

Lean back or lean in? Exploring social loafing in human-robot teams

Introduction: Thanks to technological advances, robots are now being used for a wide range of tasks in the workplace. They are often introduced as team partners to assist workers. This teaming is typically associated with positive effects on work performance and outcomes. However, little is known about whether typical performance-reducing effects that occur in human teams also occur in human-robot teams. For example, it is not clear whether social loafing, defined as reduced individual effort on a task performed in a team compared to a task performed alone, can also occur in human-robot teams. Methods: We investigated this question in an experimental study in which participants worked on an industrial defect inspection task that required them to search for manufacturing defects on circuit boards. One group of participants worked on the task alone, while the other group worked with a robot team partner, receiving boards that had already been inspected by the robot. The robot was quite reliable and marked defects on the boards before handing them over to the human. However, it missed 5 defects. The dependent behavioural measures of interest were effort, operationalised as inspection time and area inspected on the board, and defect detection performance. In addition, subjects rated their subjective effort, performance, and perceived responsibility for the task. Results: Participants in both groups inspected almost the entire board surface, took their time searching, and rated their subjective effort as high. However, participants working in a team with the robot found on average 3.3 defects. People working alone found significantly more defects on these 5 occasions-an average of 4.2. Discussion: This suggests that participants may have searched the boards less attentively when working with a robot team partner. The participants in our study seemed to have maintained the motor effort to search the boards, but it appears that the search was carried out with less mental effort and less attention to the information being sampled. Changes in mental effort are much harder to measure, but need to be minimised to ensure good performance.

The Synergy Zone: Connecting the Mind, Brain, and Heart for the Ideal Classroom Learning Environment

This paper proposes a new perspective on implementing neuroeducation in the classroom. The pandemic exacerbated the mental health issues of faculty and students, creating a mental health crisis that impairs learning. It is important to get our students back in "the zone", both cognitively and emotionally, by creating an ideal learning environment for capturing our students and keeping them-the Synergy Zone. Research that examines the classroom environment often focuses on the foreground-instructors' organizational and instructional aspects and content. However, the emotional climate of the classroom affects student well-being. This emotional climate would ideally exhibit the brain states of engagement, attention, connection, and enjoyment by addressing the mind, brain, and heart. This ideal learning environment would be achieved by combining proposed practices derived from three areas of research: flow theory, brain synchronization, and positive emotion with heart engagement. Each of these enhances the desired brain states in a way that the whole is greater than the sum of the individual parts. I call this the Synergy Zone. A limitation of this proposed model is that implementation of some aspects may be challenging, and professional development resources might be needed. This essay presenting this perspective provides the relevant scientific research and the educational implications of implementation.

Crew Autonomy During Simulated Medical Event Management on Long Duration Space Exploration Missions

OBJECTIVE

Our primary aim was to investigate crew performance during medical emergencies with and without ground-support from a flight surgeon located at mission control.

BACKGROUND

There are gaps in knowledge regarding the potential for unanticipated in-flight medical events to affect crew health and capacity, and potentially compromise mission success. Additionally, ground support may be impaired or periodically absent during long duration missions.

METHOD

We reviewed video recordings of 16 three-person flight crews each managing four unique medical events in a fully immersive spacecraft simulator. Crews were randomized to two conditions: with and without telemedical flight surgeon (FS) support. We assessed differences in technical performance, behavioral skills, and cognitive load between groups.

RESULTS

Crews with FS support performed better clinically, were rated higher on technical skills, and completed more clinical tasks from the medical checklists than crews without FS support. Crews with FS support also had better behavioral/non-technical skills (information exchange) and reported significantly lower cognitive demand during the medical event scenarios on the NASA-TLX scale, particularly in mental demand and temporal demand. There was no significant difference between groups in time to treat or in objective measures of cognitive demand derived from heart rate variability and electroencephalography.

CONCLUSION

Medical checklists are necessary but not sufficient to support high levels of autonomous crew performance in the absence of real-time flight surgeon support.

APPLICATION

Potential applications of this research include developing ground-based and in-flight training countermeasures; informing policy regarding autonomous spaceflight, and design of autonomous clinical decision support systems.

Ratio Indexes Based on Spectral Electroencephalographic Brainwaves for Assessment of Mental Involvement: A Systematic Review

BACKGROUND

This review systematically examined the scientific literature about electroencephalogram-derived ratio indexes used to assess human mental involvement, in order to deduce what they are, how they are defined and used, and what their best fields of application are. (2) Methods: The review was carried out according to the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines. (3) Results: From the search query, 82 documents resulted. The majority (82%) were classified as related to mental strain, while 12% were classified as related to sensory and emotion aspects, and 6% to movement. The electroencephalographic electrode montage used was low-density in 13%, high-density in 6% and very-low-density in 81% of documents. The most used electrode positions for computation of involvement indexes were in the frontal and prefrontal cortex. Overall, 37 different formulations of involvement indexes were found. None of them could be directly related to a specific field of application. (4) Conclusions: Standardization in the definition of these indexes is missing, both in the considered frequency bands and in the exploited electrodes. Future research may focus on the development of indexes with a unique definition to monitor and characterize mental involvement.

EEG and peripheral markers of viewer ratings: a study of short films

Introduction

Cinema is an important part of modern culture, influencing millions of viewers. Research suggested many models for the prediction of film success, one of them being the use of neuroscientific tools. The aim of our study was to find physiological markers of viewer perception and correlate them to short film ratings given by our subjects. Short films are used as a test case for directors and screenwriters and can be created to raise funding for future projects; however, they have not been studied properly with physiological methods.

Methods

We recorded electroencephalography (18 sensors), facial electromyography (corrugator supercilii and zygomaticus major), photoplethysmography, and skin conductance in 21 participants while watching and evaluating 8 short films (4 dramas and 4 comedies). Also, we used machine learning (CatBoost, SVR) to predict the exact rating of each film (from 1 to 10), based on all physiological indicators. In addition, we classified each film as low or high rated by our subjects (with Logistic Regression, KNN, decision tree, CatBoost, and SVC).

Results

The results showed that ratings did not differ between genres. Corrugator supercilii activity ("frowning" muscle) was larger when watching dramas; whereas zygomaticus major ("smiling" muscle) activity was larger during the watching of comedies. Of all somatic and vegetative markers, only zygomaticus major activity, PNN50, SD1/SD2 (heart rate variability parameters) positively correlated to the film ratings. The EEG engagement indices, beta/(alpha+theta) and beta/alpha correlated positively with the film ratings in the majority of sensors. Arousal (beta_F3 + beta_F4)/(alpha_F3 + alpha_F4), and valence (alpha_F4/beta_F4) - (alpha_F3/beta_F3) indices also correlated positively to film ratings. When we attempted to predict exact ratings, MAPE was 0.55. As for the binary classification, logistic regression yielded the best values (area under the ROC curve = 0.62) than other methods (0.51-0.60).

Discussion

Overall, we revealed EEG and peripheral markers, which reflect viewer ratings and can predict them to a certain extent. In general, high film ratings can reflect a fusion of high arousal and different valence, positive valence being more important. These findings broaden our knowledge about the physiological basis of viewer perception and can be potentially used at the stage of film production.

Neurophysiological correlates of tactile width discrimination in humans

Introduction

Tactile information processing requires the integration of sensory, motor, and cognitive information. Width discrimination has been extensively studied in rodents, but not in humans.

Methods

Here, we describe Electroencephalography (EEG) signals in humans performing a tactile width discrimination task. The first goal of this study was to describe changes in neural activity occurring during the discrimination and the response periods. The second goal was to relate specific changes in neural activity to the performance in the task.

Results

Comparison of changes in power between two different periods of the task, corresponding to the discrimination of the tactile stimulus and the motor response, revealed the engagement of an asymmetrical network associated with fronto-temporo-parieto-occipital electrodes and across multiple frequency bands. Analysis of ratios of higher [Ratio 1: (0.5-20 Hz)/(0.5-45 Hz)] or lower frequencies [Ratio 2: (0.5-4.5 Hz)/(0.5-9 Hz)], during the discrimination period revealed that activity recorded from frontal-parietal electrodes was correlated to tactile width discrimination performance between-subjects, independently of task difficulty. Meanwhile, the dynamics in parieto-occipital electrodes were correlated to the changes in performance within-subjects (i.e., between the first and the second blocks) independently of task difficulty. In addition, analysis of information transfer, using Granger causality, further demonstrated that improvements in performance between blocks were characterized by an overall reduction in information transfer to the ipsilateral parietal electrode (P4) and an increase in information transfer to the contralateral parietal electrode (P3).

Discussion

The main finding of this study is that fronto-parietal electrodes encoded between-subjects' performances while parieto-occipital electrodes encoded within-subjects' performances, supporting the notion that tactile width discrimination processing is associated with a complex asymmetrical network involving fronto-parieto-occipital electrodes.

Do you like it or not? Identifying preference using an electroencephalogram during the viewing of short videos

Accurately predicting whether a short video will be liked by viewers is a topic of interest to media researchers. This study used an electroencephalogram (EEG) to record neural activity in 109 participants as they watched short videos (16 clips per person) to see which neural signals reflected viewers' preferences. The results showed that, compared with the short videos they disliked, individuals would experience positive emotions [indexed by a higher theta power, lower (beta - theta)/(beta + theta) score], more relaxed states (indexed by a lower beta power), lower levels of mental engagement and alertness [indexed by a lower beta/(alpha + theta) score], and devote more attention (indexed by lower alpha/theta) when watching short videos they liked. We further used artificial neural networks to classify the neural signals of different preferences induced by short videos. The classification accuracy was the highest when using data from bands over the whole brain, which was 75.78%. These results may indicate the potential of EEG measurement to evaluate the subjective preferences of individuals for short videos.

BCI Applications to Creativity: Review and Future Directions, from little-c to C2

BCI devices are increasingly being used to create interactive interfaces between users and their own psychophysiological signals. Over the years, these systems have seen strong development as they can enable people with limited mobility to make certain decisions to alter their environment. Additionally, their portability and ease of use have allowed a field of research to flourish for the study of cognitive and emotional processes in natural settings. The study of creativity, especially little creativity (little-c), is one example, although the results of this cutting-edge research are often poorly systematized. The purpose of the present paper, therefore, was to conduct a scoping review to describe and systematize the various studies that have been conducted on the application potential of BCI to the field of creativity. Twenty-two papers were selected that collect information on different aspects of creativity, including clinical applications; art experience in settings with high ecological validity; BCI for creative content creation, and participants' engagement. Critical issues and potentialities of this promising area of study are also presented. Implications for future developments towards multi-brain creativity settings and C² are discussed.

Confused or not: decoding brain activity and recognizing confusion in reasoning learning using EEG

Objective.Confusion is the primary epistemic emotion in the learning process, influencing students' engagement and whether they become frustrated or bored. However, research on confusion in learning is still in its early stages, and there is a need to better understand how to recognize it and what electroencephalography (EEG) signals indicate its occurrence. The present work investigates confusion during reasoning learning using EEG, and aims to fill this gap with a multidisciplinary approach combining educational psychology, neuroscience and computer science.Approach.First, we design an experiment to actively and accurately induce confusion in reasoning. Second, we propose a subjective and objective joint labeling technique to address the label noise issue. Third, to confirm that the confused state can be distinguished from the non-confused state, we compare and analyze the mean band power of confused and unconfused states across five typical bands. Finally, we present an EEG database for confusion analysis, together with benchmark results from conventional (Naive Bayes, Support Vector Machine, Random Forest, and Artificial Neural Network) and end-to-end (Long Short Term Memory, Residual Network, and EEGNet) machine learning methods.Main results.Findings revealed: 1. Significant differences in the power of delta, theta, alpha, beta and lower gamma between confused and non-confused conditions; 2. A higher attentional and cognitive load when participants were confused; and 3. The Random Forest algorithm with time-domain features achieved a high accuracy/F1 score (88.06%/0.88 for the subject-dependent approach and 84.43%/0.84 for the subject-independent approach) in the binary classification of the confused and non-confused states.Significance.The study advances our understanding of confusion and provides practical insights for recognizing and analyzing it in the learning process. It extends existing theories on the differences between confused and non-confused states during learning and contributes to the cognitive-affective model. The research enables researchers, educators, and practitioners to monitor confusion, develop adaptive systems, and test recognition approaches.

Effects of reading contextualized physics problems among men and women: A psychophysiological approach

To counteract declining interest in science, contextualizing course material has been suggested, despite little evidence supporting this strategy. We assessed how reading physics problems in different contexts-none, technical, or humanistic-impacted performance and implicit cognitive and affective situational interest (SI) among undergraduate men and women (n = 60). We hypothesized that contextualized problems would increase cognitive SI, boosting performance. We also investigated existing hypotheses that this influence would be stronger when contexts matched stereotypical gender interests. Pupillometric and electroencephalographic data served to indicate cognitive SI, while electrodermal activity (EDA) and valence were measures of affective SI. Significantly higher valence was observed in decontextualized than humanistic problems (p = 0.003) specifically among men (p < 0.001). Greater EDA (p = 0.019) and decontextualized problems (p < 0.001) yielded greater performance than contextualized problems for all participants. Results emphasize the importance of affective SI and of avoiding gender biases in curricular development. This study encourages caution if implementing contextualization.

On the accuracy of code complexity metrics: A neuroscience-based guideline for improvement

Complexity is the key element of software quality. This article investigates the problem of measuring code complexity and discusses the results of a controlled experiment to compare different views and methods to measure code complexity. Participants (27 programmers) were asked to read and (try to) understand a set of programs, while the complexity of such programs is assessed through different methods and perspectives: (a) classic code complexity metrics such as McCabe and Halstead metrics, (b) cognitive complexity metrics based on scored code constructs, (c) cognitive complexity metrics from state-of-the-art tools such as SonarQube, (d) human-centered metrics relying on the direct assessment of programmers' behavioral features (e.g., reading time, and revisits) using eye tracking, and (e) cognitive load/mental effort assessed using electroencephalography (EEG). The human-centered perspective was complemented by the subjective evaluation of participants on the mental effort required to understand the programs using the NASA Task Load Index (TLX). Additionally, the evaluation of the code complexity is measured at both the program level and, whenever possible, at the very low level of code constructs/code regions, to identify the actual code elements and the code context that may trigger a complexity surge in the programmers' perception of code comprehension difficulty. The programmers' cognitive load measured using EEG was used as a reference to evaluate how the different metrics can express the (human) difficulty in comprehending the code. Extensive experimental results show that popular metrics such as V(g) and the complexity metric from SonarSource tools deviate considerably from the programmers' perception of code complexity and often do not show the expected monotonic behavior. The article summarizes the findings in a set of guidelines to improve existing code complexity metrics, particularly state-of-the-art metrics such as cognitive complexity from SonarSource tools.

The mitigation of the executive vigilance decrement via HD-tDCS over the right posterior parietal cortex and its association with neural oscillations

Vigilance-maintaining a prolonged state of preparation to detect and respond to specific yet unpredictable environmental changes-usually decreases across prolonged tasks, causing potentially severe real-life consequences, which could be mitigated through transcranial direct current stimulation (tDCS). The present study aimed at replicating previous mitigatory effects observed with anodal high-definition tDCS (HD-tDCS) over the right posterior parietal cortex (rPPC) while extending the analyses on electrophysiological measures associated with vigilance. In sum, 60 participants completed the ANTI-Vea task while receiving anodal (1.5 mA, n = 30) or sham (0 mA, n = 30) HD-tDCS over the rPPC for ~ 28 min. EEG recordings were completed before and after stimulation. Anodal HD-tDCS specifically mitigated executive vigilance (EV) and reduced the alpha power increment across time-on-task while increasing the gamma power increment. To further account for the observed behavioral and physiological outcomes, a new index of Alphaparietal/Gammafrontal is proposed. Interestingly, the increment of this Alphaparietal/Gammafrontal Index with time-on-task is associated with a steeper EV decrement in the sham group, which was mitigated by anodal HD-tDCS. We highlight the relevance of replicating mitigatory effects of tDCS and the need to integrate conventional and novel physiological measures to account for how anodal HD-tDCS can be used to modulate cognitive performance.

The neurophysiology of intraoperative error: An EEG study of trainee surgeons during robotic-assisted surgery simulations

Surgeons operate in mentally and physically demanding workspaces where the impact of error is highly consequential. Accurately characterizing the neurophysiology of surgeons during intraoperative error will help guide more accurate performance assessment and precision training for surgeons and other teleoperators. To better understand the neurophysiology of intraoperative error, we build and deploy a system for intraoperative error detection and electroencephalography (EEG) signal synchronization during robot-assisted surgery (RAS). We then examine the association between EEG data and detected errors. Our results suggest that there are significant EEG changes during intraoperative error that are detectable irrespective of surgical experience level.

Central EEG Beta/Alpha Ratio Predicts the Population-Wide Efficiency of Advertisements

Recent studies have demonstrated that the brain activity of a group of people can be used to forecast choices at the population level. In this study, we attempted to neuroforecast aggregate consumer behavior of Internet users. During our electroencephalography (EEG) and eye-tracking study, participants were exposed to 10 banners that were also used in the real digital marketing campaign. In the separate online study, we additionally collected self-reported preferences for the same banners. We explored the relationship between the EEG, eye-tracking, and behavioral indexes obtained in our studies and the banners' aggregate efficiency provided by the large food retailer based on the decisions of 291,301 Internet users. An EEG-based engagement index (central beta/alpha ratio) significantly correlated with the aggregate efficiency of banners. Furthermore, our multiple linear regression models showed that a combination of eye-tracking, EEG and behavioral measurements better explained the market-level efficiency of banner advertisements than each measurement alone. Overall, our results confirm that neural signals of a relatively small number of individuals can forecast aggregate behavior at the population level.

Predictions of task using neural modeling

Introduction

A well-designed brain-computer interface (BCI) can make accurate and reliable predictions of a user's state through the passive assessment of their brain activity; in turn, BCI can inform an adaptive system (such as artificial intelligence, or AI) to intelligently and optimally aid the user to maximize the human-machine team (HMT) performance. Various groupings of spectro-temporal neural features have shown to predict the same underlying cognitive state (e.g., workload) but vary in their accuracy to generalize across contexts, experimental manipulations, and beyond a single session. In our work we address an outstanding challenge in neuroergonomic research: we quantify if (how) identified neural features and a chosen modeling approach will generalize to various manipulations defined by the same underlying psychological construct, (multi)task cognitive workload.

Methods

To do this, we train and test 20 different support vector machine (SVM) models, each given a subset of neural features as recommended from previous research or matching the capabilities of commercial devices. We compute each model's accuracy to predict which (monitoring, communications, tracking) and how many (one, two, or three) task(s) were completed simultaneously. Additionally, we investigate machine learning model accuracy to predict task(s) within- vs. between-sessions, all at the individual-level.

Results

Our results indicate gamma activity across all recording locations consistently outperformed all other subsets from the full model. Our work demonstrates that modelers must consider multiple types of manipulations which may each influence a common underlying psychological construct.

Discussion

We offer a novel and practical modeling solution for system designers to predict task through brain activity and suggest next steps in expanding our framework to further contribute to research and development in the neuroergonomics community. Further, we quantified the cost in model accuracy should one choose to deploy our BCI approach using a mobile EEG-systems with fewer electrodes-a practical recommendation from our work.

Assessing student engagement from facial behavior in on-line learning

The automatic monitoring and assessment of the engagement level of learners in distance education may help in understanding problems and providing personalized support during the learning process. This article presents a research aiming to investigate how student engagement level can be assessed from facial behavior and proposes a model based on Long Short-Term Memory (LSTM) networks to predict the level of engagement from facial action units, gaze, and head poses. The dataset used to learn the model is the one of the EmotiW 2019 challenge datasets. In order to test its performance in learning contexts, an experiment, involving students attending an online lecture, was performed. The aim of the study was to compare the self-evaluation of the engagement perceived by the students with the one assessed by the model. During the experiment we collected videos of students behavior and, at the end of each session, we asked students to answer a questionnaire for assessing their perceived engagement. Then, the collected videos were analyzed automatically with a software that implements the model and provides an interface for the visual analysis of the model outcome. Results show that, globally, engagement prediction from students' facial behavior was weakly correlated to their subjective answers. However, when considering only the emotional dimension of engagement, this correlation is stronger and the analysis of facial action units and head pose (facial movements) are positively correlated with it, while there is an inverse correlation with the gaze, meaning that the more the student's feels engaged the less are the gaze movements.

Ambient Light Conveying Reliability Improves Drivers’ Takeover Performance without Increasing Mental Workload

Drivers of L3 automated vehicles (AVs) are not required to continuously monitor the AV system. However, they must be prepared to take over when requested. Therefore, it is necessary to design an in-vehicle environment that allows drivers to adapt their levels of preparedness to the likelihood of control transition. This study evaluates ambient in-vehicle lighting that continuously communicates the current level of AV reliability, specifically on how it could influence drivers’ take-over performance and mental workload (MW). We conducted an experiment in a driving simulator with 42 participants who experienced 10 take-over requests (TORs). The experimental group experienced a four-stage ambient light display that communicated the current level of AV reliability, which was not provided to the control group. The experimental group demonstrated better take-over performance, based on lower vehicle jerks. Notably, perceived MW did not differ between the groups, and the EEG indices of MW (frontal theta power, parietal alpha power, Task–Load Index) did not differ between the groups. These findings suggest that communicating the current level of reliability using ambient light might help drivers be better prepared for TORs and perform better without increasing their MW.

Understanding Clinical Reasoning through Visual Scanpath and Brain Activity Analysis

This paper presents an experimental study that analyzes learners’ visual behaviour and brain activity in clinical reasoning. An acquisition protocol was defined to record eye tracking and EEG data from 15 participants as they interact with a computer-based learning environment called Amnesia, a medical simulation system that assesses the analytical skills of novice medicine students while they solve patient cases. We use gaze data to assess learners’ visual focus and present our methodology to track learners’ reasoning process through scanpath pattern analysis. We also describe our methodology for examining learners’ cognitive states using mental engagement and workload neural indexes. Finally, we discuss the relationship between gaze path information and EEG and how our analyses can lead to new forms of clinical diagnostic reasoning assessment.

Emotional Activity Is Negatively Associated With Cognitive Load in Multimedia Learning: A Case Study With EEG Signals

We aimed to investigate the relationship between emotional activity and cognitive load during multimedia learning from an emotion dynamics perspective using electroencephalography (EEG) signals. Using a between-subjects design, 42 university students were randomly assigned to two video lecture conditions (color-coded vs. grayscale). While the participants watched the assigned video, their EEG signals were recorded. After processing the EEG signals, we employed the correlation-based feature selector (CFS) method to identify emotion-related subject-independent features. We then put these features into the Isomap model to obtain a one-dimensional trajectory of emotional changes. Next, we used the zero-crossing rate (ZCR) as the quantitative characterization of emotional changes ZCR EC . Meanwhile, we extracted cognitive load-related features to analyze the degree of cognitive load (CLI). We employed a linear regression fitting method to study the relationship between ZCR EC and CLI. We conducted this study from two perspectives. One is the frequency domain method (wavelet feature), and the other is the non-linear dynamic method (entropy features). The results indicate that emotional activity is negatively associated with cognitive load. These findings have practical implications for designing video lectures for multimedia learning. Learning material should reduce learners' cognitive load to keep their emotional experience at optimal levels to enhance learning.

Virtual Reality Adaptation Using Electrodermal Activity to Support the User Experience

Virtual reality is increasingly used for tasks such as work and education. Thus, rendering scenarios that do not interfere with such goals and deplete user experience are becoming progressively more relevant. We present a physiologically adaptive system that optimizes the virtual environment based on physiological arousal, i.e., electrodermal activity. We investigated the usability of the adaptive system in a simulated social virtual reality scenario. Participants completed an n-back task (primary) and a visual detection (secondary) task. Here, we adapted the visual complexity of the secondary task in the form of the number of non-player characters of the secondary task to accomplish the primary task. We show that an adaptive virtual reality can improve users’ comfort by adapting to physiological arousal regarding the task complexity. Our findings suggest that physiologically adaptive virtual reality systems can improve users’ experience in a wide range of scenarios.

Distraction classification during target tracking tasks involving target and cursor flickering using EEGNet

Keeping patients from being distracted while performing motor rehabilitation is important. An EEG-based biofeedback strategy has been introduced to help encourage participants to focus their attention on rehabilitation tasks. Here, we suggest a BCI-based monitoring method using a flickering cursor and target that can evoke a steady-state visually evoked potential (SSVEP) using the fact that the SSVEP is modulated by a patient's attention. Fifteen healthy individuals performed a tracking task where the target and cursor flickered. There were two tracking sessions, one with and one without flickering stimuli, and each session had four conditions in which each had no distractor (non-D), a visual (vis-D) or cognitive distractor (cog-D), and both distractors (both-D). An EEGNet was trained as a classifier using only non-D and both-D conditions to classify whether it was distracted and validated with a leave-one-subject-out scheme. The results reveal that the proposed classifier demonstrates superior performance when using data from the task with the flickering stimuli compared to the case without the flickering stimuli. Furthermore, the observed classification likelihood was between those corresponding to the non-D and both-D when using the trained EEGNet. This suggests that the classifier trained for the two conditions could also be used to measure the level of distraction by windowing and averaging the outcomes. Therefore, the proposed method is advantageous because it can reveal a robust and continuous level of patient distraction. This facilitates its successful application to the rehabilitation systems that use computerized technology, such as virtual reality to encourage patient engagement.

EEG-based measurement system for monitoring student engagement in learning 4.0

A wearable system for the personalized EEG-based detection of engagement in learning 4.0 is proposed. In particular, the effectiveness of the proposed solution is assessed by means of the classification accuracy in predicting engagement. The system can be used to make an automated teaching platform adaptable to the user, by managing eventual drops in the cognitive and emotional engagement. The effectiveness of the learning process mainly depends on the engagement level of the learner. In case of distraction, lack of interest or superficial participation, the teaching strategy could be personalized by an automatic modulation of contents and communication strategies. The system is validated by an experimental case study on twenty-one students. The experimental task was to learn how a specific human-machine interface works. Both the cognitive and motor skills of participants were involved. De facto standard stimuli, namely (1) cognitive task (Continuous Performance Test), (2) music background (Music Emotion Recognition-MER database), and (3) social feedback (Hermans and De Houwer database), were employed to guarantee a metrologically founded reference. In within-subject approach, the proposed signal processing pipeline (Filter bank, Common Spatial Pattern, and Support Vector Machine), reaches almost 77% average accuracy, in detecting both cognitive and emotional engagement.

Degraded States of Engagement in Air Traffic Control

Safety studies have identified attention as a recurring cause of incidents and accidents in air traffic control. However, little is known of the precise attentional states that lead to degraded ATC performance. Therefore, we surveyed 150 French en route air traffic controllers on the causes of and impacts on perceived cooperation, safety, and performance of seven degraded attentional states from the literature: task-related and task-unrelated mind wandering, mental overload, inattentional deafness and blindness, attentional entropy, and perseveration. Our findings indicated that task-related and task-unrelated mind wandering were the most prevalent but had the least impact on perceived safety. Conversely, inattentional blindness and attentional entropy were less reported but were considered a significant safety concern, while inattentional deafness affected cooperation. Most states were experienced in workload levels consistent with the literature. However, no other factor such as shift work was identified as a cause of these states. Overall, these findings suggest that “attention” is not a specific enough subject for ATC, as attentional issues can occur in various conditions and have different impacts. As far as safety is concerned, inattentional blindness should be the prime target for further research. Neuroergonomics in particular could help develop dynamic countermeasures to mitigate its impact.

Effectiveness of Electricity-Saving Communication Campaigns: Neurophysiological Approach

Public communication campaigns are among the tools for promoting electricity saving. A crucial task in the process of creating a campaign is to design a simple message to effectively reach the average consumer. It is a beneficial practice to create alternative messages and pretest them to find the most effective. The research methodology during pretesting includes both quantitative and qualitative methods. However, it is believed that the outcomes obtained with the use of conventional techniques are not fully reliable. Therefore, the following question arises: What additional research methods should be applied at the stage of testing the message of a communication campaign so that its effectiveness can be assessed more reliably and/or improved even before its broadcast? In this study, we aim to present the possibility of applying cognitive neuroscience methods in conjunction with a questionnaire to experimentally check the effectiveness of the message using the example of selected electricity-saving communication campaigns. The key results of this study indicate that merging conscious and subconscious reactions to media messages allows us to gain new knowledge that can be used in the future to improve the communication campaign effectiveness. Our investigation showed the benefits that can be obtained by synergizing traditional research methods with neuroscientific approaches.

Cognitive Workload of Tugboat Captains in Realistic Scenarios: Adaptive Spatial Filtering for Transfer Between Conditions

Changing and often class-dependent non-stationarities of signals are a big challenge in the transfer of common findings in cognitive workload estimation using Electroencephalography (EEG) from laboratory experiments to realistic scenarios or other experiments. Additionally, it often remains an open question whether actual cognitive workload reflected by brain signals was the main contribution to the estimation or discriminative and class-dependent muscle and eye activity, which can be secondary effects of changing workload levels. Within this study, we investigated a novel approach to spatial filtering based on beamforming adapted to changing settings. We compare it to no spatial filtering and Common Spatial Patterns (CSP). We used a realistic maneuvering task, as well as an auditory n-back secondary task on a tugboat simulator as two different conditions to induce workload changes on professional tugboat captains. Apart from the typical within condition classification, we investigated the ability of the different classification methods to transfer between the n-back condition and the maneuvering task. The results show a clear advantage of the proposed approach over the others in the challenging transfer setting. While no filtering leads to lowest within-condition normalized classification loss on average in two scenarios (22 and 10%), our approach using adaptive beamforming (30 and 18%) performs comparably to CSP (33 and 15%). Importantly, in the transfer from one to another setting, no filtering and CSP lead to performance around chance level (45 to 53%), while our approach in contrast is the only one capable of classifying in all other scenarios (34 and 35%) with a significant difference from chance level. The changing signal composition over the scenarios leads to a need to adapt the spatial filtering in order to be transferable. With our approach, the transfer is successful due to filtering being optimized for the extraction of neural components and additional investigation of their scalp patterns revealed mainly neural origin. Interesting findings are that rather the patterns slightly change between conditions. We conclude that the approaches with low normalized loss depend on eye and muscle activity which is successful for classification within conditions, but fail in the classifier transfer since eye and muscle contributions are highly condition-specific.

Measuring and Modeling the Effect of Audio on Human Focus in Everyday Environments Using Brain-Computer Interface Technology

The goal of this study was to investigate the effect of audio listened to through headphones on subjectively reported human focus levels, and to identify through objective measures the properties that contribute most to increasing and decreasing focus in people within their regular, everyday environment. Participants (N = 62, 18–65 years) performed various tasks on a tablet computer while listening to either no audio (silence), popular audio playlists designed to increase focus (pre-recorded music arranged in a particular sequence of songs), or engineered soundscapes that were personalized to individual listeners (digital audio composed in real-time based on input parameters such as heart rate, time of day, location, etc.). Audio stimuli were delivered to participants through headphones while their brain signals were simultaneously recorded by a portable electroencephalography headband. Participants completed four 1-h long sessions at home during which different audio played continuously in the background. Using brain-computer interface technology for brain decoding and based on an individual’s self-report of their focus, we obtained individual focus levels over time and used this data to analyze the effects of various properties of the sounds contained in the audio content. We found that while participants were working, personalized soundscapes increased their focus significantly above silence (p = 0.008), while music playlists did not have a significant effect. For the young adult demographic (18–36 years), all audio tested was significantly better than silence at producing focus (p = 0.001–0.009). Personalized soundscapes increased focus the most relative to silence, but playlists of pre-recorded songs also increased focus significantly during specific time intervals. Ultimately we found it is possible to accurately predict human focus levels a priori based on physical properties of audio content. We then applied this finding to compare between music genres and revealed that classical music, engineered soundscapes, and natural sounds were the best genres for increasing focus, while pop and hip-hop were the worst. These insights can enable human and artificial intelligence composers to produce increases or decreases in listener focus with high temporal (millisecond) precision. Future research will include real-time adaptation of audio for other functional objectives beyond affecting focus, such as affecting listener enjoyment, drowsiness, stress and memory.

Neurophysiological Verbal Working Memory Patterns in Children: Searching for a Benchmark of Modality Differences in Audio/Video Stimuli Processing

Exploration of specific brain areas involved in verbal working memory (VWM) is a powerful but not widely used tool for the study of different sensory modalities, especially in children. In this study, for the first time, we used electroencephalography (EEG) to investigate neurophysiological similarities and differences in response to the same verbal stimuli, expressed in the auditory and visual modality during the n-back task with varying memory load in children. Since VWM plays an important role in learning ability, we wanted to investigate whether children elaborated the verbal input from auditory and visual stimuli through the same neural patterns and if performance varies depending on the sensory modality. Performance in terms of reaction times was better in visual than auditory modality (p = 0.008) and worse as memory load increased regardless of the modality (p < 0.001). EEG activation was proportionally influenced by task level and was evidenced in theta band over the prefrontal cortex (p = 0.021), along the midline (p = 0.003), and on the left hemisphere (p = 0.003). Differences in the effects of the two modalities were seen only in gamma band in the parietal cortices (p = 0.009). The values of a brainwave-based engagement index, innovatively used here to test children in a dual-modality VWM paradigm, varied depending on n-back task level (p = 0.001) and negatively correlated (p = 0.002) with performance, suggesting its computational effectiveness in detecting changes in mental state during memory tasks involving children. Overall, our findings suggest that auditory and visual VWM involved the same brain cortical areas (frontal, parietal, occipital, and midline) and that the significant differences in cortical activation in theta band were more related to memory load than sensory modality, suggesting that VWM function in the child's brain involves a cross-modal processing pattern.

Making the Best Out of IT: Design and Development of Exergames for Older Adults With Mild Neurocognitive Disorder - A Methodological Paper

Background: Utilizing information technology (IT) systems, for example in form of computerized cognitive screening or exergame-based (also called active videogames) training, has gained growing interest for supporting healthy aging and to detect, prevent and treat neurocognitive disorders (NCD). To ameliorate the effectiveness of exergaming, the neurobiological mechanisms as well as the most effective components for exergame-based training remain to be established. At the same time, it is important to account for the end-users' capabilities, preferences, and therapeutic needs during the design and development process to foster the usability and acceptance of the resulting program in clinical practice. This will positively influence adherence to the resulting exergame-based training program, which, in turn, favors more distinct training-related neurobiological effects. Objectives and Methods: This methodological paper describes the design and development process of novel exergame-based training concepts guided by a recently proposed methodological framework: The 'Multidisciplinary Iterative Design of Exergames (MIDE): A Framework for Supporting the Design, Development, and Evaluation of Exergames for Health' (Li et al., 2020). Case Study: A step-by-step application of the MIDE-framework as a specific guidance in an ongoing project aiming to design, develop, and evaluate an exergame-based training concept with the aim to halt and/or reduce cognitive decline and improve quality of life in older adults with mild neurocognitive disorder (mNCD) is illustrated. Discussion and Conclusion: The development of novel exergame-based training concepts is greatly facilitated when it is based on a theoretical framework (e.g., the MIDE-framework). Applying this framework resulted in a structured, iterative, and evidence-based approach that led to the identification of multiple key requirements for the exergame design as well as the training components that otherwise may have been overlooked or neglected. This is expected to foster the usability and acceptance of the resulting exergame intervention in "real life" settings. Therefore, it is strongly recommended to implement a theoretical framework (e.g., the MIDE-framework) for future research projects in line with well-known checklists to improve completeness of reporting and replicability when serious games for motor-cognitive rehabilitation purposes are to be developed.

Multi-Parameter Physiological State Monitoring in Target Detection Under Real-World Settings

Mental state changes induced by stimuli under experimental settings or by daily events in real life affect task performance and are entwined with physical and mental health. In this study, we developed a physiological state indicator with five parameters that reflect the subject's real-time physiological states based on online EEG signal processing. These five parameters are attention, fatigue, stress, and the brain activity shifts of the left and right hemispheres. We designed a target detection experiment modified by a cognitive attention network test for validating the effectiveness of the proposed indicator, as such conditions would better approximate a real chaotic environment. Results demonstrated that attention levels while performing the target detection task were significantly higher than during rest periods, but also exhibited a decay over time. In contrast, the fatigue level increased gradually and plateaued by the third rest period. Similar to attention levels, the stress level decreased as the experiment proceeded. These parameters are therefore shown to be highly correlated to different stages of the experiment, suggesting their usage as primary factors in passive brain-computer interfaces (BCI). In addition, the left and right brain activity indexes reveal the EEG neural modulations of the corresponding hemispheres, which set a feasible reference of activation for an active BCI control system, such as one executing motor imagery tasks. The proposed indicator is applicable to potential passive and active BCI applications for monitoring the subject's physiological state change in real-time, along with providing a means of evaluating the associated signal quality to enhance the BCI performance.

Effect of BCI-Controlled Pedaling Training System With Multiple Modalities of Feedback on Motor and Cognitive Function Rehabilitation of Early Subacute Stroke Patients

Brain-computer interfaces (BCIs) are currently integrated into traditional rehabilitation interventions after stroke. Although BCIs bring many benefits to the rehabilitation process, their effects are limited since many patients cannot concentrate during training. Despite this outcome post-stroke motor-attention dual-task training using BCIs has remained mostly unexplored. This study was a randomized placebo-controlled blinded-endpoint clinical trial to investigate the effects of a BCI-controlled pedaling training system (BCI-PT) on the motor and cognitive function of stroke patients during rehabilitation. A total of 30 early subacute ischemic stroke patients with hemiplegia and cognitive impairment were randomly assigned to the BCI-PT or traditional pedaling training. We used single-channel Fp1 to collect electroencephalography data and analyze the attention index. The BCI-PT system timely provided visual, auditory, and somatosensory feedback to enhance the patient's participation to pedaling based on the real-time attention index. After 24 training sessions, the attention index of the experimental group was significantly higher than that of the control group. The lower limbs motor function (FMA-L) increased by an average of 4.5 points in the BCI-PT group and 2.1 points in the control group (P = 0.022) after treatments. The difference was still significant after adjusting for the baseline indicators ( β = 2.41 , 95%CI: 0.48-4.34, P = 0.024). We found that BCI-PT significantly improved the patient's lower limb motor function by increasing the patient's participation. (clinicaltrials.gov: NCT04612426).