Prediction of subjective states from psychophysiology: A multivariate approach

Combined Effects of Moderate Hypoxia and Sleep Restriction on Mental Workload

Aircraft pilots face a high mental workload (MW) under environmental constraints induced by high altitude and sometimes sleep restriction (SR). Our aim was to assess the combined effects of hypoxia and sleep restriction on cognitive and physiological responses to different MW levels using the Multi-Attribute Test Battery (MATB)-II with an additional auditory Oddball-like task. Seventeen healthy subjects were subjected in random order to three 12-min periods of increased MW level (low, medium, and high): sleep restriction (SR, <3 h of total sleep time (TST)) vs. habitual sleep (HS, >6 h TST), hypoxia (HY, 2 h, FIO2 = 13.6%, ~3500 m vs. normoxia, NO, FIO2 = 21%). Following each MW level, participants completed the NASA-TLX subjective MW scale. Increasing MW decreases performance on the MATB-II Tracking task (p = 0.001, MW difficulty main effect) and increases NASA-TLX (p = 0.001). In the combined HY/SR condition, MATB-II performance was lower, and the NASA-TLX score was higher compared with the NO/HS condition, while no effect of hypoxia alone was observed. In the accuracy of the auditory task, there is a significant interaction between hypoxia and MW difficulty (F(2-176) = 3.14, p = 0.04), with lower values at high MW under hypoxic conditions. Breathing rate, pupil size, and amplitude of pupil dilation response (PDR) to auditory stimuli are associated with increased MW. These parameters are the best predictors of increased MW, independently of physiological constraints. Adding ECG, SpO2, or electrodermal conductance does not improve model performance. In conclusion, hypoxia and sleep restriction have an additive effect on MW. Physiological and electrophysiological responses must be taken into account when designing a MW predictive model and cross-validation.

Granular estimation of user cognitive workload using multi-modal physiological sensors

Mental workload (MWL) is a crucial area of study due to its significant influence on task performance and potential for significant operator error. However, measuring MWL presents challenges, as it is a multi-dimensional construct. Previous research on MWL models has focused on differentiating between two to three levels. Nonetheless, tasks can vary widely in their complexity, and little is known about how subtle variations in task difficulty influence workload indicators. To address this, we conducted an experiment inducing MWL in up to 5 levels, hypothesizing that our multi-modal metrics would be able to distinguish between each MWL stage. We measured the induced workload using task performance, subjective assessment, and physiological metrics. Our simulated task was designed to induce diverse MWL degrees, including five different math and three different verbal tiers. Our findings indicate that all investigated metrics successfully differentiated between various MWL levels induced by different tiers of math problems. Notably, performance metrics emerged as the most effective assessment, being the only metric capable of distinguishing all the levels. Some limitations were observed in the granularity of subjective and physiological metrics. Specifically, the subjective overall mental workload couldn't distinguish lower levels of workload, while all physiological metrics could detect a shift from lower to higher levels, but did not distinguish between workload tiers at the higher or lower ends of the scale (e.g., between the easy and the easy-medium tiers). Despite these limitations, each pair of levels was effectively differentiated by one or more metrics. This suggests a promising avenue for future research, exploring the integration or combination of multiple metrics. The findings suggest that subtle differences in workload levels may be distinguishable using combinations of subjective and physiological metrics.

Executive Functions Assessment Based on Wireless EEG and 3D Gait Analysis During Dual-Task: A Feasibility Study

Executive functions (EFs) are neurocognitive processes planning and regulating daily life actions. Performance of two simultaneous tasks, requiring the same cognitive resources, lead to a cognitive fatigue. Several studies investigated cognitive-motor task and the interference during walking, highlighting an increasing risk of falls especially in elderly and people with neurological diseases. A few studies instrumentally explored relationship between activation-no-activation of two EFs (working memory and inhibition) and spatial-temporal gait parameters. Aim of our study was to detect activation of inhibition and working memory during progressive difficulty levels of cognitive tasks and spontaneous walking using, respectively, wireless electroencephalography (EEG) and 3D-gait analysis. Thirteen healthy subjects were recruited. Two cognitive tasks were performed, activating inhibition (Go-NoGo) and working memory (N-back). EEG features (absolute and relative power in different bands) and kinematic parameters (7 spatial-temporal ones and Gait Variable Score for 9 range of motion of lower limbs) were analyzed. A significant decrease of stride length and an increase of external-rotation of foot progression were found during dual task with Go-NoGo. Moreover, a significant correlation was found between the relative power in the delta band at channels Fz, C4 and progressive difficulty levels of Go-NoGo (activating inhibition) during walking, whereas working memory showed no correlation. This study reinforces the hypothesis of the prevalent involvement of inhibition with respect to working memory during dual task walking and reveals specific kinematic adaptations. The foundations for EEG-based monitoring of cognitive processes involved in gait are laid. Clinical and Translational Impact Statement: Clinical and instrumental evaluation and training of executive functions (as inhibition), during cognitive-motor task, could be useful for rehabilitation treatment of gait disorder in elderly and people with neurological disease.

Physiological Metrics of Surgical Difficulty and Multi-Task Requirement during Robotic Surgery Skills

Previous studies in robotic-assisted surgery (RAS) have studied cognitive workload by modulating surgical task difficulty, and many of these studies have relied on self-reported workload measurements. However, contributors to and their effects on cognitive workload are complex and may not be sufficiently summarized by changes in task difficulty alone. This study aims to understand how multi-task requirement contributes to the prediction of cognitive load in RAS under different task difficulties. Multimodal physiological signals (EEG, eye-tracking, HRV) were collected as university students performed simulated RAS tasks consisting of two types of surgical task difficulty under three different multi-task requirement levels. EEG spectral analysis was sensitive enough to distinguish the degree of cognitive workload under both surgical conditions (surgical task difficulty/multi-task requirement). In addition, eye-tracking measurements showed differences under both conditions, but significant differences of HRV were observed in only multi-task requirement conditions. Multimodal-based neural network models have achieved up to 79% accuracy for both surgical conditions.

Mathematical-heuristic modelling for human performance envelope

BACKGROUND: Using the theory of complex systems, some human functions (thinking, memory, language) and human relationships have been analyzed and explained. In order to study the limits of human performance (in Air Traffic Controllers and pilots) a new concept was created, called the Human Performance Envelope (HPE). OBJECTIVE: The aim of this paper is to apply the principles of the complex system to the analysis of the human factors of the HPE concept. Moreover, this paper’s objective is to create a mathematical model that will give the opportunity to study all the physiological ergonomic factors, not only the ones that are most commonly studied. The most studied factors are mental workload, stress and situation awareness (SA). By applying the mathematical model, it is possible to analyze all the physiological factors (stress, mental workload, fatigue, attention, vigilance and SA). METHODS: In the present paper the theory of complex systems (hybrid modelling) was applied to the Human Performance Envelope concept. A mathematical model was created, then it was validated and solved based on previous researches. RESULTS: Firstly, a literature analysis was performed on the complex systems application by the present researchers concerning pilots’ HPE. The proportional and inverse proportional relationships between the nine human factors were visually illustrated. Finally, a mathematical model was proposed, consisting of a set of equations, which were partially solved and validated by the experiments on pilots done by other researchers. CONCLUSIONS: Further research is required to validate the whole mathematical model, including physiological measurements (experiments) for the six ergonomic factors and the applied heuristic psychosocial methods for the others.

EEG microstate changes according to mental fatigue induced by aircraft piloting simulation: An exploratory study

BACKGROUND

A continuous flight task load can induce fatigue and lead to changes in electroencephalography (EEG). EEG microstates can reflect the activities of large-scale neural networks during mental fatigue. This exploratory experiment explored the effects of mental fatigue induced by continuous simulated flight multitasking on EEG microstate indices.

METHODS

Twenty-four participants performed continuous 2-hour aircraft piloting simulation while EEG were recorded. The Stanford sleepiness scale (SSS) and critical flicker fusion frequency (CFF) were measured before and after the task. Microstate analysis was applied to EEG. Four microstate classes (A-D) were identified during the pre-task, post-task, beginning, and end phases. The effects of mental fatigue were analyzed.

RESULTS

Compared with the pre-task, the post-task had a higher global explained variance (GEV) and time parameters of class C but lower occurrence and coverage of class D. The end had a higher GEV but lower duration and coverage of class D than at the beginning. After 2 h of multitasking, the transition probability between A and D, and between B and D decreased but between A and C increased. Subjective fatigue scores were negatively correlated with occurrence and coverage of class D. Task performance was negatively correlated with duration and coverage of class C but positively correlated with duration and occurrence of class B.

CONCLUSION

Time parameters and transition probability of EEG microstates can detect mental fatigue induced by continuous aircraft piloting simulation. The global brain network activation of mental fatigue can be detected by EEG microstates that can evaluate flight fatigue.

Using machine learning methods and EEG to discriminate aircraft pilot cognitive workload during flight

Pilots of aircraft face varying degrees of cognitive workload even during normal flight operations. Periods of low cognitive workload may be followed by periods of high cognitive workload and vice versa. During such changing demands, there exists potential for increased error on behalf of the pilots due to periods of boredom or excessive cognitive task demand. To further understand cognitive workload in aviation, the present study involved collection of electroencephalogram (EEG) data from ten (10) collegiate aviation students in a live-flight environment in a single-engine aircraft. Each pilot possessed a Federal Aviation Administration (FAA) commercial pilot certificate and either FAA class I or class II medical certificate. Each pilot flew a standardized flight profile representing an average instrument flight training sequence. For data analysis, we used four main sub-bands of the recorded EEG signals: delta, theta, alpha, and beta. Power spectral density (PSD) and log energy entropy of each sub-band across 20 electrodes were computed and subjected to two feature selection algorithms (recursive feature elimination (RFE) and lasso cross-validation (LassoCV), and a stacking ensemble machine learning algorithm composed of support vector machine, random forest, and logistic regression. Also, hyperparameter optimization and tenfold cross-validation were used to improve the model performance, reliability, and generalization. The feature selection step resulted in 15 features that can be considered an indicator of pilots' cognitive workload states. Then these features were applied to the stacking ensemble algorithm, and the highest results were achieved using the selected features by the RFE algorithm with an accuracy of 91.67% (± 0.11), a precision of 93.89% (± 0.09), recall of 91.67% (± 0.11), F-score of 91.22% (± 0.12), and the mean ROC-AUC of 0.93 (± 0.06). The achieved results indicated that the combination of PSD and log energy entropy, along with well-designed machine learning algorithms, suggest the potential for the use of EEG to discriminate periods of the low, medium, and high workload to augment aircraft system design, including flight automation features to improve aviation safety.

Neuroergonomics on the Go: An Evaluation of the Potential of Mobile EEG for Workplace Assessment and Design

OBJECTIVE

We demonstrate and discuss the use of mobile electroencephalogram (EEG) for neuroergonomics. Both technical state of the art as well as measures and cognitive concepts are systematically addressed.

BACKGROUND

Modern work is increasingly characterized by information processing. Therefore, the examination of mental states, mental load, or cognitive processing during work is becoming increasingly important for ergonomics.

RESULTS

Mobile EEG allows to measure mental states and processes under real live conditions. It can be used for various research questions in cognitive neuroergonomics. Besides measures in the frequency domain that have a long tradition in the investigation of mental fatigue, task load, and task engagement, new approaches-like blink-evoked potentials-render event-related analyses of the EEG possible also during unrestricted behavior.

CONCLUSION

Mobile EEG has become a valuable tool for evaluating mental states and mental processes on a highly objective level during work. The main advantage of this technique is that working environments don't have to be changed while systematically measuring brain functions at work. Moreover, the workflow is unaffected by such neuroergonomic approaches.

The influence of experience on cognitive load during simultaneous interpretation

Simultaneous interpretation is a complex task that is assumed to be associated with a high workload. To corroborate this association, we measured workload during three tasks of increasing complexity: listening, shadowing, and interpreting, using electroencephalography and self-assessments in four groups of participants with varying experience in simultaneous interpretation. The self-assessment data showed that professional interpreters perceived the most workload-inducing condition, namely the interpreting task, as less demanding compared to the less experienced participants. This higher subjectively perceived workload in non-interpreters was paralleled by increasing frontal theta power values from listening to interpreting, whereas such a modulation was less pronounced in professional interpreters. Furthermore, regarding both workload measures, trainee interpreters were situated between professional interpreters and non-interpreters. Since the non-interpreters demonstrated high proficiencies and exposure in their second language, too, our findings provide evidence for an influence of interpretation training on experienced workload during simultaneous interpretation.

Assessing student engagement from facial behavior in on-line learning

The automatic monitoring and assessment of the engagement level of learners in distance education may help in understanding problems and providing personalized support during the learning process. This article presents a research aiming to investigate how student engagement level can be assessed from facial behavior and proposes a model based on Long Short-Term Memory (LSTM) networks to predict the level of engagement from facial action units, gaze, and head poses. The dataset used to learn the model is the one of the EmotiW 2019 challenge datasets. In order to test its performance in learning contexts, an experiment, involving students attending an online lecture, was performed. The aim of the study was to compare the self-evaluation of the engagement perceived by the students with the one assessed by the model. During the experiment we collected videos of students behavior and, at the end of each session, we asked students to answer a questionnaire for assessing their perceived engagement. Then, the collected videos were analyzed automatically with a software that implements the model and provides an interface for the visual analysis of the model outcome. Results show that, globally, engagement prediction from students' facial behavior was weakly correlated to their subjective answers. However, when considering only the emotional dimension of engagement, this correlation is stronger and the analysis of facial action units and head pose (facial movements) are positively correlated with it, while there is an inverse correlation with the gaze, meaning that the more the student's feels engaged the less are the gaze movements.

Ambient Light Conveying Reliability Improves Drivers’ Takeover Performance without Increasing Mental Workload

Drivers of L3 automated vehicles (AVs) are not required to continuously monitor the AV system. However, they must be prepared to take over when requested. Therefore, it is necessary to design an in-vehicle environment that allows drivers to adapt their levels of preparedness to the likelihood of control transition. This study evaluates ambient in-vehicle lighting that continuously communicates the current level of AV reliability, specifically on how it could influence drivers’ take-over performance and mental workload (MW). We conducted an experiment in a driving simulator with 42 participants who experienced 10 take-over requests (TORs). The experimental group experienced a four-stage ambient light display that communicated the current level of AV reliability, which was not provided to the control group. The experimental group demonstrated better take-over performance, based on lower vehicle jerks. Notably, perceived MW did not differ between the groups, and the EEG indices of MW (frontal theta power, parietal alpha power, Task–Load Index) did not differ between the groups. These findings suggest that communicating the current level of reliability using ambient light might help drivers be better prepared for TORs and perform better without increasing their MW.

Methods and Measures for Mental Stress Assessment in Surgery: A Systematic Review of 20 Years of Literature

Real-time mental stress monitoring from surgeons and surgical staff in operating rooms may reduce surgical injuries, improve performance and quality of medical care, and accelerate implementation of stress-management strategies. Motivated by the increase in usage of objective and subjective metrics for cognitive monitoring and by the gap in reviews of experimental design setups and data analytics, a systematic review of 71 studies on mental stress and workload measurement in surgical settings, published in 2001-2020, is presented. Almost 61% of selected papers used both objective and subjective measures, followed by 25% that only administered subjective tools - mostly consisting of validated instruments and customized surveys. An overall increase in the total number of publications on intraoperative stress assessment was observed from mid-2010 s along with a momentum in the use of both subjective and real-time objective measures. Cardiac activity, including heart-rate variability metrics, stress hormones, and eye-tracking metrics were the most frequently and electroencephalography (EEG) was the least frequently used objective measures. Around 40% of selected papers collected at least two objective measures, 41% used wearable devices, 23% performed synchronization and annotation, and 76% conducted baseline or multi-point data acquisition. Furthermore, 93% used a variety of statistical techniques, 14% applied regression models, and only one study released a public, anonymized dataset. This review of data modalities, experimental setups, and analysis techniques for intraoperative stress monitoring highlights the initiatives of surgical data science and motivates research on computational techniques for mental and surgical skills assessment and cognition-guided surgery.

Virtual Reality Adaptation Using Electrodermal Activity to Support the User Experience

Virtual reality is increasingly used for tasks such as work and education. Thus, rendering scenarios that do not interfere with such goals and deplete user experience are becoming progressively more relevant. We present a physiologically adaptive system that optimizes the virtual environment based on physiological arousal, i.e., electrodermal activity. We investigated the usability of the adaptive system in a simulated social virtual reality scenario. Participants completed an n-back task (primary) and a visual detection (secondary) task. Here, we adapted the visual complexity of the secondary task in the form of the number of non-player characters of the secondary task to accomplish the primary task. We show that an adaptive virtual reality can improve users’ comfort by adapting to physiological arousal regarding the task complexity. Our findings suggest that physiologically adaptive virtual reality systems can improve users’ experience in a wide range of scenarios.

Unravelling the Physiological Correlates of Mental Workload Variations in Tracking and Collision Prediction Tasks

Modern work environments have extensive interactions with technology and greater cognitive complexity of the tasks, which results in human operators experiencing increased mental workload. Air traffic control operators routinely work in such complex environments, and we designed tracking and collision prediction tasks to emulate their elementary tasks. The physiological response to the workload variations in these tasks was elucidated to untangle the impact of workload variations experienced by operators. Electroencephalogram (EEG), eye activity, and heart rate variability (HRV) data were recorded from 24 participants performing tracking and collision prediction tasks with three levels of difficulty. Our findings indicate that variations in task load in both these tasks are sensitively reflected in EEG, eye activity and HRV data. Multiple regression results also show that operators' performance in both tasks can be predicted using the corresponding EEG, eye activity and HRV data. The results also demonstrate that the brain dynamics during each of these tasks can be estimated from the corresponding eye activity, HRV and performance data. Furthermore, the markedly distinct neurometrics of workload variations in the tracking and collision prediction tasks indicate that neurometrics can provide insights on the type of mental workload. These findings have applicability to the design of future mental workload adaptive systems that integrate neurometrics in deciding not just "when" but also "what" to adapt. Our study provides compelling evidence in the viability of developing intelligent closed-loop mental workload adaptive systems that ensure efficiency and safety in complex work environments.

Study of EEG characteristics while solving scientific problems with different mental effort

Studying the mental effort in problem-solving is important to the understanding of how the brain allocates cognitive resources to process information. The electroencephalogram is a promising physiological approach to assessing the online mental effort. In this study, we investigate the EEG indicators of mental effort while solving scientific problems. By manipulating the complexity of the scientific problem, the level of mental effort also changes. With the increase of mental effort, theta synchronization in the frontal region and lower alpha desynchronization in the parietal and occipital regions significantly increase. Also, upper alpha desynchronization demonstrates a widespread enhancement across the whole brain. According to the functional topography of brain activity in the theta and alpha frequency, our results suggest that the mental effort while solving scientific problems is related to working memory, visuospatial processing, semantic processing and magnitude manipulation. This study suggests the reliability of EEG to evaluate the mental effort in an educational context and provides valuable insights into improving the problem-solving abilities of students in educational practice.

Examining the effect of depressive symptoms on habit formation and habit-breaking

BACKGROUND AND OBJECTIVES

Dysfunction in reward processing is a hallmark feature of depression. In the context of reinforcement learning, previous research has linked depression with reliance on simple habit-driven ('model-free') learning strategies over more complex, goal-directed ('model-based') strategies. However, the relationship between depression and habit-breaking remains an under-explored research area. The current study sought to bridge this gap by investigating the effect of depressive symptoms on habit formation and habit-breaking under monetary and social feedback conditions. Additionally, we examined whether spontaneous eyeblink rate (EBR), an indirect marker for striatal dopamine levels, would modulate such effects.

METHODS

Depressive symptoms were operationalized using self-report measures. To examine differences in habit formation and habit breaking, undergraduate participants (N = 156) completed a two-stage reinforcement learning task with a devaluation procedure using either monetary or social feedback.

RESULTS

Regression results showed that in the monetary feedback condition, spontaneous EBR moderated the relationship between depressive symptoms and model-free strategies; individuals with more depressive symptomatology and high EBR (higher dopamine levels) exhibited increased reliance on model-free strategies. Depressive symptoms negatively predicted devaluation sensitivity, indicative of difficulty in habit-breaking, in both monetary and social feedback contexts.

LIMITATIONS

Social feedback relied on fixed feedback rather than real-time peer evaluations; depressive symptoms were measured using self-report rather than diagnostic criteria for Major Depressive Disorder; dopaminergic functioning was measured using EBR rather than PET imaging; potential confounds were not controlled for.

CONCLUSIONS

These findings have implications for identifying altered patterns of habit formation and deficits in habit-breaking among those experiencing depressive symptoms.

Quantitative assessment of pilot-endured workloads during helicopter flying emergencies: an analysis of physiological parameters during an autorotation

The procedures to be performed after sudden engine failure of a single-engine helicopter impose high workload on pilots. The maneuver to regain aircraft control and safe landing is called autorotation. The safety limits to conduct this maneuver are based on the aircraft height versus speed diagram, which is also known as "Dead Man’s Curve”. Flight-test pilots often use subjective methods to assess the difficulty to conduct maneuvers in the vicinity of this curve. We carried out an extensive flight test campaign to verify the feasibility of establishing quantitative physiological parameters to better assess the workload endured by pilots undergoing those piloting conditions. Eleven pilots were fully instrumented with sensors and had their physiological reactions collected during autorotation maneuvers. Our analyses suggested that physiological measurements (heart rate and electrodermal activity) can be successfully recorded and useful to capture the most effort-demanding effects during the maneuvers. Additionally, the helicopter’s flight controls displacements were also recorded, as well as the pilots’ subjective responses evaluated by the Handling Qualities Rate scale. Our results revealed that the degree of cognitive workload was associated with the helicopter’s flight profile concerning the Height-Speed diagram and that the strain intensity showed a correlation with measurable physiological responses. Recording flight controls displacement and quantifying the pilot's subjective responses show themselves as natural effective candidates to evaluate the intensity of cognitive workload in such maneuvers.

Symptoms of convergence and accommodative insufficiency predict engagement and cognitive fatigue during complex task performance with and without automation

Deficits in the accommodative and/or vergence responses have been linked with inattentive behavioral symptoms. While using automated systems (e.g., self-driving cars, autopilot), operators (e.g., drivers, pilots, soldiers) visually monitor displays for critical changes, making deficits in the accommodative and/or vergence responses potentially hazardous for individuals remaining actively engaged in the task at hand. The purpose of this study was to determine if symptoms of accommodative-vergence deficits predict an individual's level of task engagement and cognitive fatigue while performing a flight simulation task with or without automation. Eighty-four participants performed a flight simulation task with or without automation. Prior to task completion, self-report accommodative-convergence deficit symptoms were assessed with the Convergence Insufficiency Symptom Survey (CISS). Before and after the flight simulation task participants rated their task engagement and cognitive fatigue. Electroencephalographic activity (EEG) was recorded concurrently during task performance. Results showed that higher scores on the CISS were related to increased feelings of fatigue and decreased ratings of task engagement. The CISS was also positively related to parietal-occipital fast alpha power during the last 10 min of the task for participants using automation, suggesting increased cortical idling. CISS scores did not predict performance. Results have implications for optimizing operator cognitive states over extended task performance.

The (b)link between amotivation and psychosis: Insights through phasic eye blink rate

Motivation deficits within Schizophrenia Spectrum Disorders (SSDs) are associated with abnormal striatal dopamine responses during reward processing. Eye blink rate (EBR) has been used as a proxy for striatal dopamine; however, it is unclear whether EBR is sensitive to individual differences in amotivation. Amotivation (clinician-rated and self-reported) and EBR during an effort-based reward task were assessed in 28 outpatients with SSDs. EBR was lower during more "active" task phases relative to rest periods. Higher EBR during reward anticipation was associated with lower self-reported, but not clinician-rated, motivation. These preliminary results support a task-engagement, rather than striatal dopamine, account of EBR.

Physiological correlates of cognitive load in laparoscopic surgery

Laparoscopic surgery can be exhausting and frustrating, and the cognitive load experienced by surgeons may have a major impact on patient safety as well as healthcare economics. As cognitive load decreases with increasing proficiency, its robust assessment through physiological data can help to develop more effective training and certification procedures in this area. We measured data from 31 novices during laparoscopic exercises to extract features based on cardiac and ocular variables. These were compared with traditional behavioural and subjective measures in a dual-task setting. We found significant correlations between the features and the traditional measures. The subjective task difficulty, reaction time, and completion time were well predicted by the physiology features. Reaction times to randomly timed auditory stimuli were correlated with the mean of the heart rate (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r = - 0.29$$\end{document}r=-0.29) and heart rate variability (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r = 0.4$$\end{document}r=0.4). Completion times were correlated with the physiologically predicted values with a correlation coefficient of 0.84. We found that the multi-modal set of physiology features was a better predictor than any individual feature and artificial neural networks performed better than linear regression. The physiological correlates studied in this paper, translated into technological products, could help develop standardised and more easily regulated frameworks for training and certification.

Measurement and identification of mental workload during simulated computer tasks with multimodal methods and machine learning

This study attempted to multimodally measure mental workload and validate indicators for estimating mental workload. A simulated computer work composed of mental arithmetic tasks with different levels of difficulty was designed and used in the experiment to measure physiological signals (heart rate, heart rate variability, electromyography, electrodermal activity, and respiration), subjective ratings of mental workload (the NASA Task Load Index), and task performance. The indices from electrodermal activity and respiration had a significant increment as task difficulty increased. There were no significant differences between the average heart rate and the low-frequency/high-frequency ratio among tasks. The classification of mental workload using combined indices as inputs showed that classification models combining physiological signals and task performance can reach satisfying accuracy at 96.4% and an accuracy of 78.3% when only using physiological indices as inputs. The present study also showed that ECG and EDA signals have good discriminating power for mental workload detection. Practitioner summary: The methods used in this study could be applied to office workers, and the findings provide preliminary support and theoretical exploration for follow-up early mental workload detection systems, whose implementation in the real world could beneficially impact worker health and company efficiency. Abbreviations: NASA-TLX: the national aeronautics and space administration-task load index; ECG: electrocardiographic; EDA: electrodermal activity; EEG: electroencephalogram; LDA: linear discriminant analysis; SVM: support vector machine; KNN: k-nearest neighbor; ANNs: artificial neural networks; EMG: electromyography; PPG: photoplethysmography; SD: standard deviation; BMI: body mass index; DSSQ: dundee stress state questionnaire; ANOVA: analysis of variance; SC: skin conductance; RMS: root mean square; AVHR: the average heart rate; HR: heart rate; LF/HF: the ratio between the low frequencies band and the high frequency band; PSD: power spectral density; MF: median frequency; HRV: heart rate variability; BPNN: backpropagation neural network.

Substance use is associated with reduced devaluation sensitivity

Substance use has been linked to impairments in reward processing and decision-making, yet empirical research on the relationship between substance use and devaluation of reward in humans is limited. We report findings from two studies that tested whether individual differences in substance use behavior predicted reward learning strategies and devaluation sensitivity in a nonclinical sample. Participants in Experiment 1 (N = 66) and Experiment 2 (N = 91) completed subscales of the Externalizing Spectrum Inventory and then performed a two-stage reinforcement learning task that included a devaluation procedure. Spontaneous eye blink rate was used as an indirect proxy for dopamine functioning. In Experiment 1, correlational analysis revealed a negative relationship between substance use and devaluation sensitivity. In Experiment 2, regression modeling revealed that while spontaneous eyeblink rate moderated the relationship between substance use and reward learning strategies, substance use alone was related to devaluation sensitivity. These results suggest that once reward-action associations are established during reinforcement learning, substance use predicted reduced sensitivity to devaluation independently of variation in eyeblink rate. Thus, substance use is not only related to increased habit formation but also to difficulty disengaging from learned habits. Implications for the role of the dopaminergic system in habitual responding in individuals with substance use problems are discussed.

A Systematic Review of Physiological Measures of Mental Workload

Mental workload (MWL) can affect human performance and is considered critical in the design and evaluation of complex human-machine systems. While numerous physiological measures are used to assess MWL, there appears no consensus on their validity as effective agents of MWL. This study was conducted to provide a comprehensive understanding of the use of physiological measures of MWL and to synthesize empirical evidence on the validity of the measures to discriminate changes in MWL. A systematical literature search was conducted with four electronic databases for empirical studies measuring MWL with physiological measures. Ninety-one studies were included for analysis. We identified 78 physiological measures, which were distributed in cardiovascular, eye movement, electroencephalogram (EEG), respiration, electromyogram (EMG) and skin categories. Cardiovascular, eye movement and EEG measures were the most widely used across varied research domains, with 76%, 66%, and 71% of times reported a significant association with MWL, respectively. While most physiological measures were found to be able to discriminate changes in MWL, they were not universally valid in all task scenarios. The use of physiological measures and their validity for MWL assessment also varied across different research domains. Our study offers insights into the understanding and selection of appropriate physiological measures for MWL assessment in varied human-machine systems.

Individual-Specific Classification of Mental Workload Levels Via an Ensemble Heterogeneous Extreme Learning Machine for EEG Modeling

In a human–machine cooperation system, assessing the mental workload (MW) of the human operator is quite crucial to maintaining safe operation conditions. Among various MW indicators, electroencephalography (EEG) signals are particularly attractive because of their high temporal resolution and sensitivity to the occupation of working memory. However, the individual difference of the EEG feature distribution may impair the machine-learning based MW classifier. In this paper, we employed a fast-training neural network, extreme learning machine (ELM), as the basis to build an individual-specific classifier ensemble to recognize binary MW. To improve the diversity of the classification committee, heterogeneous member classifiers were adopted by fusing multiple ELMs and Bayesian models. Specifically, a deep network structure was applied in each weak model aiming at finding informative EEG feature representations. The structure of hyper-parameters of the proposed heterogeneous ensemble ELM (HE-ELM) was then identified and then its performance was compared against several competitive MW classifiers. We found that the HE-ELM model was superior for improving the individual-specific accuracy of MW assessments.

Examining the link between reward and response inhibition in individuals with substance abuse tendencies

BACKGROUND

Substance use problems are often characterized by dysregulation in reward sensitivity and inhibitory control. In line with this representation, the goal of this investigation was to determine how substance abuse tendencies among university students affect incentivized response inhibition. Additionally, this study examined whether striatal dopamine moderates the impact of substance use on response inhibition performance.

METHODS

The sample included ninety-eight university students. Participants completed this prospective experimental study at an on-campus laboratory. All participants completed substance abuse and disinhibition subscales of the Externalizing Spectrum Inventory-Brief Form. Using a within-subjects design, participants then performed the Stop Signal Task under both neutral (unrewarded) and reward conditions, in which correct response cancellations resulted in a monetary reward. Striatal tonic dopamine levels were operationalized using spontaneous eyeblink rate.

RESULTS

The outcome measures were Stop Signal Reaction Time (SSRT) performance in the unrewarded and rewarded phases of the task. A hierarchical linear regression analysis, controlling for trait disinhibition, age, gender, and cigarette smoking status, identified an interactive effect of substance use and striatal dopamine levels on incentivized SSRT. Substance abuse tendencies were associated with slower SSRT and thus poorer inhibitory control under reward conditions among individuals with low levels of striatal dopamine (F = 7.613, p = .007).

CONCLUSIONS

This work has implications for research examining advanced drug use trajectories. In situations in which rewards are at stake, drug users with low tonic dopamine may be more motivated to seek those rewards at the expense of regulating inhibitory control.

Adaptações do cérebro durante uma tarefa de longa duração: Um estudo de Potencial Relacionado a Evento

Resumo O objetivo deste estudo é investigar o efeito da demanda cognitiva prolongada na modulação do Potencial Relacionado a Evento (ERP) em um paradigma de controle inibitório. Os dados foram coletados em 19 voluntários destros, com a média de idade de 21,21 (±1,77) anos, que realizaram o paradigma do Go/NoGo durante 50 minutos, com gravação sincronizada do eletroencefalograma para obtenção dos ERPs. O efeito do tempo de realização da tarefa provocou alterações significativas nas variáveis subjetivas, de desempenho cognitivo e nas amplitudes máximas dos componentes N2 e P3. Nossos resultados sugerem que quando nosso cérebro está submetido a demandas cognitivas extensas, ocorrem adaptações para a manutenção do desempenho comportamental através da estratégia de realocação de recursos energéticos.

Measuring mental workload using physiological measures: A systematic review

Technological advances have led to physiological measurement being increasingly used to measure and predict operator states. Mental workload (MWL) in particular has been characterised using a variety of physiological sensor data. This systematic review contributes a synthesis of the literature summarising key findings to assist practitioners to select measures for use in evaluation of MWL. We also describe limitations of the methods to assist selection when being deployed in applied or laboratory settings. We detail fifty-eight peer reviewed journal articles which present original data using physiological measures to include electrocardiographic, respiratory, dermal, blood pressure and ocular. Electroencephalographic measures have been included if they are presented with another measure to constrain scope. The literature reviewed covers a range of applied and experimental studies across various domains, safety-critical applications being highly represented in the sample of applied literature reviewed. We present a summary of the six measures and provide an evidence base which includes how to deploy each measure, and characteristics that can affect or preclude the use of a measure in research. Measures can be used to discriminate differences in MWL caused by task type, task load, and in some cases task difficulty. Varying ranges of sensitivity to sudden or gradual changes in taskload are also evident across the six measures. We conclude that there is no single measure that clearly discriminates mental workload but there is a growing empirical basis with which to inform both science and practice.

Evaluating Pro- and Re-Active Driving Behavior by Means of the EEG

Traffic safety essentially depends on the drivers' alertness and vigilance, especially in monotonous or demanding driving situations. Brain oscillatory EEG activity offers insight into a drivers' mental state and has therefore attracted much attention in the past. However, EEG measures do not only vary with internal factors like attentional engagement and vigilance but might also interact with external factors like time on task, task demands, or the degree to which a traffic situation is predictable. In order to identify EEG parameters for cognitive mechanisms involved in tasks of high and low controllability, the present study investigated the interaction of time on task, task load, and cognitive controllability in simulated driving scenarios, using an either re-active or pro-active driving task. Participants performed a lane-keeping task, half of them compensating varying levels of crosswind (re-active task), and the other half driving along a winding road (pro-active task). Both driving tasks were adjusted with respect to difficulty. The analysis of oscillatory EEG parameters showed an increase in total power (1-30 Hz) with time on task, with decreasing task load, and in the re-active compared to the pro-active task. Furthermore, the relative power in Alpha band increased with decreasing task load and time on task, while relative Theta power showed the opposite pattern. Moreover, relative Alpha power was also higher in the re-active, than pro-active, driving situation, an effect that even increased with time on task. The results demonstrate that the controllability of a driving situation has a similar effect on oscillatory EEG activity like time on task and task load.

Effects of platooning on signal-detection performance, workload, and stress: A driving simulator study

Platooning, whereby automated vehicles travel closely together in a group, is attractive in terms of safety and efficiency. However, concerns exist about the psychological state of the platooning driver, who is exempted from direct control, yet remains responsible for monitoring the outside environment to detect potential threats. By means of a driving simulator experiment, we investigated the effects on recorded and self-reported measures of workload and stress for three task-instruction conditions: (1) No Task, in which participants had to monitor the road, (2) Voluntary Task, in which participants could do whatever they wanted, and (3) Detection Task, in which participants had to detect red cars. Twenty-two participants performed three 40-min runs in a constant-speed platoon, one condition per run in counterbalanced order. Contrary to some classic literature suggesting that humans are poor monitors, in the Detection Task condition participants attained a high mean detection rate (94.7%) and a low mean false alarm rate (0.8%). Results of the Dundee Stress State Questionnaire indicated that automated platooning was less distressing in the Voluntary Task than in the Detection Task and No Task conditions. In terms of heart rate variability, the Voluntary Task condition yielded a lower power in the low-frequency range relative to the high-frequency range (LF/HF ratio) than the Detection Task condition. Moreover, a strong time-on-task effect was found, whereby the mean heart rate dropped from the first to the third run. In conclusion, participants are able to remain attentive for a prolonged platooning drive, and the type of monitoring task has effects on the driver's psychological state.

Dopamine, depressive symptoms, and decision-making: the relationship between spontaneous eye blink rate and depressive symptoms predicts Iowa Gambling Task performance

Depressive symptomatology has been associated with alterations in decision-making, although conclusions have been mixed, with depressed individuals showing impairments in some contexts but advantages in others. The dopaminergic system may link depressive symptoms with decision-making performance. We assessed the role of striatal dopamine D2 receptor density, using spontaneous eye blink rates, in moderating the relationship between depressive symptoms and decision-making performance in a large undergraduate sample that had not been screened for mental illness (N = 104). The regression results revealed that eye blink rate moderated the relationship between depressive symptoms and advantageous decisions on the Iowa Gambling Task, in which individuals with more depressive symptomatology and high blink rates (higher striatal dopamine D2 receptor density) performed better on the task. Our computational modeling results demonstrated that depressive symptoms alone were associated with enhanced loss-aversive behavior, whereas individuals with high blink rates and elevated depressive symptoms tended to persevere in selecting options that led to net gains (avoiding options with net losses). These findings suggest that variation in striatal dopamine D2 receptor availability in individuals with depressive symptoms may contribute to differences in decision-making behavior.

Cerebral Blood Flow Velocity and Task Engagement as Predictors of Vigilance Performance

Responses to a brief six-min screening battery involving high-workload tracking, verbal working memory, and line discrimination tasks were used to predict subsequent performance on a 36-min vigilance task. Two predictors of interest were subjective state, as indexed by the Dundee Stress State Questionnaire (DSSQ), and cerebral blood flow velocity (CBFV), measured via transcranial Doppler sonography. The results testify to the importance of assessing task-induced responses for predicting vigilance performance. They also indicate that forecasting vigilance performance is a complex endeavor requiring a set of multidimensional predictors. Specifically, higher post-battery task engagement scores on the DSSQ in this study and higher levels of CBFV in the left hemisphere during performance of the screening battery predicted more correct detections on the subsequent vigilance task. These findings are interpreted in the light of the resource-workload model of vigilance, and their practical significance is discussed.

Striatal Dopamine, Externalizing Proneness, and Substance Abuse: Effects on Wanting and Learning during Reward-Based Decision Making

We examined whether striatal dopamine moderates the impact of externalizing proneness (disinhibition) on reward-based decision-making. Participants completed disinhibition and substance abuse subscales of the brief form Externalizing Spectrum Inventory, and then performed a delay discounting task to assess preference for immediate rewards along with a dynamic decision-making task that assessed long-term reward learning (i.e., inclination to choose larger delayed versus smaller immediate rewards). Striatal tonic dopamine levels were operationalized using spontaneous eyeblink rate. Regression analyses revealed that high disinhibition predicted greater delay discounting among participants with lower levels of striatal dopamine only, while substance abuse was associated with poorer long-term learning among individuals with lower levels of striatal dopamine, but better long-term learning in those with higher levels of striatal dopamine. These results suggest that disinhibition is more strongly associated with the wanting component of reward-based decision-making, whereas substance abuse behavior is associated more with learning of long-term action-reward contingencies.

Eye-blinks in choice response tasks uncover hidden aspects of information processing

Spontaneous eye-blinks occur much more often than it would be necessary to maintain the tear film on the eyes. Various factors like cognitive demand, task engagement, or fatigue are influencing spontaneous blink rate. During cognitive information processing there is evidence that blinks occur preferably at moments that can be assigned to input stream segmentation. We investigated blinking behavior in three different visual choice response experiments (Experiment 1: spatial Stimulus-Response correspondence, Experiment 2: Change Detection, Experiment 3: Continuous performance Test - AX version). Blinks during the experimental tasks were suppressed when new information was expected, as well as during cognitive processing until the response was executed. Blinks in go trials occurred within a short and relatively constant interval after manual responses. However, blinks were not a side effect of manual behavior, as they occurred in a similar manner in no-go trials in which no manual response was executed. In these trials, blinks were delayed when a prepared response had to be inhibited, compared to trials in which no response was intended. Additionally, time on task effects for no-go blinks mirrored those obtained in go trials. Thus, blinks seem to provide a reliable measure for cognitive processing beyond (or rather additional to) manual responses.

Cluster-based analysis for personalized stress evaluation using physiological signals

Technology development in wearable sensors and biosignal processing has made it possible to detect human stress from the physiological features. However, the intersubject difference in stress responses presents a major challenge for reliable and accurate stress estimation. This research proposes a novel cluster-based analysis method to measure perceived stress using physiological signals, which accounts for the intersubject differences. The physiological data are collected when human subjects undergo a series of task-rest cycles, incurring varying levels of stress that is indicated by an index of the State Trait Anxiety Inventory. Next, a quantitative measurement of stress is developed by analyzing the physiological features in two steps: 1) a k -means clustering process to divide subjects into different categories (clusters), and 2) cluster-wise stress evaluation using the general regression neural network. Experimental results show a significant improvement in evaluation accuracy as compared to traditional methods without clustering. The proposed method is useful in developing intelligent, personalized products for human stress management.

Predictors of Neurocognitive Syndromes in Combat Veterans

Traumatic brain injury, depression and posttraumatic stress disorder (PTSD) are neurocognitive syndromes often associated with impairment of physical and mental health, as well as functional status. These syndromes are also frequent in military service members (SMs) after combat, although their presentation is often delayed until months after their return. The objective of this prospective cohort study was the identification of independent predictors of neurocognitive syndromes upon return from deployment could facilitate early intervention to prevent disability. We completed a comprehensive baseline assessment, followed by serial evaluations at three, six, and 12 months, to assess for new-onset PTSD, depression, or postconcussive syndrome (PCS) in order to identify baseline factors most strongly associated with subsequent neurocognitive syndromes. On serial follow-up, seven participants developed at least one neurocognitive syndrome: five with PTSD, one with depression and PTSD, and one with PCS. On univariate analysis, 60 items were associated with syndrome development at p < 0.15. Decision trees and ensemble tree multivariate models yielded four common independent predictors of PTSD: right superior longitudinal fasciculus tract volume on MRI; resting state connectivity between the right amygdala and left superior temporal gyrus (BA41/42) on functional MRI; and single nucleotide polymorphisms in the genes coding for myelin basic protein as well as brain-derived neurotrophic factor. Our findings require follow-up studies with greater sample size and suggest that neuroimaging and molecular biomarkers may help distinguish those at high risk for post-deployment neurocognitive syndromes.

Effective Indices for Monitoring Mental Workload While Performing Multiple Tasks

This study identified several physiological indices that can accurately monitor mental workload while participants performed multiple tasks with the strategy of maintaining stable performance and maximizing accuracy. Thirty male participants completed three 10-min. simulated multitasks: MATB (Multi-Attribute Task Battery) with three workload levels. Twenty-five commonly used mental workload measures were collected, including heart rate, 12 HRV (heart rate variability), 10 EEG (electroencephalography) indices (α, β, θ, α/θ, θ/β from O1-O2 and F4-C4), and two subjective measures. Analyses of index sensitivity showed that two EEG indices, θ and α/θ (F4-C4), one time-domain HRV-SDNN (standard deviation of inter-beat intervals), and four frequency-domain HRV: VLF (very low frequency), LF (low frequency), %HF (percentage of high frequency), and LF/HF were sensitive to differentiate high workload. EEG α/θ (F4-C4) and LF/HF were most effective for monitoring high mental workload. LF/HF showed the highest correlations with other physiological indices. EEG α/θ (F4-C4) showed strong correlations with subjective measures across diff erent mental workload levels. Operation strategy would affect the sensitivity of EEG α (F4-C4) and HF.

Traces of unconscious mental processes in introspective reports and physiological responses

Unconscious mental processes have recently started gaining attention in a number of scientific disciplines. One of the theoretical frameworks for describing unconscious processes was introduced by Jung as a part of his model of the psyche. This framework uses the concept of archetypes that represent prototypical experiences associated with objects, people, and situations. Although the validity of Jungian model remains an open question, this framework is convenient from the practical point of view. Moreover, archetypes found numerous applications in the areas of psychology and marketing. Therefore, observation of both conscious and unconscious traces related to archetypal experiences seems to be an interesting research endeavor. In a study with 36 subjects, we examined the effects of experiencing conglomerations of unconscious emotions associated with various archetypes on the participants' introspective reports and patterns of physiological activations. Our hypothesis for this experiment was that physiological data may predict archetypes more precisely than introspective reports due to the implicit nature of archetypal experiences. Introspective reports were collected using the Self-Assessment Manikin (SAM) technique. Physiological measures included cardiovascular, electrodermal, respiratory responses and skin temperature of the subjects. The subjects were stimulated to feel four archetypal experiences and four explicit emotions by means of film clips. The data related to the explicit emotions served as a reference in analysis of archetypal experiences. Our findings indicated that while prediction models trained on the collected physiological data could recognize the archetypal experiences with accuracy of 55 percent, similar models built based on the SAM data demonstrated performance of only 33 percent. Statistical tests enabled us to confirm that physiological observations are better suited for observation of implicit psychological constructs like archetypes than introspective reports.

Cardiovascular state changes in simulated work environments

The usefulness of cardiovascular measures as indicators of changes in cognitive workload has been addressed in several studies. In this paper the question is explored whether cardiovascular patterns in heart rate, blood pressure, baroreflex sensitivity and HRV that are found are consistent within and between two simulated working environments. Two studies, were performed, both with 21 participants: one in an ambulance dispatch simulation and one in a driving simulator. In the ambulance dispatcher task an initial strong increase in blood pressure is followed by a moderate on-going increase in blood pressure during the next hour of task performance. This pattern is accompanied by a strong increase in baroreflex sensitivity while heart rate decreases. In the driving simulator study, blood pressure initially increases but decreases almost to baseline level in the next hour. This pattern is accompanied by a decrease in baroreflex sensitivity, while heart rate decreases. Results of both studies are interpreted in terms of autonomic control (related to both sympathetic and para-sympathetic effects), using a simplified simulation of a baroreflex regulation model. Interpretation of the results leads to the conclusion that the cardiovascular response patterns in both tasks are a combination of an initial defensive reaction, in combination with compensatory blood pressure control. The level of compensatory blood pressure control, however, is quite different for the two tasks. This helps to understand the differences in response patterns between the two studies in this paper and may be helpful as well for understanding differences in cardiovascular response patterns in general. A substantial part of the effects observed during task performance are regulatory effects and are not always directly related to workload manipulations. Making this distinction may also contribute to the understanding of differences in cardiovascular response patterns during cognitive workload.