Classification of cross task cognitive workload using deep recurrent network with modelling of temporal dynamics

Comparison of concurrent cognitive load measures during n-back tasks

The cognitive load experienced by humans is an important factor affecting their performance. Cognitive overload or underload may result in suboptimal human performance and may compromise safety in emerging human-in-the-loop systems. In driving, cognitive overload, due to various secondary tasks, such as texting, results in driver distraction. On the other hand, cognitive underload may result in fatigue. In automated manufacturing systems, a distracted operator may be prone to muscle injuries. Similar outcomes are possible in many other fields of human performance such as aviation, healthcare, and learning environments. The challenge with such human-centred applications is that the cognitive load is not directly measurable. Only the change in cognitive load is measured indirectly through various physiological, behavioural, performance-based and subjective means. A method to objectively assess the performance of such diverse measures of cognitive load is lacking in the literature. In this paper, a performance metric for the comparison of different measures to determine the cognitive workload is proposed in terms of the signal-to-noise ratio. Using this performance metric, several measures of cognitive load, that fall under the four broad groups were compared on the same scale for their ability to measure changes in cognitive load. Using the proposed metrics, the cognitive load measures were compared based on data collected from 28 participants while they underwent n-back tasks of varying difficulty. The results show that the proposed performance evaluation method can be useful to individually assess different measures of cognitive load.

Assessing Cognitive Workload in Motor Decision-Making through Functional Connectivity Analysis: Towards Early Detection and Monitoring of Neurodegenerative Diseases

Neurodegenerative diseases (NDs), such as Alzheimer's, Parkinson's, amyotrophic lateral sclerosis, and frontotemporal dementia, among others, are increasingly prevalent in the global population. The clinical diagnosis of these NDs is based on the detection and characterization of motor and non-motor symptoms. However, when these diagnoses are made, the subjects are often in advanced stages where neuromuscular alterations are frequently irreversible. In this context, we propose a methodology to evaluate the cognitive workload (CWL) of motor tasks involving decision-making processes. CWL is a concept widely used to address the balance between task demand and the subject's available resources to complete that task. In this study, multiple models for motor planning during a motor decision-making task were developed by recording EEG and EMG signals in n=17 healthy volunteers (9 males, 8 females, age 28.66±8.8 years). In the proposed test, volunteers have to make decisions about which hand should be moved based on the onset of a visual stimulus. We computed functional connectivity between the cortex and muscles, as well as among muscles using both corticomuscular and intermuscular coherence. Despite three models being generated, just one of them had strong performance. The results showed two types of motor decision-making processes depending on the hand to move. Moreover, the central processing of decision-making for the left hand movement can be accurately estimated using behavioral measures such as planning time combined with peripheral recordings like EMG signals. The models provided in this study could be considered as a methodological foundation to detect neuromuscular alterations in asymptomatic patients, as well as to monitor the process of a degenerative disease.

Cross-task cognitive workload recognition using a dynamic residual network with attention mechanism based on neurophysiological signals

BACKGROUND AND OBJECTIVE

Evaluation of human cognitive workload (CW) helps improve the user experience of human-centered systems. To provide a continuous estimation of the CW, we built a CW recognizer that maps human electroencephalograms (EEGs) to discrete CW levels with deep learning tools. However, the EEG distribution varies when humans perform different cognitive tasks. There is thus a question on the capacity for generalizing the CW recognizer across tasks. In this study, we examined the CW's performance when it was trained and tested on two EEG databases corresponding to different human-machine tasks.

METHODS

A novel deep neural network-based EEG recognizer, dynamic residual network with attention mechanism (DRNA-Net), is proposed in the present study. By taking advantage of recurrent networks, the DRNA-Net further incorporates a self-attention mechanism in discovering robust EEG patterns across different cognitive tasks.

RESULTS

We designed an experiment that applied a multidimensional N-back task to induce the CW that consists of visual and auditory memory tasks. We validated the cross-task generalization capability of the DRNA-Net based on the EEG features extracted from the N-back task and a public database. The results show that the DRNA-Net achieves classification accuracy and Macro-F1 values are 0.6055 and 0.6067, respectively.

CONCLUSIONS

The performance of the DRNA-Net indicates that it has a certain ability of cross-task cognitive workload classification, which outperforms several shallow learners and deep convolutional neural networks under various conditions of the feature subsets.

Predictions of task using neural modeling

Introduction

A well-designed brain-computer interface (BCI) can make accurate and reliable predictions of a user's state through the passive assessment of their brain activity; in turn, BCI can inform an adaptive system (such as artificial intelligence, or AI) to intelligently and optimally aid the user to maximize the human-machine team (HMT) performance. Various groupings of spectro-temporal neural features have shown to predict the same underlying cognitive state (e.g., workload) but vary in their accuracy to generalize across contexts, experimental manipulations, and beyond a single session. In our work we address an outstanding challenge in neuroergonomic research: we quantify if (how) identified neural features and a chosen modeling approach will generalize to various manipulations defined by the same underlying psychological construct, (multi)task cognitive workload.

Methods

To do this, we train and test 20 different support vector machine (SVM) models, each given a subset of neural features as recommended from previous research or matching the capabilities of commercial devices. We compute each model's accuracy to predict which (monitoring, communications, tracking) and how many (one, two, or three) task(s) were completed simultaneously. Additionally, we investigate machine learning model accuracy to predict task(s) within- vs. between-sessions, all at the individual-level.

Results

Our results indicate gamma activity across all recording locations consistently outperformed all other subsets from the full model. Our work demonstrates that modelers must consider multiple types of manipulations which may each influence a common underlying psychological construct.

Discussion

We offer a novel and practical modeling solution for system designers to predict task through brain activity and suggest next steps in expanding our framework to further contribute to research and development in the neuroergonomics community. Further, we quantified the cost in model accuracy should one choose to deploy our BCI approach using a mobile EEG-systems with fewer electrodes-a practical recommendation from our work.

Recognition of the Mental Workloads of Pilots in the Cockpit Using EEG Signals

The commercial flightdeck is a naturally multi-tasking work environment, one in which interruptions are frequent come in various forms, contributing in many cases to aviation incident reports. Automatic characterization of pilots’ workloads is essential to preventing these kind of incidents. In addition, minimizing the physiological sensor network as much as possible remains both a challenge and a requirement. Electroencephalogram (EEG) signals have shown high correlations with specific cognitive and mental states, such as workload. However, there is not enough evidence in the literature to validate how well models generalize in cases of new subjects performing tasks with workloads similar to the ones included during the model’s training. In this paper, we propose a convolutional neural network to classify EEG features across different mental workloads in a continuous performance task test that partly measures working memory and working memory capacity. Our model is valid at the general population level and it is able to transfer task learning to pilot mental workload recognition in a simulated operational environment.