Monitoring task loading with multivariate EEG measures during complex forms of human-computer interaction

Cognitive and behavioral markers for human detection error in AI-assisted bridge inspection

Integrating Artificial Intelligence (AI) and drone technology into bridge inspections offers numerous advantages, including increased efficiency and enhanced safety. However, it is essential to recognize that this integration changes the cognitive ergonomics of the inspection task. Gaining a deeper understanding of how humans process information and behave when collaborating with drones and AI systems is necessary for designing and implementing effective AI-assisted inspection drones. To further understand human-drone-AI intricate dynamics, an experiment was conducted in which participants' biometric and behavioral data were collected during a simulated drone-enabled bridge inspection under two conditions: with an 80% accurate AI assistance and with no AI assistance. Results indicate that cognitive and behavioral factors, including vigilance, cognitive processing intensity, gaze patterns, and visual scanning efficiency can influence inspectors' performance respectively in either condition. This highlights the importance of designing inspection protocols, drones and AI systems based on a comprehensive understanding of the cognitive processes required in each condition to prevent cognitive overload and minimize errors. We also remark on the visual scanning and gaze patterns associated with a higher chance of missing critical information in each condition, insights that inspectors can use to enhance their inspection performance.

Air Traffic Controller Workload Detection Based on EEG Signals

The assessment of the cognitive workload experienced by air traffic controllers is a complex and prominent issue in the research community. This study introduces new indicators related to gamma waves to detect controllers' workload and develops experimental protocols to capture their EEG data and NASA-TXL data. Then, statistical tests, including the Shapiro-Wilk test and ANOVA, were used to verify whether there was a significant difference between the workload data of the controllers in different scenarios. Furthermore, the Support Vector Machine (SVM) classifier was employed to assess the detection accuracy of these indicators across four categorizations. According to the outcomes, hypotheses suggesting a strong correlation between gamma waves and an air traffic controller's workload were put forward and subsequently verified; meanwhile, compared with traditional indicators, the indicators associated with gamma waves proposed in this paper have higher accuracy. In addition, to explore the applicability of the indicator, sensitive channels were selected based on the mRMR algorithm for the indicator with the highest accuracy, β + θ + α + γ, showcasing a recognition rate of a single channel exceeding 95% of the full channel, which meets the requirements of convenience and accuracy in practical applications. In conclusion, this study demonstrates that utilizing EEG gamma wave-associated indicators can offer valuable insights into analyzing workload levels among air traffic controllers.

From E-Commerce to the Metaverse: A Neuroscientific Analysis of Digital Consumer Behavior

The growing interest in consumer behavior in the digital environment is leading scholars and companies to focus on consumer behavior and choices on digital platforms, such as the metaverse. On this immersive digital shopping platform, consumer neuroscience provides an optimal opportunity to explore consumers' emotions and cognitions. In this study, neuroscience techniques (EEG, SC, BVP) were used to compare emotional and cognitive aspects of shopping between metaverse and traditional e-commerce platforms. Participants were asked to purchase the same product once on a metaverse platform (Second Life, SL) and once via an e-commerce website (EC). After each task, questionnaires were administered to measure perceived enjoyment, informativeness, ease of use, cognitive effort, and flow. Statistical analyses were conducted to examine differences between SL and EC at the neurophysiological and self-report levels, as well as between different stages of the purchase process. The results show that SL elicits greater cognitive engagement than EC, but it is also more mentally demanding, with a higher workload and more memorization, and fails to elicit a strong positive emotional response, leading to a poorer shopping experience. These findings provide insights not only for digital-related consumer research but also for companies to improve their metaverse shopping experience. Before investing in the platform or creating a digital retail space, companies should thoroughly analyze it, focusing on how to enhance users' cognition and emotions, ultimately promoting a better consumer experience. Despite its limitations, this pilot study sheds light on the emotional and cognitive aspects of metaverse shopping and suggests potential for further research with a consumer neuroscience approach in the metaverse field.

Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study

While the term task load (TL) refers to external task demands, the amount of work, or the number of tasks to be performed, mental workload (MWL) refers to the individual's effort, mental capacity, or cognitive resources utilized while performing a task. MWL in multitasking scenarios is often closely linked with the quantity of tasks a person is handling within a given timeframe. In this study, we challenge this hypothesis from the perspective of electroencephalography (EEG) using a deep learning approach. We conducted an EEG experiment with 50 participants performing NASA Multi-Attribute Task Battery II (MATB-II) under 4 different task load levels. We designed a convolutional neural network (CNN) to help with two distinct classification tasks. In one setting, the CNN was used to classify EEG segments based on their task load level. In another setting, the same CNN architecture was trained again to detect the presence of individual MATB-II subtasks. Results show that, while the model successfully learns to detect whether a particular subtask is active in a given segment (i.e., to differentiate between different subtasks-related EEG patterns), it struggles to differentiate between the two highest levels of task load (i.e., to distinguish MWL-related EEG patterns). We speculate that the challenge comes from two factors: first, the experiment was designed in a way that these two highest levels differed only in the quantity of work within a given timeframe; and second, the participants' effective adaptation to increased task demands, as evidenced by low error rates. Consequently, this indicates that under such conditions in multitasking, EEG may not reflect distinct enough patterns to differentiate higher levels of task load.

Investigating mental workload caused by NDRTs in highly automated driving with deep learning

OBJECTIVE

This study aimed to examine the impact of non-driving-related tasks (NDRTs) on drivers in highly automated driving scenarios and sought to develop a deep learning model for classifying mental workload using electroencephalography (EEG) signals.

METHODS

The experiment involved recruiting 28 participants who engaged in simulations within a driving simulator while exposed to 4 distinct NDRTs: (1) reading, (2) listening to radio news, (3) watching videos, and (4) texting. EEG data collected during NDRTs were categorized into 3 levels of mental workload, high, medium, and low, based on the NASA Task Load Index (NASA-TLX) scores. Two deep learning methods, namely, long short-term memory (LSTM) and bidirectional long short-term memory (BLSTM), were employed to develop the classification model.

RESULTS

A series of correlation analyses revealed that the channels and frequency bands are linearly correlated with mental workload. The comparative analysis of classification results demonstrates that EEG data featuring significantly correlated frequency bands exhibit superior classification accuracy compared to the raw EEG data.

CONCLUSIONS

This research offers a reference for assessing mental workload resulting from NDRTs in the context of highly automated driving. Additionally, it delves into the development of deep learning classifiers for EEG signals with heightened accuracy.

EEG-based detection of modality-specific visual and auditory sensory processing

Objective.A passive brain-computer interface (pBCI) is a system that enhances a human-machine interaction by monitoring the mental state of the user and, based on this implicit information, making appropriate modifications to the interaction. Key to the development of such a system is the ability to reliably detect the mental state of interest via neural signals. Many different mental states have been investigated, including fatigue, attention and various emotions, however one of the most commonly studied states is mental workload, i.e. the amount of attentional resources required to perform a task. The emphasis of mental workload studies to date has been almost exclusively on detecting and predicting the 'level' of cognitive resources required (e.g. high vs. low), but we argue that having information regarding the specific 'type' of resources (e.g. visual or auditory) would allow the pBCI to apply more suitable adaption techniques than would be possible knowing just the overall workload level.Approach.15 participants performed carefully designed visual and auditory tasks while electroencephalography (EEG) data was recorded. The tasks were designed to be as similar as possible to one another except for the type of attentional resources required. The tasks were performed at two different levels of demand. Using traditional machine learning algorithms, we investigated, firstly, if EEG can be used to distinguish between auditory and visual processing tasks and, secondly, what effect level of sensory processing demand has on the ability to distinguish between auditory and visual processing tasks.Main results.The results show that at the high level of demand, the auditory vs. visual processing tasks could be distinguished with an accuracy of 77.1% on average. However, in the low demand condition in this experiment, the tasks were not classified with an accuracy exceeding chance.Significance.These results support the feasibility of developing a pBCI for detecting not only the level, but also the type, of attentional resources being required of the user at a given time. Further research is required to determine if there is a threshold of demand under which the type of sensory processing cannot be detected, but even if that is the case, these results are still promising since it is the high end of demand that is of most concern in safety critical scenarios. Such a BCI could help improve safety in high risk occupations by initiating the most effective and efficient possible adaptation strategies when high workload conditions are detected.

Open multi-session and multi-task EEG cognitive Dataset for passive brain-computer Interface Applications

Brain-Computer Interfaces and especially passive Brain-Computer interfaces (pBCI), with their ability to estimate and monitor user mental states, are receiving increasing attention from both the fundamental research and the applied research and development communities. Testing new pipelines and benchmarking classifiers and feature extraction algorithms is central to further research within this domain. Unfortunately, data sharing in pBCI research is still scarce. The COG-BCI database encompasses the recordings of 29 participants over 3 separate sessions with 4 different tasks (MATB, N-Back, PVT, Flanker) designed to elicit different mental states, for a total of over 100 hours of open EEG data. This dataset was validated on a subjective, behavioral and physiological level, to ensure its usefulness to the pBCI community. Furthermore, a proof of concept is given with an example of mental workload estimation pipeline and results, to ensure that the data can be used for the design and evaluation of pBCI pipelines. This body of work presents a large effort to promote the use of pBCIs in an open science framework.

The mitigation of the executive vigilance decrement via HD-tDCS over the right posterior parietal cortex and its association with neural oscillations

Vigilance-maintaining a prolonged state of preparation to detect and respond to specific yet unpredictable environmental changes-usually decreases across prolonged tasks, causing potentially severe real-life consequences, which could be mitigated through transcranial direct current stimulation (tDCS). The present study aimed at replicating previous mitigatory effects observed with anodal high-definition tDCS (HD-tDCS) over the right posterior parietal cortex (rPPC) while extending the analyses on electrophysiological measures associated with vigilance. In sum, 60 participants completed the ANTI-Vea task while receiving anodal (1.5 mA, n = 30) or sham (0 mA, n = 30) HD-tDCS over the rPPC for ~ 28 min. EEG recordings were completed before and after stimulation. Anodal HD-tDCS specifically mitigated executive vigilance (EV) and reduced the alpha power increment across time-on-task while increasing the gamma power increment. To further account for the observed behavioral and physiological outcomes, a new index of Alphaparietal/Gammafrontal is proposed. Interestingly, the increment of this Alphaparietal/Gammafrontal Index with time-on-task is associated with a steeper EV decrement in the sham group, which was mitigated by anodal HD-tDCS. We highlight the relevance of replicating mitigatory effects of tDCS and the need to integrate conventional and novel physiological measures to account for how anodal HD-tDCS can be used to modulate cognitive performance.

Predictions of task using neural modeling

Introduction

A well-designed brain-computer interface (BCI) can make accurate and reliable predictions of a user's state through the passive assessment of their brain activity; in turn, BCI can inform an adaptive system (such as artificial intelligence, or AI) to intelligently and optimally aid the user to maximize the human-machine team (HMT) performance. Various groupings of spectro-temporal neural features have shown to predict the same underlying cognitive state (e.g., workload) but vary in their accuracy to generalize across contexts, experimental manipulations, and beyond a single session. In our work we address an outstanding challenge in neuroergonomic research: we quantify if (how) identified neural features and a chosen modeling approach will generalize to various manipulations defined by the same underlying psychological construct, (multi)task cognitive workload.

Methods

To do this, we train and test 20 different support vector machine (SVM) models, each given a subset of neural features as recommended from previous research or matching the capabilities of commercial devices. We compute each model's accuracy to predict which (monitoring, communications, tracking) and how many (one, two, or three) task(s) were completed simultaneously. Additionally, we investigate machine learning model accuracy to predict task(s) within- vs. between-sessions, all at the individual-level.

Results

Our results indicate gamma activity across all recording locations consistently outperformed all other subsets from the full model. Our work demonstrates that modelers must consider multiple types of manipulations which may each influence a common underlying psychological construct.

Discussion

We offer a novel and practical modeling solution for system designers to predict task through brain activity and suggest next steps in expanding our framework to further contribute to research and development in the neuroergonomics community. Further, we quantified the cost in model accuracy should one choose to deploy our BCI approach using a mobile EEG-systems with fewer electrodes-a practical recommendation from our work.

Cognitive performance and electroencephalographic variations in air traffic controllers under various mental workload and time of day

The aim of this study was to investigate the effects of mental workload (MWL) and time of day on cognitive performance and electroencephalographic (EEG) parameters of air traffic controllers. EEG signals recorded while 20 professional air traffic controllers performed cognitive tasks [A-X Continuous Performance Test (AX-CPT) and 3-back working memory task] after they were exposed to two levels of task difficulty (high and low MWL) in the morning and afternoon. Significant decreases in cognitive performance were found when the levels of task difficulty increased in both tasks. The results confirmed the sensitivity of the theta and beta activities to levels of task difficulty in the 3-back task, while they were not affected in the AX-CPT. Theta and beta activities were influenced by time of day in the AX-CPT. The findings provide guidance for application of changes in EEG parameters when MWL level is manipulated during the day that could be implemented in future for the development of real-time monitoring systems to improve aviation safety.

Unravelling the Physiological Correlates of Mental Workload Variations in Tracking and Collision Prediction Tasks

Modern work environments have extensive interactions with technology and greater cognitive complexity of the tasks, which results in human operators experiencing increased mental workload. Air traffic control operators routinely work in such complex environments, and we designed tracking and collision prediction tasks to emulate their elementary tasks. The physiological response to the workload variations in these tasks was elucidated to untangle the impact of workload variations experienced by operators. Electroencephalogram (EEG), eye activity, and heart rate variability (HRV) data were recorded from 24 participants performing tracking and collision prediction tasks with three levels of difficulty. Our findings indicate that variations in task load in both these tasks are sensitively reflected in EEG, eye activity and HRV data. Multiple regression results also show that operators' performance in both tasks can be predicted using the corresponding EEG, eye activity and HRV data. The results also demonstrate that the brain dynamics during each of these tasks can be estimated from the corresponding eye activity, HRV and performance data. Furthermore, the markedly distinct neurometrics of workload variations in the tracking and collision prediction tasks indicate that neurometrics can provide insights on the type of mental workload. These findings have applicability to the design of future mental workload adaptive systems that integrate neurometrics in deciding not just "when" but also "what" to adapt. Our study provides compelling evidence in the viability of developing intelligent closed-loop mental workload adaptive systems that ensure efficiency and safety in complex work environments.

Cognitive Workload of Tugboat Captains in Realistic Scenarios: Adaptive Spatial Filtering for Transfer Between Conditions

Changing and often class-dependent non-stationarities of signals are a big challenge in the transfer of common findings in cognitive workload estimation using Electroencephalography (EEG) from laboratory experiments to realistic scenarios or other experiments. Additionally, it often remains an open question whether actual cognitive workload reflected by brain signals was the main contribution to the estimation or discriminative and class-dependent muscle and eye activity, which can be secondary effects of changing workload levels. Within this study, we investigated a novel approach to spatial filtering based on beamforming adapted to changing settings. We compare it to no spatial filtering and Common Spatial Patterns (CSP). We used a realistic maneuvering task, as well as an auditory n-back secondary task on a tugboat simulator as two different conditions to induce workload changes on professional tugboat captains. Apart from the typical within condition classification, we investigated the ability of the different classification methods to transfer between the n-back condition and the maneuvering task. The results show a clear advantage of the proposed approach over the others in the challenging transfer setting. While no filtering leads to lowest within-condition normalized classification loss on average in two scenarios (22 and 10%), our approach using adaptive beamforming (30 and 18%) performs comparably to CSP (33 and 15%). Importantly, in the transfer from one to another setting, no filtering and CSP lead to performance around chance level (45 to 53%), while our approach in contrast is the only one capable of classifying in all other scenarios (34 and 35%) with a significant difference from chance level. The changing signal composition over the scenarios leads to a need to adapt the spatial filtering in order to be transferable. With our approach, the transfer is successful due to filtering being optimized for the extraction of neural components and additional investigation of their scalp patterns revealed mainly neural origin. Interesting findings are that rather the patterns slightly change between conditions. We conclude that the approaches with low normalized loss depend on eye and muscle activity which is successful for classification within conditions, but fail in the classifier transfer since eye and muscle contributions are highly condition-specific.

Simultaneous Classification of Both Mental Workload and Stress Level Suitable for an Online Passive Brain-Computer Interface

Research studies on EEG-based mental workload detection for a passive BCI generally focus on classifying cognitive states associated with the performance of tasks at different levels of difficulty, with no other aspects of the user's mental state considered. However, in real-life situations, different aspects of the user's state such as their cognitive (e.g., level of mental workload) and affective (e.g., level of stress/anxiety) states will often change simultaneously, and performance of a BCI system designed considering just one state may be unreliable. Moreover, multiple mental states may be relevant to the purposes of the BCI-for example both mental workload and stress level might be related to an aircraft pilot's risk of error-and the simultaneous prediction of states may be critical in maximizing the practical effectiveness of real-life online BCI systems. In this study we investigated the feasibility of performing simultaneous classification of mental workload and stress level in an online passive BCI. We investigated both subject-specific and cross-subject classification approaches, the latter with and without the application of a transfer learning technique to align the distributions of data from the training and test subjects. Using cross-subject classification with transfer learning in a simulated online analysis, we obtained accuracies of 77.5 ± 6.9% and 84.1 ± 5.9%, across 18 participants for mental workload and stress level detection, respectively.

Measuring the Domain Specificity of Workload Using EEG: Auditory and Visual Domains in Rotary-Wing Simulated Flight

OBJECTIVE

The overarching objective was to evaluate whether workload sensory-domain specificity could be identified through electroencephalogram (EEG) recordings during simulated rotary-wing operations.

BACKGROUND

Rotary-wing aviators experience workload from different sensory domains, although predominantly through auditory and visual domains. Development of real-time monitoring tools using psychophysiological indices, such as EEG recordings, could enable identification of aviator overload in real time.

METHOD

Two studies were completed, both of which recorded EEG, task performance, and self-report data. In Study 1, 16 individuals completed a basic auditory and a basic visual laboratory task where workload was manipulated. In Study 2, 23 Army aviators completed simulated aviation flights where workload was manipulated within auditory and visual sensory domains.

RESULTS

Results from Study 1 found differences in frontal alpha activity during the auditory task, and that alpha and beta activities were associated with perceived workload. Frontal theta activity was found to differ during the visual task while frontal alpha was associated with perceived workload. Study 2 found support for frontal beta activity and the ratio of beta to alpha + theta to differentiate level of workload within the auditory domain.

CONCLUSION

There is likely a role of frontal alpha and beta activities in response to workload manipulations within the auditory domain; however, this role becomes more equivocal when examined in a multifaceted flight scenario.

APPLICATION

Results from this study provide a basis for understanding changes in EEG activity when workload is manipulated in sensory domains that can be used in furthering the development of real-time monitoring tools.

Tracking of Mental Workload with a Mobile EEG Sensor

The aim of the present investigation was to assess if a mobile electroencephalography (EEG) setup can be used to track mental workload, which is an important aspect of learning performance and motivation and may thus represent a valuable source of information in the evaluation of cognitive training approaches. Twenty five healthy subjects performed a three-level N-back test using a fully mobile setup including tablet-based presentation of the task and EEG data collection with a self-mounted mobile EEG device at two assessment time points. A two-fold analysis approach was chosen including a standard analysis of variance and an artificial neural network to distinguish the levels of cognitive load. Our findings indicate that the setup is feasible for detecting changes in cognitive load, as reflected by alterations across lobes in different frequency bands. In particular, we observed a decrease of occipital alpha and an increase in frontal, parietal and occipital theta with increasing cognitive load. The most distinct levels of cognitive load could be discriminated by the integrated machine learning models with an accuracy of 86%.

Real-time prediction of short-timescale fluctuations in cognitive workload

Human operators often experience large fluctuations in cognitive workload over seconds timescales that can lead to sub-optimal performance, ranging from overload to neglect. Adaptive automation could potentially address this issue, but to do so it needs to be aware of real-time changes in operators' spare cognitive capacity, so it can provide help in times of peak demand and take advantage of troughs to elicit operator engagement. However, it is unclear whether rapid changes in task demands are reflected in similarly rapid fluctuations in spare capacity, and if so what aspects of responses to those demands are predictive of the current level of spare capacity. We used the ISO standard detection response task (DRT) to measure cognitive workload approximately every 4 s in a demanding task requiring monitoring and refueling of a fleet of simulated unmanned aerial vehicles (UAVs). We showed that the DRT provided a valid measure that can detect differences in workload due to changes in the number of UAVs. We used cross-validation to assess whether measures related to task performance immediately preceding the DRT could predict detection performance as a proxy for cognitive workload. Although the simple occurrence of task events had weak predictive ability, composite measures that tapped operators' situational awareness with respect to fuel levels were much more effective. We conclude that cognitive workload does vary rapidly as a function of recent task events, and that real-time predictive models of operators' cognitive workload provide a potential avenue for automation to adapt without an ongoing need for intrusive workload measurements.

Effect of tonal noise and task difficulty on electroencephalography and cognitive performance

Objectives. The present study aimed to investigate the effect of tonal noise and task difficulty on electroencephalography (EEG) and cognitive performance. Methods. Twelve healthy volunteers participated in the present study. Four noise signals were generated by four prominence tone levels (0, 2, 5 and 9) at background noise levels of 55 dBA and frequency of 500 Hz using the Test Tone Generator from Esser Audio (USA). The participants were asked to perform the tasks with low, moderate and high levels of difficulty while exposed to the noises in an acoustics laboratory. The values of reaction time, correct rate and missed numbers were recorded during each step. Moreover, the EEG signals were measured. Results. The results showed that higher tone level and more task difficulty significantly decreased the correct rate, and increased the miss numbers. However, no significant effect was observed on reaction times. Furthermore, tone level and task difficulty significantly increased activity of the θ and β bands and decreased activity of the α band. Conclusion. Task difficulty and tone level could significantly affect the parameters of performance and the activity of EEG bands. Therefore, noise control can help sustain appropriate performance.

A Review of Evidence on the Role of Digital Technology in Shaping Attention and Cognitive Control in Children

The role of digital technology in shaping attention and cognitive development has been at the centre of public discourse for decades. The current review presents findings from three main bodies of literature on the implications of technology use for attention and cognitive control: television, video games, and digital multitasking. The aim is to identify key lessons from prior research that are relevant for the current generation of digital users. In particular, the lack of scientific consensus on whether digital technologies are good or bad for children reflects that effects depend on users' characteristics, the form digital technologies take, the circumstances in which use occurs and the interaction between the three factors. Some features of digital media may be particularly problematic, but only for certain users and only in certain contexts. Similarly, individual differences mediate how, when and why individuals use technology, as well as how much benefit or harm can be derived from its use. The finding emerging from the review on the large degree of heterogeneity in associations is especially relevant due to the rapid development and diffusion of a large number of different digital technologies and contents, and the increasing variety of user experiences. We discuss the importance of leveraging existing knowledge and integrating past research findings into a broader organizing framework in order to guide emerging technology-based research and practice. We end with a discussion of some of the challenges and unaddressed issues in the literature and propose directions for future research.

Symptoms of convergence and accommodative insufficiency predict engagement and cognitive fatigue during complex task performance with and without automation

Deficits in the accommodative and/or vergence responses have been linked with inattentive behavioral symptoms. While using automated systems (e.g., self-driving cars, autopilot), operators (e.g., drivers, pilots, soldiers) visually monitor displays for critical changes, making deficits in the accommodative and/or vergence responses potentially hazardous for individuals remaining actively engaged in the task at hand. The purpose of this study was to determine if symptoms of accommodative-vergence deficits predict an individual's level of task engagement and cognitive fatigue while performing a flight simulation task with or without automation. Eighty-four participants performed a flight simulation task with or without automation. Prior to task completion, self-report accommodative-convergence deficit symptoms were assessed with the Convergence Insufficiency Symptom Survey (CISS). Before and after the flight simulation task participants rated their task engagement and cognitive fatigue. Electroencephalographic activity (EEG) was recorded concurrently during task performance. Results showed that higher scores on the CISS were related to increased feelings of fatigue and decreased ratings of task engagement. The CISS was also positively related to parietal-occipital fast alpha power during the last 10 min of the task for participants using automation, suggesting increased cortical idling. CISS scores did not predict performance. Results have implications for optimizing operator cognitive states over extended task performance.

Applications of EEG indices for the quantification of human cognitive performance: A systematic review and bibliometric analysis

BACKGROUND

Neuroergonomics combines neuroscience with ergonomics to study human performance using recorded brain signals. Such neural signatures of performance can be measured using a variety of neuroimaging techniques, including functional magnetic resonance imaging (fMRI), functional near-infrared spectroscopy (fNIRS), and electroencephalography (EEG). EEG has an excellent temporal resolution, and EEG indices are highly sensitive to human brain activity fluctuations.

OBJECTIVE

The focus of this systematic review was to explore the applications of EEG indices for quantifying human performance in a variety of cognitive tasks at the macro and micro scales. To identify trends and the state of the field, we examined global patterns among selected articles, such as journal contributions, highly cited papers, affiliations, and high-frequency keywords. Moreover, we discussed the most frequently used EEG indices and synthesized current knowledge regarding the EEG signatures of associated human performance measurements.

METHODS

In this systematic review, we analyzed articles published in English (from peer-reviewed journals, proceedings, and conference papers), Ph.D. dissertations, textbooks, and reference books. All articles reviewed herein included exclusively EEG-based experimental studies in healthy participants. We searched Web-of-Science and Scopus databases using specific sets of keywords.

RESULTS

Out of 143 papers, a considerable number of cognitive studies focused on quantifying human performance with respect to mental fatigue, mental workload, mental effort, visual fatigue, emotion, and stress. An increasing trend for publication in this area was observed, with the highest number of publications in 2017. Most studies applied linear methods (e.g., EEG power spectral density and the amplitude of event-related potentials) to evaluate human cognitive performance. A few papers utilized nonlinear methods, such as fractal dimension, largest Lyapunov exponent, and signal entropy. More than 50% of the studies focused on evaluating an individual's mental states while operating a vehicle. Several different methods of artifact removal have also been noted. Based on the reviewed articles, research gaps, trends, and potential directions for future research were explored.

CONCLUSION

This systematic review synthesized current knowledge regarding the application of EEG indices for quantifying human performance in a wide variety of cognitive tasks. This knowledge is useful for understanding the global patterns of applications of EEG indices for the analysis and design of cognitive tasks.

Evaluating effectiveness of information visualizations using cognitive fit theory: A neuroergonomics approach

Information visualizations may be evaluated from the perspective of how they match tasks that must be performed with them, a cognitive fit perspective. However, there is a gap between the high-level references made to cognitive fit and the low-level ability to identify and measure it during human interaction with visualizations. We bridge this gap by using an electroencephalography metric derived from frontal midline theta power and parietal alpha power, known as the task load index, to determine if cognitive effort measured at the level of cortical activity is less when cognitive fit is present compared to when cognitive fit is not. We found that when there is cognitive fit between the type of problem to be solved and the information displayed by a system, the task load index is lower compared to when cognitive fit is not present. We support this finding with subjective (NASA task load index) and performance (response time and accuracy) measures. Our approach, using electroencephalography, provides supplemental information to self-report and performance measures. Findings from this study are important because they (1) provide more validity to the cognitive fit theory using a neurophysiological measure, and (2) use the electroencephalography task load index metric as a means to assess cognitive workload and effort in general.

A Cyber-Physical-Human System for One-to-Many UAS Operations: Cognitive Load Analysis

The continuing development of avionics for Unmanned Aircraft Systems (UASs) is introducing higher levels of intelligence and autonomy both in the flight vehicle and in the ground mission control, allowing new promising operational concepts to emerge. One-to-Many (OTM) UAS operations is one such concept and its implementation will require significant advances in several areas, particularly in the field of Human-Machine Interfaces and Interactions (HMI2). Measuring cognitive load during OTM operations, in particular Mental Workload (MWL), is desirable as it can relieve some of the negative effects of increased automation by providing the ability to dynamically optimize avionics HMI2 to achieve an optimal sharing of tasks between the autonomous flight vehicles and the human operator. The novel Cognitive Human Machine System (CHMS) proposed in this paper is a Cyber-Physical Human (CPH) system that exploits the recent technological developments of affordable physiological sensors. This system focuses on physiological sensing and Artificial Intelligence (AI) techniques that can support a dynamic adaptation of the HMI2 in response to the operators' cognitive state (including MWL), external/environmental conditions and mission success criteria. However, significant research gaps still exist, one of which relates to a universally valid method for determining MWL that can be applied to UAS operational scenarios. As such, in this paper we present results from a study on measuring MWL on five participants in an OTM UAS wildfire detection scenario, using Electroencephalogram (EEG) and eye tracking measurements. These physiological data are compared with a subjective measure and a task index collected from mission-specific data, which serves as an objective task performance measure. The results show statistically significant differences for all measures including the subjective, performance and physiological measures performed on the various mission phases. Additionally, a good correlation is found between the two physiological measurements and the task index. Fusing the physiological data and correlating with the task index gave the highest correlation coefficient (CC = 0.726 ± 0.14) across all participants. This demonstrates how fusing different physiological measurements can provide a more accurate representation of the operators' MWL, whilst also allowing for increased integrity and reliability of the system.

Detecting cognitive hacking in visual inspection with physiological measurements

Cyber threats are targeting vulnerabilities of human workers performing tasks in manufacturing processes, including visual inspection to bias their decision-making, thereby sabotaging product quality. This article examines the use of priming as a form of "cognitive hacking" to adversely affect quality inspection decisions in manufacturing, and investigates physiological measurements as means to detect such intrusion. In a within-subject design experiment, twenty participants inspected surface roughness of a manufactured component with and without exposure to priming on the display of an inspection logging system. The results show that the presence of primes impacted accuracy on surface roughness, cortical activities at parietal lobe P4, and eye gaze for inspecting components. The experiment provides supporting evidence that basic hacking of a worker display can be an effective method to alter decision making in inspection. The findings also illustrate that cortical activities and eye gaze can be useful indicators of cognitive hacking. A major implication of the study results is that physiological indicators can be effective at revealing unconscious cognitive influence in visual inspection.

EEG Theta Power Activity Reflects Workload among Army Combat Drivers: An Experimental Study

We aimed to evaluate the effects of mental workload variations, as a function of the road environment, on the brain activity of army drivers performing combat and non-combat scenarios in a light multirole vehicle dynamic simulator. Forty-one non-commissioned officers completed three standardized driving exercises with different terrain complexities (low, medium, and high) while we recorded their electroencephalographic (EEG) activity. We focused on variations in the theta EEG power spectrum, a well-known index of mental workload. We also assessed performance and subjective ratings of task load. The theta EEG power spectrum in the frontal, temporal, and occipital areas were higher during the most complex scenarios. Performance (number of engine stops) and subjective data supported these findings. Our findings strengthen previous results found in civilians on the relationship between driver mental workload and the theta EEG power spectrum. This suggests that EEG activity can give relevant insight into mental workload variations in an objective, unbiased fashion, even during real training and/or operations. The continuous monitoring of the warfighter not only allows instantaneous detection of over/underload but also might provide online feedback to the system (either automated equipment or the crew) to take countermeasures and prevent fatal errors.

Effects of map design characteristics on users’ search performance and cognitive load

Purpose

This paper aims to investigate the influence of map design characteristics on users’ cognitive load and search performance. Two design conditions (symbolic vs non-symbolic) were used to evaluate users’ ability to locate a place of interest.

Design/methodology/approach

A total of 19 students (10 male and 9 female, 20-23 years old) participated in this study. The time required for subjects to find a place in the two conditions was used to estimate their searching performance. An electroencephalogram (EEG) device was used to examine students’ cognitive load using event-related desynchronization percentages of alpha, beta and theta brain wave rhythms.

Findings

The results showed that subjects needed more time to find a place in the non-symbolic condition than the symbolic condition. The EEG data, however, revealed that users experienced higher cognitive load when searching for a place in the symbolic condition. The authors found that the design characteristics of the map significantly influenced users’ brain activity, thus impacting their search performance.

Originality/value

Outcomes from this study can be used by cartographic designers and scholars to understand how certain design characteristics can trigger cognitive activity to improve users' searching experience and efficiency.

Sensor Networks for Aerospace Human-Machine Systems

Intelligent automation and trusted autonomy are being introduced in aerospace cyber-physical systems to support diverse tasks including data processing, decision-making, information sharing and mission execution. Due to the increasing level of integration/collaboration between humans and automation in these tasks, the operational performance of closed-loop human-machine systems can be enhanced when the machine monitors the operator's cognitive states and adapts to them in order to maximise the effectiveness of the Human-Machine Interfaces and Interactions (HMI²). Technological developments have led to neurophysiological observations becoming a reliable methodology to evaluate the human operator's states using a variety of wearable and remote sensors. The adoption of sensor networks can be seen as an evolution of this approach, as there are notable advantages if these sensors collect and exchange data in real-time, while their operation is controlled remotely and synchronised. This paper discusses recent advances in sensor networks for aerospace cyber-physical systems, focusing on Cognitive HMI² (CHMI²) implementations. The key neurophysiological measurements used in this context and their relationship with the operator's cognitive states are discussed. Suitable data analysis techniques based on machine learning and statistical inference are also presented, as these techniques allow processing both neurophysiological and operational data to obtain accurate cognitive state estimations. Lastly, to support the development of sensor networks for CHMI² applications, the paper addresses the performance characterisation of various state-of-the-art sensors and the propagation of measurement uncertainties through a machine learning-based inference engine. Results show that a proper sensor selection and integration can support the implementation of effective human-machine systems for various challenging aerospace applications, including Air Traffic Management (ATM), commercial airliner Single-Pilot Operations (SIPO), one-to-many Unmanned Aircraft Systems (UAS), and space operations management.

Different oscillatory entrainment of cortical networks during motor imagery and neurofeedback in right and left handers

Volitional modulation and neurofeedback of sensorimotor oscillatory activity is currently being evaluated as a strategy to facilitate motor restoration following stroke. Knowledge on the interplay between this regional brain self-regulation, distributed network entrainment and handedness is, however, limited. In a randomized cross-over design, twenty-one healthy subjects (twelve right-handers [RH], nine left-handers [LH]) performed kinesthetic motor imagery of left (48 trials) and right finger extension (48 trials). A brain-machine interface turned event-related desynchronization in the beta frequency-band (16-22 Hz) during motor imagery into passive hand opening by a robotic orthosis. Thereby, every participant subsequently activated either the dominant (DH) or non-dominant hemisphere (NDH) to control contralateral hand opening. The task-related cortical networks were studied with electroencephalography. The magnitude of the induced oscillatory modulation range in the sensorimotor cortex was independent of both handedness (RH, LH) and hemispheric specialization (DH, NDH). However, the regional beta-band modulation was associated with different alpha-band networks in RH and LH: RH presented a stronger inter-hemispheric connectivity, while LH revealed a stronger intra-hemispheric interaction. Notably, these distinct network entrainments were independent of hemispheric specialization. In healthy subjects, sensorimotor beta-band activity can be robustly modulated by motor imagery and proprioceptive feedback in both hemispheres independent of handedness. However, right and left handers show different oscillatory entrainment of cortical alpha-band networks during neurofeedback. This finding may inform neurofeedback interventions in future to align them more precisely with the underlying physiology.

Age-Related Differences in Pro-active Driving Behavior Revealed by EEG Measures

Healthy aging is associated with a decline in cognitive functions. This may become an issue when complex tasks have to be performed like driving a car in a demanding traffic situation. On the other hand, older people are able to compensate for age-related deficits, e.g., by deploying extra mental effort and other compensatory strategies. The present study investigated the interplay of age, task workload, and mental effort using EEG measures and a proactive driving task, in which 16 younger and 16 older participants had to keep a virtual car on track on a curvy road. Total oscillatory power and relative power in Theta and Alpha bands were analyzed, as well as event-related potentials (ERPs) to task-irrelevant regular and irregular sound stimuli. Steering variability and Theta power increased with increasing task load (i.e., with shaper bends of the road), while Alpha power decreased. This pattern of workload and mental effort was found in both age groups. However, only in the older group a relationship between steering variability and Theta power occurred: better steering performance was associated with higher Theta power, reflecting higher mental effort. Higher Theta power while driving was also associated with a stronger increase in reported subjective fatigue in the older group. In the younger group, lower steering variability came along with lower ERP responses to deviant sound stimuli, reflecting reduced processing of task-irrelevant environmental stimuli. In sum, better performance in proactive driving (i.e., more alert steering behavior) was associated with increased mental effort in the older group, and higher attentional focus on the task in the younger group, indicating age-specific strategies in the way younger and older drivers manage demanding (driving) tasks.

Mental State Assessment and Validation Using Personalized Physiological Biometrics

Mental state monitoring is a critical component of current and future human-machine interfaces, including semi-autonomous driving and flying, air traffic control, decision aids, training systems, and will soon be integrated into ubiquitous products like cell phones and laptops. Current mental state assessment approaches supply quantitative measures, but their only frame of reference is generic population-level ranges. What is needed are physiological biometrics that are validated in the context of task performance of individuals. Using curated intake experiments, we are able to generate personalized models of three key biometrics as useful indicators of mental state; namely, mental fatigue, stress, and attention. We demonstrate improvements to existing approaches through the introduction of new features. Furthermore, addressing the current limitations in assessing the efficacy of biometrics for individual subjects, we propose and employ a multi-level validation scheme for the biometric models by means of k-fold cross-validation for discrete classification and regression testing for continuous prediction. The paper not only provides a unified pipeline for extracting a comprehensive mental state evaluation from a parsimonious set of sensors (only EEG and ECG), but also demonstrates the use of validation techniques in the absence of empirical data. Furthermore, as an example of the application of these models to novel situations, we evaluate the significance of correlations of personalized biometrics to the dynamic fluctuations of accuracy and reaction time on an unrelated threat detection task using a permutation test. Our results provide a path toward integrating biometrics into augmented human-machine interfaces in a judicious way that can help to maximize task performance.

A comprehensive prediction and evaluation method of pilot workload

BACKGROUND:

The prediction and evaluation of pilot workload is a key problem in human factor airworthiness of cockpit.

OBJECTIVE:

A pilot traffic pattern task was designed in a flight simulation environment in order to carry out the pilot workload prediction and improve the evaluation method.

METHODS:

The prediction of typical flight subtasks and dynamic workloads (cruise, approach, and landing) were built up based on multiple resource theory, and a favorable validity was achieved by the correlation analysis verification between sensitive physiological data and the predicted value.

RESULTS:

Statistical analysis indicated that eye movement indices (fixation frequency, mean fixation time, saccade frequency, mean saccade time, and mean pupil diameter), Electrocardiogram indices (mean normal-to-normal interval and the ratio between low frequency and sum of low frequency and high frequency), and Electrodermal Activity indices (mean tonic and mean phasic) were all sensitive to typical workloads of subjects.

CONCLUSION:

A multinominal logistic regression model based on combination of physiological indices (fixation frequency, mean normal-to-normal interval, the ratio between low frequency and sum of low frequency and high frequency, and mean tonic) was constructed, and the discriminate accuracy was comparatively ideal with a rate of 84.85%.

Mental Fatigue Alters Cortical Activation and Psychological Responses, Impairing Performance in a Distance-Based Cycling Trial

Purpose: We sought to verify if alterations in prefrontal cortex (PFC) activation and psychological responses would play along with impairments in pacing and performance of mentally fatigued cyclists.

Materials and Methods: Eight recreational cyclists performed two preliminary sessions to familiarize them with the rapid visual information processing (RVP) test, psychological scales and 20 km cycling time trial (TT_20km) (session 1), as well as to perform a VO_2MAX test (session 2). Thereafter, they performed a TT_20km either after a RVP test (30 min) or a time-matched rest control session (session 3 and 4 in counterbalanced order). Performance and psychological responses were obtained throughout the TT_20km while PFC electroencephalography (EEG) was obtained at 10 and 20 km of the TT_20km and throughout the RVP test. Increases in EEG theta band power indicated a mental fatigue condition. Repeated-measures mixed models design and post-hoc effect size (ES) were used in comparisons.

Results: Cyclists completed the trial ~2.7% slower in mental fatigue (34.3 ± 1.3 min) than in control (33.4 ± 1.1 min, p = 0.02, very large ES), with a lower W_MEAN (224.5 ± 17.9 W vs. 240.2 ± 20.9 W, respectively; p = 0.03; extremely large ES). There was a higher EEG theta band power during RVP test (p = 0.03; extremely large ES), which remained during the TT_20km (p = 0.01; extremely large ES). RPE increased steeper in mental fatigue than in control, together with isolated reductions in motivation at 2th km (p = 0.04; extremely large ES), felt arousal at the 2nd and 4th km (p = 0.01; extremely large ES), and associative thoughts to exercise at the 6th and 16th km (p = 0.02; extremely large ES) of the TT_20km.

Conclusions: Mentally fatigued recreational cyclists showed impaired performance, altered PFC activation and faster increase in RPE during a TT_20km.

Change in hippocampal theta oscillation associated with multiple lever presses in a bimanual two-lever choice task for robot control in rats

Hippocampal theta oscillations have been implicated in working memory and attentional process, which might be useful for the brain-machine interface (BMI). To further elucidate the properties of the hippocampal theta oscillations that can be used in BMI, we investigated hippocampal theta oscillations during a two-lever choice task. During the task body-restrained rats were trained with a food reward to move an e-puck robot towards them by pressing the correct lever, ipsilateral to the robot several times, using the ipsilateral forelimb. The robot carried food and moved along a semicircle track set in front of the rat. We demonstrated that the power of hippocampal theta oscillations gradually increased during a 6-s preparatory period before the start of multiple lever pressing, irrespective of whether the correct lever choice or forelimb side were used. In addition, there was a significant difference in the theta power after the first choice, between correct and incorrect trials. During the correct trials the theta power was highest during the first lever-releasing period, whereas in the incorrect trials it occurred during the second correct lever-pressing period. We also analyzed the hippocampal theta oscillations at the termination of multiple lever pressing during the correct trials. Irrespective of whether the correct forelimb side was used, the power of hippocampal theta oscillations gradually decreased with the termination of multiple lever pressing. The frequency of theta oscillation also demonstrated an increase and decrease, before and after multiple lever pressing, respectively. There was a transient increase in frequency after the first lever press during the incorrect trials, while no such increase was observed during the correct trials. These results suggested that hippocampal theta oscillations reflect some aspects of preparatory and cognitive neural activities during the robot controlling task, which could be used for BMI.

Effect of Passive Hyperthermia on Working Memory Resources during Simple and Complex Cognitive Tasks

The aim of this study was to verify the hypothesis that hyperthermia represents a cognitive load limiting available resources for executing concurrent cognitive tasks. Electroencephalographic activity (EEG: alpha and theta power) was obtained in 10 hyperthermic participants in HOT (50°C, 50% RH) conditions and in a normothermic state in CON (25°C, 50% RH) conditions in counterbalanced order. In each trial, EEG was measured over the frontal lobe prior to task engagement (PRE) in each condition and during simple (One Touch Stockings of Cambridge, OTS-4) and complex (OTS-6) cognitive tasks. Core (39.5 ± 0.5 vs. 36.9 ± 0.2°C) and mean skin (39.06 ± 0.3 vs. 31.6 ± 0.6°C) temperatures were significantly higher in HOT than CON (p < 0.005). Theta power significantly increased with task demand (p = 0.017, η² = 0.36) and was significantly higher in HOT than CON (p = 0.041, η² = 0.39). The difference between HOT and CON was large (η² = 0.40) and significant (p = 0.036) PRE, large (η² = 0.20) but not significant (p = 0.17) during OTS-4, and disappeared during OTS-6 (p = 0.87, η² = 0.00). Those changes in theta power suggest that hyperthermia may act as an additional cognitive load. However, this load disappeared during OTS-6 together with an impaired performance, suggesting a potential saturation of the available resources.

Dual Frequency Head Maps: A New Method for Indexing Mental Workload Continuously during Execution of Cognitive Tasks

One goal of advanced information and communication technology is to simplify work. However, there is growing consensus regarding the negative consequences of inappropriate workload on employee's health and the safety of persons. In order to develop a method for continuous mental workload monitoring, we implemented a task battery consisting of cognitive tasks with diverse levels of complexity and difficulty. We conducted experiments and registered the electroencephalogram (EEG), performance data, and the NASA-TLX questionnaire from 54 people. Analysis of the EEG spectra demonstrates an increase of the frontal theta band power and a decrease of the parietal alpha band power, both under increasing task difficulty level. Based on these findings we implemented a new method for monitoring mental workload, the so-called Dual Frequency Head Maps (DFHM) that are classified by support vectors machines (SVMs) in three different workload levels. The results are in accordance with the expected difficulty levels arising from the requirements of the tasks on the executive functions. Furthermore, this article includes an empirical validation of the new method on a secondary subset with new subjects and one additional new task without any adjustment of the classifiers. Hence, the main advantage of the proposed method compared with the existing solutions is that it provides an automatic, continuous classification of the mental workload state without any need for retraining the classifier—neither for new subjects nor for new tasks. The continuous workload monitoring can help ensure good working conditions, maintain a good level of performance, and simultaneously preserve a good state of health.

Assessment of mental workload by EEG+FNIRS

We investigated the use of a multimodal functional neuroimaging system in quantifying mental workload of healthy human volunteers. We recorded behavioral performance measures as well as electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) simultaneously from subjects performing n-back tasks. The EEG and fNIRS signals were used in feature generation and classification offline using support vector machines. We examined the classification accuracy of three distinct systems: EEG based; fNIRS based; and Hybrid, which contained features from the first two systems as based on their interactions. The classification accuracy of the Hybrid system was observed to be greater than that of either system, indicating the synergistic role played by multimodal signals and by neurovascular coupling in quantifying mental workload.

Domain-general neural correlates of dependency formation: Using complex tones to simulate language

There is an ongoing debate whether the P600 event-related potential component following syntactic anomalies reflects syntactic processes per se, or if it is an instance of the P300, a domain-general ERP component associated with attention and cognitive reorientation. A direct comparison of both components is challenging because of the huge discrepancy in experimental designs and stimulus choice between language and 'classic' P300 experiments. In the present study, we develop a new approach to mimic the interplay of sequential position as well as categorical and relational information in natural language syntax (word category and agreement) in a non-linguistic target detection paradigm using musical instruments. Participants were instructed to (covertly) detect target tones which were defined by instrument change and pitch rise between subsequent tones at the last two positions of four-tone sequences. We analysed the EEG using event-related averaging and time-frequency decomposition. Our results show striking similarities to results obtained from linguistic experiments. We found a P300 that showed sensitivity to sequential position and a late positivity sensitive to stimulus type and position. A time-frequency decomposition revealed significant effects of sequential position on the theta band and a significant influence of stimulus type on the delta band. Our results suggest that the detection of non-linguistic targets defined via complex feature conjunctions in the present study and the detection of syntactic anomalies share the same underlying processes: attentional shift and memory based matching processes that act upon multi-feature conjunctions. We discuss the results as supporting domain-general accounts of the P600 during natural language comprehension.

Human Factors and Neurophysiological Metrics in Air Traffic Control: A Critical Review

This paper provides a focused and organized review of the research progress on neurophysiological indicators, also called "neurometrics," to show how they can effectively address some of the most important human factors (HFs) needs in the air traffic management (ATM) field. In order to better understand and highlight available opportunities of such neuroscientific applications, state of the art on the most involved HFs and related cognitive processes (e.g., mental workload and cognitive training) are presented together with examples of possible applications in current and future ATM scenarios. Furthermore, this paper will discuss the potential enhancements that further research and development activities could bring to the efficiency and safety of the ATM service.

Automated Task Load Detection with Electroencephalography: Towards Passive Brain–Computer Interfacing in Robotic Surgery

Automatic detection of the current task load of a surgeon in the theatre in real time could provide helpful information, to be used in supportive systems. For example, such information may enable the system to automatically support the surgeon when critical or stressful periods are detected, or to communicate to others when a surgeon is engaged in a complex maneuver and should not be disturbed. Passive brain–computer interfaces (BCI) infer changes in cognitive and affective state by monitoring and interpreting ongoing brain activity recorded via an electroencephalogram. The resulting information can then be used to automatically adapt a technological system to the human user. So far, passive BCI have mostly been investigated in laboratory settings, even though they are intended to be applied in real-world settings. In this study, a passive BCI was used to assess changes in task load of skilled surgeons performing both simple and complex surgical training tasks. Results indicate that the introduced methodology can reliably and continuously detect changes in task load in this realistic environment.

The Berlin Brain-Computer Interface: Progress Beyond Communication and Control

The combined effect of fundamental results about neurocognitive processes and advancements in decoding mental states from ongoing brain signals has brought forth a whole range of potential neurotechnological applications. In this article, we review our developments in this area and put them into perspective. These examples cover a wide range of maturity levels with respect to their applicability. While we assume we are still a long way away from integrating Brain-Computer Interface (BCI) technology in general interaction with computers, or from implementing neurotechnological measures in safety-critical workplaces, results have already now been obtained involving a BCI as research tool. In this article, we discuss the reasons why, in some of the prospective application domains, considerable effort is still required to make the systems ready to deal with the full complexity of the real world.

Toward a better understanding of task demands, workload, and performance during physician-computer interactions

OBJECTIVE

To assess the relationship between (1) task demands and workload, (2) task demands and performance, and (3) workload and performance, all during physician-computer interactions in a simulated environment.

METHODS

Two experiments were performed in 2 different electronic medical record (EMR) environments: WebCIS (n = 12) and Epic (n = 17). Each participant was instructed to complete a set of prespecified tasks on 3 routine clinical EMR-based scenarios: urinary tract infection (UTI), pneumonia (PN), and heart failure (HF). Task demands were quantified using behavioral responses (click and time analysis). At the end of each scenario, subjective workload was measured using the NASA-Task-Load Index (NASA-TLX). Physiological workload was measured using pupillary dilation and electroencephalography (EEG) data collected throughout the scenarios. Performance was quantified based on the maximum severity of omission errors.

RESULTS

Data analysis indicated that the PN and HF scenarios were significantly more demanding than the UTI scenario for participants using WebCIS (P < .01), and that the PN scenario was significantly more demanding than the UTI and HF scenarios for participants using Epic (P < .01). In both experiments, the regression analysis indicated a significant relationship only between task demands and performance (P < .01).

DISCUSSION

Results suggest that task demands as experienced by participants are related to participants' performance. Future work may support the notion that task demands could be used as a quality metric that is likely representative of performance, and perhaps patient outcomes.

CONCLUSION

The present study is a reasonable next step in a systematic assessment of how task demands and workload are related to performance in EMR-evolving environments.

Effective Indices for Monitoring Mental Workload While Performing Multiple Tasks

This study identified several physiological indices that can accurately monitor mental workload while participants performed multiple tasks with the strategy of maintaining stable performance and maximizing accuracy. Thirty male participants completed three 10-min. simulated multitasks: MATB (Multi-Attribute Task Battery) with three workload levels. Twenty-five commonly used mental workload measures were collected, including heart rate, 12 HRV (heart rate variability), 10 EEG (electroencephalography) indices (α, β, θ, α/θ, θ/β from O1-O2 and F4-C4), and two subjective measures. Analyses of index sensitivity showed that two EEG indices, θ and α/θ (F4-C4), one time-domain HRV-SDNN (standard deviation of inter-beat intervals), and four frequency-domain HRV: VLF (very low frequency), LF (low frequency), %HF (percentage of high frequency), and LF/HF were sensitive to differentiate high workload. EEG α/θ (F4-C4) and LF/HF were most effective for monitoring high mental workload. LF/HF showed the highest correlations with other physiological indices. EEG α/θ (F4-C4) showed strong correlations with subjective measures across diff erent mental workload levels. Operation strategy would affect the sensitivity of EEG α (F4-C4) and HF.