Sex-specific Neural Strategies During Fatiguing Work in Older Adults

BACKGROUND

Historical biases in ergonomics-related studies have been attributed to lack of participant diversity and sensitivity of measurements to capture variability between diverse groups. We posit that a neuroergonomics approach, that is, study of brain-behavior relationships during fatiguing work, allows for unique insights on sex differences in fatigue mechanisms that are not available via traditional "neck down" measurement approaches.

OBJECTIVE

This study examined the supraspinal mechanisms of exercise performance under fatigue and determined if there were any sex differences in these mechanisms.

METHODS

Fifty-nine older adults performed submaximal handgrip contractions until voluntary fatigue. Traditional ergonomics measures, namely, force variability, electromyography (EMG) of arm muscles, and strength and endurance times, and prefrontal and motor cortex hemodynamic responses were recorded.

RESULTS

There were no significant differences observed between older males and females in fatigability outcomes (i.e., endurance times, strength loss, and EMG activity) and brain activation. Effective connectivity from prefrontal to motor areas was significant for both sexes throughout the task, but during fatigue, males had higher interregional connectivity than females.

DISCUSSION

While traditional metrics of fatigue were comparable between the sexes, we observed distinct sex-specific neuromotor strategies (i.e., information flow between frontal-motor regions) that were adopted by older adults to maintain motor performance.

APPLICATION

The findings from this study offer insights into the capabilities and adaptation strategies of older men and women under fatiguing conditions. This knowledge can facilitate in the development of effective and targeted ergonomic strategies that accommodate for the varying physical capacities of diverse worker demographics.

Reproducible machine learning research in mental workload classification using EEG

This study addresses concerns about reproducibility in scientific research, focusing on the use of electroencephalography (EEG) and machine learning to estimate mental workload. We established guidelines for reproducible machine learning research using EEG and used these to assess the current state of reproducibility in mental workload modeling. We first started by summarizing the current state of reproducibility efforts in machine learning and in EEG. Next, we performed a systematic literature review on Scopus, Web of Science, ACM Digital Library, and Pubmed databases to find studies about reproducibility in mental workload prediction using EEG. All of this previous work was used to formulate guidelines, which we structured along the widely recognized Cross-Industry Standard Process for Data Mining (CRISP-DM) framework. By using these guidelines, researchers can ensure transparency and comprehensiveness of their methodologies, therewith enhancing collaboration and knowledge-sharing within the scientific community, and enhancing the reliability, usability and significance of EEG and machine learning techniques in general. A second systematic literature review extracted machine learning studies that used EEG to estimate mental workload. We evaluated the reproducibility status of these studies using our guidelines. We highlight areas studied and overlooked and identify current challenges for reproducibility. Our main findings include limitations on reporting performance on unseen test data, open sharing of data and code, and reporting of resources essential for training and inference processes.

Mental workload assessment by monitoring brain, heart, and eye with six biomedical modalities during six cognitive tasks

Introduction

The efficiency and safety of complex high precision human-machine systems such as in aerospace and robotic surgery are closely related to the cognitive readiness, ability to manage workload, and situational awareness of their operators. Accurate assessment of mental workload could help in preventing operator error and allow for pertinent intervention by predicting performance declines that can arise from either work overload or under stimulation. Neuroergonomic approaches based on measures of human body and brain activity collectively can provide sensitive and reliable assessment of human mental workload in complex training and work environments.

Methods

In this study, we developed a new six-cognitive-domain task protocol, coupling it with six biomedical monitoring modalities to concurrently capture performance and cognitive workload correlates across a longitudinal multi-day investigation. Utilizing two distinct modalities for each aspect of cardiac activity (ECG and PPG), ocular activity (EOG and eye-tracking), and brain activity (EEG and fNIRS), 23 participants engaged in four sessions over 4 weeks, performing tasks associated with working memory, vigilance, risk assessment, shifting attention, situation awareness, and inhibitory control.

Results

The results revealed varying levels of sensitivity to workload within each modality. While certain measures exhibited consistency across tasks, neuroimaging modalities, in particular, unveiled meaningful differences between task conditions and cognitive domains.

Discussion

This is the first comprehensive comparison of these six brain-body measures across multiple days and cognitive domains. The findings underscore the potential of wearable brain and body sensing methods for evaluating mental workload. Such comprehensive neuroergonomic assessment can inform development of next generation neuroadaptive interfaces and training approaches for more efficient human-machine interaction and operator skill acquisition.

Blink-Related Oscillations Provide Naturalistic Assessments of Brain Function and Cognitive Workload within Complex Real-World Multitasking Environments

BACKGROUND

There is a significant need to monitor human cognitive performance in complex environments, with one example being pilot performance. However, existing assessments largely focus on subjective experiences (e.g., questionnaires) and the evaluation of behavior (e.g., aircraft handling) as surrogates for cognition or utilize brainwave measures which require artificial setups (e.g., simultaneous auditory stimuli) that intrude on the primary tasks. Blink-related oscillations (BROs) are a recently discovered neural phenomenon associated with spontaneous blinking that can be captured without artificial setups and are also modulated by cognitive loading and the external sensory environment-making them ideal for brain function assessment within complex operational settings.

METHODS

Electroencephalography (EEG) data were recorded from eight adult participants (five F, M = 21.1 years) while they completed the Multi-Attribute Task Battery under three different cognitive loading conditions. BRO responses in time and frequency domains were derived from the EEG data, and comparisons of BRO responses across cognitive loading conditions were undertaken. Simultaneously, assessments of blink behavior were also undertaken.

RESULTS

Blink behavior assessments revealed decreasing blink rate with increasing cognitive load (p < 0.001). Prototypical BRO responses were successfully captured in all participants (p < 0.001). BRO responses reflected differences in task-induced cognitive loading in both time and frequency domains (p < 0.05). Additionally, reduced pre-blink theta band desynchronization with increasing cognitive load was also observed (p < 0.05).

CONCLUSION

This study confirms the ability of BRO responses to capture cognitive loading effects as well as preparatory pre-blink cognitive processes in anticipation of the upcoming blink during a complex multitasking situation. These successful results suggest that blink-related neural processing could be a potential avenue for cognitive state evaluation in operational settings-both specialized environments such as cockpits, space exploration, military units, etc. and everyday situations such as driving, athletics, human-machine interactions, etc.-where human cognition needs to be seamlessly monitored and optimized.

Cognitive Effort during Visuospatial Problem Solving in Physical Real World, on Computer Screen, and in Virtual Reality

Spatial cognition plays a crucial role in academic achievement, particularly in science, technology, engineering, and mathematics (STEM) domains. Immersive virtual environments (VRs) have the growing potential to reduce cognitive load and improve spatial reasoning. However, traditional methods struggle to assess the mental effort required for visuospatial processes due to the difficulty in verbalizing actions and other limitations in self-reported evaluations. In this neuroergonomics study, we aimed to capture the neural activity associated with cognitive workload during visuospatial tasks and evaluate the impact of the visualization medium on visuospatial task performance. We utilized functional near-infrared spectroscopy (fNIRS) wearable neuroimaging to assess cognitive effort during spatial-reasoning-based problem-solving and compared a VR, a computer screen, and a physical real-world task presentation. Our results reveal a higher neural efficiency in the prefrontal cortex (PFC) during 3D geometry puzzles in VR settings compared to the settings in the physical world and on the computer screen. VR appears to reduce the visuospatial task load by facilitating spatial visualization and providing visual cues. This makes it a valuable tool for spatial cognition training, especially for beginners. Additionally, our multimodal approach allows for progressively increasing task complexity, maintaining a challenge throughout training. This study underscores the potential of VR in developing spatial skills and highlights the value of comparing brain data and human interaction across different training settings.

A Window Into the Tired Brain: Neurophysiological Dynamics of Visuospatial Working Memory Under Fatigue

OBJECTIVE

We examine the spatiotemporal dynamics of neural activity and its correlates in heart rate and its variability (HR/HRV) during a fatiguing visuospatial working memory task.

BACKGROUND

The neural and physiological drivers of fatigue are complex, coupled, and poorly understood. Investigations that combine the fidelity of neural indices and the field-readiness of physiological measures can facilitate measurements of fatigue states in operational settings.

METHOD

Sixteen healthy adults, balanced by sex, completed a 60-minute fatiguing visuospatial working memory task. Changes in task performance, subjective measures of effort and fatigue, cerebral hemodynamics, and HR/HRV were analyzed. Peak brain activation, functional and effective connections within relevant brain networks were contrasted against spectral and temporal features of HR/HRV.

RESULTS

Task performance elicited increased neural activation in regions responsible for maintaining working memory capacity. With the onset of time-on-task effects, resource utilization was seen to increase beyond task-relevant networks. Over time, functional connections in the prefrontal cortex were seen to weaken, with changes in the causal relationships between key regions known to drive working memory. HR/HRV indices were seen to closely follow activity in the prefrontal cortex.

CONCLUSION

This investigation provided a window into the neurophysiological underpinnings of working memory under the time-on-task effect. HR/HRV was largely shown to mirror changes in cortical networks responsible for working memory, therefore supporting the possibility of unobtrusive state recognition under ecologically valid conditions.

APPLICATIONS

Findings here can inform the development of a fieldable index for cognitive fatigue.

Towards ubiquitous and nonintrusive measurements of brain function in the real world: assessing blink-related oscillations during simulated flight using portable low-cost EEG

Blink-related oscillations (BRO) are newly discovered neurophysiological phenomena associated with spontaneous blinking and represent cascading neural mechanisms including visual sensory, episodic memory, and information processing responses. These phenomena have been shown to be present at rest and during tasks and are modulated by cognitive load, creating the possibility for brain function assessments that can be integrated seamlessly into real-world settings. Prior works have largely examined the BRO phenomenon within controlled laboratory environments using magnetoencephalography and high-density electroencephalography (EEG) that are ill-suited for real-world deployment. Investigating BROs using low-density EEG within complex environments reflective of the real-world would further our understanding of how BRO responses can be utilized in real-world settings. We evaluated whether the BRO response could be captured in a high-fidelity flight simulation environment using a portable, low-density wireless EEG system. The effects of age and task demands on BRO responses were also examined. EEG data from 30 licensed pilots (age 43.37 +/- 17.86, 2 females) were collected during simulated flights at two cognitive workload levels. Comparisons of signal amplitudes were undertaken to confirm the presence of BRO responses and mixed model ANOVAs quantified the effects of workload and age group on BRO amplitudes. Significant increases in neural activity were observed post-blink compared to the baseline period (p < 0.05), confirming the presence of BRO responses. In line with prior studies, results showed BRO time-domain responses from the delta band (0.5-4 Hz) consisting of an early negative peak followed by a positive peak post-blink in temporal and parietal electrodes. Additionally, task workload and age-related effects were also found, with observations of the enhancement of BRO amplitudes with older age and attenuation of BRO responses in high workloads (p < 0.05). These findings demonstrate that it is possible to capture BRO responses within simulated flight environments using portable, low-cost, easy-to-use EEG systems. Furthermore, biological and task salience were reflected in these BRO responses. The successful detection and demonstration of both task-and age-related modulation of BRO responses in this study open the possibility of assessing human brain function across the lifespan with BRO responses in complex and realistic environments.

Effect of temperature reduction of the prefrontal area on accuracy of visual sustained attention

Objectives. Detection of sensitive signs in many work environments with automated systems (aviation industry, flight safety tower, maritime industry, monitoring in the military industry, etc.) is essential and requires constant visual attention. Therefore, the aim of this study was to investigate the effect of forehead cooling on the accuracy of stable visual attention. Methods. This interventional study was performed on 34 male students. The sampling method was a randomized block design. Subjects were assessed by demographic questionnaire, Snellen chart, Spielberger state-trait anxiety inventory (STAI) and physiological and cognitive measurements. Results. Prefrontal cortex (PFC) cooling caused significant changes in sublingual temperature during four measurements in the intervention group. There were no significant changes in heart rate, diastolic blood pressure and saturation of peripheral oxygen (%SpO2) between the two groups. The critical flicker frequency (CFF) as an indicator of cognitive fatigue showed that cognitive improvement after PFC cooling occurred following a reduction in cognitive fatigue. Conclusions. Considering the importance of choosing non-invasive methods to improve the operator's cognitive skills while performing cognitive tasks in the field of neuroergonomics, it can be concluded that PFC cooling is an effective and safe way to improve some cognitive skills such as visual attention.

Modeling Brain Dynamics During Virtual Reality-Based Emergency Response Learning Under Stress

BACKGROUND

Stress affects learning during training, and virtual reality (VR) based training systems that manipulate stress can improve retention and retrieval performance for firefighters. Brain imaging using functional Near Infrared Spectroscopy (fNIRS) can facilitate development of VR-based adaptive training systems that can continuously assess the trainee's states of learning and cognition.

OBJECTIVE

The aim of this study was to model the neural dynamics associated with learning and retrieval under stress in a VR-based emergency response training exercise.

METHODS

Forty firefighters underwent an emergency shutdown training in VR and were randomly assigned to either a control or a stress group. The stress group experienced stressors including smoke, fire, and explosions during the familiarization and training phase. Both groups underwent a stress memory retrieval and no-stress memory retrieval condition. Participant's performance scores, fNIRS-based neural activity, and functional connectivity between the prefrontal cortex (PFC) and motor regions were obtained for the training and retrieval phases.

RESULTS

The performance scores indicate that the rate of learning was slower in the stress group compared to the control group, but both groups performed similarly during each retrieval condition. Compared to the control group, the stress group exhibited suppressed PFC activation. However, they showed stronger connectivity within the PFC regions during the training and between PFC and motor regions during the retrieval phases.

DISCUSSION

While stress impaired performance during training, adoption of stress-adaptive neural strategies (i.e., stronger brain connectivity) were associated with comparable performance between the stress and the control groups during the retrieval phase.

Hands off, brain off? A meta-analysis of neuroimaging data during active and passive driving

BACKGROUND

Car driving is more and more automated, to such an extent that driving without active steering control is becoming a reality. Although active driving requires the use of visual information to guide actions (i.e., steering the vehicle), passive driving only requires looking at the driving scene without any need to act (i.e., the human is passively driven).

MATERIALS & METHODS

After a careful search of the scientific literature, 11 different studies, providing 17 contrasts, were used to run a comprehensive meta-analysis contrasting active driving with passive driving.

RESULTS

Two brain regions were recruited more consistently for active driving compared to passive driving, the left precentral gyrus (BA3 and BA4) and the left postcentral gyrus (BA4 and BA3/40), whereas a set of brain regions was recruited more consistently in passive driving compared to active driving: the left middle frontal gyrus (BA6), the right anterior lobe and the left posterior lobe of the cerebellum, the right sub-lobar thalamus, the right anterior prefrontal cortex (BA10), the right inferior occipital gyrus (BA17/18/19), the right inferior temporal gyrus (BA37), and the left cuneus (BA17).

DISCUSSION

From a theoretical perspective, these findings support the idea that the output requirement of the visual scanning process engaged for the same activity can trigger different cerebral pathways, associated with different cognitive processes. A dorsal stream dominance was found during active driving, whereas a ventral stream dominance was obtained during passive driving. From a practical perspective, and contrary to the dominant position in the Human Factors community, our findings support the idea that a transition from passive to active driving would remain challenging as passive and active driving engage distinct neural networks.

A Neuroergonomic Approach Fostered by Wearable EEG for the Multimodal Assessment of Drivers Trainees

When assessing trainees' progresses during a driving training program, instructors can only rely on the evaluation of a trainee's explicit behavior and their performance, without having any insight about the training effects at a cognitive level. However, being able to drive does not imply knowing how to drive safely in a complex scenario such as the road traffic. Indeed, the latter point involves mental aspects, such as the ability to manage and allocate one's mental effort appropriately, which are difficult to assess objectively. In this scenario, this study investigates the validity of deploying an electroencephalographic neurometric of mental effort, obtained through a wearable electroencephalographic device, to improve the assessment of the trainee. The study engaged 22 young people, without or with limited driving experience. They were asked to drive along five different but similar urban routes, while their brain activity was recorded through electroencephalography. Moreover, driving performance, subjective and reaction times measures were collected for a multimodal analysis. In terms of subjective and performance measures, no driving improvement could be detected either through the driver's subjective measures or through their driving performance. On the other side, through the electroencephalographic neurometric of mental effort, it was possible to catch their improvement in terms of mental performance, with a decrease in experienced mental demand after three repetitions of the driving training tasks. These results were confirmed by the analysis of reaction times, that significantly improved from the third repetition as well. Therefore, being able to measure when a task is less mentally demanding, and so more automatic, allows to deduce the degree of users training, becoming capable of handling additional tasks and reacting to unexpected events.

Decoding Mental Effort in a Quasi-Realistic Scenario: A Feasibility Study on Multimodal Data Fusion and Classification

Humans' performance varies due to the mental resources that are available to successfully pursue a task. To monitor users' current cognitive resources in naturalistic scenarios, it is essential to not only measure demands induced by the task itself but also consider situational and environmental influences. We conducted a multimodal study with 18 participants (nine female, M = 25.9 with SD = 3.8 years). In this study, we recorded respiratory, ocular, cardiac, and brain activity using functional near-infrared spectroscopy (fNIRS) while participants performed an adapted version of the warship commander task with concurrent emotional speech distraction. We tested the feasibility of decoding the experienced mental effort with a multimodal machine learning architecture. The architecture comprised feature engineering, model optimisation, and model selection to combine multimodal measurements in a cross-subject classification. Our approach reduces possible overfitting and reliably distinguishes two different levels of mental effort. These findings contribute to the prediction of different states of mental effort and pave the way toward generalised state monitoring across individuals in realistic applications.

Simultaneous fMRI and tDCS for Enhancing Training of Flight Tasks

There is a gap in our understanding of how best to apply transcranial direct-current stimulation (tDCS) to enhance learning in complex, realistic, and multifocus tasks such as aviation. Our goal is to assess the effects of tDCS and feedback training on task performance, brain activity, and connectivity using functional magnetic resonance imaging (fMRI). Experienced glider pilots were recruited to perform a one-day, three-run flight-simulator task involving varying difficulty conditions and a secondary auditory task, mimicking real flight requirements. The stimulation group (versus sham) received 1.5 mA high-definition HD-tDCS to the right dorsolateral prefrontal cortex (DLPFC) for 30 min during the training. Whole-brain fMRI was collected before, during, and after stimulation. Active stimulation improved piloting performance both during and post-training, particularly in novice pilots. The fMRI revealed a number of tDCS-induced effects on brain activation, including an increase in the left cerebellum and bilateral basal ganglia for the most difficult conditions, an increase in DLPFC activation and connectivity to the cerebellum during stimulation, and an inhibition in the secondary task-related auditory cortex and Broca's area. Here, we show that stimulation increases activity and connectivity in flight-related brain areas, particularly in novices, and increases the brain's ability to focus on flying and ignore distractors. These findings can guide applied neurostimulation in real pilot training to enhance skill acquisition and can be applied widely in other complex perceptual-motor real-world tasks.

The role of brain-localized gamma and alpha oscillations in inattentional deafness: implications for understanding human attention

Introduction

The processes involved in how the attention system selectively focuses on perceptual and motor aspects related to a specific task, while suppressing features of other tasks and/or objects in the environment, are of considerable interest for cognitive neuroscience. The goal of this experiment was to investigate neural processes involved in selective attention and performance under multi-task situations. Several studies have suggested that attention-related gamma-band activity facilitates processing in task-specific modalities, while alpha-band activity inhibits processing in non-task-related modalities. However, investigations into the phenomenon of inattentional deafness/blindness (inability to observe stimuli in non-dominant task when primary task is demanding) have yet to observe gamma-band activity.

Methods

This EEG experiment utilizes an engaging whole-body perceptual motor task while carrying out a secondary auditory detection task to investigate neural correlates of inattentional deafness in natural immersive high workload conditions. Differences between hits and misses on the auditory detection task in the gamma (30-50 Hz) and alpha frequency (8-12 Hz) range were carried out at the cortical source level using LORETA.

Results

Participant auditory task performance correlated with an increase in gamma-band activity for hits over misses pre- and post-stimulus in left auditory processing regions. Alpha-band activity was greater for misses relative to hits in right auditory processing regions pre- and post-stimulus onset. These results are consistent with the facilitatory/inhibitory role of gamma/alpha-band activity for neural processing. Additional gamma- and alpha-band activity was found in frontal and parietal brain regions which are thought to reflect various attentional monitoring, selection, and switching processes.

Discussion

The results of this study help to elucidate the role of gamma and alpha frequency bands in frontal and modality-specific regions involved with selective attention in multi-task immersive situations.

Using EEG signals to assess workload during memory retrieval in a real-world scenario

OBJECTIVE

The Electroencephalogram (EEG) is gaining popularity as a physiological measure for neuroergonomics in human factor studies because it is objective, less prone to bias, and capable of assessing the dynamics of cognitive states. This study investigated the associations between memory workload and EEG during participants' typical office tasks on a single-monitor and dual-monitor arrangement. We expect a higher memory workload for the single-monitor arrangement.&#xD;&#xD;Approach: We designed an experiment that mimics the scenario of a subject performing some office work and examined whether the subjects experienced various levels of memory workload in two different office setups: 1) a single-monitor setup and 2) a dual-monitor&#xD;setup. We used EEG band power, mutual information, and coherence as features to train machine learning models to classify high versus low memory workload states.&#xD;&#xD;Main results: The study results showed that these characteristics exhibited significant differences that were consistent across all participants. We also verified the robustness and consistency of these EEG signatures in a different data set collected during a Sternberg task in a prior study.&#xD;&#xD;Significance: The study found the EEG correlates of memory workload across individuals, demonstrating the effectiveness of using EEG analysis in conducting real-world neuroergonomic studies.

Performance, Hemodynamics, and Stress in a Two-Day Vigilance Task: Practical and Theoretical Implications

OBJECTIVE

To explore vigilance task performance, cerebral blood flow velocity (CBFV), workload, and stress in a within-subjects, two-session experiment.

BACKGROUND

Vigilance, or sustained attention, tasks are often characterized by a decline in operator performance and CBFV with time on task, and high workload and stress. Though performance is known to improve with practice, past research has not included measures of CBFV, stress, and workload in a within-subjects multi-session design, which may also provide insight into ongoing theoretical debate.

METHOD

Participants performed a vigilance task on two separate occasions. Performance, CBFV, workload, and self-reported stress were measured.

RESULTS

Within each session, results were consistent with the vigilance profile found in prior research. Across sessions, performance improved but the time on task decrement remained. Mean CBFV and workload ratings did not differ between sessions, but participants reported significantly less distress, worry, and engagement after session two compared to one.

CONCLUSION

Though practice may not disrupt the standard vigilance profile, it may serve to improve overall performance and reduce stress. However, repeated exposure may have negative implications for engagement and mind-wandering.

APPLICATION

It is important to better understand the relationship between experience, performance, physiological response, and self-reported stress and workload in vigilance because real-world environments often require operators to do the same task over many occasions. While performance improvement and reduced distress is an encouraging result, the decline in engagement requires further research. Results across sessions fail to provide support to the mind-wandering theory of vigilance.

Neuroergonomics on the Go: An Evaluation of the Potential of Mobile EEG for Workplace Assessment and Design

OBJECTIVE

We demonstrate and discuss the use of mobile electroencephalogram (EEG) for neuroergonomics. Both technical state of the art as well as measures and cognitive concepts are systematically addressed.

BACKGROUND

Modern work is increasingly characterized by information processing. Therefore, the examination of mental states, mental load, or cognitive processing during work is becoming increasingly important for ergonomics.

RESULTS

Mobile EEG allows to measure mental states and processes under real live conditions. It can be used for various research questions in cognitive neuroergonomics. Besides measures in the frequency domain that have a long tradition in the investigation of mental fatigue, task load, and task engagement, new approaches-like blink-evoked potentials-render event-related analyses of the EEG possible also during unrestricted behavior.

CONCLUSION

Mobile EEG has become a valuable tool for evaluating mental states and mental processes on a highly objective level during work. The main advantage of this technique is that working environments don't have to be changed while systematically measuring brain functions at work. Moreover, the workflow is unaffected by such neuroergonomic approaches.

Neurophysiological and emotional influences on team communication and metacognitive cyber situational awareness during a cyber engineering exercise

Background: Cyber operations unfold at superhuman speeds where cyber defense decisions are based on human-to-human communication aiming to achieve a shared cyber situational awareness. The recently proposed Orient, Locate, Bridge (OLB) model suggests a three-phase metacognitive approach for successful communication of cyber situational awareness for good cyber defense decision-making. Successful OLB execution implies applying cognitive control to coordinate self-referential and externally directed cognitive processes. In the brain, this is dependent on the frontoparietal control network and its connectivity to the default mode network. Emotional reactions may increase default mode network activity and reduce attention allocation to analytical processes resulting in sub-optimal decision-making. Vagal tone is an indicator of activity in the dorsolateral prefrontal node of the frontoparietal control network and is associated with functional connectivity between the frontoparietal control network and the default mode network. Aim: The aim of the present study was to assess whether indicators of neural activity relevant to the processes outlined by the OLB model were related to outcomes hypothesized by the model. Methods: Cyber cadets (N = 36) enrolled in a 3-day cyber engineering exercise organized by the Norwegian Defense Cyber Academy participated in the study. Differences in prospective metacognitive judgments of cyber situational awareness, communication demands, and mood were compared between cyber cadets with high and low vagal tone. Vagal tone was measured at rest prior to the exercise. Affective states, communication demands, cyber situational awareness, and metacognitive accuracy were measured on each day of the exercise. Results: We found that cyber cadets with higher vagal tone had better metacognitive judgments of cyber situational awareness, imposed fewer communication demands on their teams, and had more neutral moods compared to cyber cadets with lower vagal tone. Conclusion: These findings provide neuroergonomic support for the OLB model and suggest that it may be useful in education and training. Future studies should assess the effect of OLB-ing as an intervention on communication and performance.

Cortical theta-gamma coupling tracks the mental workload as an indicator of mental schema development during simulated quadrotor UAV operation

Objective. In the emerging field of neuroergonomics, mental workload assessment is one of the most important problems. Previous studies have made some progress on the relationship between task difficulties and mental workload, but how the mental schema, a reflection of the understanding and mastery degree of a task, affects mental workload has not been clearly discussed.Approach. There is emerging appreciation for the role of theta-gamma coupling (TGC) in high-level cognitive functions. Here, we attempt to further our understanding of how mental schema development and task difficulty had an impact on mental workload from the perspective of TGC. Specifically, the variation of TGC coupling strength and coupling pattern was estimated with different test orders and task difficulties performed by 51 students in a ten-day simulated quadrotor unmanned aerial vehicle flight training and test tasks.Main results. During the training, TGC increased with mental schema development. For the test tasks, TGC did not change with increasing task difficulty before the operator formed a mental schema but decreased with the increasing mental workload after the formation of the mental schema.Significance. Our results suggest that TGC was a robust indicator of mental schema development and could be biased by task difficulty. In conclusion, TGC can be a promising measure of mental workload, but only for experienced operators.

Evaluating Effective Connectivity of Trust in Human-Automation Interaction: A Dynamic Causal Modeling (DCM) Study

OBJECTIVE

Using dynamic causal modeling (DCM), we examined how credibility and reliability affected the way brain regions exert causal influence over each other-effective connectivity (EC)-in the context of trust in automation.

BACKGROUND

Multiple brain regions of the central executive network (CEN) and default mode network (DMN) have been implicated in trust judgment. However, the neural correlates of trust judgment are still relatively unexplored in terms of the directed information flow between brain regions.

METHOD

Sixteen participants observed the performance of four computer algorithms, which differed in credibility and reliability, of the system monitoring subtask of the Air Force Multi-Attribute Task Battery (AF-MATB). Using six brain regions of the CEN and DMN commonly identified to be activated in human trust, a total of 30 (forward, backward, and lateral) connection models were developed. Bayesian model averaging (BMA) was used to quantify the connectivity strength among the brain regions.

RESULTS

Relative to the high trust condition, low trust showed unique presence of specific connections, greater connectivity strengths from the prefrontal cortex, and greater network complexity. High trust condition showed no backward connections.

CONCLUSION

Results indicated that trust and distrust can be two distinctive neural processes in human-automation interaction-distrust being a more complex network than trust, possibly due to the increased cognitive load.

APPLICATION

The causal architecture of distributed brain regions inferred using DCM can help not only in the design of a balanced human-automation interface design but also in the proper use of automation in real-life situations.

Driver's turning intent recognition model based on brain activation and contextual information

Traffic situations like turning at intersections are destined for safety-critical situations and accidents. Human errors are one of the main reasons for accidents in these situations. A model that recognizes the driver's turning intent could help to reduce accidents by warning the driver or stopping the vehicle before a dangerous turning maneuver. Most models that aim at predicting the probability of a driver's turning intent use only contextual information, such as gap size or waiting time. The objective of this study is to investigate whether the combination of context information and brain activation measurements enhances the recognition of turning intent. We conducted a driving simulator study while simultaneously measuring brain activation using high-density fNIRS. A neural network model for turning intent recognition was trained on the fNIRS and contextual data. The input variables were analyzed using SHAP (SHapley Additive exPlanations) feature importance analysis to show the positive effect of the inclusion of brain activation data. Both the model's evaluation and the feature importance analysis suggest that the combination of context information and brain activation leads to an improved turning intent recognition. The fNIRS results showed increased brain activation differences during the "turn" decision-making phase before turning execution in parts of the left motor cortices, such as the primary motor cortex (PMC; putative BA 4), premotor area (PMA; putative BA 6), and supplementary motor area (SMA; putative BA 8). Furthermore, we also observed increased activation differences in the left prefrontal areas, potentially in the left middle frontal gyrus (putative BA 9), which has been associated with the control of executive functions, such as decision-making and action planning. We hypothesize that brain activation measurements could be a more direct indicator with potentially high specificity for the turning behavior and thus help to increase the recognition model's performance.

Current Evidence and Prospects for Future Research

Neuroergonomics focuses on the brain signatures and associated mental states underlying behavior to design human-machine interfaces enhancing performance in the cognitive and physical domains. Brain imaging techniques such as functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) have been considered key methods for achieving this goal. Recent research stresses the value of combining EEG and fNIRS in improving these interface systems' mental state decoding abilities, but little is known about whether these improvements generalize over different paradigms and methodologies, nor about the potentialities for using these systems in the real world. We review 33 studies comparing mental state decoding accuracy between bimodal EEG-fNIRS and unimodal EEG and fNIRS in several subdomains of neuroergonomics. In light of these studies, we also consider the challenges of exploiting wearable versions of these systems in real-world contexts. Overall the studies reviewed suggest that bimodal EEG-fNIRS outperforms unimodal EEG or fNIRS despite major differences in their conceptual and methodological aspects. Much work however remains to be done to reach practical applications of bimodal EEG-fNIRS in naturalistic conditions. We consider these points to identify aspects of bimodal EEG-fNIRS research in which progress is expected or desired.

EEG-Based Index for Timely Detecting User's Drowsiness Occurrence in Automotive Applications

Human errors are widely considered among the major causes of road accidents. Furthermore, it is estimated that more than 90% of vehicle crashes causing fatal and permanent injuries are directly related to mental tiredness, fatigue, and drowsiness of the drivers. In particular, driving drowsiness is recognized as a crucial aspect in the context of road safety, since drowsy drivers can suddenly lose control of the car. Moreover, the driving drowsiness episodes mostly appear suddenly without any prior behavioral evidence. The present study aimed at characterizing the onset of drowsiness in car drivers by means of a multimodal neurophysiological approach to develop a synthetic electroencephalographic (EEG)-based index, able to detect drowsy events. The study involved 19 participants in a simulated scenario structured in a sequence of driving tasks under different situations and traffic conditions. The experimental conditions were designed to induce prominent mental drowsiness in the final part. The EEG-based index, so-called “MDrow index”, was developed and validated to detect the driving drowsiness of the participants. The MDrow index was derived from the Global Field Power calculated in the Alpha EEG frequency band over the parietal brain sites. The results demonstrated the reliability of the proposed MDrow index in detecting the driving drowsiness experienced by the participants, resulting also more sensitive and timely sensible with respect to more conventional autonomic parameters, such as the EyeBlinks Rate and the Heart Rate Variability, and to subjective measurements (self-reports).

Mental Workload in Neuropsychology: An Example With the NASA-TLX in Adults With HIV

A preliminary set of analyses are presented, where workload was examined in 32 adults infected with the human immunodeficiency virus (HIV). Like the current COVID-19 pandemic (caused by the SARS-CoV-2 virus), HIV can produce a wide variety of symptoms, including various levels of cognitive dysfunction. In fact, a recent meta-analysis estimates that of the 39 million adults infected globally with HIV, 42.6% exhibit some form of HIV-associated neurocognitive disorder. A common cognitive symptom in HIV is decline in attention and executive functioning. Though typically examined by clinicians with less precise traditional paper-and-pencil neuropsychological tests, we examined this aspect of cognitive functioning using a more psychometrically sophisticated task as we had HIV-positive adults perform a computerized tracking task in single, dual, and tri-task conditions via the Multi-Attribute Task (MAT) Battery. Also assessed was mental workload, with the NASA-Task Load Index (NASA-TLX), rarely used in neuropsychology but a standard tool in human factors and neuroergonomics research. As expected, tracking performance declined with task condition difficulty (p < 0.001). Although no direct statistical comparisons were made, MAT performance here appeared worse than the MAT performance of various other groups reported in the research literature and in our laboratory. Ratings of workload also tended to increase as a function of task condition difficulty (p < 0.001). Plotting MAT tracking performance against the Mental Demand subscale scores, large individual differences in this aspect of workload were evident in both optimal and sub-optimal tracking performance. To examine likely variables with a potential impact on Mental Demand, a variety of variables (nadir CD4 count, viral load, depression symptoms, diagnosis of AIDS, presence of opportunistic infection, general cognitive status, etc.) were examined in relation to the Mental Demand scale, with age showing a significant association (r = 0.41, p = 0.022) and a diagnosis of AIDS showing trend associations (ps ≥ 0.066). Findings suggesting a deficit in metacognition or insight are also discussed. It is argued that assessment of workload (and its various aspects or components) can provide valuable additional information in neuropsychology.

Neuroadaptive Training via fNIRS in Flight Simulators

Training to master a new skill often takes a lot of time, effort, and financial resources, particularly when the desired skill is complex, time sensitive, or high pressure where lives may be at risk. Professions such as aircraft pilots, surgeons, and other mission-critical operators that fall under this umbrella require extensive domain-specific dedicated training to enable learners to meet real-world demands. In this study, we describe a novel neuroadaptive training protocol to enhance learning speed and efficiency using a neuroimaging-based cognitive workload measurement system in a flight simulator. We used functional near-infrared spectroscopy (fNIRS), which is a wearable, mobile, non-invasive neuroimaging modality that can capture localized hemodynamic response and has been used extensively to monitor the anterior prefrontal cortex to estimate cognitive workload. The training protocol included four sessions over 2 weeks and utilized realistic piloting tasks with up to nine levels of difficulty. Learners started at the lowest level and their progress adapted based on either behavioral performance and fNIRS measures combined (neuroadaptive) or performance measures alone (control). Participants in the neuroadaptive group were found to have significantly more efficient training, reaching higher levels of difficulty or significantly improved performance depending on the task, and showing consistent patterns of hemodynamic-derived workload in the dorsolateral prefrontal cortex. The results of this study suggest that a neuroadaptive personalized training protocol using non-invasive neuroimaging is able to enhance learning of new tasks. Finally, we outline here potential avenues for further optimization of this fNIRS based neuroadaptive training approach. As fNIRS mobile neuroimaging is becoming more practical and accessible, the approaches developed here can be applied in the real world in scale.

Benchmarking cEEGrid and Solid Gel-Based Electrodes to Classify Inattentional Deafness in a Flight Simulator

Transfer from experiments in the laboratory to real-life tasks is challenging due notably to the inability to reproduce the complexity of multitasking dynamic everyday life situations in a standardized lab condition and to the bulkiness and invasiveness of recording systems preventing participants from moving freely and disturbing the environment. In this study, we used a motion flight simulator to induce inattentional deafness to auditory alarms, a cognitive difficulty arising in complex environments. In addition, we assessed the possibility of two low-density EEG systems a solid gel-based electrode Enobio (Neuroelectrics, Barcelona, Spain) and a gel-based cEEGrid (TMSi, Oldenzaal, Netherlands) to record and classify brain activity associated with inattentional deafness (misses vs. hits to odd sounds) with a small pool of expert participants. In addition to inducing inattentional deafness (missing auditory alarms) at much higher rates than with usual lab tasks (34.7% compared to the usual 5%), we observed typical inattentional deafness-related activity in the time domain but also in the frequency and time-frequency domains with both systems. Finally, a classifier based on Riemannian Geometry principles allowed us to obtain more than 70% of single-trial classification accuracy for both mobile EEG, and up to 71.5% for the cEEGrid (TMSi, Oldenzaal, Netherlands). These results open promising avenues toward detecting cognitive failures in real-life situations, such as real flight.

Effects of Workstation Type on Mental Stress: FNIRS Study

OBJECTIVE

The purpose of this study is to examine the effect of the workstation type on the severity of mental stress by means of measuring prefrontal cortex (PFC) activation using functional near-infrared spectroscopy.

BACKGROUND

Workstation type is known to influence worker's health and performance. Despite the practical implications of ergonomic workstations, limited information is available regarding their impact on brain activity and executive functions.

METHOD

Ten healthy participants performed a Montreal imaging stress task (MIST) in ergonomic and nonergonomic workstations to investigate their effects on the severity of the induced mental stress.

RESULTS

Cortical hemodynamic changes in the PFC were observed during the MIST in both the ergonomic and nonergonomic workstations. However, the ergonomic workstation exhibited improved MIST performance, which was positively correlated with the cortical activation on the right ventrolateral and the left dorsolateral PFC, as well as a marked decrease in salivary alpha-amylase activity compared with that of the nonergonomic workstation. Further analysis using the NASA Task Load Index revealed a higher weighted workload score in the nonergonomic workstation than that in the ergonomic workstation.

CONCLUSION

The findings suggest that ergonomic workstations could significantly improve cognitive functioning and human capabilities at work compared to a nonergonomic workstation.

APPLICATION

Such a study could provide critical information on workstation design and development of mental stress that can be overlooked during traditional workstation design and mental stress assessments.

Is Physiobehavioral Monitoring Nonintrusive? An Examination of Transcranial Doppler Sonography in a Vigilance Task

OBJECTIVE

The current study was designed to determine whether continuous, physiobehavioral monitoring via transcranial Doppler sonography (TCD) has negative effects on human performance or user state in a vigilance task.

BACKGROUND

Physiobehavioral measures have been identified as a promising method of user state assessment, in part because they are thought to be relatively nonintrusive. The notion that physiobehavioral measures are nonintrusive should not be taken for granted and needs to be tested empirically. It is possible that, even though physiobehavioral measures do not require input from a user, they may still hinder performance by causing discomfort, distraction, or interfering with physical activities required for task performance.

METHOD

The current study employed TCD, a common method of monitoring user vigilance. Participants completed a 40-min vigilance task. During the task, 50% wore TCD apparatus, while 50% did not. Intrusiveness was measured in terms of vigilance performance as well as workload, stress, and simulator sickness.

RESULTS

Analyses revealed results that mirrored prototypical vigilance findings: performance declined over time, workload was high, distress and reported simulator sickness symptomology increased during the task, while engagement decreased. The presence or absence of TCD monitoring had no direct or interactive effects on performance or user state.

CONCLUSION

TCD monitoring of user vigilance appears to be nonintrusive.

APPLICATION

Findings support the recommendation that TCD should be used in research and operational settings where user vigilance is of paramount importance. More broadly, when developing and fielding physiobehavioral state measurement systems, intrusiveness should be considered and evaluated.

Measuring the Domain Specificity of Workload Using EEG: Auditory and Visual Domains in Rotary-Wing Simulated Flight

OBJECTIVE

The overarching objective was to evaluate whether workload sensory-domain specificity could be identified through electroencephalogram (EEG) recordings during simulated rotary-wing operations.

BACKGROUND

Rotary-wing aviators experience workload from different sensory domains, although predominantly through auditory and visual domains. Development of real-time monitoring tools using psychophysiological indices, such as EEG recordings, could enable identification of aviator overload in real time.

METHOD

Two studies were completed, both of which recorded EEG, task performance, and self-report data. In Study 1, 16 individuals completed a basic auditory and a basic visual laboratory task where workload was manipulated. In Study 2, 23 Army aviators completed simulated aviation flights where workload was manipulated within auditory and visual sensory domains.

RESULTS

Results from Study 1 found differences in frontal alpha activity during the auditory task, and that alpha and beta activities were associated with perceived workload. Frontal theta activity was found to differ during the visual task while frontal alpha was associated with perceived workload. Study 2 found support for frontal beta activity and the ratio of beta to alpha + theta to differentiate level of workload within the auditory domain.

CONCLUSION

There is likely a role of frontal alpha and beta activities in response to workload manipulations within the auditory domain; however, this role becomes more equivocal when examined in a multifaceted flight scenario.

APPLICATION

Results from this study provide a basis for understanding changes in EEG activity when workload is manipulated in sensory domains that can be used in furthering the development of real-time monitoring tools.

Brain Activity-Based Metrics for Assessing Learning States in VR under Stress among Firefighters: An Explorative Machine Learning Approach in Neuroergonomics

The nature of firefighters` duties requires them to work for long periods under unfavorable conditions. To perform their jobs effectively, they are required to endure long hours of extensive, stressful training. Creating such training environments is very expensive and it is difficult to guarantee trainees' safety. In this study, firefighters are trained in a virtual environment that includes virtual perturbations such as fires, alarms, and smoke. The objective of this paper is to use machine learning methods to discern encoding and retrieval states in firefighters during a visuospatial episodic memory task and explore which regions of the brain provide suitable signals to solve this classification problem. Our results show that the Random Forest algorithm could be used to distinguish between information encoding and retrieval using features extracted from fNIRS data. Our algorithm achieved an F-1 score of 0.844 and an accuracy of 79.10% if the training and testing data are obtained at similar environmental conditions. However, the algorithm's performance dropped to an F-1 score of 0.723 and accuracy of 60.61% when evaluated on data collected under different environmental conditions than the training data. We also found that if the training and evaluation data were recorded under the same environmental conditions, the RPM, LDLPFC, RDLPFC were the most relevant brain regions under non-stressful, stressful, and a mix of stressful and non-stressful conditions, respectively.

Neuroergonomics: A Perspective from Neuropsychology, with a Proposal about Workload

In a brief overview of neuroergonomics, including some personal reminiscences of Raja Parasuraman, it is recognized that the field of human factors and ergonomics has benefitted greatly from the inclusion and integration of neuroscientific methods and theory. It is argued that such synergistic success can work in the other direction as well with the inclusion of methods and theory of human factors by a neuro field, in this case, neuropsychology. More specifically, it is proposed that neuropsychology can benefit from the inclusion of workload measures and theory. Preliminary studies on older adults, persons living with HIV, and patients with a traumatic brain injury or multiple sclerosis, are reviewed. As an adjunct measure to neuropsychological tests, the construct of workload seems perfectly suited to provide an additional vector of information on patient status, capturing some of the large individual differences evident in clinical populations and facilitating the early detection of cognitive change.

Direct Communication Between Brains: A Systematic PRISMA Review of Brain-To-Brain Interface

This paper aims to review the current state of brain-to-brain interface (B2BI) technology and its potential. B2BIs function via a brain-computer interface (BCI) to read a sender's brain activity and a computer-brain interface (CBI) to write a pattern to a receiving brain, transmitting information. We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) to systematically review current literature related to B2BI, resulting in 15 relevant publications. Experimental papers primarily used transcranial magnetic stimulation (tMS) for the CBI portion of their B2BI. Most targeted the visual cortex to produce phosphenes. In terms of study design, 73.3% (11) are unidirectional and 86.7% (13) use only a 1:1 collaboration model (subject to subject). Limitations are apparent, as the CBI method varied greatly between studies indicating no agreed upon neurostimulatory method for transmitting information. Furthermore, only 12.4% (2) studies are more complicated than a 1:1 model and few researchers studied direct bidirectional B2BI. These studies show B2BI can offer advances in human communication and collaboration, but more design and experiments are needed to prove potential. B2BIs may allow rehabilitation therapists to pass information mentally, activating a patient's brain to aid in stroke recovery and adding more complex bidirectionality may allow for increased behavioral synchronization between users. The field is very young, but applications of B2BI technology to neuroergonomics and human factors engineering clearly warrant more research.

Brain Oscillation Entrainment by Perceptible and Non-perceptible Rhythmic Light Stimulation

Objective and Background: Decades of research in the field of steady-state visual evoked potentials (SSVEPs) have revealed great potential of rhythmic light stimulation for brain-computer interfaces. Additionally, rhythmic light stimulation provides a non-invasive method for entrainment of oscillatory activity in the brain. Especially effective protocols enabling non-perceptible rhythmic stimulation and, thereby, reducing eye fatigue and user discomfort are favorable. Here, we investigate effects of (1) perceptible and (2) non-perceptible rhythmic light stimulation as well as attention-based effects of the stimulation by asking participants to focus (a) on the stimulation source directly in an overt attention condition or (b) on a cross-hair below the stimulation source in a covert attention condition. Method: SSVEPs at 10 Hz were evoked with a light-emitting diode (LED) driven by frequency-modulated signals and amplitudes of the current intensity either below or above a previously estimated individual threshold. Furthermore, we explored the effect of attention by asking participants to fixate on the LED directly in the overt attention condition and indirectly attend it in the covert attention condition. By measuring electroencephalography, we analyzed differences between conditions regarding the detection of reliable SSVEPs via the signal-to-noise ratio (SNR) and functional connectivity in occipito-frontal(-central) regions. Results: We could observe SSVEPs at 10 Hz for the perceptible and non-perceptible rhythmic light stimulation not only in the overt but also in the covert attention condition. The SNR and SSVEP amplitudes did not differ between the conditions and SNR values were in all except one participant above significance thresholds suggested by previous literature indicating reliable SSVEP responses. No difference between the conditions could be observed in the functional connectivity in occipito-frontal(-central) regions. Conclusion: The finding of robust SSVEPs even for non-intrusive rhythmic stimulation protocols below an individual perceptibility threshold and without direct fixation on the stimulation source reveals strong potential as a safe stimulation method for oscillatory entrainment in naturalistic applications.

Motor Training Using Mental Workload (MWL) With an Assistive Soft Exoskeleton System: A Functional Near-Infrared Spectroscopy (fNIRS) Study for Brain-Machine Interface (BMI)

Mental workload is a neuroergonomic human factor, which is widely used in planning a system's safety and areas like brain-machine interface (BMI), neurofeedback, and assistive technologies. Robotic prosthetics methodologies are employed for assisting hemiplegic patients in performing routine activities. Assistive technologies' design and operation are required to have an easy interface with the brain with fewer protocols, in an attempt to optimize mobility and autonomy. The possible answer to these design questions may lie in neuroergonomics coupled with BMI systems. In this study, two human factors are addressed: designing a lightweight wearable robotic exoskeleton hand that is used to assist the potential stroke patients with an integrated portable brain interface using mental workload (MWL) signals acquired with portable functional near-infrared spectroscopy (fNIRS) system. The system may generate command signals for operating a wearable robotic exoskeleton hand using two-state MWL signals. The fNIRS system is used to record optical signals in the form of change in concentration of oxy and deoxygenated hemoglobin (HbO and HbR) from the pre-frontal cortex (PFC) region of the brain. Fifteen participants participated in this study and were given hand-grasping tasks. Two-state MWL signals acquired from the PFC region of the participant's brain are segregated using machine learning classifier-support vector machines (SVM) to utilize in operating a robotic exoskeleton hand. The maximum classification accuracy is 91.31%, using a combination of mean-slope features with an average information transfer rate (ITR) of 1.43. These results show the feasibility of a two-state MWL (fNIRS-based) robotic exoskeleton hand (BMI system) for hemiplegic patients assisting in the physical grasping tasks.

Effects of Neural Mechanisms of Pretask Resting EEG Alpha Information on Situational Awareness: A Functional Connectivity Approach

OBJECTIVE

In this study, the influence of pretask resting neural mechanisms on situational awareness (SA)-task is studied.

BACKGROUND

Pretask electroencephalography (EEG) information and Stroop effect are known to influence task engagement independently. However, neural mechanisms of pretask resting absolute alpha (PRAA) and pretask resting alpha frontal asymmetry (PRAFA) in influencing SA-task which is undergoing Stroop effect is still not understood.

METHOD

The study involved pretask resting EEG measurements from 18 healthy individuals followed by functional magnetic resonance imaging (fMRI) acquisition during SA-task. To understand the effect of pretask alpha information and Stroop effect on SA, a robust correlation between mean reaction time, SA Index, PRAA, and PRAFA were assessed. Furthermore, neural underpinnings of PRAA, PRAFA in SA-task, and functional connectivity were analyzed through the EEG-informed fMRI approach.

RESULTS

Significant robust correlation of reaction time was observed with SA Index (Pearson: r = .50, pcorr = .05) and PRAFA (Pearson: r = .63; pcorr = .01), respectively. Similarly, SA Index significantly correlated with PRAFA (Pearson: r = .56, pcorr = .01; Spearman: r = .61, pcorr = .007), and PRAA (Pearson: r = .59, pcorr = .005; Spearman: r = .59, pcorr = .002). Neural underpinnings of SA-task revealed regions involved in visual-processing and higher-order cognition. PRAA was primarily underpinned at frontal-temporal areas and functionally connected to SA-task regions pertaining to the emotional regulation. PRAFA has correlated with limbic and parietal regions, which are involved in integration of visual, emotion, and memory information of SA-task.

CONCLUSION

The results suggest a strong association of reaction time with SA-task and PRAFA and strongly support the hypothesis that PRAFA, PRAA, and associated neural mechanisms significantly influence the outcome of SA-task.

APPLICATION

It is beneficial to study the effect of pretask resting information on SA-task to improve SA.

Autonomic Activity and Surgical Flow Disruptions in Healthcare Providers during Cardiac Surgery

Cardiac surgery represents a complex sociotechnical environment relying on a combination of technical and non-technical team-based expertise. Surgical flow disruptions (SFDs) may be influenced by a variety of sources, including social, environmental, and emotional factors affecting healthcare providers (HCPs). Many of these factors can be readily observed, except for emotional factors (i.e. distress), which represents an underappreciated yet critical source of SFDs. The aim of this study was to demonstrate the sensitivity of autonomic activity metrics to detect an SFD during cardiac surgery. We integrated heart rate variability (HRV) analysis with observation-based annotations to allow data triangulation. Following a critical medication administration error by the anesthesiologist in-training, data sources were consulted to identify events precipitating this near-miss event. Using pyphysio, an open-source physiological signal processing package, we analyzed the attending anesthesiologists' HRV, specifically the low frequency (LF) power, high frequency (HF) power, LF/HF ratio, standard deviation of normal-to-normal (SDNN), and root mean square of the successive differences (RMSSD) as indicators of ANS activity. A heightened SNS response in the attending anesthesiologists' physiological arousal was observed as elevations in LF power and LF/HF ratio, as well as depressions in HF power, SDNN, and RMSSD prior to the near-miss event. The attending anesthesiologist subjectively confirmed a state of high distress induced by task-irrelevant environmental factors during this time. Qualitative analysis of audio/video recordings objectively revealed that the autonomic nervous system (ANS) activation detected was temporally associated with an argument over operating room management. This study confirms that it is possible to recognize detrimental psychophysiological influences in cardiac surgery procedures via advanced HRV analysis. To our knowledge, ours is the first such case demonstrating ANS activity coinciding with strong self-reported emotion during live surgery using HRV. Despite extensive experience in the cardiac OR, transient but intense emotional changes may have the potential to disrupt attention processes in even the most experienced HCP. A primary implication of this work is the possibility to detect real-time ANS activity, which could enable personalized interventions to proactively mitigate downstream adverse events. Additional studies on our large database of surgical cases are underway and new studies are actively being planned to confirm this preliminary observation.

A Cyber-Physical-Human System for One-to-Many UAS Operations: Cognitive Load Analysis

The continuing development of avionics for Unmanned Aircraft Systems (UASs) is introducing higher levels of intelligence and autonomy both in the flight vehicle and in the ground mission control, allowing new promising operational concepts to emerge. One-to-Many (OTM) UAS operations is one such concept and its implementation will require significant advances in several areas, particularly in the field of Human-Machine Interfaces and Interactions (HMI2). Measuring cognitive load during OTM operations, in particular Mental Workload (MWL), is desirable as it can relieve some of the negative effects of increased automation by providing the ability to dynamically optimize avionics HMI2 to achieve an optimal sharing of tasks between the autonomous flight vehicles and the human operator. The novel Cognitive Human Machine System (CHMS) proposed in this paper is a Cyber-Physical Human (CPH) system that exploits the recent technological developments of affordable physiological sensors. This system focuses on physiological sensing and Artificial Intelligence (AI) techniques that can support a dynamic adaptation of the HMI2 in response to the operators' cognitive state (including MWL), external/environmental conditions and mission success criteria. However, significant research gaps still exist, one of which relates to a universally valid method for determining MWL that can be applied to UAS operational scenarios. As such, in this paper we present results from a study on measuring MWL on five participants in an OTM UAS wildfire detection scenario, using Electroencephalogram (EEG) and eye tracking measurements. These physiological data are compared with a subjective measure and a task index collected from mission-specific data, which serves as an objective task performance measure. The results show statistically significant differences for all measures including the subjective, performance and physiological measures performed on the various mission phases. Additionally, a good correlation is found between the two physiological measurements and the task index. Fusing the physiological data and correlating with the task index gave the highest correlation coefficient (CC = 0.726 ± 0.14) across all participants. This demonstrates how fusing different physiological measurements can provide a more accurate representation of the operators' MWL, whilst also allowing for increased integrity and reliability of the system.

Effects of Semantic Congruence on Sign Identification: An ERP Study

OBJECTIVE

The purpose of this study was to investigate the effects of semantic congruence and incongruence on sign identification by using event-related potentials (ERPs).

BACKGROUND

Sign systems have crucial roles in public spaces and traffic facilities. Poorly designed signs can easily confuse pedestrians and drivers and reduce the efficiency of public activities and urban administration.

METHOD

Thirty-one participants completed a sign identification experiment independently in a laboratory setting. Experimental materials were selected from GB/T 10001, a Chinese national recommendation standard that is officially named Public Information Graphical Symbols for Use on Signs. All ERP data were processed using MATLAB 13b, and behavioral data were analyzed using Stata 14.

RESULTS

N170, P200, N300, and N400 components were induced during semantic processing. Statistical analysis revealed that semantic congruence has a main effect on N300 in the frontal region and has a main effect on N400 at FZ in the frontal region, CPZ in the parietal-central region, and PZ in the parietal region. Amplitudes of N300 induced by picture-word matching were considerably different between the two experimental conditions at electrodes FZ and FCZ. Amplitudes of N400 were significantly larger in the incongruent condition than in the congruent condition.

CONCLUSION

The study demonstrated that N300 and N400 are promising indicators for measuring semantic congruence in future sign design.

APPLICATION

Our findings provide ERP indicators for measuring the semantic congruence of sign design, which can be easily applied to improve the efficiency of sign design and sign comprehension.

Methodological Approaches and Recommendations for Functional Near-Infrared Spectroscopy Applications in HF/E Research

OBJECTIVE

The objective of this study was to systematically document current methods and protocols employed when using functional near-infrared spectroscopy (fNIRS) techniques in human factors and ergonomics (HF/E) research and generate recommendations for conducting and reporting fNIRS findings in HF/E applications.

METHOD

A total of 1,687 articles were identified through Ovid-MEDLINE, PubMed, Web of Science, and Scopus databases, of which 37 articles were included in the review based on review inclusion/exclusion criteria.

RESULTS

A majority of the HF/E fNIRS investigations were found in transportation, both ground and aviation, and in assessing cognitive (e.g., workload, working memory) over physical constructs. There were large variations pertaining to data cleaning, processing, and analysis approaches across the studies that warrant standardization of methodological approaches. The review identified major challenges in transparency and reporting of important fNIRS data collection and analyses specifications that diminishes study replicability, introduces potential biases, and increases likelihood of inaccurate results. As such, results reported in existing fNIRS studies need to be cautiously approached.

CONCLUSION

To improve the quality of fNIRS investigations and/or to facilitate its adoption and integration in different HF/E applications, such as occupational ergonomics and rehabilitation, recommendations for fNIRS data collection, processing, analysis, and reporting are provided.

Neuroergonomic Assessment of Wheelchair Control Using Mobile fNIRS

For over two centuries, the wheelchair has been one of the most common assistive devices for individuals with locomotor impairments without many modifications. Wheelchair control is a complex motor task that increases both the physical and cognitive workload. New wheelchair interfaces, including Power Assisted devices, can further augment users by reducing the required physical effort, however little is known on the mental effort implications. In this study, we adopted a neuroergonomic approach utilizing mobile and wireless functional near infrared spectroscopy (fNIRS) based brain monitoring of physically active participants. 48 volunteers (30 novice and 18 experienced) self-propelled on a wheelchair with and without a PowerAssist interface in both simple and complex realistic environments. Results indicated that as expected, the complex more difficult environment led to lower task performance complemented by higher prefrontal cortex activity compared to the simple environment. The use of the PowerAssist feature had significantly lower brain activation compared to traditional manual control only for novices. Expertise led to a lower brain activation pattern within the middle frontal gyrus, complemented by performance metrics that involve lower cognitive workload. Results here confirm the potential of the Neuroergonomic approach and that direct neural activity measures can complement and enhance task performance metrics. We conclude that the cognitive workload benefits of PowerAssist are more directed to new users and difficult settings. The approach demonstrated here can be utilized in future studies to enable greater personalization and understanding of mobility interfaces within real-world dynamic environments.

Neuroergonomic Assessment of Hot Beverage Preparation and Consumption: An EEG and EDA Study

Neuroergonomics is an emerging field that investigates the human brain about behavioral performance in natural environments and everyday settings. This study investigated the body and brain activity correlates of a typical daily activity, hot beverage preparation, and consumption in a realistic office environment where participants performed natural daily tasks. Using wearable, battery operated and wireless Electroencephalogram (EEG) and Electrodermal activity (EDA) sensors, neural and physiological responses were measured in untethered, freely moving participants who prepared hot beverages using two different machines (a market leader and follower as determined by annual US sales). They later consumed the drinks they had prepared in three blocks. Emotional valence was estimated using frontal asymmetry in EEG alpha band power and emotional arousal was estimated from EDA tonic and phasic activity. Results from 26 participants showed that the market-leading coffee machine was more efficient to use based on self-reports, behavioral performance measures, and there were significant within-subject differences in valence between the two machine use. Moreover, the market leader user interface led to greater self-reported product preference, which was further supported by significant differences in measured arousal and valence (EDA and EEG, respectively) during coffee production and consumption. This is the first study that uses a multimodal and comprehensive assessment of coffee machine use and beverage consumption in a naturalistic work environment. Approaches described in this study can be adapted in the future to other task-specific machine usability and consumer neuroscience studies.

A Neuroergonomics Approach to Mental Workload, Engagement and Human Performance

The assessment and prediction of cognitive performance is a key issue for any discipline concerned with human operators in the context of safety-critical behavior. Most of the research has focused on the measurement of mental workload but this construct remains difficult to operationalize despite decades of research on the topic. Recent advances in Neuroergonomics have expanded our understanding of neurocognitive processes across different operational domains. We provide a framework to disentangle those neural mechanisms that underpin the relationship between task demand, arousal, mental workload and human performance. This approach advocates targeting those specific mental states that precede a reduction of performance efficacy. A number of undesirable neurocognitive states (mind wandering, effort withdrawal, perseveration, inattentional phenomena) are identified and mapped within a two-dimensional conceptual space encompassing task engagement and arousal. We argue that monitoring the prefrontal cortex and its deactivation can index a generic shift from a nominal operational state to an impaired one where performance is likely to degrade. Neurophysiological, physiological and behavioral markers that specifically account for these states are identified. We then propose a typology of neuroadaptive countermeasures to mitigate these undesirable mental states.

EEG Theta Power Activity Reflects Workload among Army Combat Drivers: An Experimental Study

We aimed to evaluate the effects of mental workload variations, as a function of the road environment, on the brain activity of army drivers performing combat and non-combat scenarios in a light multirole vehicle dynamic simulator. Forty-one non-commissioned officers completed three standardized driving exercises with different terrain complexities (low, medium, and high) while we recorded their electroencephalographic (EEG) activity. We focused on variations in the theta EEG power spectrum, a well-known index of mental workload. We also assessed performance and subjective ratings of task load. The theta EEG power spectrum in the frontal, temporal, and occipital areas were higher during the most complex scenarios. Performance (number of engine stops) and subjective data supported these findings. Our findings strengthen previous results found in civilians on the relationship between driver mental workload and the theta EEG power spectrum. This suggests that EEG activity can give relevant insight into mental workload variations in an objective, unbiased fashion, even during real training and/or operations. The continuous monitoring of the warfighter not only allows instantaneous detection of over/underload but also might provide online feedback to the system (either automated equipment or the crew) to take countermeasures and prevent fatal errors.