Correlation between Situational Awareness and EEG signals

EEG and fNIRS are associated with situation awareness (hazard) prediction during a driving task

Situation awareness (SA) is important in many demanding tasks (e.g. driving). Assessing SA during training can indicate whether someone is ready to perform in the real world. SA is typically assessed by interrupting the task to ask questions about the situation or asking questions after task completion, assessing only momentary SA. An objective and continuous means of detecting SA is needed. We examined whether neurophysiological sensors are useful to objectively measure Level 3 SA (projection of events into the future) during a driving task. We measured SA by the speed at which participants responded to SA questions and the accuracy of responses. For EEG, beta and theta power were most sensitive to SA response time. For fNIRS, oxygenated haemoglobin (HbO) was most sensitive to accuracy. This is the first evidence to our knowledge that neurophysiological measures are useful for assessing Level 3 SA during an ecologically valid task.

Evidence of elevated situational awareness for active duty soldiers during navigation of a virtual environment

U.S. service members maintain constant situational awareness (SA) due to training and experience operating in dynamic and complex environments. Work examining how military experience impacts SA during visual search of a complex naturalistic environment, is limited. Here, we compare Active Duty service members and Civilians' physiological behavior during a navigational visual search task in an open-world virtual environment (VE) while cognitive load was manipulated. We measured eye-tracking and electroencephalogram (EEG) outcomes from Active Duty (N = 21) and Civilians (N = 15) while they navigated a desktop VE at a self-regulated pace. Participants searched and counted targets (N = 15) presented among distractors, while cognitive load was manipulated with an auditory Math Task. Results showed Active Duty participants reported significantly greater/closer to the correct number of targets compared to Civilians. Overall, Active Duty participants scanned the VE with faster peak saccade velocities and greater average saccade magnitudes compared to Civilians. Convolutional Neural Network (CNN) response (EEG P-300) was significantly weighted more to initial fixations for the Active Duty group, showing reduced attentional resources on object refixations compared to Civilians. There were no group differences in fixation outcomes or overall CNN response when comparing targets versus distractor objects. When cognitive load was manipulated, only Civilians significantly decreased their average dwell time on each object and the Active Duty group had significantly fewer numbers of correct answers on the Math Task. Overall, the Active Duty group explored the VE with increased scanning speed and distance and reduced cognitive re-processing on objects, employing a different, perhaps expert, visual search strategy indicative of increased SA. The Active Duty group maintained SA in the main visual search task and did not appear to shift focus to the secondary Math Task. Future work could compare how a stress inducing environment impacts these groups' physiological or cognitive markers and performance for these groups.

Physiological records-based situation awareness evaluation under aviation context: A comparative analysis

Situational Awareness (SA) assessment is of paramount importance in various domains, with particular significance in the military for safe aviation decision-making. It involves encompassing perception, comprehension, and projection levels in human beings. Accurate evaluation of SA statuses across these three levels is crucial for mitigating human false-positive and false-negative rates in monitoring complex scenarios in the aviation context. This study proposes a comprehensive comparative analysis by involving two types of physiological records: electroencephalogram (EEG) signals and brain electrical activity mapping (BEAM) images. These two modalities are leveraged to automate precise SA evaluation using both conventional machine learning and advanced deep learning techniques. Benchmarking experiments reveal that the BEAM-based deep learning models attain state-of-the-art performance scores of 0.955 for both SA perception and comprehension levels, respectively. Conversely, the EEG signals-based manual feature extraction, selection, and classification approach achieved a superior accuracy of 0.929 for the projection level of SA. These findings collectively highlight the potential of deploying diverse physiological records as valuable computational tools for enhancing SA evaluation throughout aviation decision-making safety.

Recognizing the situation awareness of forklift operators based on EEG techniques in a field experiment

Lack of situation awareness (SA) is the primary cause of human errors when operating forklifts, so determining the SA level of the forklift operator is crucial to the safety of forklift operations. An EEG recognition approach of forklift operator SA in actual settings was presented in order to address the issues with invasiveness, subjectivity, and intermittency of existing measuring methods. In this paper, we conducted a field experiment that mimicked a typical forklift operation scenario to verify the differences in EEG states of forklift operators with different SA levels and investigate the correlation of multi-band combination features of each brain region of forklift operators with SA. Based on the sensitive EEG combination indexes, Support Vector Mechanism was used to construct a forklift operator SA recognition model. The results revealed that there were differences between forklift operators with high and low SA in the θ, α, and β frequency bands in zones F, C, P, and O; combined EEG indicators θ/β, (α + θ)/(α + β), and θ/(α + β) in zones F, P, and C were significantly correlated with SA; the recognition accuracy of the model reached 88.64% in the case of combined EEG indicators of zones C & F & P as input. It could provide a reference for SA measurement, contributing to the improvement of SA.

Detection of Pilot's Mental Workload Using a Wireless EEG Headset in Airfield Traffic Pattern Tasks

Elevated mental workload (MWL) experienced by pilots can result in increased reaction times or incorrect actions, potentially compromising flight safety. This study aims to develop a functional system to assist administrators in identifying and detecting pilots' real-time MWL and evaluate its effectiveness using designed airfield traffic pattern tasks within a realistic flight simulator. The perceived MWL in various situations was assessed and labeled using NASA Task Load Index (NASA-TLX) scores. Physiological features were then extracted using a fast Fourier transformation with 2-s sliding time windows. Feature selection was conducted by comparing the results of the Kruskal-Wallis (K-W) test and Sequential Forward Floating Selection (SFFS). The results proved that the optimal input was all PSD features. Moreover, the study analyzed the effects of electroencephalography (EEG) features from distinct brain regions and PSD changes across different MWL levels to further assess the proposed system's performance. A 10-fold cross-validation was performed on six classifiers, and the optimal accuracy of 87.57% was attained using a multi-class K-Nearest Neighbor (KNN) classifier for classifying different MWL levels. The findings indicate that the wireless headset-based system is reliable and feasible. Consequently, numerous wireless EEG device-based systems can be developed for application in diverse real-driving scenarios. Additionally, the current system contributes to future research on actual flight conditions.

EEG-based analysis for pilots' at-risk cognitive competency identification using RF-CNN algorithm

Cognitive competency is an essential complement to the existing ship pilot screening system that should be focused on. Situation awareness (SA), as the cognitive foundation of unsafe behaviors, is susceptible to influencing piloting performance. To address this issue, this paper develops an identification model based on random forest- convolutional neural network (RF-CNN) method for detecting at-risk cognitive competency (i.e., low SA level) using wearable EEG signal acquisition technology. In the poor visibility scene, the pilots' SA levels were correlated with EEG frequency metrics in frontal (F) and central (C) regions, including α/β (p = 0.071 < 0.1 in F and p = 0.042 < 0.05 in C), θ/(α + θ) (p = 0.048 < 0.05 in F and p = 0.026 < 0.05 in C) and (α + θ)/β (p = 0.046 < 0.05 in F and p = 0.012 < 0.05 in C), and then a total of 12 correlation features were obtained based on a 5 s sliding time window. Using the RF algorithm developed by principal component analysis (PCA) for further feature combination, these salient combinations are used as input sets to obtain the CNN algorithm with optimal parameters for identification. The comparative results of the proposed RF-CNN (accuracy is 84.8%) against individual RF (accuracy is 78.1%) and CNN (accuracy is 81.6%) methods demonstrate that the RF-CNN with feature optimization provides the best identification of at-risk cognitive competency (accuracy increases 6.7%). Overall, the results of this paper provide key technical support for the development of an adaptive evaluation system of pilots' cognitive competency based on intelligent technology, and lay the foundation and framework for monitoring the cognitive process and competency of ship piloting operation in China.

Mental stress and safety awareness during human-robot collaboration - Review

Human-robot collaboration (HRC) is an emerging research area that has gained tremendous attention in both academia and industry. Yet, the feature that humans and robots sharing the workplace has led to safety concerns. In particular, the mental stress or safety awareness of human teammates during HRC remains unclear but is also of great importance to workplace safety. In this manuscript, we reviewed twenty-five studies for understanding the relationships between HRC and workers' mental stress or safety awareness. Specifically, we aimed to understand: (1) robot-related factors that may affect human workers' mental stress or safety awareness, (2) a number of measurements that could be used to evaluate workers' mental stress in HRC, and (3) various methods for measuring safety awareness that had been adopted or could be applied in HRC. According to our literature review, robot-related factors including robot characteristics, social touching and trajectory have relationships with workers' mental stress or safety awareness. For the measurement of mental stress and safety awareness, each method mentioned has its validity and rationality. Additionally, a discussion related to the potential co-robot actions to lower mental stress or improve safety awareness as well as future implications were provided.

A Digital Twin-Based Platform towards Intelligent Automation with Virtual Counterparts of Flight and Air Traffic Control Operations

Automation technologies have been deployed widely to boost the efficiency of production and operations, to trim the complicated process, and to reduce the human error involved. Nevertheless, aviation remains human-centred and requires collaboration between different parties. Given the lack of a collaborative decision-making training platform for air traffic operations in the industry, this study utilises the concept of cyber-physical systems (CPS) to formulate a system architecture for pilots and air traffic control officers training in collaborative decision making by linking and integrating the virtual counterparts of flights and air traffic control operations. Collaborative decision-making training and the corresponding intelligent automation aids could be realised and supported. A performance analysis via a flight task undertaken with different computational load settings was prepared to evaluate the platform’s latency and integrity. The latency is presented using its 95% confidence interval, and integrity is presented using the percentage of data loss during wireless transmission. The results demonstrated convincing performance and a promising system robustness in both domains.