Human-Machine Interaction Assessment by Neurophysiological Measures: A Study on Professional Air Traffic Controllers

Mental Effort Estimation by Passive BCI: A Cross-Subject Analysis

Despite the technological advancements, the employment of passive brain computer interface (BCI) out of the laboratory context is still challenging. This is largely due to methodological reasons. On the one hand, machine learning methods have shown their potential in maximizing performance for user mental states classification. On the other hand, the issues related to the necessary and frequent calibration of algorithms and to the temporal resolution of the measurement (i.e. how long it takes to have a reliable state measure) are still unsolved. This work explores the performances of a passive BCI system for mental effort monitoring consisting of three frontal electroencephalographic (EEG) channels. In particular, three calibration approaches have been tested: an intra-subject approach, a cross-subject approach, and a free-calibration procedure based on the simple average of theta activity over the three employed channels. A Random Forest model has been employed in the first two cases. The results obtained during multi-tasking have shown that the cross-subject approach allows the classification of low and high mental effort with an AUC higher than 0.9, with a related time resolution of 45 seconds. Moreover, these performances are not significantly different from the intra-subject approach although they are significantly higher than the calibration-free approach. In conclusion, these results suggest that a light (three EEG channels) passive BCI system based on a Random Forest algorithm and cross-subject calibration could be a simple and reliable tool for out-of-the-lab employment.

A multimodal and signals fusion approach for assessing the impact of stressful events on Air Traffic Controllers

Stress is a word used to describe human reactions to emotionally, cognitively and physically challenging experiences. A hallmark of the stress response is the activation of the autonomic nervous system, resulting in the "fight-freeze-flight" response to a threat from a dangerous situation. Consequently, the capability to objectively assess and track a controller's stress level while dealing with air traffic control (ATC) activities would make it possible to better tailor the work shift and maintain high safety levels, as well as to preserve the operator's health. In this regard, sixteen controllers were asked to perform a realistic air traffic management (ATM) simulation during which subjective data (i.e. stress perception) and neurophysiological data (i.e. brain activity, heart rate, and galvanic skin response) were collected with the aim of accurately characterising the controller's stress level experienced in the various experimental conditions. In addition, external supervisors regularly evaluated the controllers in terms of manifested stress, safety, and efficiency throughout the ATM scenario. The results demonstrated 1) how the stressful events caused both supervisors and controllers to underestimate the experienced stress level, 2) the advantage of taking into account both cognitive and hormonal processes in order to define a reliable stress index, and 3) the importance of the points in time at which stress is measured owing to the potential transient effect once the stressful events have ceased.

Prediction of Individual User's Dynamic Ranges of EEG Features from Resting-State EEG Data for Evaluating Their Suitability for Passive Brain-Computer Interface Applications

With the recent development of low-cost wearable electroencephalogram (EEG) recording systems, passive brain-computer interface (pBCI) applications are being actively studied for a variety of application areas, such as education, entertainment, and healthcare. Various EEG features have been employed for the implementation of pBCI applications; however, it is frequently reported that some individuals have difficulty fully enjoying the pBCI applications because the dynamic ranges of their EEG features (i.e., its amplitude variability over time) were too small to be used in the practical applications. Conducting preliminary experiments to search for the individualized EEG features associated with different mental states can partly circumvent this issue; however, these time-consuming experiments were not necessary for the majority of users whose dynamic ranges of EEG features are large enough to be used for pBCI applications. In this study, we tried to predict an individual user's dynamic ranges of the EEG features that are most widely employed for pBCI applications from resting-state EEG (RS-EEG), with the ultimate goal of identifying individuals who might need additional calibration to become suitable for the pBCI applications. We employed a machine learning-based regression model to predict the dynamic ranges of three widely used EEG features known to be associated with the brain states of valence, relaxation, and concentration. Our results showed that the dynamic ranges of EEG features could be predicted with normalized root mean squared errors of 0.2323, 0.1820, and 0.1562, respectively, demonstrating the possibility of predicting the dynamic ranges of the EEG features for pBCI applications using short resting EEG data.

How Neurophysiological Measures Can be Used to Enhance the Evaluation of Remote Tower Solutions

New solutions in operational environments are often, among objective measurements, evaluated by using subjective assessment and judgment from experts. Anyhow, it has been demonstrated that subjective measures suffer from poor resolution due to a high intra and inter-operator variability. Also, performance measures, if available, could provide just partial information, since an operator could achieve the same performance but experiencing a different workload. In this study, we aimed to demonstrate: (i) the higher resolution of neurophysiological measures in comparison to subjective ones; and (ii) how the simultaneous employment of neurophysiological measures and behavioral ones could allow a holistic assessment of operational tools. In this regard, we tested the effectiveness of an electroencephalography (EEG)-based neurophysiological index (W_EEG index) in comparing two different solutions (i.e., Normal and Augmented) in terms of experienced workload. In this regard, 16 professional air traffic controllers (ATCOs) have been asked to perform two operational scenarios. Galvanic Skin Response (GSR) has also been recorded to evaluate the level of arousal (i.e., operator involvement) during the two scenarios execution. NASA-TLX questionnaire has been used to evaluate the perceived workload, and an expert was asked to assess performance achieved by the ATCOs. Finally, reaction times on specific operational events relevant for the assessment of the two solutions, have also been collected. Results highlighted that the Augmented solution induced a local increase in subjects performance (Reaction times). At the same time, this solution induced an increase in the workload experienced by the participants (W_EEG). Anyhow, this increase is still acceptable, since it did not negatively impact the performance and has to be intended only as a consequence of the higher engagement of the ATCOs. This behavioral effect is totally in line with physiological results obtained in terms of arousal (GSR), that increased during the scenario with augmentation. Subjective measures (NASA-TLX) did not highlight any significant variation in perceived workload. These results suggest that neurophysiological measure provide additional information than behavioral and subjective ones, even at a level of few seconds, and its employment during the pre-operational activities (e.g., design process) could allow a more holistic and accurate evaluation of new solutions.

Antismoking Campaigns' Perception and Gender Differences: A Comparison among EEG Indices

Human factors' aim is to understand and evaluate the interactions between people and tasks, technologies, and environment. Among human factors, it is possible then to include the subjective reaction to external stimuli, due to individual's characteristics and states of mind. These processes are also involved in the perception of antismoking public service announcements (PSAs), the main tool for governments to contrast the first cause of preventable deaths in the world: tobacco addiction. In the light of that, in the present article, it has been investigated through the comparison of different electroencephalographic (EEG) indices a typical item known to be able of influencing PSA perception, that is gender. In order to investigate the neurophysiological underpinnings of such different perception, we tested two PSAs: one with a female character and one with a male character. Furthermore, the experimental sample was divided into men and women, as well as smokers and nonsmokers. The employed EEG indices were the mental engagement (ME: the ratio between beta activity and the sum of alpha and theta activity); the approach/withdrawal (AW: the frontal alpha asymmetry in the alpha band); and the frontal theta activity and the spectral asymmetry index (SASI: the ratio between beta minus theta and beta plus theta). Results suggested that the ME and the AW presented an opposite trend, with smokers showing higher ME and lower AW than nonsmokers. The ME and the frontal theta also evidenced a statistically significant interaction between the kind of the PSA and the gender of the observers; specifically, women showed higher ME and frontal theta activity for the male character PSA. This study then supports the usefulness of the ME and frontal theta for purposes of PSAs targeting on the basis of gender issues and of the ME and the AW and for purposes of PSAs targeting on the basis of smoking habits.