EEG-Based Cognitive Control Behaviour Assessment: an Ecological study with Professional Air Traffic Controllers

Correlations of pilot trainees' brainwave dynamics with subjective performance evaluations: insights from EEG microstate analysis

Objective

This study aims to investigate the relationship between the subjective performance evaluations on pilot trainees' aircraft control abilities and their brainwave dynamics reflected in the results from EEG microstate analysis. Specifically, we seek to identify correlations between distinct microstate patterns and each dimension included in the subjective flight control evaluations, shedding light on the neurophysiological mechanisms underlying aviation expertise and possible directions for future improvements in pilot training.

Background

Proficiency in aircraft control is crucial for aviation safety and modern aviation where pilots need to maneuver aircraft through an array of situations, ranging from routine takeoffs and landings to complex weather conditions and emergencies. However, the neurophysiological aspects of aviation expertise remain largely unexplored. This research bridges the gap by examining the relationship between pilot trainees' specific brainwave patterns and their subjective evaluations of flight control levels, offering insights into the cognitive underpinnings of pilot skill efficiency and development.

Method

EEG microstate analysis was employed to examine the brainwave dynamics of pilot trainees while they performed aircraft control tasks under a flight simulator-based pilot training process. Trainees' control performance was evaluated by experienced instructors across five dimensions and their EEG data were analyzed to investigate the associations between the parameters of specific microstates with successful aircraft control.

Results

The experimental results revealed significant associations between aircraft control levels and the parameters of distinct EEG microstates. Notably, these associations varied across control dimensions, highlighting the multifaceted nature of control proficiency. Noteworthy correlations included positive correlations between microstate class E and class G with aircraft control, emphasizing the role of attentional processes, perceptual integration, working memory, cognitive flexibility, decision-making, and executive control in aviation expertise. Conversely, negative correlations between microstate class C and class F with aircraft control indicated links between pilot trainees' cognitive control and their control performance on flight tasks.

Conclusion

The findings underscore the multidimensional nature of aircraft control proficiency and emphasize the significance of attentional and cognitive processes in achieving aviation expertise. These neurophysiological markers offer a basis for designing targeted pilot training programs and interventions to enhance trainees' aircraft control skills.

P300 to Low and High Frequency Stimuli Are Not Influenced by Intensity in Adults with Normal Hearing

Background/Objectives: Since high frequencies are susceptible to disruption in various types of hearing loss, a symptom which is common in people with tinnitus, the aim of the study was to investigate EEG cortical auditory evoked and P300 responses to both a high- and low frequency-centered oddball paradigm to begin to establish the most suitable cognitive physiologic testing conditions for those with both unimpaired hearing and those with hearing impairments. Methods: Cortical auditory evoked potential (CAEP) P1, N1, P2 and P300 (subtraction wave) peaks were identified in response to high- (standard: 6000 Hz, deviant: 8000 Hz) and low frequency (Standard: 375 Hz, Deviant: 500 Hz) oddball paradigms. Each paradigm was presented at various intensity levels. Latencies and amplitudes were then computed for each condition to assess the effects of frequency and intensity. Results: Stimulus intensity had no effect on either the high- or low frequency paradigms of P300 characteristics. In contrast, for the low frequency paradigm, intensity influenced the N1 latency and P2 amplitude, while for the high frequency paradigm intensity influenced P1 and P2 latency and P2 amplitude. Conclusions: Obligatory CAEP components responded more readily to stimulus frequency and intensity changes, and one possible consideration is that higher frequencies could play a role in the response characteristics exhibited by N1 (except for N1 amplitude) and P2, given their involvement in attentional processes linked to the detection of warning cues. P300 latency and amplitude were not influenced by such factors. These findings support the hypothesis that disentangling the cognitive from the more sensory-based response is possible, even in those with hearing loss, provided that the patient's hearing loss is considered when determining the presentation level. While the present study was performed in participants with unimpaired hearing, these data set up future studies investigating the effectiveness of using similar methods in hearing-impaired persons.

From data to decisions: AI and functional connectivity for diagnosis, prognosis, and recovery prediction in stroke

Stroke is a severe medical condition which may lead to permanent disability conditions. The initial 8 weeks following a stroke are crucial for rehabilitation, as most recovery occurs during this period. Personalized approaches and predictive biomarkers are needed for tailored rehabilitation. In this context, EEG brain connectivity and Artificial Intelligence (AI) can play a crucial role in diagnosing and predicting stroke outcomes efficiently. In the present study, 127 patients with subacute ischemic lesions and 90 age- and gender-matched healthy controls were enrolled. EEG recordings were obtained from each participant within 15 days of stroke onset. Clinical evaluations were performed at baseline and at 40-days follow-up using the National Institutes of Health Stroke Scale (NIHSS). Functional connectivity analysis was conducted using Total Coherence (TotCoh) and Small Word (SW). Quadratic support vector machines (SVM) algorithms were implemented to classify healthy subjects compared to stroke patients (Healthy vs Stroke), determine the affected hemisphere (Left vs Right Hemisphere), and predict functional recovery (Functional Recovery Prediction). In the classification for Functional Recovery Prediction, an accuracy of 94.75%, sensitivity of 96.27% specificity of 92.33%, and AUC of 0.95 were achieved; for Healthy vs Stroke, an accuracy of 99.09%, sensitivity of 100%, specificity of 98.46%, and AUC of 0.99 were achieved. For Left vs Right Hemisphere classification, accuracy was 86.77%, sensitivity was 91.44%, specificity was 80.33%, and AUC was 0.87. These findings highlight the potential of utilizing functional connectivity measures based on EEG in combination with AI algorithms to improve patient outcomes by targeted rehabilitation interventions.

Monitoring pilot trainees' cognitive control under a simulator-based training process with EEG microstate analysis

The objective of pilot training is to equip trainees with the knowledge, judgment, and skills to maintain control of an aircraft and respond to critical flight tasks. The present research aims to investigate changes in trainees' cognitive control levels during a pilot training process while they underwent basic flight maneuvers. EEG microstate analysis was applied together with spectral power features to quantitatively monitor trainees' cognitive control under varied flight tasks during different training sessions on a flight simulator. Not only could EEG data provide an objective measure of cognitive control to complement the current subjective assessments, but the application of EEG microstate analysis is particularly well-suited for capturing rapid dynamic changes in cognitive states that may happen under complex human activities in conducting flight maneuvers. Comparisons were conducted between two types of tasks and across different training stages to monitor how pilot trainees' cognitive control responds to varied flight task types and training stages. The present research provides insights into the changes in trainees' cognitive control during a pilot training process and highlights the potential of EEG microstate analysis for monitoring cognitive control.

Decoding influences of indoor temperature and light on neural activity: entropy analysis of electroencephalographic signals

Understanding the neural responses to indoor characteristics like temperature and light is crucial for comprehending how the physical environment influences the human brain. Our study introduces an innovative approach using entropy analysis, specifically, approximate entropy (ApEn), applied to electroencephalographic (EEG) signals to investigate neural responses to temperature and light variations in indoor environments. By strategically placing electrodes over specific brain regions linked to temperature and light processing, we show how ApEn can be influenced by indoor factors. We also integrate heart indices from a multi-sensor bracelet to create a machine learning classifier for temperature conditions. Results showed that in anterior frontal and temporoparietal areas, neutral temperature conditions yield higher ApEn values. The anterior frontal area showed a trend of gradually decreasing ApEn values from neutral to warm conditions, with cold being in an intermediate position. There was a significant interaction between light and site factors, only evident in the temporoparietal region. Here, the neutral light condition had higher ApEn values compared to blue and red light conditions. Positive correlations between anterior frontal ApEn and thermal comfort scores suggest a link between entropy and perceived thermal comfort. Our quadratic SVM classifier, incorporating entropy and heart features, demonstrates strong performance (until 90% in terms of AUC, accuracy, sensitivity, and specificity) in classifying temperature sensations. This study offers insights into neural responses to indoor factors and presents a novel approach for temperature classification using EEG entropy and heart features.

Investigation of Deficits in Auditory Emotional Content Recognition by Adult Cochlear Implant Users through the Study of Electroencephalographic Gamma and Alpha Asymmetry and Alexithymia Assessment

BACKGROUND/OBJECTIVES

Given the importance of emotion recognition for communication purposes, and the impairment for such skill in CI users despite impressive language performances, the aim of the present study was to investigate the neural correlates of emotion recognition skills, apart from language, in adult unilateral CI (UCI) users during a music in noise (happy/sad) recognition task. Furthermore, asymmetry was investigated through electroencephalographic (EEG) rhythm, given the traditional concept of hemispheric lateralization for emotional processing, and the intrinsic asymmetry due to the clinical UCI condition.

METHODS

Twenty adult UCI users and eight normal hearing (NH) controls were recruited. EEG gamma and alpha band power was assessed as there is evidence of a relationship between gamma and emotional response and between alpha asymmetry and tendency to approach or withdraw from stimuli. The TAS-20 questionnaire (alexithymia) was completed by the participants.

RESULTS

The results showed no effect of background noise, while supporting that gamma activity related to emotion processing shows alterations in the UCI group compared to the NH group, and that these alterations are also modulated by the etiology of deafness. In particular, relative higher gamma activity in the CI side corresponds to positive processes, correlated with higher emotion recognition abilities, whereas gamma activity in the non-CI side may be related to positive processes inversely correlated with alexithymia and also inversely correlated with age; a correlation between TAS-20 scores and age was found only in the NH group.

CONCLUSIONS

EEG gamma activity appears to be fundamental to the processing of the emotional aspect of music and also to the psychocognitive emotion-related component in adults with CI.

Limited Consistency and Strength of Neural Oscillations During Sustained Visual Attention in Schizophrenia

BACKGROUND

Neural oscillations support perception, attention, and higher-order decision making. Aberrations in the strength or consistency of these oscillations in response to stimuli may underlie impaired visual perception and attention in schizophrenia. Here, we examined the phase and power of alpha oscillations (8-12 Hz) as well as aspects of beta and theta frequency oscillations during a demanding visual sustained attention task.

METHODS

Patients with schizophrenia (n = 74) and healthy control participants (n = 68) completed the degraded stimulus continuous performance task during electroencephalography. We used time-frequency analysis to evaluate the consistency (intertrial phase coherence) of the alpha cycle shortly after stimulus presentation (50-250 ms). For oscillation strength, we examined event-related desynchronization in a later window associated with decision making (360-700 ms).

RESULTS

Alpha intertrial phase coherence was reduced in schizophrenia, and similar reductions were observed in theta (4-7 Hz) and beta (13-20 Hz), suggesting a lack of responsiveness in slower oscillations to visual stimuli. Alpha and beta event-related desynchronization were also reduced in schizophrenia and associated with worse task performance, increased symptoms, and poorer cognition, suggesting that limited responsiveness of oscillations is related to impairments in the disorder. Individuals with lower intertrial phase coherence had slower resting-state alpha rhythms consistent with dysfunctional oscillations persisting across default and task-related brain states.

CONCLUSIONS

In schizophrenia, abnormalities in the phase consistency and strength of slower oscillations during visual perception are related to symptoms and cognitive functioning. Altered visual perception and impaired attention in the disorder may be the consequence of aberrant slower oscillations that fail to dynamically reset and modulate in response to stimuli.

A Neuroergonomic Approach Fostered by Wearable EEG for the Multimodal Assessment of Drivers Trainees

When assessing trainees' progresses during a driving training program, instructors can only rely on the evaluation of a trainee's explicit behavior and their performance, without having any insight about the training effects at a cognitive level. However, being able to drive does not imply knowing how to drive safely in a complex scenario such as the road traffic. Indeed, the latter point involves mental aspects, such as the ability to manage and allocate one's mental effort appropriately, which are difficult to assess objectively. In this scenario, this study investigates the validity of deploying an electroencephalographic neurometric of mental effort, obtained through a wearable electroencephalographic device, to improve the assessment of the trainee. The study engaged 22 young people, without or with limited driving experience. They were asked to drive along five different but similar urban routes, while their brain activity was recorded through electroencephalography. Moreover, driving performance, subjective and reaction times measures were collected for a multimodal analysis. In terms of subjective and performance measures, no driving improvement could be detected either through the driver's subjective measures or through their driving performance. On the other side, through the electroencephalographic neurometric of mental effort, it was possible to catch their improvement in terms of mental performance, with a decrease in experienced mental demand after three repetitions of the driving training tasks. These results were confirmed by the analysis of reaction times, that significantly improved from the third repetition as well. Therefore, being able to measure when a task is less mentally demanding, and so more automatic, allows to deduce the degree of users training, becoming capable of handling additional tasks and reacting to unexpected events.

Music in noise recognition: An EEG study of listening effort in cochlear implant users and normal hearing controls

Despite the plethora of studies investigating listening effort and the amount of research concerning music perception by cochlear implant (CI) users, the investigation of the influence of background noise on music processing has never been performed. Given the typical speech in noise recognition task for the listening effort assessment, the aim of the present study was to investigate the listening effort during an emotional categorization task on musical pieces with different levels of background noise. The listening effort was investigated, in addition to participants' ratings and performances, using EEG features known to be involved in such phenomenon, that is alpha activity in parietal areas and in the left inferior frontal gyrus (IFG), that includes the Broca's area. Results showed that CI users performed worse than normal hearing (NH) controls in the recognition of the emotional content of the stimuli. Furthermore, when considering the alpha activity corresponding to the listening to signal to noise ratio (SNR) 5 and SNR10 conditions subtracted of the activity while listening to the Quiet condition-ideally removing the emotional content of the music and isolating the difficulty level due to the SNRs- CI users reported higher levels of activity in the parietal alpha and in the homologous of the left IFG in the right hemisphere (F8 EEG channel), in comparison to NH. Finally, a novel suggestion of a particular sensitivity of F8 for SNR-related listening effort in music was provided.

Prediction of mental effort derived from an automated vocal biomarker using machine learning in a large-scale remote sample

Introduction

Biomarkers of mental effort may help to identify subtle cognitive impairments in the absence of task performance deficits. Here, we aim to detect mental effort on a verbal task, using automated voice analysis and machine learning.

Methods

Audio data from the digit span backwards task were recorded and scored with automated speech recognition using the online platform NeuroVocalixTM, yielding usable data from 2,764 healthy adults (1,022 male, 1,742 female; mean age 31.4 years). Acoustic features were aggregated across each trial and normalized within each subject. Cognitive load was dichotomized for each trial by categorizing trials at >0.6 of each participants' maximum span as "high load." Data were divided into training (60%), test (20%), and validate (20%) datasets, each containing different participants. Training and test data were used in model building and hyper-parameter tuning. Five classification models (Logistic Regression, Naive Bayes, Support Vector Machine, Random Forest, and Gradient Boosting) were trained to predict cognitive load ("high" vs. "low") based on acoustic features. Analyses were limited to correct responses. The model was evaluated using the validation dataset, across all span lengths and within the subset of trials with a four-digit span. Classifier discriminant power was examined with Receiver Operating Curve (ROC) analysis.

Results

Participants reached a mean span of 6.34 out of 8 items (SD = 1.38). The Gradient Boosting classifier provided the best performing model on test data (AUC = 0.98) and showed excellent discriminant power for cognitive load on the validation dataset, across all span lengths (AUC = 0.99), and for four-digit only utterances (AUC = 0.95).

Discussion

A sensitive biomarker of mental effort can be derived from vocal acoustic features in remotely administered verbal cognitive tests. The use-case of this biomarker for improving sensitivity of cognitive tests to subtle pathology now needs to be examined.

Raw Electroencephalogram-Based Cognitive Workload Classification Using Directed and Nondirected Functional Connectivity Analysis and Deep Learning

With the phenomenal rise in internet-of-things devices, the use of electroencephalogram (EEG) based brain-computer interfaces (BCIs) can empower individuals to control equipment with thoughts. These allow BCI to be used and pave the way for pro-active health management and the development of internet-of-medical-things architecture. However, EEG-based BCIs have low fidelity, high variance, and EEG signals are very noisy. These challenges compel researchers to design algorithms that can process big data in real-time while being robust to temporal variations and other variations in the data. Another issue in designing a passive BCI is the regular change in user's cognitive state (measured through cognitive workload). Though considerable amount of research has been conducted on this front, methods that could withstand high variability in EEG data and still reflect the neuronal dynamics of cognitive state variations are lacking and much needed in literature. In this research, we evaluate the efficacy of a combination of functional connectivity algorithms and state-of-the-art deep learning algorithms for the classification of three different levels of cognitive workload. We acquire 64-channel EEG data from 23 participants executing the n-back task at three different levels; 1-back (low-workload condition), 2-back (medium-workload condition), and 3-back (high-workload condition). We compared two different functional connectivity algorithms, namely phase transfer entropy (PTE) and mutual information (MI). PTE is a directed functional connectivity algorithm, whereas MI is non-directed. Both methods are suitable for extracting functional connectivity matrices in real-time, which could eventually be used for rapid, robust, and efficient classification. For classification, we use the recently proposed BrainNetCNN deep learning model, designed specifically to classify functional connectivity matrices. Results reveal a classification accuracy of 92.81% with MI and BrainNetCNN and a staggering 99.50% with PTE and BrainNetCNN on test data. PTE can yield a higher classification accuracy due to its robustness to linear mixing of the data and its ability to detect functional connectivity across a range of analysis lags.

Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities

Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.

Do you like it or not? Identifying preference using an electroencephalogram during the viewing of short videos

Accurately predicting whether a short video will be liked by viewers is a topic of interest to media researchers. This study used an electroencephalogram (EEG) to record neural activity in 109 participants as they watched short videos (16 clips per person) to see which neural signals reflected viewers' preferences. The results showed that, compared with the short videos they disliked, individuals would experience positive emotions [indexed by a higher theta power, lower (beta - theta)/(beta + theta) score], more relaxed states (indexed by a lower beta power), lower levels of mental engagement and alertness [indexed by a lower beta/(alpha + theta) score], and devote more attention (indexed by lower alpha/theta) when watching short videos they liked. We further used artificial neural networks to classify the neural signals of different preferences induced by short videos. The classification accuracy was the highest when using data from bands over the whole brain, which was 75.78%. These results may indicate the potential of EEG measurement to evaluate the subjective preferences of individuals for short videos.

Virtual and Reality: A Neurophysiological Pilot Study of the Sarcophagus of the Spouses

Art experience is not solely the observation of artistic objects, but great relevance is also placed on the environment in which the art experience takes place, often in museums and galleries. Interestingly, in the last few years, the introduction of some forms of virtual reality (VR) in museum contexts has been increasing. This has solicited enormous research interest in investigating any eventual differences between looking at the same artifact either in a real context (e.g. a museum) and in VR. To address such a target, a neuroaesthetic study was performed in which electroencephalography (EEG) and autonomic signals (heart rate and skin conductance) were recorded during the observation of the Etruscan artifact "Sarcophagus of the Spouses", both in the museum and in a VR reproduction. Results from EEG analysis showed a higher level of the Workload Index during observation in the museum compared to VR (p = 0.04), while the Approach-Withdrawal Index highlighted increased levels during the observation in VR compared to the observation in the museum (p = 0.03). Concerning autonomic indices, the museum elicited a higher Emotional Index response than the VR (p = 0.03). Overall, preliminary results suggest a higher engagement potential of the museum compared to VR, although VR could also favour higher embodiment than the museum.

Task difficulty and physiological measures of mental workload in air traffic control: a scoping review

This study provides a systematic synthesis of empirical research on mental workload (MWL) in air traffic control (ATC). MWL is a key concept in research on innovative technologies, because the assessment of MWL is crucial to the evaluation of such technologies. Our specific focus was on physiological measures of MWL. The used search strategy identified 39 peer-reviewed publications that analysed ATC tasks, examined different levels of difficulty of the ATC task, and considered at least one physiological measure of MWL. Positive relations between measures of MWL and task difficulty were observed most frequently, indicating that the measures indeed allowed the assessment of MWL. The most commonly used physiological measures were brain measures (EEG and fNIR) and heart rate measures. The review revealed a need for more precise descriptions of crucial experimental parameters in order to permit a transition of the field towards more interactive and dynamic types of analysis. Practitioner summary: Research on innovative technology in air traffic control (ATC) depends on assessments of mental workload (MWL). We reviewed empirical research on MWL in ATC. Brain and heart measures often allow assessments of MWL. Better descriptions of experiments are needed to allow comparisons among studies and more dynamic and interactive analyses.

Wearable Cardiorespiratory Sensors for Aerospace Applications

Emerging Air Traffic Management (ATM) and avionics human-machine system concepts require the real-time monitoring of the human operator to support novel task assessment and system adaptation features. To realise these advanced concepts, it is essential to resort to a suite of sensors recording neurophysiological data reliably and accurately. This article presents the experimental verification and performance characterisation of a cardiorespiratory sensor for ATM and avionics applications. In particular, the processed physiological measurements from the designated commercial device are verified against clinical-grade equipment. Compared to other studies which only addressed physical workload, this characterisation was performed also looking at cognitive workload, which poses certain additional challenges to cardiorespiratory monitors. The article also addresses the quantification of uncertainty in the cognitive state estimation process as a function of the uncertainty in the input cardiorespiratory measurements. The results of the sensor verification and of the uncertainty propagation corroborate the basic suitability of the commercial cardiorespiratory sensor for the intended aerospace application but highlight the relatively poor performance in respiratory measurements during a purely mental activity.

Air Force Pilot Expertise Assessment with Regard to Mental Effort Requested during Unusual Attitude Recovery Flight Training Simulations

Pilot training and expertise are key aspects in aviation. A traditional way of evaluating pilot expertise is to measure performance output. However, this approach provides a narrow view of the pilot’s capacity, especially with regard to mental and emotional profile. The aim of this study is hence to investigate whether neurophysiological data can be employed as an additional objective measure to assess the expertise of pilots. In this regard, it has been demonstrated that mental effort can be used as an indirect measure of operator expertise and capacity. An increase in mental effort, for instance, can automatically result in a decrease in the remaining capacity of the operator. To better investigate this aspect, we ask two groups of Italian Air Force pilots, experienced (Experts) and unexperienced (Novices), to undergo unusual attitude recovery flight training simulations. Their behavioral (unusual attitude recovery time), subjective (mental effort demand perception) and neurophysiological data (Electroencephalogram, EEG; Electrocardiogram, ECG) are collected during the entire flight simulations. Although the two groups do not exhibit differences in terms of unusual attitude recovery time and mental effort demand perception, the EEG-based mental effort index shows how Novices request significantly higher mental effort during unusual conditions.

Simultaneous Classification of Both Mental Workload and Stress Level Suitable for an Online Passive Brain-Computer Interface

Research studies on EEG-based mental workload detection for a passive BCI generally focus on classifying cognitive states associated with the performance of tasks at different levels of difficulty, with no other aspects of the user's mental state considered. However, in real-life situations, different aspects of the user's state such as their cognitive (e.g., level of mental workload) and affective (e.g., level of stress/anxiety) states will often change simultaneously, and performance of a BCI system designed considering just one state may be unreliable. Moreover, multiple mental states may be relevant to the purposes of the BCI-for example both mental workload and stress level might be related to an aircraft pilot's risk of error-and the simultaneous prediction of states may be critical in maximizing the practical effectiveness of real-life online BCI systems. In this study we investigated the feasibility of performing simultaneous classification of mental workload and stress level in an online passive BCI. We investigated both subject-specific and cross-subject classification approaches, the latter with and without the application of a transfer learning technique to align the distributions of data from the training and test subjects. Using cross-subject classification with transfer learning in a simulated online analysis, we obtained accuracies of 77.5 ± 6.9% and 84.1 ± 5.9%, across 18 participants for mental workload and stress level detection, respectively.

Neurophysiological Verbal Working Memory Patterns in Children: Searching for a Benchmark of Modality Differences in Audio/Video Stimuli Processing

Exploration of specific brain areas involved in verbal working memory (VWM) is a powerful but not widely used tool for the study of different sensory modalities, especially in children. In this study, for the first time, we used electroencephalography (EEG) to investigate neurophysiological similarities and differences in response to the same verbal stimuli, expressed in the auditory and visual modality during the n-back task with varying memory load in children. Since VWM plays an important role in learning ability, we wanted to investigate whether children elaborated the verbal input from auditory and visual stimuli through the same neural patterns and if performance varies depending on the sensory modality. Performance in terms of reaction times was better in visual than auditory modality (p = 0.008) and worse as memory load increased regardless of the modality (p < 0.001). EEG activation was proportionally influenced by task level and was evidenced in theta band over the prefrontal cortex (p = 0.021), along the midline (p = 0.003), and on the left hemisphere (p = 0.003). Differences in the effects of the two modalities were seen only in gamma band in the parietal cortices (p = 0.009). The values of a brainwave-based engagement index, innovatively used here to test children in a dual-modality VWM paradigm, varied depending on n-back task level (p = 0.001) and negatively correlated (p = 0.002) with performance, suggesting its computational effectiveness in detecting changes in mental state during memory tasks involving children. Overall, our findings suggest that auditory and visual VWM involved the same brain cortical areas (frontal, parietal, occipital, and midline) and that the significant differences in cortical activation in theta band were more related to memory load than sensory modality, suggesting that VWM function in the child's brain involves a cross-modal processing pattern.

Smoke signals: A study of the neurophysiological reaction of smokers and non-smokers to smoking cues inserted into antismoking public service announcements

Tobacco addiction is one of the biggest health emergencies in the world, Antismoking Public Service Announcements (PSAs) represent the main public tool against smoking; however, smoking-related cues (SCs) often included in PSAs can trigger ambiguous cerebral reactions that could impact the persuasiveness and efficacy of the antismoking message. This study aimed to investigate the electroencephalographic (EEG) response in adult smokers and non-smokers during the exposure to SCs presented in antismoking PSAs video, in order to identify eventual neurophysiological features of SCs' 'boomerang effect' elicited in smokers. EEG frontal Alpha asymmetry and frontal Theta were analyzed in 92 adults (30 no smokers, 31 low smokers, 31 high smokers) from EEG recorded during the vision of 3 antismoking PSAs, statistical analysis was conducted using ANOVA. Main results showed a significant interaction between smoking cue condition (Pre and Post) and smoking habit (in particular for female heavy smokers) for the frontal Alpha asymmetry. Since the relative higher right frontal Alpha activity is associated with approach towards a stimulus, it is suggested that the relative left frontal Alpha increase in response to SCs might reflect an appetitive approach in response to it. In the light of the Incentive Sensitization Theory, this pattern can be interpreted as a neurophysiological signal in response to SCs that could undermine the message's effectiveness contributing to the maintenance of the addiction.

Explanation of Machine-Learning Solutions in Air-Traffic Management

Advances in the trusted autonomy of air-traffic management (ATM) systems are currently being pursued to cope with the predicted growth in air-traffic densities in all classes of airspace. Highly automated ATM systems relying on artificial intelligence (AI) algorithms for anomaly detection, pattern identification, accurate inference, and optimal conflict resolution are technically feasible and demonstrably able to take on a wide variety of tasks currently accomplished by humans. However, the opaqueness and inexplicability of most intelligent algorithms restrict the usability of such technology. Consequently, AI-based ATM decision-support systems (DSS) are foreseen to integrate eXplainable AI (XAI) in order to increase interpretability and transparency of the system reasoning and, consequently, build the human operators’ trust in these systems. This research presents a viable solution to implement XAI in ATM DSS, providing explanations that can be appraised and analysed by the human air-traffic control operator (ATCO). The maturity of XAI approaches and their application in ATM operational risk prediction is investigated in this paper, which can support both existing ATM advisory services in uncontrolled airspace (Classes E and F) and also drive the inflation of avoidance volumes in emerging performance-driven autonomy concepts. In particular, aviation occurrences and meteorological databases are exploited to train a machine learning (ML)-based risk-prediction tool capable of real-time situation analysis and operational risk monitoring. The proposed approach is based on the XGBoost library, which is a gradient-boost decision tree algorithm for which post-hoc explanations are produced by SHapley Additive exPlanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME). Results are presented and discussed, and considerations are made on the most promising strategies for evolving the human–machine interactions (HMI) to strengthen the mutual trust between ATCO and systems. The presented approach is not limited only to conventional applications but also suitable for UAS-traffic management (UTM) and other emerging applications.

Wearable Technologies for Mental Workload, Stress, and Emotional State Assessment during Working-Like Tasks: A Comparison with Laboratory Technologies

The capability of monitoring user’s performance represents a crucial aspect to improve safety and efficiency of several human-related activities. Human errors are indeed among the major causes of work-related accidents. Assessing human factors (HFs) could prevent these accidents through specific neurophysiological signals’ evaluation but laboratory sensors require highly-specialized operators and imply a certain grade of invasiveness which could negatively interfere with the worker’s activity. On the contrary, consumer wearables are characterized by their ease of use and their comfortability, other than being cheaper compared to laboratory technologies. Therefore, wearable sensors could represent an ideal substitute for laboratory technologies for a real-time assessment of human performances in ecological settings. The present study aimed at assessing the reliability and capability of consumer wearable devices (i.e., Empatica E4 and Muse 2) in discriminating specific mental states compared to laboratory equipment. The electrooculographic (EOG), electrodermal activity (EDA) and photoplethysmographic (PPG) signals were acquired from a group of 17 volunteers who took part to the experimental protocol in which different working scenarios were simulated to induce different levels of mental workload, stress, and emotional state. The results demonstrated that the parameters computed by the consumer wearable and laboratory sensors were positively and significantly correlated and exhibited the same evidences in terms of mental states discrimination.

Spectral Resting-State EEG (rsEEG) in Chronic Aphasia Is Reliable, Sensitive, and Correlates With Functional Behavior

We investigated spectral resting-state EEG in persons with chronic stroke-induced aphasia to determine its reliability, sensitivity, and relationship to functional behaviors. Resting-state EEG has not yet been characterized in this population and was selected given the demonstrated potential of resting-state investigations using other neuroimaging techniques to guide clinical decision-making. Controls and persons with chronic stroke-induced aphasia completed two EEG recording sessions, separated by approximately 1 month, as well as behavioral assessments of language, sensorimotor, and cognitive domains. Power in the classic frequency bands (delta, theta, alpha, and beta) was examined via spectral analysis of resting-state EEG data. Results suggest that power in the theta, alpha, and beta bands is reliable for use as a repeated measure. Significantly greater theta and lower beta power was observed in persons with aphasia (PWAs) than controls. Finally, in PWAs theta power negatively correlated with performance on a discourse informativeness measure, while alpha and beta power positively correlated with performance on the same measure. This indicates that spectral rsEEG slowing observed in PWAs in the chronic stage is pathological and suggests a possible avenue for directly altering brain activation to improve behavioral function. Taken together, these results suggest that spectral resting-state EEG holds promise for sensitive measurement of functioning and change in persons with chronic aphasia. Future studies investigating the utility of these measures as biomarkers of frank or latent aphasic deficits and treatment response in chronic stroke-induced aphasia are warranted.

Higher Right Hemisphere Gamma Band Lateralization and Suggestion of a Sensitive Period for Vocal Auditory Emotional Stimuli Recognition in Unilateral Cochlear Implant Children: An EEG Study

In deaf children, huge emphasis was given to language; however, emotional cues decoding and production appear of pivotal importance for communication capabilities. Concerning neurophysiological correlates of emotional processing, the gamma band activity appears a useful tool adopted for emotion classification and related to the conscious elaboration of emotions. Starting from these considerations, the following items have been investigated: (i) whether emotional auditory stimuli processing differs between normal-hearing (NH) children and children using a cochlear implant (CI), given the non-physiological development of the auditory system in the latter group; (ii) whether the age at CI surgery influences emotion recognition capabilities; and (iii) in light of the right hemisphere hypothesis for emotional processing, whether the CI side influences the processing of emotional cues in unilateral CI (UCI) children. To answer these matters, 9 UCI (9.47 ± 2.33 years old) and 10 NH (10.95 ± 2.11 years old) children were asked to recognize nonverbal vocalizations belonging to three emotional states: positive (achievement, amusement, contentment, relief), negative (anger, disgust, fear, sadness), and neutral (neutral, surprise). Results showed better performances in NH than UCI children in emotional states recognition. The UCI group showed increased gamma activity lateralization index (LI) (relative higher right hemisphere activity) in comparison to the NH group in response to emotional auditory cues. Moreover, LI gamma values were negatively correlated with the percentage of correct responses in emotion recognition. Such observations could be explained by a deficit in UCI children in engaging the left hemisphere for more demanding emotional task, or alternatively by a higher conscious elaboration in UCI than NH children. Additionally, for the UCI group, there was no difference between the CI side and the contralateral side in gamma activity, but a higher gamma activity in the right in comparison to the left hemisphere was found. Therefore, the CI side did not appear to influence the physiologic hemispheric lateralization of emotional processing. Finally, a negative correlation was shown between the age at the CI surgery and the percentage of correct responses in emotion recognition and then suggesting the occurrence of a sensitive period for CI surgery for best emotion recognition skills development.

Objective Assessments in Diagnoses and Treatment: A Proposed Change in Paradigm

For patients with psychiatric disorders, current diagnostic and treatment approaches are far from optimal. The clinical interview drives the standard approach-matching symptoms to diagnostic criteria-and results in standardized pharmacological and behavioral treatments, often, with inadequate outcome; but now, recent imaging advances can correlate behavioral assessments with brain function and measure them against normative databases to provide data critical for the reevaluation of patient diagnosis and treatment. This article addresses the data that support a redefinition of our current paradigm. We believe a neurobehavioral approach provides for more personalized treatment approaches unbound from classically defined diagnostic biases.

NeuroDante: Poetry Mentally Engages More Experts but Moves More Non-Experts, and for Both the Cerebral Approach Tendency Goes Hand in Hand with the Cerebral Effort

Neuroaesthetics, the science studying the biological underpinnings of aesthetic experience, recently extended its area of investigation to literary art; this was the humus where neurocognitive poetics blossomed. Divina Commedia represents one of the most important, famous and studied poems worldwide. Poetry stimuli are characterized by elements (meter and rhyme) promoting the processing fluency, a core aspect of neuroaesthetics theories. In addition, given the evidence of different neurophysiological reactions between experts and non-experts in response to artistic stimuli, the aim of the present study was to investigate, in poetry, a different neurophysiological cognitive and emotional reaction between Literature (L) and Non-Literature (NL) students. A further aim was to investigate whether neurophysiological underpinnings would support explanation of behavioral data. Investigation methods employed: self-report assessments (recognition, appreciation, content recall) and neurophysiological indexes (approach/withdrawal (AW), cerebral effort (CE) and galvanic skin response (GSR)). The main behavioral results, according to fluency theories in aesthetics, suggested in the NL but not in the L group that the appreciation/liking went hand by hand with the self-declared recognition and with the content recall. The main neurophysiological results were: (i) higher galvanic skin response in NL, whilst higher CE values in L; (ii) a positive correlation between AW and CE indexes in both groups. The present results extended previous evidence relative to figurative art also to auditory poetry stimuli, suggesting an emotional attenuation “expertise-specific” showed by experts, but increased cognitive processing in response to the stimuli.

Forefront Users' Experience Evaluation by Employing Together Virtual Reality and Electroencephalography: A Case Study on Cognitive Effects of Scents

Scents have the ability to affect peoples' mental states and task performance with to different extents. It has been widely demonstrated that the lemon scent, included in most all-purpose cleaners, elicits stimulation and activation, while the lavender scent elicits relaxation and sedative effects. The present study aimed at investigating and fostering a novel approach to evaluate users' experience with respect to scents' effects through the joint employment of Virtual Reality and users' neurophysiological monitoring, in particular Electroencephalography. In particular, this study, involving 42 participants, aimed to compare the effects of lemon and lavender scents on the deployment of cognitive resources during a daily life experience consisting in a train journey carried out in virtual reality. Our findings showed a significant higher request of cognitive resources during the processing of an informative message for subjects exposed to the lavender scent with respect to the lemon exposure. No differences were found between lemon and lavender conditions on the self-reported items of pleasantness and involvement; as this study demonstrated, the employment of the lavender scent preserves the quality of the customer experience to the same extent as the more widely used lemon scent.

Toward a Subject-Independent EEG-Based Neural Indicator of Task Proficiency During Training

This study explores the feasibility of developing an EEG-based neural indicator of task proficiency based on subject-independent mental state classification. Such a neural indicator could be used in the development of a passive brain-computer interface to potentially enhance training effectiveness and efficiency. A spatial knowledge acquisition training protocol was used in this study. Fifteen participants acquired spatial knowledge in a novel virtual environment via 60 navigation trials (divided into ten blocks). Task performance (time required to complete trials), perceived task certainty, and EEG signal data were collected. For each participant, 1 s epochs of EEG data were classified as either from the "low proficiency, 0" or "high proficiency, 1" state using a support vector machine classifier trained on data from the remaining 14 participants. The average epoch classification per trial was used to calculate a neural indicator (NI) ranging from 0 ("low proficiency") to 1 ("high proficiency"). Trends in the NI throughout the session-from the first to the last trial-were analyzed using a repeated measure mixed model linear regression. There were nine participants for whom the neural indicator was quite effective in tracking the progression from low to high proficiency. These participants demonstrated a significant (p < 0.001) increase in the neural indicator throughout the training from NI = 0.15 in block 1 to NI = 0.81 (on average) in block 10, with the average NI reaching a plateau after block 7. For the remaining participants, the NI did not effectively track the progression of task proficiency. The results support the potential of a subject-independent EEG-based neural indicator of task proficiency and encourage further research toward this objective.

Adaptive Human-Robot Interactions for Multiple Unmanned Aerial Vehicles

Advances in unmanned aircraft systems (UAS) have paved the way for progressively higher levels of intelligence and autonomy, supporting new modes of operation, such as the one-to-many (OTM) concept, where a single human operator is responsible for monitoring and coordinating the tasks of multiple unmanned aerial vehicles (UAVs). This paper presents the development and evaluation of cognitive human-machine interfaces and interactions (CHMI2) supporting adaptive automation in OTM applications. A CHMI2 system comprises a network of neurophysiological sensors and machine-learning based models for inferring user cognitive states, as well as the adaptation engine containing a set of transition logics for control/display functions and discrete autonomy levels. Models of the user’s cognitive states are trained on past performance and neurophysiological data during an offline calibration phase, and subsequently used in the online adaptation phase for real-time inference of these cognitive states. To investigate adaptive automation in OTM applications, a scenario involving bushfire detection was developed where a single human operator is responsible for tasking multiple UAV platforms to search for and localize bushfires over a wide area. We present the architecture and design of the UAS simulation environment that was developed, together with various human-machine interface (HMI) formats and functions, to evaluate the CHMI2 system’s feasibility through human-in-the-loop (HITL) experiments. The CHMI2 module was subsequently integrated into the simulation environment, providing the sensing, inference, and adaptation capabilities needed to realise adaptive automation. HITL experiments were performed to verify the CHMI2 module’s functionalities in the offline calibration and online adaptation phases. In particular, results from the online adaptation phase showed that the system was able to support real-time inference and human-machine interface and interaction (HMI2) adaptation. However, the accuracy of the inferred workload was variable across the different participants (with a root mean squared error (RMSE) ranging from 0.2 to 0.6), partly due to the reduced number of neurophysiological features available as real-time inputs and also due to limited training stages in the offline calibration phase. To improve the performance of the system, future work will investigate the use of alternative machine learning techniques, additional neurophysiological input features, and a more extensive training stage.

Electro-Encephalography and Electro-Oculography in Aeronautics: A Review Over the Last Decade (2010-2020)

Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges.

A Novel Mutual Information Based Feature Set for Drivers' Mental Workload Evaluation Using Machine Learning

Analysis of physiological signals, electroencephalography more specifically, is considered a very promising technique to obtain objective measures for mental workload evaluation, however, it requires a complex apparatus to record, and thus, with poor usability in monitoring in-vehicle drivers’ mental workload. This study proposes a methodology of constructing a novel mutual information-based feature set from the fusion of electroencephalography and vehicular signals acquired through a real driving experiment and deployed in evaluating drivers’ mental workload. Mutual information of electroencephalography and vehicular signals were used as the prime factor for the fusion of features. In order to assess the reliability of the developed feature set mental workload score prediction, classification and event classification tasks were performed using different machine learning models. Moreover, features extracted from electroencephalography were used to compare the performance. In the prediction of mental workload score, expert-defined scores were used as the target values. For classification tasks, true labels were set from contextual information of the experiment. An extensive evaluation of every prediction tasks was carried out using different validation methods. In predicting the mental workload score from the proposed feature set lowest mean absolute error was 0.09 and for classifying mental workload highest accuracy was 94%. According to the outcome of the study, it can be stated that the novel mutual information based features developed through the proposed approach can be employed to classify and monitor in-vehicle drivers’ mental workload.

P300 Measures and Drive-Related Risks: A Systematic Review and Meta-Analysis

Detecting signs for an increased level of risk during driving are critical for the effective prevention of road traffic accidents. The current study searched for literature through major databases such as PubMed, EBSCO, IEEE, and ScienceDirect. A total of 14 articles that measured P300 components in relation to driving tasks were included for a systematic review and meta-analysis. The risk factors investigated in the reviewed articles were summarized in five categories, including reduced attention, distraction, alcohol, challenging situations on the road, and negative emotion. A meta-analysis was conducted at both behavioral and neural levels. Behavioral performance was measured by the reaction time and driving performance, while the neural response was measured by P300 amplitude and latency. A significant increase in reaction time was identified when drivers were exposed to the risk factors. In addition, the significant effects of a reduced P300 amplitude and prolonged P300 latency indicated a reduced capacity for cognitive information processing. There was a tendency of driving performance decrement in relation to the risk factors, however, the effect was non-significant due to considerable variations and heterogeneity across the included studies. The results led to the conclusion that the P300 amplitude and latency are reliable indicators and predictors of the increased risk in driving. Future applications of the P300-based brain–computer interface (BCI) system may make considerable contributions toward preventing road traffic accidents.

A multimodal and signals fusion approach for assessing the impact of stressful events on Air Traffic Controllers

Stress is a word used to describe human reactions to emotionally, cognitively and physically challenging experiences. A hallmark of the stress response is the activation of the autonomic nervous system, resulting in the "fight-freeze-flight" response to a threat from a dangerous situation. Consequently, the capability to objectively assess and track a controller's stress level while dealing with air traffic control (ATC) activities would make it possible to better tailor the work shift and maintain high safety levels, as well as to preserve the operator's health. In this regard, sixteen controllers were asked to perform a realistic air traffic management (ATM) simulation during which subjective data (i.e. stress perception) and neurophysiological data (i.e. brain activity, heart rate, and galvanic skin response) were collected with the aim of accurately characterising the controller's stress level experienced in the various experimental conditions. In addition, external supervisors regularly evaluated the controllers in terms of manifested stress, safety, and efficiency throughout the ATM scenario. The results demonstrated 1) how the stressful events caused both supervisors and controllers to underestimate the experienced stress level, 2) the advantage of taking into account both cognitive and hormonal processes in order to define a reliable stress index, and 3) the importance of the points in time at which stress is measured owing to the potential transient effect once the stressful events have ceased.

Indexing Mental Workload During Simulated Air Traffic Control Tasks by Means of Dual Frequency Head Maps

In our digitized society, advanced information and communication technology and highly interactive work environments impose high demands on cognitive capacity. Optimal workload conditions are important for assuring employee's health and safety of other persons. This is particularly relevant in safety-critical occupations, such as air traffic control. For measuring mental workload using the EEG, we have developed the method of Dual Frequency Head Maps (DFHM). The method was tested and validated already under laboratory conditions. However, validation of the method regarding reliability and reproducibility of results under realistic settings and real world scenarios was still required. In our study, we examined 21 air traffic controllers during arrival management tasks. Mental workload variations were achieved by simulation scenarios with different number of aircraft and the occurrence of a priority-flight request as an exceptional event. The workload was assessed using the EEG-based DFHM-workload index and instantaneous self-assessment questionnaire. The DFHM-workload index gave stable results with highly significant correlations between scenarios with similar traffic-load conditions (r between 0.671 and 0.809, p ≤ 0.001). For subjects reporting that they experienced workload variation between the different scenarios, the DFHM-workload index yielded significant differences between traffic-load levels and priority-flight request conditions. For subjects who did not report to experience workload variations between the scenarios, the DFHM-workload index did not yield any significant differences for any of the factors. We currently conclude that the DFHM-workload index reveals potential for applications outside the laboratory and yields stable results without retraining of the classifiers neither regarding new subjects nor new tasks.

Neurophysiological Vigilance Characterisation and Assessment: Laboratory and Realistic Validations Involving Professional Air Traffic Controllers

Vigilance degradation usually causes significant performance decrement. It is also considered the major factor causing the out-of-the-loop phenomenon (OOTL) occurrence. OOTL is strongly related to a high level of automation in operative contexts such as the Air Traffic Management (ATM), and it could lead to a negative impact on the Air Traffic Controllers’ (ATCOs) engagement. As a consequence, being able to monitor the ATCOs’ vigilance would be very important to prevent risky situations. In this context, the present study aimed to characterise and assess the vigilance level by using electroencephalographic (EEG) measures. The first study, involving 13 participants in laboratory settings allowed to find out the neurophysiological features mostly related to vigilance decrements. Those results were also confirmed under realistic ATM settings recruiting 10 professional ATCOs. The results demonstrated that (i) there was a significant performance decrement related to vigilance reduction; (ii) there were no substantial differences between the identified neurophysiological features in controlled and ecological settings, and the EEG-channel configuration defined in laboratory was able to discriminate and classify vigilance changes in ATCOs’ vigilance with high accuracy (up to 84%); (iii) the derived two EEG-channel configuration was able to assess vigilance variations reporting only slight accuracy reduction.

A Comparison of Mental Workload in Individuals with Transtibial and Transfemoral Lower Limb Loss during Dual-Task Walking under Varying Demand

OBJECTIVES

This study aimed to evaluate the influence of lower limb loss (LL) on mental workload by assessing neurocognitive measures in individuals with unilateral transtibial (TT) versus those with transfemoral (TF) LL while dual-task walking under varying cognitive demand.

METHODS

Electroencephalography (EEG) was recorded as participants performed a task of varying cognitive demand while being seated or walking (i.e., varying physical demand).

RESULTS

The findings revealed both groups of participants (TT LL vs. TF LL) exhibited a similar EEG theta synchrony response as either the cognitive or the physical demand increased. Also, while individuals with TT LL maintained similar performance on the cognitive task during seated and walking conditions, those with TF LL exhibited performance decrements (slower response times) on the cognitive task during the walking in comparison to the seated conditions. Furthermore, those with TF LL neither exhibited regional differences in EEG low-alpha power while walking, nor EEG high-alpha desynchrony as a function of cognitive task difficulty while walking. This lack of alpha modulation coincided with no elevation of theta/alpha ratio power as a function of cognitive task difficulty in the TF LL group.

CONCLUSIONS

This work suggests that both groups share some common but also different neurocognitive features during dual-task walking. Although all participants were able to recruit neural mechanisms critical for the maintenance of cognitive-motor performance under elevated cognitive or physical demands, the observed differences indicate that walking with a prosthesis, while concurrently performing a cognitive task, imposes additional cognitive demand in individuals with more proximal levels of amputation.

Consumer Behaviour through the Eyes of Neurophysiological Measures: State-of-the-Art and Future Trends

The new technological advances achieved during the last decade allowed the scientific community to investigate and employ neurophysiological measures not only for research purposes but also for the study of human behaviour in real and daily life situations. The aim of this review is to understand how and whether neuroscientific technologies can be effectively employed to better understand the human behaviour in real decision-making contexts. To do so, firstly, we will describe the historical development of neuromarketing and its main applications in assessing the sensory perceptions of some marketing and advertising stimuli. Then, we will describe the main neuroscientific tools available for such kind of investigations (e.g., measuring the cerebral electrical or hemodynamic activity, the eye movements, and the psychometric responses). Also, this review will present different brain measurement techniques, along with their pros and cons, and the main cerebral indexes linked to the specific mental states of interest (used in most of the neuromarketing research). Such indexes have been supported by adequate validations from the scientific community and are largely employed in neuromarketing research. This review will also discuss a series of papers that present different neuromarketing applications, such us in-store choices and retail, services, pricing, brand perception, web usability, neuropolitics, evaluation of the food and wine taste, and aesthetic perception of artworks. Furthermore, this work will face the ethical issues arisen on the use of these tools for the evaluation of the human behaviour during decision-making tasks. In conclusion, the main challenges that neuromarketing is going to face, as well as future directions and possible scenarios that could be derived by the use of neuroscience in the marketing field, will be identified and discussed.

Brain-Computer Interface-Based Adaptive Automation to Prevent Out-Of-The-Loop Phenomenon in Air Traffic Controllers Dealing With Highly Automated Systems

Increasing the level of automation in air traffic management is seen as a measure to increase the performance of the service to satisfy the predicted future demand. This is expected to result in new roles for the human operator: he will mainly monitor highly automated systems and seldom intervene. Therefore, air traffic controllers (ATCos) would often work in a supervisory or control mode rather than in a direct operating mode. However, it has been demonstrated how human operators in such a role are affected by human performance issues, known as Out-Of-The-Loop (OOTL) phenomenon, consisting in lack of attention, loss of situational awareness and de-skilling. A countermeasure to this phenomenon has been identified in the adaptive automation (AA), i.e., a system able to allocate the operative tasks to the machine or to the operator depending on their needs. In this context, psychophysiological measures have been highlighted as powerful tool to provide a reliable, unobtrusive and real-time assessment of the ATCo's mental state to be used as control logic for AA-based systems. In this paper, it is presented the so-called "Vigilance and Attention Controller", a system based on electroencephalography (EEG) and eye-tracking (ET) techniques, aimed to assess in real time the vigilance level of an ATCo dealing with a highly automated human-machine interface and to use this measure to adapt the level of automation of the interface itself. The system has been tested on 14 professional ATCos performing two highly realistic scenarios, one with the system disabled and one with the system enabled. The results confirmed that (i) long high automated tasks induce vigilance decreasing and OOTL-related phenomena; (ii) EEG measures are sensitive to these kinds of mental impairments; and (iii) AA was able to counteract this negative effect by keeping the ATCo more involved within the operative task. The results were confirmed by EEG and ET measures as well as by performance and subjective ones, providing a clear example of potential applications and related benefits of AA.