Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness

Cognitive load during driving: EEG microstate metrics are sensitive to task difficulty and predict safety outcomes

Engaging in phone conversations or other cognitively challenging tasks while driving detrimentally impacts cognitive functions and has been associated with increased risk of accidents. Existing EEG methods have been shown to differentiate between load and no load, but not between different levels of cognitive load. Furthermore, it has not been investigated whether EEG measurements of load can be used to predict safety outcomes in critical events. EEG microstates analysis, categorizing EEG signals into a concise set of prototypical functional states, has been used in other task contexts with good results, but has not been applied in the driving context. Here, this gap is addressed by means of a driving simulation experiment. Three phone use conditions (no phone use, hands-free, and handheld), combined with two task difficulty levels (single- or double-digit addition and subtraction), were tested before and during a rear-end collision conflict. Both conventional EEG spectral power and EEG microstates were analyzed. The results showed that different levels of cognitive load influenced EEG microstates differently, while EEG spectral power remained unaffected. A distinct EEG pattern emerged when drivers engaged in phone tasks while driving, characterized by a simultaneous increase and decrease in two of the EEG microstates, suggesting a heightened focus on auditory information, potentially at a cost to attention reorientation ability. The increase and decrease in these two microstates follow a monotonic sequence from baseline to hands-free simple, hands-free complex, handheld simple, and finally handheld complex, showing sensitivity to task difficulty. This pattern was found both before and after the lead vehicle braked. Furthermore, EEG microstates prior to the lead vehicle braking improved predictions of safety outcomes in terms of minimum time headway after the lead vehicle braked, clearly suggesting that these microstates measure brain states which are indicative of impaired driving. Additionally, EEG microstates are more predictive of safety outcomes than task difficulty, highlighting individual differences in task effects. These findings enhance our understanding of the neural dynamics involved in distracted driving and can be used in methods for evaluating the cognitive load induced by in-vehicle systems.

Cardiac autonomic responses in relation to cognitive workload during simulated military flight

Understanding the operator's cognitive workload is crucial for efficiency and safety in human-machine systems. This study investigated how cognitive workload modulates cardiac autonomic regulation during a standardized military simulator flight. Military student pilots completed simulated flight tasks in a Hawk flight simulator. Continuous electrocardiography was recorded to analyze time and frequency domain heart rate variability (HRV). After the simulation, a flight instructor used a standardized method to evaluate student pilot's individual cognitive workload from video-recorded flight simulator data. Results indicated that HRV was able to differentiate flight phases that induced varying levels of cognitive workload; an increasing level of cognitive workload caused significant decreases in many HRV variables, mainly reflecting parasympathetic deactivation of cardiac autonomic regulation. In conclusion, autonomic physiological responses can be used to examine reactions to increased cognitive workload during simulated military flights. HRV could be beneficial in assessing individual responses to cognitive workload and pilot performance during simulator training.

o-CLEAN: a novel multi-stage algorithm for the ocular artifacts' correction from EEG data in out-of-the-lab applications

In the context of electroencephalographic (EEG) signal processing, artifacts generated by ocular movements, such as blinks, are significant confounding factors. These artifacts overwhelm informative EEG features and may occur too frequently to simply remove affected epochs without losing valuable data. Correcting these artifacts remains a challenge, particularly in out-of-lab and online applications using wearable EEG systems (i.e. with low number of EEG channels, without any additional channels to track EOG).Objective.The main objective of the present work consisted in validating a novel ocular blinks artefacts correction method, named multi-stage OCuLar artEfActs deNoising algorithm (o-CLEAN), suitable for online processing with minimal EEG channels.Approach.The research was conducted considering one EEG dataset collected in highly controlled environment, and a second one collected in real environment. The analysis was performed by comparing the o-CLEAN method with previously validated state-of-art techniques, and by evaluating its performance along two dimensions: (a) the ocular artefacts correction performance (IN-Blink), and (b) the EEG signal preservation when the method was applied without any ocular artefacts occurrence (OUT-Blink).Main results.Results highlighted that (i) o-CLEAN algorithm resulted to be, at least, significantly reliable as the most validated approaches identified in scientific literature in terms of ocular blink artifacts correction, (ii) o-CLEAN showed the best performances in terms of EEG signal preservation especially with a low number of EEG channels.Significance.The testing and validation of the o-CLEAN addresses a relevant open issue in bioengineering EEG processing, especially within out-of-the-lab application. In fact, the method offers an effective solution for correcting ocular artifacts in EEG signals with a low number of available channels, for online processing, and without any specific template of the EOG. It was demonstrated to be particularly effective for EEG data gathered in real environments using wearable systems, a rapidly expanding area within applied neuroscience.

Brain-computer Interaction in the Smart Era

The brain-computer interface (BCI) system serves as a critical link between external output devices and the human brain. A monitored object's mental state, sensory cognition, and even higher cognition are reflected in its electroencephalography (EEG) signal. Nevertheless, unprocessed EEG signals are frequently contaminated with a variety of artifacts, rendering the analysis and elimination of impurities from the collected EEG data exceedingly challenging, not to mention the manual adjustment thereof. Over the last few decades, the rapid advancement of artificial intelligence (AI) technology has contributed to the development of BCI technology. Algorithms derived from AI and machine learning have significantly enhanced the ability to analyze and process EEG electrical signals, thereby expanding the range of potential interactions between the human brain and computers. As a result, the present BCI technology with the help of AI can assist physicians in gaining a more comprehensive understanding of their patients' physical and psychological status, thereby contributing to improvements in their health and quality of life.

Research on brain functional network property analysis and recognition methods targeting brain fatigue

At present, researches on brain fatigue recognition are still in the stage of single task and simple brain region network features, while researches on high-order brain functional network features and brain region state mechanisms during fatigue in multi-task scenarios are still insufficient, making it difficult to meet the needs of fatigue recognition under complex conditions. Therefore, this study utilized functional near-infrared spectroscopy (fNIRS) technology to explore the correlation and differences in the low-order and high-order brain functional network attributes of three task induced mental fatigue, and to explore the brain regions that have a major impact on mental fatigue. Self-training algorithms were used to identify the three levels of brain fatigue. The results showed that during the fatigue development, the overall connection strength of the endothelial cell metabolic activity and neural activity frequency bands of the low-order brain functional network first decreased and then increased, while the myogenic activity and heart rate activity frequency bands showed the opposite pattern. Network topology analysis indicated that from no fatigue to mild fatigue, the clustering coefficient of endothelial cell metabolic activity and myogenic activity frequency bands significantly decreased, while the characteristic path length of myogenic activity significantly increased; when experiencing severe fatigue, the small-world attribute of the neural frequency band significantly weakened. However, each frequency band maintained its small-world attribute, reflecting the self-optimization and adaptability of the network during the fatigue process. During mild fatigue, neuronal activity bands' node degree, cluster coefficient, and efficiency rose in high-order brain networks, while low-order networks showed no significant changes. As fatigue progressed, the myogenic activity bands of high-order network properties dominated, but neural bands gained prominence in mild fatigue, approaching the level of myogenic bands in severe fatigue, indicating that brain fatigue orchestrated a shift from myogenic to neural dominance in frequency bands. In addition, during the process of fatigue, the four network attributes of the high-order network cluster composed of low-order nodes related to the prefrontal cortex region, left anterior motor region, motor assist region, and left frontal lobe eye movement region significantly increased, indicating that these brain regions had a significant impact on brain fatigue status. The accuracy of using both high-order and low-order features to identify fatigue levels reached 88.095%, indicating that the combined network features of both high-order and low-order fNIRS signals could effectively detect multi-level mental fatigue, providing innovative ideas for fatigue warning.

A performance-based mental workload identification method for special vehicle crews

Detecting the mental workload state of armored vehicle crews is of great significance for monitoring the driving state of the crew and improving comprehensive combat effectiveness. In this manuscript, we propose a performance-based mental workload identification method and carry out experimental validation to improve the accuracy of crew mental workload identification and realize the effective classification of mental workload. Based on the virtual simulation system of the special vehicle crew task, this manuscript selects 20 subjects for the mental workload experiment of special vehicle crews. The experiment collected NASA-TLX scale, EEG, eye-tracking data, and performance data. The results show that the mental workload of the crews fluctuates in the segmented tasks of complex operations in typical scenes of special vehicles. The method of determining mental workload using NASA-TLX generates label noise in classification, which is not suitable for special vehicle tasks. Performance-based mental workload identification method is able to recognize fluctuations in the crew's mental workload during segmented tasks. Performance-based and NASA-TXL-based methods were classified using linear discriminant analysis. The results show that the accuracy of the method based on performance is improved by 15.72 %. This manuscript found the NASA-TXL scale is not suitable for the complex tasks of special vehicles, and proposed a performance-based identification method that can help to categorize the mental workload states of special vehicle crews.

Reliable but multi-dimensional cognitive demand in operating partially automated vehicles: implications for real-world automation research

The reliability of cognitive demand measures in controlled laboratory settings is well-documented; however, limited research has directly established their stability under real-life and high-stakes conditions, such as operating automated technology on actual highways. Partially automated vehicles have advanced to become an everyday mode of transportation, and research on driving these advanced vehicles requires reliable tools for evaluating the cognitive demand on motorists to sustain optimal engagement in the driving process. This study examined the reliability of five cognitive demand measures, while participants operated partially automated vehicles on real roads across four occasions. Seventy-one participants (aged 18-64 years) drove on actual highways while their heart rate, heart rate variability, electroencephalogram (EEG) alpha power, and behavioral performance on the Detection Response Task were measured simultaneously. Findings revealed that EEG alpha power had excellent test-retest reliability, heart rate and its variability were good, and Detection Response Task reaction time and hit-rate had moderate reliabilities. Thus, the current study addresses concerns regarding the reliability of these measures in assessing cognitive demand in real-world automation research, as acceptable test-retest reliabilities were found across all measures for drivers across occasions. Despite the high reliability of each measure, low intercorrelations among measures were observed, and internal consistency was better when cognitive demand was estimated as a multi-factorial construct. This suggests that they tap into different aspects of cognitive demand while operating automation in real life. The findings highlight that a combination of psychophysiological and behavioral methods can reliably capture multi-faceted cognitive demand in real-world automation research.

Audiovisual messages may improve the processing of traffic information and driver attention during partially automated driving: An EEG study

Partially autonomous vehicles can help minimize human errors. However, being free from some driving subtasks can result in a low vigilance state, which can affect the driver's attention towards the road. The present study first tested whether drivers of partially autonomous vehicles would benefit from the addition of auditory versions of the messages presented in variable message signs (VMS), particularly, when they find themselves in a monotonous driving situation. A second aim was to test whether the addition of auditory messages would also produce an indirect effect on the driver's vigilance, improving performance on other driving subtasks not related to the message processing. Forty-three volunteers participated in a driving simulator study. They completed two tasks: (a) a VMS task, where they had to regain manual control of the car if the VMS message was critical, and (b) a car-following task, where they had to pay attention to the preceding car to respond to occasional brake events. Behavioral and EEG data were registered. Overall, results indicated that the addition of audio messages helped drivers process VMS information more effectively and maintain a higher level of vigilance throughout the driving time. These findings would provide useful information for the development of partially automated vehicles, as their design must guarantee that the driver remains attentive enough to assume control when necessary.

Research on fatigue detection of flight trainees based on face EMF feature model combination with PSO-CNN algorithm

Even though the capability of aircraft manufacturing has improved, human factors still play a pivotal role in flight accidents. For example, fatigue-related accidents are a common factor in human-led accidents. Hence, pilots' precise fatigue detections could help increase the flight safety of airplanes. The article suggests a model to recognize fatigue by implementing the convolutional neural network (CNN) by implementing flight trainees' face attributions. First, the flight trainees' face attributions are derived by a method called the land-air call process when the flight simulation is run. Then, sixty-eight points of face attributions are detected by employing the Dlib package. Fatigue attribution points were derived based on the face attribution points to construct a model called EMF to detect face fatigue. Finally, the proposed PSO-CNN algorithm is implemented to learn and train the dataset, and the network algorithm achieves a recognition ratio of 93.9% on the test set, which can efficiently pinpoint the flight trainees' fatigue level. Also, the reliability of the proposed algorithm is validated by comparing two machine learning models.

Electrophysiological correlation between executive vigilance and attention network based on cognitive resource control theory

Attention is comprised of three independent and interacting attention networks: phasic alertness, orienting, and executive control. Previous studies have explored event-related potentials associated with these attention networks and executive vigilance, there is a lack of research on the relationship between executive vigilance and the three attention networks. However, there is a lack of research on the relationship between executive vigilance and the three attention networks. The present study aims to investigate this relationship. Based on the theory of cognitive resource control, two experimental blocks were designed with the vigilance task as the control variable. A total of 39 participants completed both ANTI and ANTI-V trials (two variants of the traditional attention network test ANT) in the same period. Through analysis of behavior measures (RT) and electrophysiological results related to phasic alertness (N1, P2, and contingent negative variation), orienting (P1, N1, and P3), and executive control (N2 and slow positive potential), we found that the reaction time of the ANTI block was lower than that of the ANTI-V block under all conditions, This suggests that adding a vigilance task may lead to reduced allocation of attention resources across all three attention networks. Furthermore, the orienting ability was weaker in the ANTI-V experimental block compared to that in the ANTI block due to effects on P1 and P3 regulation by the vigilance task. The N2 amplitude of the ANTI-V block was consistently reduced under similar conditions, indicating a weakening of executive control ability. The electrophysiological results revealed that executive vigilance inhibited the component of early attention perception related to the orienting network and was also related to the ability to detect conflict in the executive control network.

The moving wave: Applications of the mobile EEG approach to study human attention

Although historically confined to traditional research laboratories, electroencephalography (EEG) paradigms are now being applied to study a wide array of behaviors, from daily activities to specialized tasks in diverse fields such as sports science, neurorehabilitation, and education. This transition from traditional to real-world mobile research can provide new tools for understanding attentional processes as they occur naturally. Early mobile EEG research has made progress, despite the large size and wired connections. Recent developments in hardware and software have expanded the possibilities of mobile EEG, enabling a broader range of applications. Despite these advancements, limitations influencing mobile EEG remain that must be overcome to achieve adequate reliability and validity. In this review, we first assess the feasibility of mobile paradigms, including electrode selection, artifact correction techniques, and methodological considerations. This review underscores the importance of ecological, construct, and predictive validity in ensuring the trustworthiness and applicability of mobile EEG findings. Second, we explore studies on attention in naturalistic settings, focusing on replicating classic P3 component studies in mobile paradigms like stationary biking in our lab, and activities such as walking, cycling, and dual-tasking outside of the lab. We emphasize how the mobile approach complements traditional laboratory paradigms and the types of insights gained in naturalistic research settings. Third, we discuss promising applications of portable EEG in workplace safety and other areas including road safety, rehabilitation medicine, and brain-computer interfaces. In summary, this review explores the expanding possibilities of mobile EEG while recognizing the existing challenges in fully realizing its potential.

Designing a practical fatigue detection system: A review on recent developments and challenges

INTRODUCTION

Fatigue is considered to have a life-threatening effect on human health and it has been an active field of research in different sectors. Deploying wearable physiological sensors helps to detect the level of fatigue objectively without any concern of bias in subjective assessment and interfering with work.

METHODS

This paper provides an in-depth review of fatigue detection approaches using physiological signals to pinpoint their main achievements, identify research gaps, and recommend avenues for future research. The review results are presented under three headings, including: signal modality, experimental environments, and fatigue detection models. Fatigue detection studies are first divided based on signal modality into uni-modal and multi-modal approaches. Then, the experimental environments utilized for fatigue data collection are critically analyzed. At the end, the machine learning models used for the classification of fatigue state are reviewed.

PRACTICAL APPLICATIONS

The directions for future research are provided based on critical analysis of past studies. Finally, the challenges of objective fatigue detection in the real-world scenario are discussed.

Reducing flight upset risk and startle response: A study of the wake vortex alert with licensed commercial pilots

The study aimed at investigating the impact of an innovative Wake Vortex Alert (WVA) avionics on pilots' operation and mental states, intending to improve aviation safety by mitigating the risks associated with wake vortex encounters (WVEs). Wake vortices, generated by jet aircraft, pose a significant hazard to trailing or crossing aircrafts. Despite existing separation rules, incidents involving WVEs continue to occur, especially affecting smaller aircrafts like business jets, resulting in aircraft upsets and occasional cabin injuries. To address these challenges, the study focused on developing and validating an alert system that can be presented to air traffic controllers, enabling them to warn flight crews. This empowers the flight crews to either avoid the wake vortex or secure the cabin to prevent injuries. The research employed a multidimensional approach including an analysis of human performance and human factors (HF) issues to determine the potential impact of the alert on pilots' roles, tasks, and mental states. It also utilizes Human Assurance Levels (HALs) to evaluate the necessary human factors support based on the safety criticality of the new system. Realistic flight simulations were conducted to collect data of pilots' behavioural, subjective and neurophysiological responses during WVEs. The data allowed for an objective evaluation of the WVA impact on pilots' operation, behaviour and mental states (mental workload, stress levels and arousal). In particular, the results highlighted the effectiveness of the alert system in facilitating pilots' preparation, awareness and crew resource management (CRM). The results also highlighted the importance of avionics able to enhance aviation safety and reducing risks associated with wake vortex encounters. In particular, we demonstrated how providing timely information and improving situational awareness, the WVA will minimize the occurrence of WVEs and contribute to safer aviation operations.

Air Traffic Controller Workload Detection Based on EEG Signals

The assessment of the cognitive workload experienced by air traffic controllers is a complex and prominent issue in the research community. This study introduces new indicators related to gamma waves to detect controllers' workload and develops experimental protocols to capture their EEG data and NASA-TXL data. Then, statistical tests, including the Shapiro-Wilk test and ANOVA, were used to verify whether there was a significant difference between the workload data of the controllers in different scenarios. Furthermore, the Support Vector Machine (SVM) classifier was employed to assess the detection accuracy of these indicators across four categorizations. According to the outcomes, hypotheses suggesting a strong correlation between gamma waves and an air traffic controller's workload were put forward and subsequently verified; meanwhile, compared with traditional indicators, the indicators associated with gamma waves proposed in this paper have higher accuracy. In addition, to explore the applicability of the indicator, sensitive channels were selected based on the mRMR algorithm for the indicator with the highest accuracy, β + θ + α + γ, showcasing a recognition rate of a single channel exceeding 95% of the full channel, which meets the requirements of convenience and accuracy in practical applications. In conclusion, this study demonstrates that utilizing EEG gamma wave-associated indicators can offer valuable insights into analyzing workload levels among air traffic controllers.

Cognitive Workload Detection of Air Traffic Controllers Based on mRMR and Fewer EEG Channels

For air traffic controllers, the extent of their cognitive workload can significantly impact their cognitive function and response time, consequently influencing their operational efficiency or even resulting in safety incidents. In order to enhance the accuracy and efficiency in determining the cognitive workload of air traffic controllers, a cognitive workload detection method for air traffic controllers based on mRMR and fewer EEG channels was proposed in this study. First of all, a set of features related to gamma waves was initially proposed; subsequently, an EEG feature evaluation method based on the mRMR algorithm was employed to pinpoint the most relevant indicators for the detection of the cognitive workload. Consequently, a model for the detection of the cognitive workload of controllers was developed, and it was optimized by filtering out channel combinations that exhibited higher sensitivity to the workload using the mRMR algorithm. The results demonstrate that the enhanced model achieves the accuracy and stability required for practical applications. Notably, in this study, only three EEG channels were employed to achieve the highly precise detection of the cognitive workload of controllers. This approach markedly increases the practicality of employing EEG equipment for the detection of the cognitive workload and streamlines the detection process.

Research on mental fatigue during long-term motor imagery: a pilot study

Mental fatigue during long-term motor imagery (MI) may affect intention recognition in MI applications. However, the current research lacks the monitoring of mental fatigue during MI and the definition of robust biomarkers. The present study aims to reveal the effects of mental fatigue on motor imagery recognition at the brain region level and explore biomarkers of mental fatigue. To achieve this, we recruited 10 healthy participants and asked them to complete a long-term motor imagery task involving both right- and left-handed movements. During the experiment, we recorded 32-channel EEG data and carried out a fatigue questionnaire for each participant. As a result, we found that mental fatigue significantly decreased the subjects' motor imagery recognition rate during MI. Additionally the theta power of frontal, central, parietal, and occipital clusters significantly increased after the presence of mental fatigue. Furthermore, the phase synchronization between the central cluster and the frontal and occipital lobes was significantly weakened. To summarize, the theta bands of frontal, central, and parieto-occipital clusters may serve as powerful biomarkers for monitoring mental fatigue during motor imagery. Additionally, changes in functional connectivity between the central cluster and the prefrontal and occipital lobes during motor imagery could be investigated as potential biomarkers.

This Is Your Brain on Autopilot 2.0: The Influence of Practice on Driver Workload and Engagement During On-Road, Partially Automated Driving

OBJECTIVE

This on-road study employed behavioral and neurophysiological measurement techniques to assess the influence of six weeks of practice driving a Level 2 partially automated vehicle on driver workload and engagement.

BACKGROUND

Level 2 partial automation requires a driver to maintain supervisory control of the vehicle to detect "edge cases" that the automation is not equipped to handle. There is mixed evidence regarding whether drivers can do so effectively. There is also an open question regarding how practice and familiarity with automation influence driver cognitive states over time.

METHOD

Behavioral and neurophysiological measures of driver workload and visual engagement were recorded from 30 participants at two testing sessions-with a six-week familiarization period in-between. At both testing sessions, participants drove a vehicle with partial automation engaged (Level 2) and not engaged (Level 0) on two interstate highways while reaction times to the detection response task (DRT) and neurophysiological (EEG) metrics of frontal theta and parietal alpha were recorded.

RESULTS

DRT results demonstrated that partially automated driving placed more cognitive load on drivers than manual driving and six weeks of practice decreased driver workload-though only when the driving environment was relatively simple. EEG metrics of frontal theta and parietal alpha showed null effects of partial automation.

CONCLUSION

Driver workload was influenced by level of automation, specific highway characteristics, and by practice over time, but only on a behavioral level and not on a neural level.

APPLICATION

These findings expand our understanding of the influence of practice on driver cognitive states under Level 2 partial automation.

A simplified method for relapsing-remitting multiple sclerosis detection: Insights from resting EEG signals

BACKGROUND AND OBJECTIVE

Multiple sclerosis (MS) is a neurodegenerative autoimmune disease affecting the central nervous system, leading to various neurological symptoms. Early detection is paramount to prevent enduring damage during MS episodes. Although magnetic resonance imaging (MRI) is a common diagnostic tool, this study aims to explore the feasibility of using electroencephalography (EEG) signals for MS detection, considering their accessibility and ease of application compared to MRI.

METHODS

The study involved the analysis of EEG signals during rest from 17 MS patients and 27 healthy volunteers to investigate MS-healthy patterns. Power spectral density features (PSD) were extracted from the 32-channel EEG signals. The study employed Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), Classification and Regression Trees (CART), and k-Nearest Neighbor (kNN) classifiers to identify channels with the highest accuracy. Notably, the study achieved 100% accuracy in MS detection using the "Fp1" and "Pz" channels with the LDA classifier. A statistical analysis, utilizing the independent sample t-test, was conducted to explore whether PSD features of these channels differed significantly between healthy individuals and those with MS.

RESULTS

The results of the study demonstrate that effective detection of MS can be achieved using PSD features from only two channels of the EEG signal. Specifically, the "Fp1" and "Pz" channels exhibited 100% accuracy in MS detection with the LDA classifier. The statistical analysis further explored and confirmed the significant differences in PSD features between healthy individuals and MS patients.

CONCLUSION

The study concludes that the proposed method, utilizing PSD features from specific EEG channels, offers a straightforward and efficient diagnostic approach for the effective detection of MS. The findings suggest the potential utility of EEG signals as a non-invasive and accessible alternative for MS detection, highlighting the importance of further research in this direction.

Utilising raw psycho-physiological data and functional data analysis for estimating mental workload in human drivers

Recent studies have focused on accurately estimating mental workload using machine learning algorithms and extracting features from physiological measures. However, feature extraction leads to the loss of valuable information and often results in binary classifications that lack specificity in the identification of optimum mental workload. This study investigates the feasibility of using raw physiological data (EEG, facial EMG, ECG, EDA, pupillometry) combined with Functional Data Analysis (FDA) to estimate the mental workload of human drivers. A driving scenario with five tasks was employed, and subjective ratings were collected. Results demonstrate that the FDA applied nine different combinations of raw physiological signals achieving a maximum 90% accuracy, outperforming extracted features by 73%. This study shows that the mental workload of human drivers can be accurately estimated without utilising burdensome feature extraction. The approach proposed in this study offers promise for mental workload assessment in real-world applications.

From E-Commerce to the Metaverse: A Neuroscientific Analysis of Digital Consumer Behavior

The growing interest in consumer behavior in the digital environment is leading scholars and companies to focus on consumer behavior and choices on digital platforms, such as the metaverse. On this immersive digital shopping platform, consumer neuroscience provides an optimal opportunity to explore consumers' emotions and cognitions. In this study, neuroscience techniques (EEG, SC, BVP) were used to compare emotional and cognitive aspects of shopping between metaverse and traditional e-commerce platforms. Participants were asked to purchase the same product once on a metaverse platform (Second Life, SL) and once via an e-commerce website (EC). After each task, questionnaires were administered to measure perceived enjoyment, informativeness, ease of use, cognitive effort, and flow. Statistical analyses were conducted to examine differences between SL and EC at the neurophysiological and self-report levels, as well as between different stages of the purchase process. The results show that SL elicits greater cognitive engagement than EC, but it is also more mentally demanding, with a higher workload and more memorization, and fails to elicit a strong positive emotional response, leading to a poorer shopping experience. These findings provide insights not only for digital-related consumer research but also for companies to improve their metaverse shopping experience. Before investing in the platform or creating a digital retail space, companies should thoroughly analyze it, focusing on how to enhance users' cognition and emotions, ultimately promoting a better consumer experience. Despite its limitations, this pilot study sheds light on the emotional and cognitive aspects of metaverse shopping and suggests potential for further research with a consumer neuroscience approach in the metaverse field.

Augmented Recognition of Distracted Driving State Based on Electrophysiological Analysis of Brain Network

In this study, we propose an electrophysiological analysis-based brain network method for the augmented recognition of different types of distractions during driving. Driver distractions, such as cognitive processing and visual disruptions during driving, lead to distinct alterations in the electroencephalogram (EEG) signals and the extracted brain networks. We designed and conducted a simulated experiment comprising 4 distracted driving subtasks. Three connectivity indices, including both linear and nonlinear synchronization measures, were chosen to construct the brain network. By computing connectivity strengths and topological features, we explored the potential relationship between brain network configurations and states of driver distraction. Statistical analysis of network features indicates substantial differences between normal and distracted states, suggesting a reconfiguration of the brain network under distracted conditions. Different brain network features and their combinations are fed into varied machine learning classifiers to recognize the distracted driving states. The results indicate that XGBoost demonstrates superior adaptability, outperforming other classifiers across all selected network features. For individual networks, features constructed using synchronization likelihood (SL) achieved the highest accuracy in distinguishing between cognitive and visual distraction. The optimal feature set from 3 network combinations achieves an accuracy of 95.1% for binary classification and 88.3% for ternary classification of normal, cognitively distracted, and visually distracted driving states. The proposed method could accomplish the augmented recognition of distracted driving states and may serve as a valuable tool for further optimizing driver assistance systems with distraction control strategies, as well as a reference for future research on the brain-computer interface in autonomous driving.

Predicting Driver's mental workload using physiological signals: A functional data analysis approach

This study investigates the impact of advanced driver-assistance systems on drivers' mental workload. Using a combination of physiological signals including ECG, EMG, EDA, EEG (af4 and fc6 channels from the theta band), and eye diameter data, this study aims to predict and categorize drivers' mental workload into low, adequate, and high levels. Data were collected from five different driving situations with varying cognitive demands. A functional linear regression model was employed for prediction, and the accuracy rate was calculated. Among the 31 tested combinations of physiological variables, 9 combinations achieved the highest accuracy result of 90%. These results highlight the potential benefits of utilizing raw physiological signal data and employing functional data analysis methods to understand and assess driver mental workload. The findings of this study have implications for the design and improvement of driver-assistance systems to optimize safety and performance.

Experimental Study on Emergency Psychophysiological and Behavioral Reactions to Coal Mining Accidents

Effective emergency responses are crucial for preventing coal mine accidents and mitigating injuries. This paper aims to investigate the characteristics of emergency psychophysiological reactions to coal mine accidents and to explore the potential of key indicators for identifying emergency behavioral patterns. Initially, virtual reality technology facilitated a simulation experiment for emergency escape during coal mine accidents. Subsequently, the characteristics of emergency reactions were analyzed through correlation analysis, hypothesis testing, and analysis of variance. The significant changes in physiological indicators were then taken as input features and fed into the three classifiers of machine learning algorithms. These classifications ultimately led to the identification of behavioral patterns, including agility, defensiveness, panic, and rigidity, that individuals may exhibit during a coal mine accident emergency. The study results revealed an intricate relationship between the mental activities induced by accident stimuli and the resulting physiological changes and behavioral performances. During the virtual reality simulation of a coal mine accident, subjects were observed to experience significant physiological changes in electrodermal activity, heart rate variability, electromyogram, respiration, and skin temperature. The random forest classification model, based on SCR + RANGE + IBI + SDNN + LF/HF, outperformed all other models, achieving accuracies of up to 92%. These findings hold promising implications for early warning systems targeting abnormal psychophysiological and behavioral reactions to emergency accidents, potentially serving as a life-saving measure in perilous situations and fostering the sustainable growth of the coal mining industry.

Individual Variability in Brain Connectivity Patterns and Driving-Fatigue Dynamics

Mental fatigue during driving poses significant risks to road safety, necessitating accurate assessment methods to mitigate potential hazards. This study explores the impact of individual variability in brain networks on driving fatigue assessment, hypothesizing that subject-specific connectivity patterns play a pivotal role in understanding fatigue dynamics. By conducting a linear regression analysis of subject-specific brain networks in different frequency bands, this research aims to elucidate the relationships between frequency-specific connectivity patterns and driving fatigue. As such, an EEG sustained driving simulation experiment was carried out, estimating individuals' brain networks using the Phase Lag Index (PLI) to capture shared connectivity patterns. The results unveiled notable variability in connectivity patterns across frequency bands, with the alpha band exhibiting heightened sensitivity to driving fatigue. Individualized connectivity analysis underscored the complexity of fatigue assessment and the potential for personalized approaches. These findings emphasize the importance of subject-specific brain networks in comprehending fatigue dynamics, while providing sensor space minimization, advocating for the development of efficient mobile sensor applications for real-time fatigue detection in driving scenarios.

Detecting and Predicting Pilot Mental Workload Using Heart Rate Variability: A Systematic Review

Measuring pilot mental workload (MWL) is crucial for enhancing aviation safety. However, MWL is a multi-dimensional construct that could be affected by multiple factors. Particularly, in the context of a more automated cockpit setting, the traditional methods of assessing pilot MWL may face challenges. Heart rate variability (HRV) has emerged as a potential tool for detecting pilot MWL during real-flight operations. This review aims to investigate the relationship between HRV and pilot MWL and to assess the performance of machine-learning-based MWL detection systems using HRV parameters. A total of 29 relevant papers were extracted from three databases for review based on rigorous eligibility criteria. We observed significant variability across the reviewed studies, including study designs and measurement methods, as well as machine-learning techniques. Inconsistent results were observed regarding the differences in HRV measures between pilots under varying levels of MWL. Furthermore, for studies that developed HRV-based MWL detection systems, we examined the diverse model settings and discovered that several advanced techniques could be used to address specific challenges. This review serves as a practical guide for researchers and practitioners who are interested in employing HRV indicators for evaluating MWL and wish to incorporate cutting-edge techniques into their MWL measurement approaches.

Measuring cognitive workload in the nuclear control room: a review

Despite the substantial literature and human factors guidance, evaluators report challenges in selecting cognitive workload measures for the evaluation of complex human-technology systems. A review of 32 articles found that self-report measures and secondary tasks were systematically sensitive to human-system interface conditions and correlated with physiological measures. Therefore, including a self-report measure of cognitive workload is recommended when evaluating human-system interfaces. Physiological measures were mainly used in method studies, and future research must demonstrate the utility of these measures for human-system evaluation in complex work settings. However, indexes of physiological measures showed promise for cognitive workload assessment. The review revealed a limited focus on the measurement of excessive cognitive workload, although this is a key topic in nuclear process control. To support human-system evaluation of adequate cognitive workload, future research on behavioural measures may be useful in the identification and analysis of underload and overload.

Development and validation of an assessment index for quantifying cognitive task load in pilots under simulated flight conditions using heart rate variability and principal component analysis

To investigate whether high cognitive task load (CTL) for aircraft pilots can be identified by analysing heart-rate variability, electrocardiograms were recorded while cadet pilots (n = 68) performed the plane tracking, anti-gravity pedalling, and reaction tasks during simulated flight missions. Data for standard electrocardiogram parameters were extracted from the R-R-interval series. In the research phase, low frequency power (LF), high frequency power (HF), normalised HF, and LF/HF differed significantly between high and low CTL conditions (p < .05 for all). A principal component analysis identified three components contributing 90.62% of cumulative heart-rate variance. These principal components were incorporated into a composite index. Validation in a separate group of cadet pilots (n = 139) under similar conditions showed that the index value significantly increased with increasing CTL (p < .05). The heart-rate variability index can be used to objectively identify high CTL flight conditions.Practitioner summary: We used principal component analysis of electrocardiogram data to construct a composite index for identifying high cognitive task load in pilots during simulated flight. We validated the index in a separate group of pilots under similar conditions. The index can be used to improve cadet training and flight safety.Abbreviations: ANOVA: a one-way analysis of variance; AP: anti-gravity pedaling task; CTL: cognitive task load; ECG: electrocardiograms; HR: heart rate; HRV: heart-rate variability; HRVI: heart-rate variability index; PT: plane-tracking task; RMSSD: root-mean square of differences between consecutive R-R intervals; RT: reaction task; SDNN: standard deviation of R-R intervals; HF: high frequency power; HFnu: normalized HF; LF: low frequency power; LFnu: normalized LF; PCA: principal component analysis.

Sheep fatigue during transport: Lost in translation?

Although sheep are commonly transported long distances, and sheep welfare during transport is a topic of research and policy discussion, the subject of their fatigue during transport has been under-researched. The current qualitative study, focused on the EU and UK, aimed to critically analyse stakeholder views on issues relating to sheep fatigue, including behavioural indications of fatigue, the interplay between fatigue and other factors, and the practicalities of identifying fatigue in commercial transport conditions. Insight into stakeholder perceptions of these issues could contribute to the body of knowledge regarding sheep fatigue during transport, potentially playing a part in future efforts to improve fatigue understanding and detection. Eighteen experts from different stakeholder groups were interviewed. Reflexive thematic analysis of interview data yielded four themes and three sub-themes. The first theme, "Let's anthropomorphise it a little bit", underscores the pervasiveness of anthropomorphism and suggests using it in a conscious and deliberate way to drive stakeholder engagement and policy change. The second theme, "We think that they're like we are and they're not", cautions against wholesale transfer of human experiences to animals. The third theme, 'See the whole animal', advocates using Qualitative Behaviour Analysis (QBA), proven reliable in other contexts, to deepen and enrich our current understanding of fatigue. The fourth theme, 'Fatigue "never comes up"', highlights the fact that fatigue is rarely if ever discussed in the context of sheep transport. These themes suggest several avenues for future research, including developing QBA-based assessments for fatigue to improve welfare during transport.

Visual perceptual processing is unaffected by cognitive fatigue

Cognitive fatigue (CF) can lead to an increase in the latency of simple reaction time, although the processes involved in this delay are unknown. One potential explanation is that a longer time may be required for sensory processing of relevant stimuli. To investigate this possibility, the current study used a visual inspection time task to measure perceptual processing speed before and after a CF (math and memory) or non-fatiguing (documentary film) intervention. Subjective fatigue and simple reaction time significantly increased following the CF, but not the non-fatiguing intervention, confirming that CF was induced. Conversely, there was no effect of CF on inspection time task performance. It was therefore concluded that the speed of perceptual processing is not significantly impacted by CF, and thus is unlikely to underlie CF-related reaction time increases. Instead, increases in simple reaction time latency in CF may be due to delays in response preparation or initiation.

Recognizing the situation awareness of forklift operators based on EEG techniques in a field experiment

Lack of situation awareness (SA) is the primary cause of human errors when operating forklifts, so determining the SA level of the forklift operator is crucial to the safety of forklift operations. An EEG recognition approach of forklift operator SA in actual settings was presented in order to address the issues with invasiveness, subjectivity, and intermittency of existing measuring methods. In this paper, we conducted a field experiment that mimicked a typical forklift operation scenario to verify the differences in EEG states of forklift operators with different SA levels and investigate the correlation of multi-band combination features of each brain region of forklift operators with SA. Based on the sensitive EEG combination indexes, Support Vector Mechanism was used to construct a forklift operator SA recognition model. The results revealed that there were differences between forklift operators with high and low SA in the θ, α, and β frequency bands in zones F, C, P, and O; combined EEG indicators θ/β, (α + θ)/(α + β), and θ/(α + β) in zones F, P, and C were significantly correlated with SA; the recognition accuracy of the model reached 88.64% in the case of combined EEG indicators of zones C & F & P as input. It could provide a reference for SA measurement, contributing to the improvement of SA.

EEG Dataset Collection for Mental Workload Predictions in Flight-Deck Environment

High mental workload reduces human performance and the ability to correctly carry out complex tasks. In particular, aircraft pilots enduring high mental workloads are at high risk of failure, even with catastrophic outcomes. Despite progress, there is still a lack of knowledge about the interrelationship between mental workload and brain functionality, and there is still limited data on flight-deck scenarios. Although recent emerging deep-learning (DL) methods using physiological data have presented new ways to find new physiological markers to detect and assess cognitive states, they demand large amounts of properly annotated datasets to achieve good performance. We present a new dataset of electroencephalogram (EEG) recordings specifically collected for the recognition of different levels of mental workload. The data were recorded from three experiments, where participants were induced to different levels of workload through tasks of increasing cognition demand. The first involved playing the N-back test, which combines memory recall with arithmetical skills. The second was playing Heat-the-Chair, a serious game specifically designed to emphasize and monitor subjects under controlled concurrent tasks. The third was flying in an Airbus320 simulator and solving several critical situations. The design of the dataset has been validated on three different levels: (1) correlation of the theoretical difficulty of each scenario to the self-perceived difficulty and performance of subjects; (2) significant difference in EEG temporal patterns across the theoretical difficulties and (3) usefulness for the training and evaluation of AI models.

Blink-Related Oscillations Provide Naturalistic Assessments of Brain Function and Cognitive Workload within Complex Real-World Multitasking Environments

BACKGROUND

There is a significant need to monitor human cognitive performance in complex environments, with one example being pilot performance. However, existing assessments largely focus on subjective experiences (e.g., questionnaires) and the evaluation of behavior (e.g., aircraft handling) as surrogates for cognition or utilize brainwave measures which require artificial setups (e.g., simultaneous auditory stimuli) that intrude on the primary tasks. Blink-related oscillations (BROs) are a recently discovered neural phenomenon associated with spontaneous blinking that can be captured without artificial setups and are also modulated by cognitive loading and the external sensory environment-making them ideal for brain function assessment within complex operational settings.

METHODS

Electroencephalography (EEG) data were recorded from eight adult participants (five F, M = 21.1 years) while they completed the Multi-Attribute Task Battery under three different cognitive loading conditions. BRO responses in time and frequency domains were derived from the EEG data, and comparisons of BRO responses across cognitive loading conditions were undertaken. Simultaneously, assessments of blink behavior were also undertaken.

RESULTS

Blink behavior assessments revealed decreasing blink rate with increasing cognitive load (p < 0.001). Prototypical BRO responses were successfully captured in all participants (p < 0.001). BRO responses reflected differences in task-induced cognitive loading in both time and frequency domains (p < 0.05). Additionally, reduced pre-blink theta band desynchronization with increasing cognitive load was also observed (p < 0.05).

CONCLUSION

This study confirms the ability of BRO responses to capture cognitive loading effects as well as preparatory pre-blink cognitive processes in anticipation of the upcoming blink during a complex multitasking situation. These successful results suggest that blink-related neural processing could be a potential avenue for cognitive state evaluation in operational settings-both specialized environments such as cockpits, space exploration, military units, etc. and everyday situations such as driving, athletics, human-machine interactions, etc.-where human cognition needs to be seamlessly monitored and optimized.

The evaluation of cEEGrids for fatigue detection in aviation

Operator fatigue poses a major concern in safety-critical industries such as aviation, potentially increasing the chances of errors and accidents. To better understand this risk, there is a need for noninvasive objective measures of fatigue. This study aimed to evaluate the performance of cEEGrids, a type of ear-EEG, for fatigue detection by analyzing the alpha and theta power before and after sleep restriction in four sessions on two separate days, employing a within-participants design. Results were compared to traditional, highly validated methods: the Karolinska Sleepiness Scale (KSS) and Psychomotor Vigilance Task (PVT). After sleep restriction and an office workday, 12 participants showed increased alpha band power in multiple electrode channels, but no channels correlated with KSS scores and PVT response speed. These findings indicate that cEEGrids can detect differences in alpha power following mild sleep loss. However, it should be noted that this capability was limited to specific channels, and no difference in theta power was observed. The study shows the potential and limitations of ear-EEG for fatigue detection as a less invasive alternative to cap-EEG. Further design and electrode configuration adjustments are necessary before ear-EEG can be implemented for fatigue detection in the field.

Assessment of Drivers' Mental Workload by Multimodal Measures during Auditory-Based Dual-Task Driving Scenarios

Assessing drivers' mental workload is crucial for reducing road accidents. This study examined drivers' mental workload in a simulated auditory-based dual-task driving scenario, with driving tasks as the main task, and auditory-based N-back tasks as the secondary task. A total of three levels of mental workload (i.e., low, medium, high) were manipulated by varying the difficulty levels of the secondary task (i.e., no presence of secondary task, 1-back, 2-back). Multimodal measures, including a set of subjective measures, physiological measures, and behavioral performance measures, were collected during the experiment. The results showed that an increase in task difficulty led to increased subjective ratings of mental workload and a decrease in task performance for the secondary N-back tasks. Significant differences were observed across the different levels of mental workload in multimodal physiological measures, such as delta waves in EEG signals, fixation distance in eye movement signals, time- and frequency-domain measures in ECG signals, and skin conductance in EDA signals. In addition, four driving performance measures related to vehicle velocity and the deviation of pedal input and vehicle position also showed sensitivity to the changes in drivers' mental workload. The findings from this study can contribute to a comprehensive understanding of effective measures for mental workload assessment in driving scenarios and to the development of smart driving systems for the accurate recognition of drivers' mental states.

An analysis of physiological responses as indicators of driver takeover readiness in conditionally automated driving

By the year 2045, it is projected that Autonomous Vehicles (AVs) will make up half of the new vehicle market. Successful adoption of AVs can reduce drivers' stress and fatigue, curb traffic congestion, and improve safety, mobility, and economic efficiency. Due to the limited intelligence in relevant technologies, human-in-the-loop modalities are still necessary to ensure the safety of AVs at current or near future stages, because the vehicles may not be able to handle all emergencies. Therefore, it is important to know the takeover readiness of the drivers to ensure the takeover quality and avoid any potential accidents. To achieve this, a comprehensive understanding of the drivers' physiological states is crucial. However, there is a lack of systematic analysis of the correlation between different human physiological responses and takeover behaviors which could serve as important references for future studies to determine the types of data to use. This paper provides a comprehensive analysis of the effects of takeover behaviors on the common physiological indicators. A program for conditional automation was developed based on a game engine and applied to a driving simulator. The experiment incorporated three types of secondary tasks, three takeover events, and two traffic densities. Brain signals, Skin Conductance Level (SCL), and Heart Rate (HR) of the participants were collected while they were performing the driving simulations. The Frontal Asymmetry Index (FAI) (as an indicator of engagement) and Mental Workload (MWL) were calculated from the brain signals to indicate the mental states of the participants. The results revealed that the FAI of the drivers would slightly decrease after the takeover alerts were issued when they were doing secondary tasks prior to the takeover activities, and the higher difficulty of the secondary tasks could lead to lower overall FAI during the takeover periods. In contrast, The MWL and SCL increased during the takeover periods. The HR also increased rapidly at the beginning of the takeover period but dropped back to a normal level quickly. It was found that a fake takeover alert would lead to lower overall HR, slower increase, and lower peak of SCL during the takeover periods. Moreover, the higher traffic density scenarios were associated with higher MWL, and a more difficult secondary task would lead to higher MWL and HR during the takeover activities. A preliminary discussion of the correlation between the physiological data, takeover scenario, and vehicle data (that relevant to takeover readiness) was then conducted, revealing that although takeover event, SCL, and HR had slightly higher correlations with the maximum acceleration and reaction time, none of them dominated the takeover readiness. In addition, the analysis of the data across different participants was conducted, which emphasized the importance of considering standardization or normalization of the data when they were further used as input features for estimating takeover readiness. Overall, the results presented in this paper offer profound insights into the patterns of physiological data changes during takeover periods. These findings can be used as benchmarks for utilizing these variables as indicators of takeover preparedness and performance in future research endeavors.

Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study

While the term task load (TL) refers to external task demands, the amount of work, or the number of tasks to be performed, mental workload (MWL) refers to the individual's effort, mental capacity, or cognitive resources utilized while performing a task. MWL in multitasking scenarios is often closely linked with the quantity of tasks a person is handling within a given timeframe. In this study, we challenge this hypothesis from the perspective of electroencephalography (EEG) using a deep learning approach. We conducted an EEG experiment with 50 participants performing NASA Multi-Attribute Task Battery II (MATB-II) under 4 different task load levels. We designed a convolutional neural network (CNN) to help with two distinct classification tasks. In one setting, the CNN was used to classify EEG segments based on their task load level. In another setting, the same CNN architecture was trained again to detect the presence of individual MATB-II subtasks. Results show that, while the model successfully learns to detect whether a particular subtask is active in a given segment (i.e., to differentiate between different subtasks-related EEG patterns), it struggles to differentiate between the two highest levels of task load (i.e., to distinguish MWL-related EEG patterns). We speculate that the challenge comes from two factors: first, the experiment was designed in a way that these two highest levels differed only in the quantity of work within a given timeframe; and second, the participants' effective adaptation to increased task demands, as evidenced by low error rates. Consequently, this indicates that under such conditions in multitasking, EEG may not reflect distinct enough patterns to differentiate higher levels of task load.

Executive Functions Assessment Based on Wireless EEG and 3D Gait Analysis During Dual-Task: A Feasibility Study

Executive functions (EFs) are neurocognitive processes planning and regulating daily life actions. Performance of two simultaneous tasks, requiring the same cognitive resources, lead to a cognitive fatigue. Several studies investigated cognitive-motor task and the interference during walking, highlighting an increasing risk of falls especially in elderly and people with neurological diseases. A few studies instrumentally explored relationship between activation-no-activation of two EFs (working memory and inhibition) and spatial-temporal gait parameters. Aim of our study was to detect activation of inhibition and working memory during progressive difficulty levels of cognitive tasks and spontaneous walking using, respectively, wireless electroencephalography (EEG) and 3D-gait analysis. Thirteen healthy subjects were recruited. Two cognitive tasks were performed, activating inhibition (Go-NoGo) and working memory (N-back). EEG features (absolute and relative power in different bands) and kinematic parameters (7 spatial-temporal ones and Gait Variable Score for 9 range of motion of lower limbs) were analyzed. A significant decrease of stride length and an increase of external-rotation of foot progression were found during dual task with Go-NoGo. Moreover, a significant correlation was found between the relative power in the delta band at channels Fz, C4 and progressive difficulty levels of Go-NoGo (activating inhibition) during walking, whereas working memory showed no correlation. This study reinforces the hypothesis of the prevalent involvement of inhibition with respect to working memory during dual task walking and reveals specific kinematic adaptations. The foundations for EEG-based monitoring of cognitive processes involved in gait are laid. Clinical and Translational Impact Statement: Clinical and instrumental evaluation and training of executive functions (as inhibition), during cognitive-motor task, could be useful for rehabilitation treatment of gait disorder in elderly and people with neurological disease.

Investigating mental workload caused by NDRTs in highly automated driving with deep learning

OBJECTIVE

This study aimed to examine the impact of non-driving-related tasks (NDRTs) on drivers in highly automated driving scenarios and sought to develop a deep learning model for classifying mental workload using electroencephalography (EEG) signals.

METHODS

The experiment involved recruiting 28 participants who engaged in simulations within a driving simulator while exposed to 4 distinct NDRTs: (1) reading, (2) listening to radio news, (3) watching videos, and (4) texting. EEG data collected during NDRTs were categorized into 3 levels of mental workload, high, medium, and low, based on the NASA Task Load Index (NASA-TLX) scores. Two deep learning methods, namely, long short-term memory (LSTM) and bidirectional long short-term memory (BLSTM), were employed to develop the classification model.

RESULTS

A series of correlation analyses revealed that the channels and frequency bands are linearly correlated with mental workload. The comparative analysis of classification results demonstrates that EEG data featuring significantly correlated frequency bands exhibit superior classification accuracy compared to the raw EEG data.

CONCLUSIONS

This research offers a reference for assessing mental workload resulting from NDRTs in the context of highly automated driving. Additionally, it delves into the development of deep learning classifiers for EEG signals with heightened accuracy.

EEG-based hierarchical classification of level of demand and modality of auditory and visual sensory processing

Objective.To date, most research on electroencephalography (EEG)-based mental workload detection for passive brain-computer interface (pBCI) applications has focused on identifying the overall level of cognitive resources required, such as whether the workload is high or low. We propose, however, that being able to determine the specific type of cognitive resources being used, such as visual or auditory, would also be useful. This would enable the pBCI to take more appropriate action to reduce the overall level of cognitive demand on the user. For example, if a high level of workload was detected and it is determined that the user is primarily engaged in visual information processing, then the pBCI could cause some information to be presented aurally instead. In our previous work we showed that EEG could be used to differentiate visual from auditory processing tasks when the level of processing is high, but the two modalities could not be distinguished when the level of cognitive processing demand was very low. The current study aims to build on this work and move toward the overall objective of developing a pBCI that is capable of predicting both the level and the type of cognitive resources being used.Approach.Fifteen individuals undertook carefully designed visual and auditory tasks while their EEG data was being recorded. In this study, we incorporated a more diverse range of sensory processing conditions including not only single-modality conditions (i.e. those requiring one of either visual or auditory processing) as in our previous study, but also dual-modality conditions (i.e. those requiring both visual and auditory processing) and no-task/baseline conditions (i.e. when the individual is not engaged in either visual or auditory processing).Main results.Using regularized linear discriminant analysis within a hierarchical classification algorithm, the overall cognitive demand was predicted with an accuracy of more than 86%, while the presence or absence of visual and auditory sensory processing were each predicted with an accuracy of approximately 70%.Significance.The findings support the feasibility of establishing a pBCI that can determine both the level and type of attentional resources required by the user at any given moment. This pBCI could assist in enhancing safety in hazardous jobs by triggering the most effective and efficient adaptation strategies when high workload conditions are detected.

In-Car Environment Control Using an SSVEP-Based Brain-Computer Interface with Visual Stimuli Presented on Head-Up Display: Performance Comparison with a Button-Press Interface

Controlling the in-car environment, including temperature and ventilation, is necessary for a comfortable driving experience. However, it often distracts the driver's attention, potentially causing critical car accidents. In the present study, we implemented an in-car environment control system utilizing a brain-computer interface (BCI) based on steady-state visual evoked potential (SSVEP). In the experiment, four visual stimuli were displayed on a laboratory-made head-up display (HUD). This allowed the participants to control the in-car environment by simply staring at a target visual stimulus, i.e., without pressing a button or averting their eyes from the front. The driving performances in two realistic driving tests-obstacle avoidance and car-following tests-were then compared between the manual control condition and SSVEP-BCI control condition using a driving simulator. In the obstacle avoidance driving test, where participants needed to stop the car when obstacles suddenly appeared, the participants showed significantly shorter response time (1.42 ± 0.26 s) in the SSVEP-BCI control condition than in the manual control condition (1.79 ± 0.27 s). No-response rate, defined as the ratio of obstacles that the participants did not react to, was also significantly lower in the SSVEP-BCI control condition (4.6 ± 14.7%) than in the manual control condition (20.5 ± 25.2%). In the car-following driving test, where the participants were instructed to follow a preceding car that runs at a sinusoidally changing speed, the participants showed significantly lower speed difference with the preceding car in the SSVEP-BCI control condition (15.65 ± 7.04 km/h) than in the manual control condition (19.54 ± 11.51 km/h). The in-car environment control system using SSVEP-based BCI showed a possibility that might contribute to safer driving by keeping the driver's focus on the front and thereby enhancing the overall driving performance.

The nudging effect of AIGC labeling on users' perceptions of automated news: evidence from EEG

Introduction

In the context of generative AI intervention in news production, this study primarily focuses on the impact of AI-generated content (AIGC) labeling cues on users' perceptions of automated news based on nudge theory.

Methods

A 2 (authorship disclosure nudge cues: with vs. without AIGC label) × 2 (automated news type: descriptive vs. evaluative news) within-subject experiment was carried out. Thirty-two participants were recruited to read automated news, evaluate the perceived content trustworthiness, and record with an EEG device.

Results

The results demonstrated that disclosure of AIGC labeling significantly reduced the trustworthiness perception of both fact-based descriptive and opinion-based evaluative news. In EEG, the delta PSD, theta PSD, alpha PSD, and beta PSD with disclosure of AIGC labeling were significantly higher than those without AIGC labeling. Meanwhile, in descriptive news conditions, TAR with AIGC labeling was higher than without AIGC labeling.

Discussion

These results suggested that AIGC labeling significantly improves the degree of attention concentration in reading and deepens the degree of cognitive processing. Users are nudged by AIGC labeling to shift their limited attention and cognitive resources to re-evaluate the information quality to obtain more prudent judgment results. This helps to supplement the theoretical perspective on transparent disclosure nudging in the Internet content governance research field, and it can offer practical guidance to use content labeling to regulate the media industry landscape in the face of AI's pervasive presence.

Enhancing EEG-based cross-day mental workload classification using periodic component of power spectrum

Objective. The day-to-day variability of electroencephalogram (EEG) poses a significant challenge to decode human brain activity in EEG-based passive brain-computer interfaces (pBCIs). Conventionally, a time-consuming calibration process is required to collect data from users on a new day to ensure the performance of the machine learning-based decoding model, which hinders the application of pBCIs to monitor mental workload (MWL) states in real-world settings.Approach. This study investigated the day-to-day stability of the raw power spectral density (PSD) and their periodic and aperiodic components decomposed by the Fitting Oscillations and One-Over-F algorithm. In addition, we validated the feasibility of using periodic components to improve cross-day MWL classification performance.Main results. Compared to the raw PSD (69.9% ± 18.5%) and the aperiodic component (69.4% ± 19.2%), the periodic component had better day-to-day stability and significantly higher cross-day classification accuracy (84.2% ± 11.0%).Significance. These findings indicate that periodic components of EEG have the potential to be applied in decoding brain states for more robust pBCIs.

Self-modulation of the sense of agency via neurofeedback enhances sensory-guided behavioral control

The sense of agency is a fundamental aspect of human self-consciousness, whose neural correlates encompass widespread brain networks. Research has explored the neuromodulatory properties of the sense of agency with noninvasive brain stimulation, which induces exogenous manipulations of brain activity; however, it is unknown whether endogenous modulation of the sense of agency is also achievable. We investigated whether the sense of agency can be self-regulated with electroencephalography-based neurofeedback. We conducted 2 experiments in which healthy humans performed a motor task while their motor control was artificially disrupted, and gave agency statements on their perceived control. We first identified the electrophysiological response to agency processing, and then applied neurofeedback in a parallel, sham-controlled design, where participants learnt to self-modulate their sense of agency. We found that behavioral measures of agency and performance on the task decreased with the increasing disruption of control. This was negatively correlated with power spectral density in the theta band, and positively correlated in the alpha and beta bands, at central and parietal electrodes. After neurofeedback training of central theta rhythms, participants improved their actual control over the task, and this was associated with a significant decrease in the frequency band trained via neurofeedback. Thus, self-regulation of theta rhythms can improve sensory-guided behavior.

Physiological measurements for driving drowsiness: A comparative study of multi-modality feature fusion and selection

A large number of traffic accidents were caused by drowsiness while driving. In-vehicle alert system based on physiological signals was one of the most promising solutions to monitor driving fatigue. However, different physiological modalities can be used, and many relative studies compared different modalities without considering the implementation feasibility of portable or wearable devices. Moreover, evaluations of each modality in previous studies were based on inconsistent choices of fatigue label and signal features, making it hard to compare the results of different studies. Therefore, the modality comparison and fusion for continuous drowsiness estimation while driving was still unclear. This work sought to comprehensively compare widely-used physiological modalities, including forehead electroencephalogram (EEG), electrooculogram (EOG), R-R intervals (RRI) and breath, in a hardware setting feasible for portable or wearable devices to monitor driving fatigue. Moreover, a more general conclusion on modality comparison and fusion was reached based on the regression of features or their combinations and the awake-to-drowsy transition. Finally, the feature subset of fused modalities was produced by feature selection method, to select the optimal feature combination and reduce computation consumption. Considering practical feasibility, the most effective combination with the highest correlation coefficient was using forehead EEG or EOG, along with RRI and RRI-derived breath. If more comfort and convenience was required, the combination of RRI and RRI-derived breath was also promising.

Neurophysiological mental fatigue assessment for developing user-centered Artificial Intelligence as a solution for autonomous driving

The human factor plays a key role in the automotive field since most accidents are due to drivers' unsafe and risky behaviors. The industry is now pursuing two main solutions to deal with this concern: in the short term, there is the development of systems monitoring drivers' psychophysical states, such as inattention and fatigue, and in the medium-long term, there is the development of fully autonomous driving. This second solution is promoted by recent technological progress in terms of Artificial Intelligence and sensing systems aimed at making vehicles more and more accurately aware of their "surroundings." However, even with an autonomous vehicle, the driver should be able to take control of the vehicle when needed, especially during the current transition from the lower (SAE < 3) to the highest level (SAE = 5) of autonomous driving. In this scenario, the vehicle has to be aware not only of its "surroundings" but also of the driver's psychophysical state, i.e., a user-centered Artificial Intelligence. The neurophysiological approach is one the most effective in detecting improper mental states. This is particularly true if considering that the more automatic the driving will be, the less available the vehicular data related to the driver's driving style. The present study aimed at employing a holistic approach, considering simultaneously several neurophysiological parameters, in particular, electroencephalographic, electrooculographic, photopletismographic, and electrodermal activity data to assess the driver's mental fatigue in real time and to detect the onset of fatigue increasing. This would ideally work as an information/trigger channel for the vehicle AI. In all, 26 professional drivers were engaged in a 45-min-lasting realistic driving task in simulated conditions, during which the previously listed biosignals were recorded. Behavioral (reaction times) and subjective measures were also collected to validate the experimental design and to support the neurophysiological results discussion. Results showed that the most sensitive and timely parameters were those related to brain activity. To a lesser extent, those related to ocular parameters were also sensitive to the onset of mental fatigue, but with a delayed effect. The other investigated parameters did not significantly change during the experimental session.

A Bibliometric Analysis of Neuroscience Tools Use in Construction Health and Safety Management

Despite longstanding traditional construction health and safety management (CHSM) methods, the construction industry continues to face persistent challenges in this field. Neuroscience tools offer potential advantages in addressing these safety and health issues by providing objective data to indicate subjects' cognition and behavior. The application of neuroscience tools in the CHSM has received much attention in the construction research community, but comprehensive statistics on the application of neuroscience tools to CHSM is lacking to provide insights for the later scholars. Therefore, this study applied bibliometric analysis to examine the current state of neuroscience tools use in CHSM. The development phases; the most productive journals, regions, and institutions; influential scholars and articles; author collaboration; reference co-citation; and application domains of the tools were identified. It revealed four application domains: monitoring the safety status of construction workers, enhancing the construction hazard recognition ability, reducing work-related musculoskeletal disorders of construction workers, and integrating neuroscience tools with artificial intelligence techniques in enhancing occupational safety and health, where magnetoencephalography (EMG), electroencephalography (EEG), eye-tracking, and electrodermal activity (EDA) are four predominant neuroscience tools. It also shows a growing interest in integrating the neuroscience tools with artificial intelligence techniques to address the safety and health issues. In addition, future studies are suggested to facilitate the applications of these tools in construction workplaces by narrowing the gaps between experimental settings and real situations, enhancing the quality of data collected by neuroscience tools and performance of data processing algorithms, and overcoming user resistance in tools adoption.

Illuminating the Neural Landscape of Pilot Mental States: A Convolutional Neural Network Approach with Shapley Additive Explanations Interpretability

Predicting pilots' mental states is a critical challenge in aviation safety and performance, with electroencephalogram data offering a promising avenue for detection. However, the interpretability of machine learning and deep learning models, which are often used for such tasks, remains a significant issue. This study aims to address these challenges by developing an interpretable model to detect four mental states-channelised attention, diverted attention, startle/surprise, and normal state-in pilots using EEG data. The methodology involves training a convolutional neural network on power spectral density features of EEG data from 17 pilots. The model's interpretability is enhanced via the use of SHapley Additive exPlanations values, which identify the top 10 most influential features for each mental state. The results demonstrate high performance in all metrics, with an average accuracy of 96%, a precision of 96%, a recall of 94%, and an F1 score of 95%. An examination of the effects of mental states on EEG frequency bands further elucidates the neural mechanisms underlying these states. The innovative nature of this study lies in its combination of high-performance model development, improved interpretability, and in-depth analysis of the neural correlates of mental states. This approach not only addresses the critical need for effective and interpretable mental state detection in aviation but also contributes to our understanding of the neural underpinnings of these states. This study thus represents a significant advancement in the field of EEG-based mental state detection.

Blink-related EEG activity measures cognitive load during proactive and reactive driving

Assessing drivers' cognitive load is crucial for driving safety in challenging situations. This research employed the occurrence of drivers' natural eye blinks as cues in continuously recorded EEG data to assess the cognitive workload while reactive or proactive driving. Twenty-eight participants performed either a lane-keeping task with varying levels of crosswind (reactive) or curve road (proactive). The blink event-related potentials (bERPs) and spectral perturbations (bERSPs) were analyzed to assess cognitive load variations. The study found that task load during reactive driving did not significantly impact bERPs or bERSPs, possibly due to enduring alertness for vehicle control. The proactive driving revealed significant differences in the occipital N1 component with task load, indicating the necessity to adapt the attentional resources allocation based on road demands. Also, increased steering complexity led to decreased frontal N2, parietal P3, occipital P2 amplitudes, and alpha power, requiring more cognitive resources for processing relevant information. Interestingly, the proactive and reactive driving scenarios demonstrated a significant interaction at the parietal P2 and occipital N1 for three difficulty levels. The study reveals that EEG measures related to natural eye blink behavior provide insights into the effect of cognitive load on different driving tasks, with implications for driver safety.

This Is Your Brain on Autopilot: Neural Indices of Driver Workload and Engagement During Partial Vehicle Automation

OBJECTIVE

This research explores the effect of partial vehicle automation on neural indices of mental workload and visual engagement during on-road driving.

BACKGROUND

There is concern that the introduction of automated technology in vehicles may lead to low driver stimulation and subsequent disengagement from the driving environment. Simulator-based studies have examined the effect of automation on a driver's cognitive state, but it is unknown how the conclusions translate to on-road driving. Electroencephalographic (EEG) measures of frontal theta and parietal alpha can provide insight into a driver's mental workload and visual engagement while driving under various conditions.

METHOD

EEG was recorded from 71 participants while driving on the roadway. We examined two age cohorts, on two different highway configurations, in four different vehicles, with partial vehicle automation both engaged and disengaged.

RESULTS

Analysis of frontal theta and parietal alpha power revealed that there was no change in mental workload or visual engagement when driving manually compared with driving under partial vehicle automation.

CONCLUSION

Drivers new to the technology remained engaged with the driving environment when operating under partial vehicle automation. These findings suggest that the concern surrounding driver disengagement under vehicle automation may need to be tempered, at least for drivers new to the experience.

APPLICATION

These findings expand our understanding of the effects of partial vehicle automation on drivers' cognitive states.

A systematic review on the influence factors, measurement, and effect of driver workload

Driver workload (DWL) is an important factor that needs to be considered in the study of traffic safety. The research focus on DWL has undergone certain shifts with the rapid development of scientific and technological advancements in the field of transportation in recent years. This study aims to grasp the state of research on DWL by both bibliometric analysis and individual critical literature review. The knowledge structure and development trend are described using bibliometric analysis. The knowledge mapping method is applied to mine the available literature in depth. It is discovered that one of the current research focus on DWL has shifted towards investigating its application in the field of autonomous driving. Subjective questionnaires and experimental tests (including both simulation technology and field study) are the main approaches to analyze DWL. An individual critical literature review of the influencing factors, measurement, and performance of DWL is provided. Research findings have shown that DWL was highly impacted by both intrinsic (e.g., age, temperament, driving experience) and external factors (e.g., vehicles, roads, tasks, environments). Scholars are actively exploring the combined effects of various factors and the level of vehicle automation on DWL. In addition to assess DWL by using subjective measures or physiological parameter measures separately, studies have started to improve classification accuracy by combining multiple measurement methods. Safety thresholds of DWL are not sufficiently studied due to the various interference items corresponding to different scenarios, but it is expected to quantify the DWL and find the threshold by establishing assessment models considering these intrinsic and external multiple-factors simultaneously. Driver or vehicle performance indicators are controversial to measure DWL directly, but they were suitable to reflect the impact of DWL in different driving conditions.