Exploring Neuro-Physiological Correlates of Drivers' Mental Fatigue Caused by Sleep Deprivation Using Simultaneous EEG, ECG, and fNIRS Data

Non-invasive detection of mental fatigue in construction equipment operators through geometric measurements of facial features

INTRODUCTION

Prolonged operation of construction equipment could lead to mental fatigue, which can increase the chances of human error-related accidents as well as operators' ill-health. The objective detection of operators' mental fatigue is crucial for reducing accident risk and ensuring operator health. Electroencephalography, photoplethysmography, electrodermal activity, and eye-tracking technology have been used to mitigate this issue. These technologies are invasive and wearable sensors that can cause irritation and discomfort. Geometric measurements of facial features can serve as a noninvasive alternative approach. Its application in detecting mental fatigue of construction equipment operators has not been reported in the literature. Although the application of facial features has been widespread in other domains, such as drivers and other occupation scenarios, their ecological validity for construction excavator operators remains a knowledge gap.

METHOD

This study proposed employing geometric measurements of facial features to detect mental fatigue in construction equipment operators' facial features. In this study, seventeen operators performed excavation operations. Mental fatigue was labeled subjectively and objectively using NASA-TLX scores and EDA values. Based on geometric measurements, facial features (eyebrow, mouth outer, mouth corners, head motion, eye area, and face area) were extracted.

RESULTS

The results showed that there was significant difference in the measured metrics for high fatigue compared to low fatigue. Specifically, the most noteworthy variation was for the eye and face area metrics, with mean differences of 45.88% and 26.9%, respectively.

CONCLUSIONS

The findings showed that geometrical measurements of facial features are a useful, noninvasive approach for detecting the mental fatigue of construction equipment operators.

Advancements in Fatigue Detection: Integrating fNIRS and Non-Voluntary Attention Brain Function Experiments

BACKGROUND

Driving fatigue is a significant concern in contemporary society, contributing to a considerable number of traffic accidents annually. This study explores novel methods for fatigue detection, aiming to enhance driving safety.

METHODS

This study utilizes electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) to monitor driver fatigue during simulated driving experiments lasting up to 7 h.

RESULTS

Analysis reveals a significant correlation between behavioral data and hemodynamic changes in the prefrontal lobe, particularly around the 4 h mark, indicating a critical period for driver performance decline. Despite a small participant cohort, the study's outcomes align closely with established fatigue standards for drivers.

CONCLUSIONS

By integrating fNIRS into non-voluntary attention brain function experiments, this research demonstrates promising efficacy in accurately detecting driving fatigue. These findings offer insights into fatigue dynamics and have implications for shaping effective safety measures and policies in various industrial settings.

Cardiac indices of driver fatigue across in-lab and on-road studies

Driver fatigue is a major contributor to road accidents. Therefore, driver assistance systems (DAS) that would monitor drivers' states may contribute to road safety. Such monitoring can potentially be achieved with input from ECG indices (e.g., heart rate). We reviewed the empirical literature on responses of cardiac measures to driver fatigue and on detecting fatigue with cardiac indices and classification algorithms. We used meta-analytical methods to explore the pooled effect sizes of different cardiac indices of fatigue, their heterogeneity, and the consistency of their responses across studies. Our large pool of studies (N = 39) allowed us to stratify the results across on-road and simulator studies. We found that despite the large heterogeneity of the effect sizes between the studies, many indices had significant pooled effect sizes across the studies, and more frequently across the on-road studies. We also found that most indices showed consistent responses across both on-road and simulator studies. Regarding the detection accuracy, we found that even on-road classification could have been as accurate as 70% with only 2-min of data. However, we could only find two on-road studies that employed fatigue classification algorithms. Overall, our findings are encouraging with respect to the prospect of using cardiac measures for detecting driver fatigue. Yet, to fully explore this possibility, there is a need for additional on-road studies that would employ a similar set of cardiac indices and detection algorithms, a unified definition of fatigue, and additional levels of fatigue than the two fatigue vs alert states.

Human Attention Restoration, Flow, and Creativity: A Conceptual Integration

In today's fast paced, attention-demanding society, executive functions and attentional resources are often taxed. Individuals need ways to sustain and restore these resources. We first review the concepts of attention and restoration, as instantiated in Attention Restoration Theory (ART). ART emphasizes the role of nature in restoring attention. We then discuss the essentials of experiments on the causal influences of nature. Next, we expand the concept of ART to include modern, designed environments. We outline a wider perspective termed attentional ecology, in which attention behavior is viewed within a larger system involving the human and their interactions with environmental demands over time. When the ecology is optimal, mental functioning can be a positive "flow" that is productive, sustainable for the individual, and sometimes creative.

Field-based longitudinal evaluation of multimodal worker fatigue assessments in offshore shiftwork

Fatigue in offshore environments is a critical safety hazard, yet the utility of daily fatigue assessments has not been longitudinally examined in these environments. The aim of this exploratory longitudinal field study across two drillships in the Gulf of Mexico was to determine the changes in subjective, performance-based, and physiological fatigue measures over time across different shift types (day, night, and swing) and to identify correlations between these multimodal fatigue assessments. Repeated measures correlation analyses of daily fatigue data from seventy offshore workers revealed that while total sleep time remained unaffected by time on rig, workers' performances on the psychomotor vigilance test (PVT) deteriorated over time across all shift types. Several correlations between the various multimodal measures were consistent with the extant literature on worker fatigue symptoms and perceptual and physiological manifestations. These findings emphasize the utility of PVT and single item self-reports to capture worker fatigue in offshore shiftwork.

Evaluation of driver drowsiness based on respiratory metrics

BACKGROUND

The transition from alertness to drowsiness can cause considerable changes in the respiratory system, providing an opportunity to detect driver drowsiness.

OBJECTIVE

The aim of this study was to determine which respiratory features indicate driver drowsiness and then use these features to classify the level of drowsiness and alertness.

METHODS

Twenty male students (mean age 25.6±2.41 years) participated in the study using a driving simulator, and eight features, including expiration duration (ED), inspiration duration (ID), peak-to-peak amplitude (PA), inspiration-to-expiration time ratio (I/E ratio), driving, timing, respiration rate (RR), and yawning, were extracted from the respiratory signal generated by abdominal motions using a belt equipped with a force sensor.

RESULTS

All eight features were statistically significant at the significance level of 0.05. Drowsiness can be detected using respiratory features with 88% accuracy, 82% precision, 86% recall, and an 90% F1 score.

CONCLUSION

The findings of this study may be useful in the development of driver drowsiness monitoring systems based on less intrusive respiratory signal analysis, particularly for specific process automation applications when vehicle control is not in the hands of the driver.

From brain to worksite: the role of fNIRS in cognitive studies and worker safety

Effective hazard recognition and decision-making are crucial factors in ensuring workplace safety in the construction industry. Workers' cognition closely relates to that hazard-handling behavior. Functional near-infrared spectroscopy (fNIRS) is a neurotechique tool that can evaluate the concentration vibration of oxygenated hemoglobin [ H b O 2 ] and deoxygenated hemoglobin [H b R ] to reflect the cognition process. It is essential to monitor workers' brain activity by fNIRS to analyze their cognitive status and reveal the mechanism in hazard recognition and decision-making process, providing guidance for capability evaluation and management enhancement. This review offers a systematic assessment of fNIRS, encompassing the basic theory, experiment analysis, data analysis, and discussion. A literature search and content analysis are conducted to identify the application of fNIRS in construction safety research, the limitations of selected studies, and the prospects of fNIRS in future research. This article serves as a guide for researchers keen on harnessing fNIRS to bolster construction safety standards and forwards insightful recommendations for subsequent studies.

Surgical stress: the muscle and cognitive demands of robotic and laparoscopic surgery

Introduction

Surgeons are among the most at-risk professionals for work-related musculoskeletal decline and experience high mental demands. This study examined the electromyographic (EMG) and electroencephalographic (EEG) activities of surgeons during surgery.

Methods

Surgeons who performed live laparoscopic (LS) and robotic (RS) surgeries underwent EMG and EEG measurements. Wireless EMG was used to measure muscle activation in four muscle groups bilaterally (biceps brachii, deltoid, upper trapezius, and latissimus dorsi), and an 8-channel wireless EEG device was used to measure cognitive demand. EMG and EEG recordings were completed simultaneously during (i) noncritical bowel dissection, (ii) critical vessel dissection, and (iii) dissection after vessel control. Robust ANOVA was used to compare the %MVC_RMS and alpha power between LS and RS.

Results

Thirteen male surgeons performed 26 laparoscopic surgeries (LS) and 28 robotic surgeries (RS). Muscle activation was significantly higher in the right deltoid (p = 0.006), upper trapezius (left, p = 0.041; right, p = 0.032), and latissimus dorsi (left, p = 0.003; right, p = 0.014) muscles in the LS group. There was greater muscle activation in the right biceps than in the left biceps in both surgical modalities (both p = 0.0001). There was a significant effect of the time of surgery on the EEG activity (p <0.0001). A significantly greater cognitive demand was observed in the RS than in the LS with alpha, beta, theta, delta, and gamma (p = 0.002 - p <0.0001).

Conclusion

These data suggest greater muscle demands in laparoscopic surgery, but greater cognitive demands in robotic surgery.

Utilizing EEG and fNIRS for the detection of sleep-deprivation-induced fatigue and its inhibition using colored light stimulation

Drowsy driving is a common, but underestimated phenomenon in terms of associated risks as it often results in crashes causing fatalities and serious injuries. It is a challenging task to alert or reduce the driver's drowsy state using non-invasive techniques. In this study, a drowsiness reduction strategy has been developed and analyzed using exposure to different light colors and recording the corresponding electrical and biological brain activities. 31 subjects were examined by dividing them into 2 classes, a control group, and a healthy group. Fourteen EEG and 42 fNIRS channels were used to gather neurological data from two brain regions (prefrontal and visual cortices). Experiments shining 3 different colored lights have been carried out on them at certain times when there is a high probability to get drowsy. The results of this study show that there is a significant increase in HbO of a sleep-deprived participant when he is exposed to blue light. Similarly, the beta band of EEG also showed an increased response. However, the study found that there is no considerable increase in HbO and beta band power in the case of red and green light exposures. In addition to that, values of other physiological signals acquired such as heart rate, eye blinking, and self-reported Karolinska Sleepiness Scale scores validated the findings predicted by the electrical and biological signals. The statistical significance of the signals achieved has been tested using repeated measures ANOVA and t-tests. Correlation scores were also calculated to find the association between the changes in the data signals with the corresponding changes in the alertness level.

Driving drowsiness detection using spectral signatures of EEG-based neurophysiology

Introduction: Drowsy driving is a significant factor causing dire road crashes and casualties around the world. Detecting it earlier and more effectively can significantly reduce the lethal aftereffects and increase road safety. As physiological conditions originate from the human brain, so neurophysiological signatures in drowsy and alert states may be investigated for this purpose. In this preface, A passive brain-computer interface (pBCI) scheme using multichannel electroencephalography (EEG) brain signals is developed for spatially localized and accurate detection of human drowsiness during driving tasks.

Methods: This pBCI modality acquired electrophysiological patterns of 12 healthy subjects from the prefrontal (PFC), frontal (FC), and occipital cortices (OC) of the brain. Neurological states are recorded using six EEG channels spread over the right and left hemispheres in the PFC, FC, and OC of the sleep-deprived subjects during simulated driving tasks. In post-hoc analysis, spectral signatures of the δ, θ, α, and β rhythms are extracted in terms of spectral band powers and their ratios with a temporal correlation over the complete span of the experiment. Minimum redundancy maximum relevance, Chi-square, and ReliefF feature selection methods are used and aggregated with a Z-score based approach for global feature ranking. The extracted drowsiness attributes are classified using decision trees, discriminant analysis, logistic regression, naïve Bayes, support vector machines, k-nearest neighbors, and ensemble classifiers. The binary classification results are reported with confusion matrix-based performance assessment metrics.

Results: In inter-classifier comparison, the optimized ensemble model achieved the best results of drowsiness classification with 85.6% accuracy and precision, 89.7% recall, 87.6% F1-score, 80% specificity, 70.3% Matthews correlation coefficient, 70.2% Cohen’s kappa score, and 91% area under the receiver operating characteristic curve with 76-ms execution time. In inter-channel comparison, the best results were obtained at the F8 electrode position in the right FC of the brain. The significance of all the results was validated with a p-value of less than 0.05 using statistical hypothesis testing methods.

Conclusions: The proposed scheme has achieved better results for driving drowsiness detection with the accomplishment of multiple objectives. The predictor importance approach has reduced the feature extraction cost and computational complexity is minimized with the use of conventional machine learning classifiers resulting in low-cost hardware and software requirements. The channel selection approach has spatially localized the most promising brain region for drowsiness detection with only a single EEG channel (F8) which reduces the physical intrusiveness in normal driving operation. This pBCI scheme has a good potential for practical applications requiring earlier, more accurate, and less disruptive drowsiness detection using the spectral information of EEG biosignals.

A New ECG Data Processing Approach to Developing an Accurate Driving Fatigue Detection Framework with Heart Rate Variability Analysis and Ensemble Learning

More than 1.3 million people are killed in traffic accidents annually. Road traffic accidents are mostly caused by human error. Therefore, an accurate driving fatigue detection system is required for drivers. Most driving fatigue detection studies concentrated on improving feature engineering and classification methods. We propose a novel driving fatigue detection framework concentrating on the development of the preprocessing, feature extraction, and classification stages to improve the classification accuracy of fatigue states. The proposed driving fatigue detection framework measures fatigue using a two-electrode ECG. The resampling method and heart rate variability analysis were used to extract features from the ECG data, and an ensemble learning model was utilized to classify fatigue states. To achieve the best model performance, 40 possible scenarios were applied: a combination of 5 resampling scenarios, 2 feature extraction scenarios, and 4 classification model scenarios. It was discovered that the combination of a resampling method with a window duration of 300 s and an overlap of 270 s, 54 extracted features, and AdaBoost yielded an optimum accuracy of 98.82% for the training dataset and 81.82% for the testing dataset. Furthermore, the preprocessing resampling method had the greatest impact on the model’s performance; it is a new approach presented in this study.

Assessing the development of mental fatigue during simulated flights with concurrent EEG-fNIRS measurement

Mental fatigue (MF) can impair pilots' performance and reactions to unforeseen events and is therefore an important concept within aviation. The physiological measurement of MF, especially with EEG and, in recent years, fNIRS, has gained much attention. However, a systematic investigation and comparison of the measurements is seldomly done. We induced MF via time on task during a 90-min simulated flight task and collected concurrent EEG-fNIRS, performance and self-report data from 31 participants. While their subjective MF increased linearly, the participants were able to keep their performance stable over the course of the experiment. EEG data showed an early increase and levelling in parietal alpha power and a slower, but steady increase in frontal theta power. No consistent trend could be observed in the fNIRS data. Thus, more research on fNIRS is needed to understand its possibilities and limits for MF assessment, and a combination with EEG is advisable to compare and validate results. Until then, EEG remains the better choice for continuous MF assessment in cockpit applications because of its high sensitivity to a transition from alert to fatigued, even before performance is impaired.

Interdisciplinary views of fNIRS: Current advancements, equity challenges, and an agenda for future needs of a diverse fNIRS research community

Functional Near-Infrared Spectroscopy (fNIRS) is an innovative and promising neuroimaging modality for studying brain activity in real-world environments. While fNIRS has seen rapid advancements in hardware, software, and research applications since its emergence nearly 30 years ago, limitations still exist regarding all three areas, where existing practices contribute to greater bias within the neuroscience research community. We spotlight fNIRS through the lens of different end-application users, including the unique perspective of a fNIRS manufacturer, and report the challenges of using this technology across several research disciplines and populations. Through the review of different research domains where fNIRS is utilized, we identify and address the presence of bias, specifically due to the restraints of current fNIRS technology, limited diversity among sample populations, and the societal prejudice that infiltrates today's research. Finally, we provide resources for minimizing bias in neuroscience research and an application agenda for the future use of fNIRS that is equitable, diverse, and inclusive.

Neuroergonomics on the Go: An Evaluation of the Potential of Mobile EEG for Workplace Assessment and Design

OBJECTIVE

We demonstrate and discuss the use of mobile electroencephalogram (EEG) for neuroergonomics. Both technical state of the art as well as measures and cognitive concepts are systematically addressed.

BACKGROUND

Modern work is increasingly characterized by information processing. Therefore, the examination of mental states, mental load, or cognitive processing during work is becoming increasingly important for ergonomics.

RESULTS

Mobile EEG allows to measure mental states and processes under real live conditions. It can be used for various research questions in cognitive neuroergonomics. Besides measures in the frequency domain that have a long tradition in the investigation of mental fatigue, task load, and task engagement, new approaches-like blink-evoked potentials-render event-related analyses of the EEG possible also during unrestricted behavior.

CONCLUSION

Mobile EEG has become a valuable tool for evaluating mental states and mental processes on a highly objective level during work. The main advantage of this technique is that working environments don't have to be changed while systematically measuring brain functions at work. Moreover, the workflow is unaffected by such neuroergonomic approaches.

Neural mechanisms underlying state mental fatigue: a systematic review and activation likelihood estimation meta-analysis

Sustained performance of cognitive tasks could lead to the development of state mental fatigue characterized by subjective sensation of mental weariness and decrease in cognitive performance. In addition to the occupational hazards associated with mental fatigue, it can also affect physical performance reducing endurance, balance, and sport-specific technical skills. Similarly, mental fatigue is a common symptom in certain chronic health conditions such as multiple sclerosis affecting quality of life of the patients. Despite its widely acknowledged negative impact, the neural mechanisms underlining this phenomenon are still not fully understood. We conducted a systematic review and activation likelihood estimation (ALE) meta-analysis of functional neuroimaging studies investigating the effect of mental fatigue due to time-on-task (TOT) on brain activity to elucidate the possible underlying mechanisms. Studies were included if they examined change in brain activity induced by experimental mental fatigue (TOT effect) or investigated the relationship between brain activity and subjective mental fatigue due to TOT. A total of 33 studies met the review's inclusion criteria, 13 of which were included in meta-analyses. Results of the meta-analyses revealed a decrease in activity with TOT in brain areas that constitute the cognitive control network. Additionally, an increased activity with TOT, as well as negative relationship with subjective mental fatigue was found in parts of the default mode network of the brain. The changes in cognitive control and the default mode networks of the brain due to state mental fatigue observed in this study were discussed in relation to the existing theories of mental fatigue.

Comprehensive evaluation of military training-induced fatigue among soldiers in China: A Delphi consensus study

Objective

Military training-induced fatigue (MIF) often results into non-combat attrition. However, standard evaluation of MIF is unavailable. This study aimed to provide credible suggestions about MIF-evaluation.

Methods

A 3-round Delphi study was performed. The authority of the experts was assessed by the authority coefficient (Aa). In round 1, categories of indicators were collected via anonymous survey of experts, then potential indicators were selected via literature search. In round 2, experts should evaluate the clinical implication, practical value, and importance of each potential indicators, or recommend new indicators based on feedback of round 1. Indicators with recommendation proportions ≥ 70% and new recommended indicators would be included in round 3 to be rated on a 5-point Likert scale. "Consensus in" was achieved when coefficient of concordance (Kendall's W) of a round was between 0.2 and 0.5 and the coefficient of variation (CV) of each aspect for an indicator was < 0.5. If round 3 could not achieve "consensus in," more rounds would be conducted iteratively based on round 3. Indicators included in the recommendation set were ultimately classified into grade I (highly recommended) or grade II (recommended) according to the mean score and CV of the aspects.

Results

Twenty-three experts participated with credible authority coefficient (mean Aa = 0.733). "Consensus in" was achieved in round 3 (Kendall's W = 0.435, p < 0.001; all CV < 0.5). Round 1 recommended 10 categories with 73 indicators identified from 2,971 articles. After 3-round consultation, consensus was reached on 28 indicators focusing on the cardiovascular system (n = 4), oxygen transport system (n = 5), energy metabolism/metabolite level (n = 6), muscle/tissue damage level (n = 3), neurological function (n = 2), neuropsychological/psychological function (n = 3), endocrine function (n = 3), and exercise capacity (n = 2). Among these, 11 indicators were recommended as grade I: basic heart rate, heart-rate recovery time, heart rate variability, hemoglobin, blood lactic acid, urine protein, creatine kinase, reaction time, Borg Rating of Perceived Exertion Scale, testosterone/cortisol, and vertical jump height.

Conclusion

This study developed a reliable foundation for the comprehensive evaluation of MIF among soldiers.

Cognitive effects of prolonged continuous human-machine interaction: The case for mental state-based adaptive interfaces

Operators of complex systems across multiple domains (e.g., aviation, automotive, and nuclear power industry) are required to perform their tasks over prolonged and continuous periods of time. Mental fatigue as well as reduced cognitive flexibility, attention, and situational awareness all result from prolonged continuous use, putting at risk the safety and efficiency of complex operations. Mental state-based adaptive systems may be a solution to this problem. These systems infer the current mental state of an operator based on a selection of metrics ranging from operator independent measures (e.g., weather and time of day), to behavioral (e.g., reaction time and lane deviation) as well as physiological markers (e.g., electroencephalography and cardiac activity). The interaction between operator and system may then be adapted in one of many ways to mitigate any detected degraded cognitive state, thereby ensuring continued safety and efficiency. Depending on the task at hand and its specific problems, possible adaptations -usually based on machine learning estimations- e.g., include modifications of information, presentation modality or stimuli salience, as well as task scheduling. Research on adaptive systems is at the interface of several domains, including neuroergonomics, human factors, and human-computer interaction in an applied and ecological context, necessitating careful consideration of each of the aforementioned aspects. This article provides an overview of some of the key questions and aspects to be considered by researchers for the design of mental state-based adaptive systems, while also promoting their application during prolonged continuous use to pave the way toward safer and more efficient human-machine interaction.

Bimodal EEG-fNIRS in Neuroergonomics. Current Evidence and Prospects for Future Research

Neuroergonomics focuses on the brain signatures and associated mental states underlying behavior to design human-machine interfaces enhancing performance in the cognitive and physical domains. Brain imaging techniques such as functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) have been considered key methods for achieving this goal. Recent research stresses the value of combining EEG and fNIRS in improving these interface systems' mental state decoding abilities, but little is known about whether these improvements generalize over different paradigms and methodologies, nor about the potentialities for using these systems in the real world. We review 33 studies comparing mental state decoding accuracy between bimodal EEG-fNIRS and unimodal EEG and fNIRS in several subdomains of neuroergonomics. In light of these studies, we also consider the challenges of exploiting wearable versions of these systems in real-world contexts. Overall the studies reviewed suggest that bimodal EEG-fNIRS outperforms unimodal EEG or fNIRS despite major differences in their conceptual and methodological aspects. Much work however remains to be done to reach practical applications of bimodal EEG-fNIRS in naturalistic conditions. We consider these points to identify aspects of bimodal EEG-fNIRS research in which progress is expected or desired.

Concurrent fNIRS and EEG for Brain Function Investigation: A Systematic, Methodology-Focused Review

Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) stand as state-of-the-art techniques for non-invasive functional neuroimaging. On a unimodal basis, EEG has poor spatial resolution while presenting high temporal resolution. In contrast, fNIRS offers better spatial resolution, though it is constrained by its poor temporal resolution. One important merit shared by the EEG and fNIRS is that both modalities have favorable portability and could be integrated into a compatible experimental setup, providing a compelling ground for the development of a multimodal fNIRS-EEG integration analysis approach. Despite a growing number of studies using concurrent fNIRS-EEG designs reported in recent years, the methodological reference of past studies remains unclear. To fill this knowledge gap, this review critically summarizes the status of analysis methods currently used in concurrent fNIRS-EEG studies, providing an up-to-date overview and guideline for future projects to conduct concurrent fNIRS-EEG studies. A literature search was conducted using PubMed and Web of Science through 31 August 2021. After screening and qualification assessment, 92 studies involving concurrent fNIRS-EEG data recordings and analyses were included in the final methodological review. Specifically, three methodological categories of concurrent fNIRS-EEG data analyses, including EEG-informed fNIRS analyses, fNIRS-informed EEG analyses, and parallel fNIRS-EEG analyses, were identified and explained with detailed description. Finally, we highlighted current challenges and potential directions in concurrent fNIRS-EEG data analyses in future research.

Predicting the Degree of Distracted Driving Based on fNIRS Functional Connectivity: A Pilot Study

Distracted driving is one of the main causes of traffic accidents. By predicting the attentional state of drivers, it is possible to prevent distractions and promote safe driving. In this study, we developed a model that could predict the degree of distracted driving based on brain activity. Changes in oxyhemoglobin concentrations were measured in drivers while driving a real car using functional near-infrared spectroscopy (fNIRS). A regression model was constructed for each participant using functional connectivity as an explanatory variable and brake reaction time to random beeps while driving as an objective variable. As a result, we were able to construct a prediction model with the mean absolute error of 5.58 × 102 ms for the BRT of the 12 participants. Furthermore, the regression model with the highest prediction accuracy for each participant was analyzed to gain a better understanding of the neural basis of distracted driving. The 11 of 12 models that showed significant accuracy were classified into five clusters by hierarchical clustering based on their functional connectivity edges used in each cluster. The results showed that the combinations of the dorsal attention network (DAN)-sensory-motor network (SMN) and DAN-ventral attention network (VAN) connections were common in all clusters and that these networks were essential to predict the degree of distraction in complex multitask driving. They also confirmed the existence of multiple types of prediction models with different within- and between-network connectivity patterns. These results indicate that it is possible to predict the degree of distracted driving based on the driver's brain activity during actual driving. These results are expected to contribute to the development of safe driving systems and elucidate the neural basis of distracted driving.

Drowsiness detection from polysomnographic data using multivariate selfsimilarity and eigen-wavelet analysis

Because drowsiness is a major cause in vehicle accidents, its automated detection is critical. Scale-free temporal dynamics is known to be typical of physiological and body rhythms. The present work quantifies the benefits of applying a recent and original multivariate selfsimilarity analysis to several modalities of polysomnographic measurements (heart rate, blood pressure, electroencephalogram and respiration), from the MIT-BIH Polysomnographic Database, to better classify drowsiness-related sleep stages. Clinical relevance- This study shows that probing jointly temporal dynamics amongst polysomnographic measurements, with a proposed original multivariate multiscale approach, yields a gain of above 5% in the Area-under-Curve quanti-fying drowsiness-related sleep stage classification performance compared to univariate analysis.

A Wearable Prefrontal Cortex Oxygen Saturation Measurement System Based on Near Infrared Spectroscopy

The measurement of blood oxygen saturation in the prefrontal cortex (PFC), especially during sleep, is of great significance for clinical research. Herein, we present a wearable PFC oxygen saturation measurement system using dual-wavelength functional near-infrared spectroscopy. The system is well designed for user-friendly donning and has the advantages of comfort, convenience, portability, and affordability. The performance of the proposed system is investigated by the calibration and experimental results. The wearable system has demonstrated great potential to conduct the physiological monitoring of PFC, and it can be widely deployed in daily life.

EEG-Based Index for Timely Detecting User's Drowsiness Occurrence in Automotive Applications

Human errors are widely considered among the major causes of road accidents. Furthermore, it is estimated that more than 90% of vehicle crashes causing fatal and permanent injuries are directly related to mental tiredness, fatigue, and drowsiness of the drivers. In particular, driving drowsiness is recognized as a crucial aspect in the context of road safety, since drowsy drivers can suddenly lose control of the car. Moreover, the driving drowsiness episodes mostly appear suddenly without any prior behavioral evidence. The present study aimed at characterizing the onset of drowsiness in car drivers by means of a multimodal neurophysiological approach to develop a synthetic electroencephalographic (EEG)-based index, able to detect drowsy events. The study involved 19 participants in a simulated scenario structured in a sequence of driving tasks under different situations and traffic conditions. The experimental conditions were designed to induce prominent mental drowsiness in the final part. The EEG-based index, so-called “MDrow index”, was developed and validated to detect the driving drowsiness of the participants. The MDrow index was derived from the Global Field Power calculated in the Alpha EEG frequency band over the parietal brain sites. The results demonstrated the reliability of the proposed MDrow index in detecting the driving drowsiness experienced by the participants, resulting also more sensitive and timely sensible with respect to more conventional autonomic parameters, such as the EyeBlinks Rate and the Heart Rate Variability, and to subjective measurements (self-reports).

Orosomucoid: a promising biomarker for the assessment of exercise-induced fatigue triggered by basic combat training

Background

Orosomucoid (ORM) is a positive acute phase protein verified to be upregulated in various forms of exercise-induced fatigued (EIF) rodents. However, its association with EIF among human beings remained unknown. This study aimed to explore the association between serum ORM and EIF triggered by military basic combat training (BCT).

Methods

The degree of EIF were measured by Borg’s Rating of Perceived Exertion Scale (Borg-RPE-Scale®) as RPE score after BCT. Fifty-three male recruits were classified into three groups according to the RPE score: (1) group 1 (slight fatigue group): RPE score after BCT < 13; (2) group 2 (moderate fatigue group): RPE score after BCT = 13 or 14; (3) group 3 (severe fatigue group): RPE score after BCT > 14. The levels of blood ORM, lactate (LAC), cortisol and C-reactive protein (CRP) were determined before and after BCT. The diagnostic value of ORM was evaluated by receiver operating characteristic (ROC) curve analysis and logistic regression.

Results

After BCT, the level of LAC, CRP, and cortisol increased among all groups, but the changes had no significant between-group difference (all p > 0.05). The level of ORM had a specific significant increase in group 3 (p = 0.039), and the changes of ORM (ΔORM) had significant difference among groups (p = 0.033). ROC curve analysis showed that the estimated area under ROC curve for ΔORM was 0.724 (p = 0.009) with the recommended optimal cut-off value as 0.2565 mg/mL. Logistic analysis showed that recruits with ΔORM ≥ 0.2565 mg/mL had higher odds for suffering from severe EIF, 5.625 times (95% CI 1.542–20.523, p = 0.009) as large as those with ΔORM < 0.2565 mg/mL.

Conclusion

ORM might be a promising biomarker of severe EIF triggered by BCT among male recruits. Its potential optimal cut-off value regarding ΔORM was recommended to be 0.2565 mg/mL.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13102-022-00490-6.

Automatic Cognitive Fatigue Detection Using Wearable fNIRS and Machine Learning

Wearable sensors have increasingly been applied in healthcare to generate data and monitor patients unobtrusively. Their application for Brain-Computer Interfaces (BCI) allows for unobtrusively monitoring one's cognitive state over time. A particular state relevant in multiple domains is cognitive fatigue, which may impact performance and attention, among other capabilities. The monitoring of this state will be applied in real learning settings to detect and advise on effective break periods. In this study, two functional near-infrared spectroscopy (fNIRS) wearable devices were employed to build a BCI to automatically detect the state of cognitive fatigue using machine learning algorithms. An experimental procedure was developed to effectively induce cognitive fatigue that included a close-to-real digital lesson and two standard cognitive tasks: Corsi-Block task and a concentration task. Machine learning models were user-tuned to account for the individual dynamics of each participant, reaching classification accuracy scores of around 70.91 ± 13.67 %. We concluded that, although effective for some subjects, the methodology needs to be individually validated before being applied. Moreover, time on task was not a particularly determining factor for classification, i.e., to induce cognitive fatigue. Further research will include other physiological signals and human-computer interaction variables.

Enhancing Emotion Recognition Using Region-Specific Electroencephalogram Data and Dynamic Functional Connectivity

Recognizing the emotional states of humans through EEG signals are of great significance to the progress of human-computer interaction. The present study aimed to perform automatic recognition of music-evoked emotions through region-specific information and dynamic functional connectivity of EEG signals and a deep learning neural network. EEG signals of 15 healthy volunteers were collected when different emotions (high-valence-arousal vs. low-valence-arousal) were induced by a musical experimental paradigm. Then a sequential backward selection algorithm combining with deep neural network called Xception was proposed to evaluate the effect of different channel combinations on emotion recognition. In addition, we also assessed whether dynamic functional network of frontal cortex, constructed through different trial number, may affect the performance of emotion cognition. Results showed that the binary classification accuracy based on all 30 channels was 70.19%, the accuracy based on all channels located in the frontal region was 71.05%, and the accuracy based on the best channel combination in the frontal region was 76.84%. In addition, we found that the classification performance increased as longer temporal functional network of frontal cortex was constructed as input features. In sum, emotions induced by different musical stimuli can be recognized by our proposed approach though region-specific EEG signals and time-varying functional network of frontal cortex. Our findings could provide a new perspective for the development of EEG-based emotional recognition systems and advance our understanding of the neural mechanism underlying emotion processing.

Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights

Monitoring pilots' cognitive states becomes increasingly important in aviation. Physiological measurement can detect increased mental workload (MWL) even before performance declines. Yet, changes in MWL are rarely varied systematically and few studies control for confounding effects of other cognitive states. The present study targets these shortcomings by analysing the effects of stepwise increased MWL on cortical activation, while controlling for mental fatigue (MF). 35 participants conducted a simulated flight with an incorporated adapted n-back and monitoring task. We recorded cortical activation with concurrent EEG and fNIRS measurement, performance, self-reported MWL and MF. Our results show the successful manipulation of MWL without confounding effects of MF. Higher task difficulty elicited higher subjective MWL ratings, performance decline, higher frontal theta activity and reduced frontal deoxyhaemoglobin (Hbr) concentration. Using both EEG and fNIRS, we could discriminate all induced MWL levels. fNIRS was more sensitive to tasks with low difficulty, and EEG to tasks with high difficulty. Our findings further suggest a plateau effect for high MWL that could present an upper boundary to individual cognitive capacity. Our results highlight the benefits of physiological measurement in aviation, both for assessment of cognitive states and as a data source for adaptive assistance systems.

Viewing neurovascular coupling through the lens of combined EEG-fNIRS: A systematic review of current methods

Neurovascular coupling is a key physiological mechanism that occurs in the healthy human brain, and understanding this process has implications for understanding the aging and neuropsychiatric populations. Combined electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) has emerged as a promising, noninvasive tool for probing neurovascular interactions in humans. However, the utility of this approach critically depends on the methodological quality used for multimodal integration. Despite a growing number of combined EEG-fNIRS applications reported in recent years, the methodological rigor of past studies remains unclear, limiting the accurate interpretation of reported findings and hindering the translational application of this multimodal approach. To fill this knowledge gap, we critically evaluated various methodological aspects of previous combined EEG-fNIRS studies performed in healthy individuals. A literature search was conducted using PubMed and PsycINFO on June 28, 2021. Studies involving concurrent EEG and fNIRS measurements in awake and healthy individuals were selected. After screening and eligibility assessment, 96 studies were included in the methodological evaluation. Specifically, we critically reviewed various aspects of participant sampling, experimental design, signal acquisition, data preprocessing, outcome selection, data analysis, and results presentation reported in these studies. Altogether, we identified several notable strengths and limitations of the existing EEG-fNIRS literature. In light of these limitations and the features of combined EEG-fNIRS, recommendations are made to improve and standardize research practices to facilitate the use of combined EEG-fNIRS when studying healthy neurovascular coupling processes and alterations in neurovascular coupling among various populations.

Development of an EEG Headband for Stress Measurement on Driving Simulators

In this paper, we designed from scratch, realized, and characterized a six-channel EEG wearable headband for the measurement of stress-related brain activity during driving. The headband transmits data over WiFi to a laptop, and the rechargeable battery life is 10 h of continuous transmission. The characterization manifested a measurement error of 6 μV in reading EEG channels, and the bandwidth was in the range [0.8, 44] Hz, while the resolution was 50 nV exploiting the oversampling technique. Thanks to the full metrological characterization presented in this paper, we provide important information regarding the accuracy of the sensor because, in the literature, commercial EEG sensors are used even if their accuracy is not provided in the manuals. We set up an experiment using the driving simulator available in our laboratory at the University of Udine; the experiment involved ten volunteers who had to drive in three scenarios: manual, autonomous vehicle with a “gentle” approach, and autonomous vehicle with an “aggressive” approach. The aim of the experiment was to assess how autonomous driving algorithms impact EEG brain activity. To our knowledge, this is the first study to compare different autonomous driving algorithms in terms of drivers’ acceptability by means of EEG signals. The obtained results demonstrated that the estimated power of beta waves (related to stress) is higher in the manual with respect to autonomous driving algorithms, either “gentle” or “aggressive”.

Optimized electroencephalogram and functional near-infrared spectroscopy-based mental workload detection method for practical applications

Background

Mental workload is a critical consideration in complex man–machine systems design. Among various mental workload detection techniques, multimodal detection techniques integrating electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) signals have attracted considerable attention. However, existing EEG–fNIRS-based mental workload detection methods have certain defects, such as complex signal acquisition channels and low detection accuracy, which restrict their practical application.

Methods

The signal acquisition configuration was optimized by analyzing the feature importance in mental workload recognition model and a more accurate and convenient EEG–fNIRS-based mental workload detection method was constructed. A classical Multi-Task Attribute Battery (MATB) task was conducted with 20 participating volunteers. Subjective scale data, 64-channel EEG data, and two-channel fNIRS data were collected.

Results

A higher number of EEG channels correspond to higher detection accuracy. However, there is no obvious improvement in accuracy once the number of EEG channels reaches 26, with a four-level mental workload detection accuracy of 76.25 ± 5.21%. Partial results of physiological analysis verify the results of previous studies, such as that the θ power of EEG and concentration of O₂Hb in the prefrontal region increase while the concentration of HHb decreases with task difficulty. It was further observed, for the first time, that the energy of each band of EEG signals was significantly different in the occipital lobe region, and the power of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta_{1}$$\end{document}β1 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta_{2}$$\end{document}β2 bands in the occipital region increased significantly with task difficulty. The changing range and the mean amplitude of O₂Hb in high-difficulty tasks were significantly higher compared with those in low-difficulty tasks.

Conclusions

The channel configuration of EEG–fNIRS-based mental workload detection was optimized to 26 EEG channels and two frontal fNIRS channels. A four-level mental workload detection accuracy of 76.25 ± 5.21% was obtained, which is higher than previously reported results. The proposed configuration can promote the application of mental workload detection technology in military, driving, and other complex human–computer interaction systems.

Is There a Difference in Brain Functional Connectivity between Chinese Coal Mine Workers Who Have Engaged in Unsafe Behavior and Those Who Have Not?

(1) Background: As a world-recognized high-risk occupation, coal mine workers need various cognitive functions to process the surrounding information to cope with a large number of perceived hazards or risks. Therefore, it is necessary to explore the connection between coal mine workers’ neural activity and unsafe behavior from the perspective of cognitive neuroscience. This study explored the functional brain connectivity of coal mine workers who have engaged in unsafe behaviors (EUB) and those who have not (NUB). (2) Methods: Based on functional near-infrared spectroscopy (fNIRS), a total of 106 workers from the Hongliulin coal mine of Shaanxi North Mining Group, one of the largest modern coal mines in China, completed the test. Pearson’s Correlation Coefficient (COR) analysis, brain network analysis, and two-sample t-test were used to investigate the difference in brain functional connectivity between the two groups. (3) Results: The results showed that there were significant differences in functional brain connectivity between EUB and NUB among the frontopolar area (p = 0.002325), orbitofrontal area (p = 0.02102), and pars triangularis Broca’s area (p = 0.02888). Small-world properties existed in the brain networks of both groups, and the dorsolateral prefrontal cortex had significant differences in clustering coefficient (p = 0.0004), nodal efficiency (p = 0.0384), and nodal local efficiency (p = 0.0004). (4) Conclusions: This study is the first application of fNIRS to the field of coal mine safety. The fNIRS brain functional connectivity analysis is a feasible method to investigate the neuropsychological mechanism of unsafe behavior in coal mine workers in the view of brain science.

Recent Advances and Current Trends in Brain-Computer Interface (BCI) Research and Their Applications

Brain-Computer Interface (BCI) provides direct communication between the brain and an external device. BCI systems have become a trendy field of research in recent years. These systems can be used in a variety of applications to help both disabled and healthy people. Concerning significant BCI progress, we may assume that these systems are not very far from real-world applications. This review has taken into account current trends in BCI research. In this survey, one hundred most cited articles from the WOS database were selected over the last four years. This survey is divided into several sectors. These sectors are Medicine, Communication and Control, Entertainment, and Other BCI applications. The application area, recording method, signal acquisition types, and countries of origin have been identified in each article. This survey provides an overview of the BCI articles published from 2016 to 2020 and their current trends and advances in different application areas.

Detecting Soldiers' Fatigue Using Eye-Tracking Glasses: Practical Field Applications and Research Opportunities

INTRODUCTION

Objectively determining soldiers' fatigue levels could help prevent injuries or accidents resulting from inattention or decreased alertness. Eye-tracking technologies, such as optical eye tracking (OET) and electrooculography (EOG), are often used to monitor fatigue. Eyeblinks-especially blink frequency and blink duration-are known as easily observable and valid biomarkers of fatigue. Currently, various eye trackers (i.e., eye-tracking glasses) are available on the market using either OET or EOG technologies. These wearable eye trackers offer several advantages, including unobtrusive functionality, practicality, and low costs. However, several challenges and limitations must be considered when implementing these technologies in the field to monitor fatigue levels. This review investigates the feasibility of eye tracking in the field focusing on the practical applications in military operational environments.

MATERIALS AND METHOD

This paper summarizes the existing literature about eyeblink dynamics and available wearable eye-tracking technologies, exposing challenges and limitations, as well as discussing practical recommendations on how to improve the feasibility of eye tracking in the field.

RESULTS

So far, no eye-tracking glasses can be recommended for use in a demanding work environment. First, eyeblink dynamics are influenced by multiple factors; therefore, environments, situations, and individual behavior must be taken into account. Second, the glasses' placement, sunlight, facial or body movements, vibrations, and sweat can drastically decrease measurement accuracy. The placement of the eye cameras for the OET and the placement of the electrodes for the EOG must be chosen consciously, the sampling rate must be minimal 200 Hz, and software and hardware must be robust to resist any factors influencing eye tracking.

CONCLUSION

Monitoring physiological and psychological readiness of soldiers, as well as other civil professionals that face higher risks when their attention is impaired or reduced, is necessary. However, improvements to eye-tracking devices' hardware, calibration method, sampling rate, and algorithm are needed in order to accurately monitor fatigue levels in the field.

A bimodal deep learning architecture for EEG-fNIRS decoding of overt and imagined speech

OBJECTIVE

Brain-computer interfaces (BCI) studies are increasingly leveraging different attributes of multiple signal modalities simultaneously. Bimodal data acquisition protocols combining the temporal resolution of electroencephalography (EEG) with the spatial resolution of functional near-infrared spectroscopy (fNIRS) require novel approaches to decoding.

METHODS

We present an EEG-fNIRS Hybrid BCI that employs a new bimodal deep neural network architecture consisting of two convolutional sub-networks (subnets) to decode overt and imagined speech. Features from each subnet are fused before further feature extraction and classification. Nineteen participants performed overt and imagined speech in a novel cue-based paradigm enabling investigation of stimulus and linguistic effects on decoding.

RESULTS

Using the hybrid approach, classification accuracies (46.31% and 34.29% for overt and imagined speech, respectively (chance: 25%)) indicated a significant improvement on EEG used independently for imagined speech (p=0.020) while tending towards significance for overt speech (p=0.098). In comparison with fNIRS, significant improvements for both speech-types were achieved with bimodal decoding (p<0.001). There was a mean difference of ~12.02% between overt and imagined speech with accuracies as high as 87.18% and 53%. Deeper subnets enhanced performance while stimulus effected overt and imagined speech in significantly different ways.

CONCLUSION

The bimodal approach was a significant improvement on unimodal results for several tasks. Results indicate the potential of multi-modal deep learning for enhancing neural signal decoding.

SIGNIFICANCE

This novel architecture can be used to enhance speech decoding from bimodal neural signals.

A Practical Application for Quantitative Brain Fatigue Evaluation Based on Machine Learning and Ballistocardiogram

Brain fatigue is often associated with inattention, mental retardation, prolonged reaction time, decreased work efficiency, increased error rate, and other problems. In addition to the accumulation of fatigue, brain fatigue has become one of the important factors that harm our mental health. Therefore, it is of great significance to explore the practical and accurate brain fatigue detection method, especially for quantitative brain fatigue evaluation. In this study, a biomedical signal of ballistocardiogram (BCG), which does not require direct contact with human body, was collected by optical fiber sensor cushion during the whole process of cognitive tasks for 20 subjects. The heart rate variability (HRV) was calculated based on BCG signal. Machine learning classification model was built based on random forest to quantify and recognize brain fatigue. The results showed that: Firstly, the heart rate obtained from BCG signal was consistent with the result displayed by the medical equipment, and the absolute difference was less than 3 beats/min, and the mean error is 1.30 ± 0.81 beats/min; secondly, the random forest classifier for brain fatigue evaluation based on HRV can effectively identify the state of brain fatigue, with an accuracy rate of 96.54%; finally, the correlation between HRV and the accuracy was analyzed, and the correlation coefficient was as high as 0.98, which indicates that the accuracy can be used as an indicator for quantitative brain fatigue evaluation during the whole task. The results suggested that the brain fatigue quantification evaluation method based on the optical fiber sensor cushion and machine learning can carry out real-time brain fatigue detection on the human brain without disturbance, reduce the risk of human accidents in human–machine interaction systems, and improve mental health among the office and driving personnel.

Research on Driving Fatigue Alleviation Using Interesting Auditory Stimulation Based on VMD-MMSE

The accurate detection and alleviation of driving fatigue are of great significance to traffic safety. In this study, we tried to apply the modified multi-scale entropy (MMSE) approach, based on variational mode decomposition (VMD), to driving fatigue detection. Firstly, the VMD was used to decompose EEG into multiple intrinsic mode functions (IMFs), then the best IMFs and scale factors were selected using the least square method (LSM). Finally, the MMSE features were extracted. Compared with the traditional sample entropy (SampEn), the VMD-MMSE method can identify the characteristics of driving fatigue more effectively. The VMD-MMSE characteristics combined with a subjective questionnaire (SQ) were used to analyze the change trends of driving fatigue under two driving modes: normal driving mode and interesting auditory stimulation mode. The results show that the interesting auditory stimulation method adopted in this paper can effectively relieve driving fatigue. In addition, the interesting auditory stimulation method, which simply involves playing interesting auditory information on the vehicle-mounted player, can effectively relieve driving fatigue. Compared with traditional driving fatigue-relieving methods, such as sleeping and drinking coffee, this interesting auditory stimulation method can relieve fatigue in real-time when the driver is driving normally.

Study on the Effect of Driving Time on Fatigue of Grassland Road Based on EEG

In order to study the change law of the fatigue degree of grassland expressway drivers over time, this paper takes the semidesert grassland landscape of Xilinhot city as the experimental environment and takes the provincial highway S101 (K278-K424) as an example to design an actual driving test. Taking Urumqi, Inner Mongolia Autonomous Region, as the experimental section, combined with the Biopac MP150 multichannel physiological instrument and its auxiliary knowledge software and mathematical statistics methods, the relationship between EEG and time was studied. The test results show that the primary fatigue factor F 1 and the secondary fatigue factor F 2 can summarize the fatigue law characterized by 96.42% of EEG information. During 130 minutes of driving on the prairie highway, the periods of high fatigue were 105 minutes and 125 minutes, respectively. Driving fatigue can be divided into three stages over time: 5-65 min fatigue-free stage, 70-85 min fatigue transition stage, and 90-130 min fatigue stage. Fatigue changes over time. The law follows the Gaussian function and the sine function.

Real-World fNIRS Brain Activity Measurements during Ashtanga Vinyasa Yoga

Functional near-infrared spectroscopy (fNIRS) is often praised for its portability and robustness towards motion artifacts. While an increasing body of fNIRS research in real-world environments is emerging, most fNIRS studies are still conducted in laboratories, and do not incorporate larger movements performed by participants. This study extends fNIRS applications in real-world environments by conducting a single-subject observational study of a yoga practice with considerable movement (Ashtanga Vinyasa Yoga) in a participant’s natural environment (their apartment). The results show differences in cognitive load (prefrontal cortex activation) when comparing technically complex postures to relatively simple ones, but also some contrasts with surprisingly little difference. This study explores the boundaries of real-world cognitive load measurements, and contributes to the empirical knowledge base of using fNIRS in realistic settings. To the best of our knowledge, this is the first demonstration of fNIRS brain imaging recorded during any moving yoga practice. Future work with fNIRS should take advantage of this by accomplishing studies with considerable real-world movement.

A Review of EEG Signal Features and their Application in Driver Drowsiness Detection Systems

Detecting drowsiness in drivers, especially multi-level drowsiness, is a difficult problem that is often approached using neurophysiological signals as the basis for building a reliable system. In this context, electroencephalogram (EEG) signals are the most important source of data to achieve successful detection. In this paper, we first review EEG signal features used in the literature for a variety of tasks, then we focus on reviewing the applications of EEG features and deep learning approaches in driver drowsiness detection, and finally we discuss the open challenges and opportunities in improving driver drowsiness detection based on EEG. We show that the number of studies on driver drowsiness detection systems has increased in recent years and that future systems need to consider the wide variety of EEG signal features and deep learning approaches to increase the accuracy of detection.

Multimodal measurement approach to identify individuals with mild cognitive impairment: study protocol for a cross-sectional trial

INTRODUCTION

The diagnosis of mild cognitive impairment (MCI), that is, the transitory phase between normal age-related cognitive decline and dementia, remains a challenging task. It was observed that a multimodal approach (simultaneous analysis of several complementary modalities) can improve the classification accuracy. We will combine three noninvasive measurement modalities: functional near-infrared spectroscopy (fNIRS), electroencephalography and heart rate variability via ECG. Our aim is to explore neurophysiological correlates of cognitive performance and whether our multimodal approach can aid in early identification of individuals with MCI.

METHODS AND ANALYSIS

This study will be a cross-sectional with patients with MCI and healthy controls (HC). The neurophysiological signals will be measured during rest and while performing cognitive tasks: (1) Stroop, (2) N-back and (3) verbal fluency test (VFT). Main aims of statistical analysis are to (1) determine the differences in neurophysiological responses of HC and MCI, (2) investigate relationships between measures of cognitive performance and neurophysiological responses and (3) investigate whether the classification accuracy can be improved by using our multimodal approach. To meet these targets, statistical analysis will include machine learning approaches.This is, to the best of our knowledge, the first study that applies simultaneously these three modalities in MCI and HC. We hypothesise that the multimodal approach improves the classification accuracy between HC and MCI as compared with a unimodal approach. If our hypothesis is verified, this study paves the way for additional research on multimodal approaches for dementia research and fosters the exploration of new biomarkers for an early detection of nonphysiological age-related cognitive decline.

ETHICS AND DISSEMINATION

Ethics approval was obtained from the local Ethics Committee (reference: 83/19). Data will be shared with the scientific community no more than 1 year following completion of study and data assembly.

TRIAL REGISTRATION NUMBER

ClinicalTrials.gov, NCT04427436, registered on 10 June 2020, https://clinicaltrials.gov/ct2/show/study/NCT04427436.

Vector Phase Analysis Approach for Sleep Stage Classification: A Functional Near-Infrared Spectroscopy-Based Passive Brain-Computer Interface

A passive brain-computer interface (BCI) based upon functional near-infrared spectroscopy (fNIRS) brain signals is used for earlier detection of human drowsiness during driving tasks. This BCI modality acquired hemodynamic signals of 13 healthy subjects from the right dorsolateral prefrontal cortex (DPFC) of the brain. Drowsiness activity is recorded using a continuous-wave fNIRS system and eight channels over the right DPFC. During the experiment, sleep-deprived subjects drove a vehicle in a driving simulator while their cerebral oxygen regulation (CORE) state was continuously measured. Vector phase analysis (VPA) was used as a classifier to detect drowsiness state along with sleep stage-based threshold criteria. Extensive training and testing with various feature sets and classifiers are done to justify the adaptation of threshold criteria for any subject without requiring recalibration. Three statistical features (mean oxyhemoglobin, signal peak, and the sum of peaks) along with six VPA features (trajectory slopes of VPA indices) were used. The average accuracies for the five classifiers are 90.9% for discriminant analysis, 92.5% for support vector machines, 92.3% for nearest neighbors, 92.4% for both decision trees, and ensembles over all subjects' data. Trajectory slopes of CORE vector magnitude and angle: m(|R|) and m(∠R) are the best-performing features, along with ensemble classifier with the highest accuracy of 95.3% and minimum computation time of 40 ms. The statistical significance of the results is validated with a p-value of less than 0.05. The proposed passive BCI scheme demonstrates a promising technique for online drowsiness detection using VPA along with sleep stage classification.

A Methodological Review of fNIRS in Driving Research: Relevance to the Future of Autonomous Vehicles

As automobile manufacturers have begun to design, engineer, and test autonomous driving systems of the future, brain imaging with functional near-infrared spectroscopy (fNIRS) can provide unique insights about cognitive processes associated with evolving levels of autonomy implemented in the automobile. Modern fNIRS devices provide a portable, relatively affordable, and robust form of functional neuroimaging that allows researchers to investigate brain function in real-world environments. The trend toward "naturalistic neuroscience" is evident in the growing number of studies that leverage the methodological flexibility of fNIRS, and in doing so, significantly expand the scope of cognitive function that is accessible to observation via functional brain imaging (i.e., from the simulator to on-road scenarios). While more than a decade's worth of study in this field of fNIRS driving research has led to many interesting findings, the number of studies applying fNIRS during autonomous modes of operation is limited. To support future research that directly addresses this lack in autonomous driving research with fNIRS, we argue that a cogent distillation of the methods used to date will help facilitate and streamline this research of tomorrow. To that end, here we provide a methodological review of the existing fNIRS driving research, with the overarching goal of highlighting the current diversity in methodological approaches. We argue that standardization of these approaches will facilitate greater overlap of methods by researchers from all disciplines, which will, in-turn, allow for meta-analysis of future results. We conclude by providing recommendations for advancing the use of such fNIRS technology in furthering understanding the adoption of safe autonomous vehicle technology.

A BCI Based Alerting System for Attention Recovery of UAV Operators

As unmanned aerial vehicles have become popular, the number of accidents caused by an operator's inattention have increased. To prevent such accidents, the operator should maintain an attention status. However, limited research has been conducted on the brain-computer interface (BCI)-based system with an alerting module for the operator's attention recovery of unmanned aerial vehicles. Therefore, we introduce a detection and alerting system that prevents an unmanned aerial vehicle operator from falling into inattention status by using the operator's electroencephalogram signal. The proposed system consists of the following three components: a signal processing module, which collects and preprocesses an electroencephalogram signal of an operator, an inattention detection module, which determines whether an inattention status occurred based on the preprocessed signal, and, lastly, an alert providing module that presents stimulus to an operator when inattention is detected. As a result of evaluating the performance with a real-world dataset, it was shown that the proposed system successfully contributed to the recovery of operator attention in the evaluating dataset, although statistical significance could not be established due to the small number of subjects.

Convolutional Neural Network for Drowsiness Detection Using EEG Signals

Drowsiness detection (DD) has become a relevant area of active research in biomedical signal processing. Recently, various deep learning (DL) researches based on the EEG signals have been proposed to detect fatigue conditions. The research presented in this paper proposes an EEG classification system for DD based on DL networks. However, the proposed DD system is mainly realized into two procedures; (i) data acquisition and (ii) model analysis. For the data acquisition procedure, two key steps are considered, which are the signal collection using a wearable Emotiv EPOC+ headset to record 14 channels of EEG, and the signal annotation. Furthermore, a data augmentation (DA) step has been added to the proposed system to overcome the problem of over-fitting and to improve accuracy. As regards the model analysis, a comparative study is also introduced in this paper to argue the choice of DL architecture and frameworks used in our DD system. In this sense, The proposed DD protocol makes use of a convolutional neural network (CNN) architecture implemented using the Keras library. The results showed a high accuracy value (90.42%) in drowsy/awake discrimination and revealed the efficiency of the proposed DD system compared to other research works.

Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination's complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to enhance brain–computer interface (BCI) performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI (hBCI) involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses (PRISMA) during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI (EEG + fNIRS) system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go.

Shedding light on the prefrontal correlates of mental workload in simulated driving: a functional near-infrared spectroscopy study

Optimal mental workload plays a key role in driving performance. Thus, driver-assisting systems that automatically adapt to a drivers current mental workload via brain-computer interfacing might greatly contribute to traffic safety. To design economic brain computer interfaces that do not compromise driver comfort, it is necessary to identify brain areas that are most sensitive to mental workload changes. In this study, we used functional near-infrared spectroscopy and subjective ratings to measure mental workload in two virtual driving environments with distinct demands. We found that demanding city environments induced both higher subjective workload ratings as well as higher bilateral middle frontal gyrus activation than less demanding country environments. A further analysis with higher spatial resolution revealed a center of activation in the right anterior dorsolateral prefrontal cortex. The area is highly involved in spatial working memory processing. Thus, a main component of drivers' mental workload in complex surroundings might stem from the fact that large amounts of spatial information about the course of the road as well as other road users has to constantly be upheld, processed and updated. We propose that the right middle frontal gyrus might be a suitable region for the application of powerful small-area brain computer interfaces.

Electro-Encephalography and Electro-Oculography in Aeronautics: A Review Over the Last Decade (2010-2020)

Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges.

Sex Differences in Time-Domain and Frequency-Domain Heart Rate Variability Measures of Fatigued Drivers

The effects of fatigue on a driver's autonomic nervous system (ANS) were investigated through heart rate variability (HRV) measures considering the difference of sex. Electrocardiogram (ECG) data from 18 drivers were recorded during a simulator-based driving experiment. Thirteen short-term HRV measures were extracted through time-domain and frequency-domain methods. First, differences in HRV measures related to mental state (alert or fatigued) were analyzed in all subjects. Then, sex-specific changes between alert and fatigued states were investigated. Finally, sex differences between alert and fatigued states were compared. For all subjects, ten measures showed significant differences (Mann-Whitney U test, p < 0.01) between different mental states. In male and female drivers, eight and four measures, respectively, showed significant differences between different mental states. Six measures showed significant differences between males and females in an alert state, while ten measures showed significant sex differences in a fatigued state. In conclusion, fatigue impacts drivers' ANS activity, and this impact differs by sex; more differences exist between male and female drivers' ANS activity in a fatigued state than in an alert state.