Towards measurement of brain function in operational environments

A novel deep-learning model based on τ-shaped convolutional network (τNet) with long short-term memory (LSTM) for physiological fatigue detection from EEG and EOG signals

In recent years, fatigue driving has become the main cause of traffic accidents, leading to increased attention towards fatigue detection systems. However, the pooling and strided convolutional operations in fatigue detection algorithm based on traditional deep learning methods may led to the loss of some useful information. This paper proposed a novel τ -shaped convolutional network ( τ Net ) aiming to address this issue. Unlike traditional network structures, τ Net incorporates the operations of upsampling features and concatenating high- and low-level features, enabling full utilization of useful information. Moreover, considering that the fatigue state is a mental state involving temporal evolution, we proposed the novel long short-term memory (LSTM)- τ -shaped convolutional network (LSTM- τ Net ), a parallel structure composed of LSTM and τ Net for fatigue detection, where τ Net extracts time-invariant features with location information, and LSTM extracts long temporal dependencies. We compared LSTM- τ Net with six competing methods based on two datasets. Results showed that the proposed algorithm achieved higher classification accuracy than the other methods, with 94.25% on EEG data (binary classification) and 82.19% on EOG data (triple classification). Additionally, the proposed algorithm exhibits low computational cost, good training stability, and robustness against insufficient training. Therefore, it is promising for further implementation of fatigue online detection systems.

Physiological Metrics of Surgical Difficulty and Multi-Task Requirement during Robotic Surgery Skills

Previous studies in robotic-assisted surgery (RAS) have studied cognitive workload by modulating surgical task difficulty, and many of these studies have relied on self-reported workload measurements. However, contributors to and their effects on cognitive workload are complex and may not be sufficiently summarized by changes in task difficulty alone. This study aims to understand how multi-task requirement contributes to the prediction of cognitive load in RAS under different task difficulties. Multimodal physiological signals (EEG, eye-tracking, HRV) were collected as university students performed simulated RAS tasks consisting of two types of surgical task difficulty under three different multi-task requirement levels. EEG spectral analysis was sensitive enough to distinguish the degree of cognitive workload under both surgical conditions (surgical task difficulty/multi-task requirement). In addition, eye-tracking measurements showed differences under both conditions, but significant differences of HRV were observed in only multi-task requirement conditions. Multimodal-based neural network models have achieved up to 79% accuracy for both surgical conditions.

An Evaluation of the EEG Alpha-to-Theta and Theta-to-Alpha Band Ratios as Indexes of Mental Workload

Many research works indicate that EEG bands, specifically the alpha and theta bands, have been potentially helpful cognitive load indicators. However, minimal research exists to validate this claim. This study aims to assess and analyze the impact of the alpha-to-theta and the theta-to-alpha band ratios on supporting the creation of models capable of discriminating self-reported perceptions of mental workload. A dataset of raw EEG data was utilized in which 48 subjects performed a resting activity and an induced task demanding exercise in the form of a multitasking SIMKAP test. Band ratios were devised from frontal and parietal electrode clusters. Building and model testing was done with high-level independent features from the frequency and temporal domains extracted from the computed ratios over time. Target features for model training were extracted from the subjective ratings collected after resting and task demand activities. Models were built by employing Logistic Regression, Support Vector Machines and Decision Trees and were evaluated with performance measures including accuracy, recall, precision and f1-score. The results indicate high classification accuracy of those models trained with the high-level features extracted from the alpha-to-theta ratios and theta-to-alpha ratios. Preliminary results also show that models trained with logistic regression and support vector machines can accurately classify self-reported perceptions of mental workload. This research contributes to the body of knowledge by demonstrating the richness of the information in the temporal, spectral and statistical domains extracted from the alpha-to-theta and theta-to-alpha EEG band ratios for the discrimination of self-reported perceptions of mental workload.

Cognitive Workload of Tugboat Captains in Realistic Scenarios: Adaptive Spatial Filtering for Transfer Between Conditions

Changing and often class-dependent non-stationarities of signals are a big challenge in the transfer of common findings in cognitive workload estimation using Electroencephalography (EEG) from laboratory experiments to realistic scenarios or other experiments. Additionally, it often remains an open question whether actual cognitive workload reflected by brain signals was the main contribution to the estimation or discriminative and class-dependent muscle and eye activity, which can be secondary effects of changing workload levels. Within this study, we investigated a novel approach to spatial filtering based on beamforming adapted to changing settings. We compare it to no spatial filtering and Common Spatial Patterns (CSP). We used a realistic maneuvering task, as well as an auditory n-back secondary task on a tugboat simulator as two different conditions to induce workload changes on professional tugboat captains. Apart from the typical within condition classification, we investigated the ability of the different classification methods to transfer between the n-back condition and the maneuvering task. The results show a clear advantage of the proposed approach over the others in the challenging transfer setting. While no filtering leads to lowest within-condition normalized classification loss on average in two scenarios (22 and 10%), our approach using adaptive beamforming (30 and 18%) performs comparably to CSP (33 and 15%). Importantly, in the transfer from one to another setting, no filtering and CSP lead to performance around chance level (45 to 53%), while our approach in contrast is the only one capable of classifying in all other scenarios (34 and 35%) with a significant difference from chance level. The changing signal composition over the scenarios leads to a need to adapt the spatial filtering in order to be transferable. With our approach, the transfer is successful due to filtering being optimized for the extraction of neural components and additional investigation of their scalp patterns revealed mainly neural origin. Interesting findings are that rather the patterns slightly change between conditions. We conclude that the approaches with low normalized loss depend on eye and muscle activity which is successful for classification within conditions, but fail in the classifier transfer since eye and muscle contributions are highly condition-specific.

Real-time prediction of short-timescale fluctuations in cognitive workload

Human operators often experience large fluctuations in cognitive workload over seconds timescales that can lead to sub-optimal performance, ranging from overload to neglect. Adaptive automation could potentially address this issue, but to do so it needs to be aware of real-time changes in operators' spare cognitive capacity, so it can provide help in times of peak demand and take advantage of troughs to elicit operator engagement. However, it is unclear whether rapid changes in task demands are reflected in similarly rapid fluctuations in spare capacity, and if so what aspects of responses to those demands are predictive of the current level of spare capacity. We used the ISO standard detection response task (DRT) to measure cognitive workload approximately every 4 s in a demanding task requiring monitoring and refueling of a fleet of simulated unmanned aerial vehicles (UAVs). We showed that the DRT provided a valid measure that can detect differences in workload due to changes in the number of UAVs. We used cross-validation to assess whether measures related to task performance immediately preceding the DRT could predict detection performance as a proxy for cognitive workload. Although the simple occurrence of task events had weak predictive ability, composite measures that tapped operators' situational awareness with respect to fuel levels were much more effective. We conclude that cognitive workload does vary rapidly as a function of recent task events, and that real-time predictive models of operators' cognitive workload provide a potential avenue for automation to adapt without an ongoing need for intrusive workload measurements.

P300 Measures and Drive-Related Risks: A Systematic Review and Meta-Analysis

Detecting signs for an increased level of risk during driving are critical for the effective prevention of road traffic accidents. The current study searched for literature through major databases such as PubMed, EBSCO, IEEE, and ScienceDirect. A total of 14 articles that measured P300 components in relation to driving tasks were included for a systematic review and meta-analysis. The risk factors investigated in the reviewed articles were summarized in five categories, including reduced attention, distraction, alcohol, challenging situations on the road, and negative emotion. A meta-analysis was conducted at both behavioral and neural levels. Behavioral performance was measured by the reaction time and driving performance, while the neural response was measured by P300 amplitude and latency. A significant increase in reaction time was identified when drivers were exposed to the risk factors. In addition, the significant effects of a reduced P300 amplitude and prolonged P300 latency indicated a reduced capacity for cognitive information processing. There was a tendency of driving performance decrement in relation to the risk factors, however, the effect was non-significant due to considerable variations and heterogeneity across the included studies. The results led to the conclusion that the P300 amplitude and latency are reliable indicators and predictors of the increased risk in driving. Future applications of the P300-based brain–computer interface (BCI) system may make considerable contributions toward preventing road traffic accidents.

Neuroergonomic Assessment of Wheelchair Control Using Mobile fNIRS

For over two centuries, the wheelchair has been one of the most common assistive devices for individuals with locomotor impairments without many modifications. Wheelchair control is a complex motor task that increases both the physical and cognitive workload. New wheelchair interfaces, including Power Assisted devices, can further augment users by reducing the required physical effort, however little is known on the mental effort implications. In this study, we adopted a neuroergonomic approach utilizing mobile and wireless functional near infrared spectroscopy (fNIRS) based brain monitoring of physically active participants. 48 volunteers (30 novice and 18 experienced) self-propelled on a wheelchair with and without a PowerAssist interface in both simple and complex realistic environments. Results indicated that as expected, the complex more difficult environment led to lower task performance complemented by higher prefrontal cortex activity compared to the simple environment. The use of the PowerAssist feature had significantly lower brain activation compared to traditional manual control only for novices. Expertise led to a lower brain activation pattern within the middle frontal gyrus, complemented by performance metrics that involve lower cognitive workload. Results here confirm the potential of the Neuroergonomic approach and that direct neural activity measures can complement and enhance task performance metrics. We conclude that the cognitive workload benefits of PowerAssist are more directed to new users and difficult settings. The approach demonstrated here can be utilized in future studies to enable greater personalization and understanding of mobility interfaces within real-world dynamic environments.

Electroencephalographic Workload Indicators During Teleoperation of an Unmanned Aerial Vehicle Shepherding a Swarm of Unmanned Ground Vehicles in Contested Environments

Background: Although many electroencephalographic (EEG) indicators have been proposed in the literature, it is unclear which of the power bands and various indices are best as indicators of mental workload. Spectral powers (Theta, Alpha, and Beta) and ratios (Beta/(Alpha + Theta), Theta/Alpha, Theta/Beta) were identified in the literature as prominent indicators of cognitive workload.

Objective: The aim of the present study is to identify a set of EEG indicators that can be used for the objective assessment of cognitive workload in a multitasking setting and as a foundational step toward a human-autonomy augmented cognition system.

Methods: The participants' perceived workload was modulated during a teleoperation task involving an unmanned aerial vehicle (UAV) shepherding a swarm of unmanned ground vehicles (UGVs). Three sources of data were recorded from sixteen participants (n = 16): heart rate (HR), EEG, and subjective indicators of the perceived workload using the Air Traffic Workload Input Technique (ATWIT).

Results: The HR data predicted the scores from ATWIT. Nineteen common EEG features offered a discriminatory power of the four workload setups with high classification accuracy (82.23%), exhibiting a higher sensitivity than ATWIT and HR.

Conclusion: The identified set of features represents EEG indicators for the objective assessment of cognitive workload across subjects. These common indicators could be used for augmented intelligence in human-autonomy teaming scenarios, and form the basis for our work on designing a closed-loop augmented cognition system for human-swarm teaming.

The neural underpinnings of music listening under different attention conditions

Most studies examining the neural underpinnings of music listening have no specific instruction on how to process the presented musical pieces. In this study, we explicitly manipulated the participants' focus of attention while they listened to the musical pieces. We used an ecologically valid experimental setting by presenting the musical stimuli simultaneously with naturalistic film sequences. In one condition, the participants were instructed to focus their attention on the musical piece (attentive listening), whereas in the second condition, the participants directed their attention to the film sequence (passive listening). We used two instrumental musical pieces: an electronic pop song, which was a major hit at the time of testing, and a classical musical piece. During music presentation, we measured electroencephalographic oscillations and responses from the autonomic nervous system (heart rate and high-frequency heart rate variability). During passive listening to the pop song, we found strong event-related synchronizations in all analyzed frequency bands (theta, lower alpha, upper alpha, lower beta, and upper beta). The neurophysiological responses during attentive listening to the pop song were similar to those of the classical musical piece during both listening conditions. Thus, the focus of attention had a strong influence on the neurophysiological responses to the pop song, but not on the responses to the classical musical piece. The electroencephalographic responses during passive listening to the pop song are interpreted as a neurophysiological and psychological state typically observed when the participants are 'drawn into the music'.

Carbon nanofiber-filled conductive silicone elastomers as soft, dry bioelectronic interfaces

Soft and pliable conductive polymer composites hold promise for application as bioelectronic interfaces such as for electroencephalography (EEG). In clinical, laboratory, and real-world EEG there is a desire for dry, soft, and comfortable interfaces to the scalp that are capable of relaying the μV-level scalp potentials to signal processing electronics. A key challenge is that most material approaches are sensitive to deformation-induced shifts in electrical impedance associated with decreased signal-to-noise ratio. This is a particular concern in real-world environments where human motion is present. The entire set of brain information outside of tightly controlled laboratory or clinical settings are currently unobtainable due to this challenge. Here we explore the performance of an elastomeric material solution purposefully designed for dry, soft, comfortable scalp contact electrodes for EEG that is specifically targeted to have flat electrical impedance response to deformation to enable utilization in real world environments. A conductive carbon nanofiber filled polydimethylsiloxane (CNF-PDMS) elastomer was evaluated at three fill ratios (3, 4 and 7 volume percent). Electromechanical testing data is presented showing the influence of large compressive deformations on electrical impedance as well as the impact of filler loading on the elastomer stiffness. To evaluate usability for EEG, pre-recorded human EEG signals were replayed through the contact electrodes subjected to quasi-static compressive strains between zero and 35%. These tests show that conductive filler ratios well above the electrical percolation threshold are desirable in order to maximize signal-to-noise ratio and signal correlation with an ideal baseline. Increasing fill ratios yield increasingly flat electrical impedance response to large applied compressive deformations with a trade in increased material stiffness, and with nominal electrical impedance tunable over greater than 4 orders of magnitude. EEG performance was independent of filler loading above 4 vol % CNF (< 103 ohms).

Electroencephalography Based Analysis of Working Memory Load and Affective Valence in an N-back Task with Emotional Stimuli

Most brain-based measures of the electroencephalogram (EEG) are used in highly controlled lab environments and only focus on narrow mental states (e.g., working memory load). However, we assume that outside the lab complex multidimensional mental states are evoked. This could potentially create interference between EEG signatures used for identification of specific mental states. In this study, we aimed to investigate more realistic conditions and therefore induced a combination of working memory load and affective valence to reveal potential interferences in EEG measures. To induce changes in working memory load and affective valence, we used a paradigm which combines an N-back task (for working memory load manipulation) with a standard method to induce affect (affective pictures taken from the International Affective Picture System (IAPS) database). Subjective ratings showed that the experimental task was successful in inducing working memory load as well as affective valence. Additionally, performance measures were analyzed and it was found that behavioral performance decreased with increasing workload as well as negative valence, showing that affective valence can have an effect on cognitive processing. These findings are supported by changes in frontal theta and parietal alpha power, parameters used for measuring of working memory load in the EEG. However, these EEG measures are influenced by the negative valence condition as well and thereby show that detection of working memory load is sensitive to affective contexts. Unexpectedly, we did not find any effects for EEG measures typically used for affective valence detection (Frontal Alpha Asymmetry (FAA)). Therefore we assume that the FAA measure might not be usable if cognitive workload is induced simultaneously. We conclude that future studies should account for potential context-specifity of EEG measures.

ICA-Derived EEG Correlates to Mental Fatigue, Effort, and Workload in a Realistically Simulated Air Traffic Control Task

Electroencephalograph (EEG) has been increasingly studied to identify distinct mental factors when persons perform cognitively demanding tasks. However, most of these studies examined EEG correlates at channel domain, which suffers the limitation that EEG signals are the mixture of multiple underlying neuronal sources due to the volume conduction effect. Moreover, few studies have been conducted in real-world tasks. To precisely probe EEG correlates with specific neural substrates to mental factors in real-world tasks, the present study examined EEG correlates to three mental factors, i.e., mental fatigue [also known as time-on-task (TOT) effect], workload and effort, in EEG component signals, which were obtained using an independent component analysis (ICA) on high-density EEG data. EEG data were recorded when subjects performed a realistically simulated air traffic control (ATC) task for 2 h. Five EEG independent component (IC) signals that were associated with specific neural substrates (i.e., the frontal, central medial, motor, parietal, occipital areas) were identified. Their spectral powers at their corresponding dominant bands, i.e., the theta power of the frontal IC and the alpha power of the other four ICs, were detected to be correlated to mental workload and effort levels, measured by behavioral metrics. Meanwhile, a linear regression analysis indicated that spectral powers at five ICs significantly increased with TOT. These findings indicated that different levels of mental factors can be sensitively reflected in EEG signals associated with various brain functions, including visual perception, cognitive processing, and motor outputs, in real-world tasks. These results can potentially aid in the development of efficient operational interfaces to ensure productivity and safety in ATC and beyond.

The Berlin Brain-Computer Interface: Progress Beyond Communication and Control

The combined effect of fundamental results about neurocognitive processes and advancements in decoding mental states from ongoing brain signals has brought forth a whole range of potential neurotechnological applications. In this article, we review our developments in this area and put them into perspective. These examples cover a wide range of maturity levels with respect to their applicability. While we assume we are still a long way away from integrating Brain-Computer Interface (BCI) technology in general interaction with computers, or from implementing neurotechnological measures in safety-critical workplaces, results have already now been obtained involving a BCI as research tool. In this article, we discuss the reasons why, in some of the prospective application domains, considerable effort is still required to make the systems ready to deal with the full complexity of the real world.

Identifying Granger causal relationships between neural power dynamics and variables of interest

Power modulations of oscillations in electro- and magnetoencephalographic (EEG/MEG) signals have been linked to a wide range of brain functions. To date, most of the evidence is obtained by correlating bandpower fluctuations to specific target variables such as reaction times or task ratings, while the causal links between oscillatory activity and behavior remain less clear. Here, we propose to identify causal relationships by the statistical concept of Granger causality, and we investigate which methods are bests suited to reveal Granger causal links between the power of brain oscillations and experimental variables. As an alternative to testing such causal links on the sensor level, we propose to linearly combine the information contained in each sensor in order to create virtual channels, corresponding to estimates of underlying brain oscillations, the Granger-causal relations of which may be assessed. Such linear combinations of sensor can be given by source separation methods such as, for example, Independent Component Analysis (ICA) or by the recently developed Source Power Correlation (SPoC) method. Here we compare Granger causal analysis on power dynamics obtained from i) sensor directly, ii) spatial filtering methods that do not optimize for Granger causality (ICA and SPoC), and iii) a method that directly optimizes spatial filters to extract sources the power dynamics of which maximally Granger causes a given target variable. We refer to this method as Granger Causal Power Analysis (GrangerCPA). Using both simulated and real EEG recordings, we find that computing Granger causality on channel-wise spectral power suffers from a poor signal-to-noise ratio due to volume conduction, while all three multivariate approaches alleviate this issue. In real EEG recordings from subjects performing self-paced foot movements, all three multivariate methods identify neural oscillations with motor-related patterns at a similar performance level. In an auditory perception task, the application of GrangerCPA reveals significant Granger-causal links between alpha oscillations and reaction times in more subjects compared to conventional methods.

Usability of four commercially-oriented EEG systems

Electroencephalography (EEG) holds promise as a neuroimaging technology that can be used to understand how the human brain functions in real-world, operational settings while individuals move freely in perceptually-rich environments. In recent years, several EEG systems have been developed that aim to increase the usability of the neuroimaging technology in real-world settings. Here, the usability of three wireless EEG systems from different companies are compared to a conventional wired EEG system, BioSemi's ActiveTwo, which serves as an established laboratory-grade 'gold standard' baseline. The wireless systems compared include Advanced Brain Monitoring's B-Alert X10, Emotiv Systems' EPOC and the 2009 version of QUASAR's Dry Sensor Interface 10-20. The design of each wireless system is discussed in relation to its impact on the system's usability as a potential real-world neuroimaging system. Evaluations are based on having participants complete a series of cognitive tasks while wearing each of the EEG acquisition systems. This report focuses on the system design, usability factors and participant comfort issues that arise during the experimental sessions. In particular, the EEG systems are assessed on five design elements: adaptability of the system for differing head sizes, subject comfort and preference, variance in scalp locations for the recording electrodes, stability of the electrical connection between the scalp and electrode, and timing integration between the EEG system, the stimulus presentation computer and other external events.

Algorithm for automatic analysis of electro-oculographic data

BACKGROUND

Large amounts of electro-oculographic (EOG) data, recorded during electroencephalographic (EEG) measurements, go underutilized. We present an automatic, auto-calibrating algorithm that allows efficient analysis of such data sets.

METHODS

The auto-calibration is based on automatic threshold value estimation. Amplitude threshold values for saccades and blinks are determined based on features in the recorded signal. The performance of the developed algorithm was tested by analyzing 4854 saccades and 213 blinks recorded in two different conditions: a task where the eye movements were controlled (saccade task) and a task with free viewing (multitask). The results were compared with results from a video-oculography (VOG) device and manually scored blinks.

RESULTS

The algorithm achieved 93% detection sensitivity for blinks with 4% false positive rate. The detection sensitivity for horizontal saccades was between 98% and 100%, and for oblique saccades between 95% and 100%. The classification sensitivity for horizontal and large oblique saccades (10 deg) was larger than 89%, and for vertical saccades larger than 82%. The duration and peak velocities of the detected horizontal saccades were similar to those in the literature. In the multitask measurement the detection sensitivity for saccades was 97% with a 6% false positive rate.

CONCLUSION

The developed algorithm enables reliable analysis of EOG data recorded both during EEG and as a separate metrics.

Comparison of dry and gel based electrodes for p300 brain-computer interfaces

Most brain–computer interfaces (BCIs) rely on one of three types of signals in the electroencephalogram (EEG): P300s, steady-state visually evoked potentials, and event-related desynchronization. EEG is typically recorded non-invasively with electrodes mounted on the human scalp using conductive electrode gel for optimal impedance and data quality. The use of electrode gel entails serious problems that are especially pronounced in real-world settings when experts are not available. Some recent work has introduced dry electrode systems that do not require gel, but often introduce new problems such as comfort and signal quality. The principal goal of this study was to assess a new dry electrode BCI system in a very common task: spelling with a P300 BCI. A total of 23 subjects used a P300 BCI to spell the word “LUCAS” while receiving real-time, closed-loop feedback. The dry system yielded classification accuracies that were similar to those obtained with gel systems. All subjects completed a questionnaire after data recording, and all subjects stated that the dry system was not uncomfortable. This is the first field validation of a dry electrode P300 BCI system, and paves the way for new research and development with EEG recording systems that are much more practical and convenient in field settings than conventional systems.

Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired?

The present study examines changes in a variety of oculomotoric variables as a function of increasing sleepiness in 129 participants, who have been passed through a broad range of subjective alertness. Up to now, spontaneous eye blinks are the most promising biosignal for in-car sleepiness warnings. Reviewing the current literature on eye movements and fatigue, experimental data are provided including additional indicative oculomotoric parameters; inter-individual differences in the experiments were also assessed. Here, self-rated alertness decreased over six steps on average and proved itself a reliable measurement. Regarding oculomotoric parameters, blink duration, delay of lid reopening, blink interval and standardised lid closure speed were identified as the best indicators of subjective as well as objective sleepiness. Saccadic parameters and fixation durations also showed specific changes with increasing sleepiness. Substantial inter-individual differences in all of these variables were illustrated. Oculomotoric parameters were linked to three different components of sleepiness while driving: a) deactivation; b) decreasing attention, resulting in disinhibition of spontaneous blinks and reflexive saccades; c) increasing attempts of self-activation. Finally, implications for the development of drowsiness detection devices were discussed.

EEG-based mental fatigue measurement using multi-class support vector machines with confidence estimate

OBJECTIVE

Automatic measurement and monitoring of mental fatigue are invaluable for preventing mental-fatigue related accidents. We test an EEG-based mental-fatigue monitoring system using a probabilistic-based support vector-machines (SVM) method.

METHODS

Ten subjects underwent 25-h sleep deprivation experiments with EEG monitoring. EEG data were segmented into 3-s long epochs and manually classified into 5 mental-fatigue levels, based on subjects' performance on an auditory vigilance task (AVT). Probabilistic-based multi-class SVM and standard multi-class SVM were compared as classifiers for distinguishing mental fatigue into the 5 mental-fatigue levels.

RESULTS

Accuracy of the probabilistic-based multi-class SVM was 87.2%, compared to 85.4% using the standard multi-class SVM. Using confidence estimates aggregation, accuracy increased to 91.2%.

CONCLUSIONS

Probabilistic-based multi-class SVM not only gives superior classification accuracy but also provides a valuable estimate of confidence in the prediction of mental fatigue level in a given 3-s EEG epoch.

SIGNIFICANCE

The work demonstrates the feasibility of an automatic EEG method for assessing and monitoring of mental fatigue. Future applications of this include traffic safety and other domains where measurement or monitoring of mental fatigue is crucial.

A feature selection method for multilevel mental fatigue EEG classification

Two feature selection approaches for multilevel mental fatigue electroencephalogram (EEG) classification are presented in this paper, in which random forest (RF) is combined with the heuristic initial feature ranking scheme (INIT) or with the recursive feature elimination scheme (RFE). In a "leave-one-proband-out" evaluation strategy, both feature selection approaches are evaluated on the recorded mental fatigue EEG time series data from 12 subjects (each for a 25-h duration) after initial feature extractions. The latter of the two approaches performs better both in classification performance and more importantly in feature reduction. RF with RFE achieved its lowest test error rate of 12.3% using 24 top-ranked features, whereas RF with INIT reached its lowest test error rate of 15.1% using 64 top-ranked features, compared to a test error rate of 22.1% using all 304 features. The results also show that 17 key features (out of 24 top-ranked features) are consistent between the subjects using RF with RFE, which is superior to the set of 64 features as determined by RF with INIT.

Development of an algorithm for an EEG-based driver fatigue countermeasure

PROBLEM

Fatigue affects a driver's ability to proceed safely. Driver-related fatigue and/or sleepiness are a significant cause of traffic accidents, which makes this an area of great socioeconomic concern. Monitoring physiological signals while driving provides the possibility of detecting and warning of fatigue. The aim of this paper is to describe an EEG-based fatigue countermeasure algorithm and to report its reliability.

METHOD

Changes in all major EEG bands during fatigue were used to develop the algorithm for detecting different levels of fatigue.

RESULTS

The software was shown to be capable of detecting fatigue accurately in 10 subjects tested. The percentage of time the subjects were detected to be in a fatigue state was significantly different than the alert phase (P<.01).

DISCUSSION

This is the first countermeasure software described that has shown to detect fatigue based on EEG changes in all frequency bands. Field research is required to evaluate the fatigue software in order to produce a robust and reliable fatigue countermeasure system.

IMPACT ON INDUSTRY

The development of the fatigue countermeasure algorithm forms the basis of a future fatigue countermeasure device. Implementation of electronic devices for fatigue detection is crucial for reducing fatigue-related road accidents and their associated costs.

Electroencephalography Activity Associated with Driver Fatigue: Implications for a Fatigue Countermeasure Device

Abstract This paper reviews the association between electroencephalography (EEG) activity and driver fatigue. The current literature shows substantial evidence of changes in brain wave activity, such as simultaneous changes in slow-wave activity (e.g., delta and theta activity) as well as alpha activity during driver fatigue. It is apparent from the literature review that EEG is a promising neurophysiological indicator of driver fatigue and has the potential to be incorporated into the development of a fatigue countermeasure device. The findings from this review are discussed in the light of directions for future fatigue research studies.

A critical review of the psychophysiology of driver fatigue

Driver fatigue is a major cause of road accidents and has implications for road safety. This review discusses the concepts of fatigue and provides a summary on psychophysiological associations with driver fatigue. A variety of psychophysiological parameters have been used in previous research as indicators of fatigue, with electroencephalography perhaps being the most promising. Most research found changes in theta and delta activity to be strongly linked to transition to fatigue. Therefore, monitoring electroencephalography during driver fatigue may be a promising variable for use in fatigue countermeasure devices. The review also identified anxiety and mood states as factors that may possibly affect driver fatigue. Furthermore, personality and temperament may also influence fatigue. Given the above, understanding the psychology of fatigue may lead to better fatigue management. The findings from this review are discussed in the light of directions for future studies and for the development of fatigue countermeasures.

Electroencephalographic imaging of higher brain function

High temporal resolution is necessary to resolve the rapidly changing patterns of brain activity that underlie mental function. Electroencephalography (EEG) provides temporal resolution in the millisecond range. However, traditional EEG technology and practice provide insufficient spatial detail to identify relationships between brain electrical events and structures and functions visualized by magnetic resonance imaging or positron emission tomography. Recent advances help to overcome this problem by recording EEGs from more electrodes, by registering EEG data with anatomical images, and by correcting the distortion caused by volume conduction of EEG signals through the skull and scalp. In addition, statistical measurements of sub-second interdependences between EEG time-series recorded from different locations can help to generate hypotheses about the instantaneous functional networks that form between different cortical regions during perception, thought and action. Example applications are presented from studies of language, attention and working memory. Along with its unique ability to monitor brain function as people perform everyday activities in the real world, these advances make modern EEG an invaluable complement to other functional neuroimaging modalities.

Problems of sensorimotor coordination in weightlessness

Previous studies about human sensorimotor coordination in space are inconclusive: it was reported that subjects in weightlessness point too high or too low, too fast or at normal speed, with increased or with normal variability; and that their tracking performance is degraded or normal. A better understanding of human performance in space would be desirable not only from the basic science perspective, but also for operational reasons. We propose a conceptual framework to explain the reported diversity, and to point out avenues for future research. We argue that exposure to weightlessness produces sensorimotor discordance, to which subjects gradually adapt through processes similar to those involved in earthbound adaptation. These processes require substantial information-processing resources in the brain, which may not be easily available during the hectic pace of a space mission. Within this framework, it is not surprising that previous data on sensorimotor performance in space were incongruent, as demand and availability of resources may have differed between missions, or even between subjects. We therefore propose that future work should control resource demand and availability, and study their effects on sensorimotor performance before and during space missions, in order to deconfound their effects from the immediate effects of gravity. A suitable hardware for such research is presented.

Neuronal cell death during sexual differentiation and lateralisation of vocal communication

A rodent analogy has been established to investigate the neural mechanisms occurring during sexual differentiation and lateralization. A sexually dimorphic hypothalamic nucleus (SDApc) is closely associated with a stereotyped, courtship vocalisation in male gerbils. Stereological analysis of SDApc cytoarchitecture reveals that neuron number and nuclear volume are asymmetric in male adults. Strikingly, neuron number on the left side of the SDApc correlates significantly with the rate of the courtship call in males. Exogenous testosterone treatment in female neonates masculinises and lateralises SDApc structure and function. Neuronal programmed cell death (apoptosis), manifested in SDApcs of neonates, is more frequent in females. Significantly, apoptosis in males is lateralised, as revealed by lateral asymmetry of neuron number at postnatal day 16. It is concluded that neuroendocrine-dependent, sexual differentiation and lateralization are concurrent and influenced by apoptotic mechanisms. It is suggested that apoptosis is the result of a genetically-driven device, inherent in postmitotic, undifferentiated cells which may have recently migrated into the SDApc. The genomic mechanism inducing lateralised apoptosis is apparently activated only neonatally in males.

Monitoring working memory load during computer-based tasks with EEG pattern recognition methods

We assessed working memory load during computer use with neural network pattern recognition applied to EEG spectral features. Eight participants performed high-, moderate-, and low-load working memory tasks. Frontal theta EEG activity increased and alpha activity decreased with increasing load. These changes probably reflect task difficulty-related increases in mental effort and the proportion of cortical resources allocated to task performance. In network analyses, test data segments from high and low load levels were discriminated with better than 95% accuracy. More than 80% of test data segments associated with a moderate load could be discriminated from high- or low-load data segments. Statistically significant classification was also achieved when applying networks trained with data from one day to data from another day, when applying networks trained with data from one task to data from another task, and when applying networks trained with data from a group of participants to data from new participants. These results support the feasibility of using EEG-based methods for monitoring cognitive load during human-computer interaction.

The future of electroencephalography in assessing neurocognitive functioning

High temporal resolution is necessary to resolve the rapidly changing patterns of brain activity underlying mental function. Additionally, simple, non-intrusive equipment is needed to routinely measure such functions in doctors' offices, at home and work and in other naturalistic contexts as people perform normal everyday activities. When compared with all other modalities for measuring higher brain functions, EEG is unique in that it has both these attributes. Two factors are limiting the further development and application of EEG for measuring cognitive functioning: a technical one that is easy to overcome and a sociological one that is more problematic. The technical limitation is that traditional EEG technology and practice provides insufficient spatial detail to identify relationships between brain electrical events and structures and functions visualized by magnetic resonance imaging (MRI) or other modalities. Recent advances overcome this problem by recording EEGs from more electrodes, by registering EEG data with anatomical information from each subject's MRI, by correcting the distortion caused by volume conduction of EEG signals through the skull and scalp, and by computing hypotheses about the sources of signals recorded at the scalp. The sociological limitation is that clinical EEGs are mostly performed by neurologists with no particular special interest in cognitive brain function, while cognitive research using EEG is largely done by psychology professors and their graduate students with no clinical ambitions. The diminishing clinical role of traditional EEGs in localizing lesions in the brain, and the obvious and insistent medical need for inexpensive and accessible tests of cognitive brain functioning may serve to soon dissipate this sociological obstruction. This will lead to a golden age of EEG in which Hans Berger's vision of the EEG as a window on the mind will be realized. Rather than slowly fading into obsolescence, EEG will retain its role as the primary means of measuring higher brain function when the purpose is not 3D localization per se, and will serve as an invaluable complement to functional MRI in those instances when both high temporal and high spatial resolution are required.

Sensori-sensory afferent conditioning with leg movement: gain control in spinal reflex and ascending paths

Studies are reviewed, predominantly involving healthy humans, on gain changes in spinal reflexes and supraspinal ascending paths during passive and active leg movement. The passive movement research shows that the pathways of H reflexes of the leg and foot are down-regulated as a consequence of movement-elicited discharge from somatosensory receptors, likely muscle spindle primary endings, both ipsi- and contralaterally. Discharge from the conditioning receptors in extensor muscles of the knee and hip appears to lead to presynaptic inhibition evoked over a spinal path, and to long-lasting attenuation when movement stops. The ipsilateral modulation is similar in phase to that seen with active movement. The contralateral conditioning does not phase modulate with passive movement and modulates to the phase of active ipsilateral movement. There are also centrifugal effects onto these pathways during movement. The pathways of the cutaneous reflexes of the human leg also are gain-modulated during active movement. The review summarizes the effects across muscles, across nociceptive and non-nociceptive stimuli and over time elapsed after the stimulus. Some of the gain changes in such reflexes have been associated with central pattern generators. However, the centripetal effect of movement-induced proprioceptive drive awaits exploration in these pathways. Scalp-recorded evoked potentials from rapidly conducting pathways that ascend to the human somatosensory cortex from stimulation sites in the leg also are gain-attenuated in relation to passive movement-elicited discharge of the extensor muscle spindle primary endings. Centrifugal influences due to a requirement for accurate active movement can partially lift the attenuation on the ascending path, both during and before movement. We suggest that a significant role for muscle spindle discharge is to control the gain in Ia pathways from the legs, consequent or prior to their movement. This control can reduce the strength of synaptic input onto target neurons from these kinesthetic receptors, which are powerfully activated by the movement, perhaps to retain the opportunity for target neuron modulation from other control sources.

Mapping cognitive brain function with modern high-resolution electroencephalography

High temporal resolution is necessary to resolve the rapidly changing patterns of brain activity that underlie mental function. While electroencephalography (EEG) provides temporal resolution in the millisecond range, which would seem to make it an ideal complement to other imaging modalities, traditional EEG technology and practice provides insufficient spatial detail to identify relationships between brain electrical events and structures and functions that are visualized by magnetic resonance imaging (MRI) or positron emission tomography (PET). Recent advances overcome this problem by recording EEGs from more electrodes, by registering EEG data with anatomical information from each subject's MRI, and by correcting the distortion that is caused by volume conduction of EEG signals through the skull and scalp. Along with its ability to record how brains think when performing everyday activities in the real world, these advances make modern EEG an invaluable complement to other functional neuroimaging modalities.