Intelligent system for drowsiness recognition based on ear canal electroencephalography with photoplethysmography and electrocardiography

Signal quality evaluation of an in-ear EEG device in comparison to a conventional cap system

Introduction

Wearable in-ear electroencephalographic (EEG) devices hold significant promise for integrating brain monitoring technologies into real-life applications. However, despite the introduction of various in-ear EEG systems, there remains a necessity for validating these technologies against gold-standard, clinical-grade devices. This study aims to evaluate the signal quality of a newly developed mobile in-ear EEG device compared to a standard scalp EEG system among healthy volunteers during wakefulness and sleep.

Methods

The study evaluated an in-ear EEG device equipped with dry electrodes in a laboratory setting, recording a single bipolar EEG channel using a cross-ear electrode configuration. Thirty healthy participants were recorded simultaneously using the in-ear EEG device and a conventional EEG cap system with 64 wet electrodes. Based on two recording protocols, one during a resting state condition involving alternating eye opening and closure with a low degree of artifact contamination and another consisting of a daytime nap, several quality measures were used for a quantitative comparison including root mean square (RMS) analysis, artifact quantification, similarities of relative spectral power (RSP), signal-to-noise ratio (SNR) based on alpha peak criteria, and cross-signal correlations of alpha activity during eyes-closed conditions and sleep activities. The statistical significance of our results was assessed through nonparametric permutation tests with False Discovery Rate (FDR) control.

Results

During the resting state, in-ear and scalp EEG signals exhibited similar fluctuations, characterized by comparable RMS values. However, intermittent signal alterations were noticed in the in-ear recordings during nap sessions, attributed to movements of the head and facial muscles. Spectral analysis indicated similar patterns between in-ear and scalp EEG, showing prominent peaks in the alpha range (8-12 Hz) during rest and in the low-frequency range during naps (particularly in the theta range of 4-7 Hz). Analysis of alpha wave characteristics during eye closures revealed smaller alpha wave amplitudes and slightly lower signal-to-noise ratio (SNR) values in the in-ear EEG compared to scalp EEG. In around 80% of cases, cross-correlation analysis between in-ear and scalp signals, using a contralateral bipolar montage of 64 scalp electrodes, revealed significant correlations with scalp EEG (p < 0.01), particularly evident in the FT11-FT12 and T7-T8 electrode derivations.

Conclusion

Our findings support the feasibility of using in-ear EEG devices with dry-contact electrodes for brain activity monitoring, compared to a standard scalp EEG, notably for wakefulness and sleep uses. Although marginal signal degradation is associated with head and facial muscle contractions, the in-ear device offers promising applications for long-term EEG recordings, particularly in scenarios requiring enhanced comfort and user-friendliness.

Wireless ear EEG to monitor drowsiness

Neural wearables can enable life-saving drowsiness and health monitoring for pilots and drivers. While existing in-cabin sensors may provide alerts, wearables can enable monitoring across more environments. Current neural wearables are promising but most require wet-electrodes and bulky electronics. This work showcases in-ear, dry-electrode earpieces used to monitor drowsiness with compact hardware. The employed system integrates additive-manufacturing for dry, user-generic earpieces, existing wireless electronics, and offline classification algorithms. Thirty-five hours of electrophysiological data were recorded across nine subjects performing drowsiness-inducing tasks. Three classifier models were trained with user-specific, leave-one-trial-out, and leave-one-user-out splits. The support-vector-machine classifier achieved an accuracy of 93.2% while evaluating users it has seen before and 93.3% when evaluating a never-before-seen user. These results demonstrate wireless, dry, user-generic earpieces used to classify drowsiness with comparable accuracies to existing state-of-the-art, wet electrode in-ear and scalp systems. Further, this work illustrates the feasibility of population-trained classification in future electrophysiological applications.

Evaluating the Electroencephalographic Signal Quality of an In-Ear Wearable Device

Wearable in-ear electroencephalographic (EEG) devices hold significant promise for advancing brain monitoring technologies into everyday applications. However, despite the current availability of several in-ear EEG devices in the market, there remains a critical need for robust validation against established clinical-grade systems. In this study, we carried out a detailed examination of the signal performance of a mobile in-ear EEG device from Naox Technologies. Our investigation had two main goals: firstly, evaluating the hardware circuit's reliability through simulated EEG signal experiments and, secondly, conducting a thorough comparison between the in-ear EEG device and gold-standard EEG monitoring equipment. This comparison assesses correlation coefficients with recognized physiological patterns during wakefulness and sleep, including alpha rhythms, eye artifacts, slow waves, spindles, and sleep stages. Our findings support the feasibility of using this in-ear EEG device for brain activity monitoring, particularly in scenarios requiring enhanced comfort and user-friendliness in various clinical and research settings.

CSF-GTNet: A Novel Multi-Dimensional Feature Fusion Network Based on Convnext-GeLU- BiLSTM for EEG-Signals-Enabled Fatigue Driving Detection

Electroencephalography (EEG) signal has been recognized as an effective fatigue detection method, which can intuitively reflect the drivers' mental state. However, the research on multi-dimensional features in existing work could be much better. The instability and complexity of EEG signals will increase the difficulty of extracting data features. More importantly, most current work only treats deep learning models as classifiers. They ignored the features of different subjects learned by the model. Aiming at the above problems, this paper proposes a novel multi-dimensional feature fusion network, CSF-GTNet, based on time and space-frequency domains for fatigue detection. Specifically, it comprises Gaussian Time Domain Network (GTNet) and Pure Convolutional Spatial Frequency Domain Network (CSFNet). The experimental results show that the proposed method effectively distinguishes between alert and fatigue states. The accuracy rates are 85.16% and 81.48% on the self-made and SEED-VIG datasets, respectively, which are higher than the state-of-the-art methods. Moreover, we analyze the contribution of each brain region for fatigue detection through the brain topology map. In addition, we explore the changing trend of each frequency band and the significance between different subjects in the alert state and fatigue state through the heat map. Our research can provide new ideas in brain fatigue research and play a specific role in promoting the development of this field.

The future of wearable EEG: a review of ear-EEG technology and its applications

Objective.This review paper provides a comprehensive overview of ear-electroencephalogram (EEG) technology, which involves recording EEG signals from electrodes placed in or around the ear, and its applications in the field of neural engineering.Approach.We conducted a thorough literature search using multiple databases to identify relevant studies related to ear-EEG technology and its various applications. We selected 123 publications and synthesized the information to highlight the main findings and trends in this field.Main results.Our review highlights the potential of ear-EEG technology as the future of wearable EEG technology. We discuss the advantages and limitations of ear-EEG compared to traditional scalp-based EEG and methods to overcome those limitations. Through our review, we found that ear-EEG is a promising method that produces comparable results to conventional scalp-based methods. We review the development of ear-EEG sensing devices, including the design, types of sensors, and materials. We also review the current state of research on ear-EEG in different application areas such as brain-computer interfaces, and clinical monitoring.Significance.This review paper is the first to focus solely on reviewing ear-EEG research articles. As such, it serves as a valuable resource for researchers, clinicians, and engineers working in the field of neural engineering. Our review sheds light on the exciting future prospects of ear-EEG, and its potential to advance neural engineering research and become the future of wearable EEG technology.

Identification of Driver Status Hazard Level and the System

According to the survey statistics, most traffic accidents are caused by the driver's behavior and status irregularities. Because there is no multi-level dangerous state grading system at home and abroad, this paper proposes a complex state grading system for real-time detection and dynamic tracking of the driver's state. The system uses OpenMV as the acquisition camera combined with the cradle head tracking system to collect the driver's current driving image in real-time dynamically, combines the YOLOX algorithm with the OpenPose algorithm to judge the driver's dangerous driving behavior by detecting unsafe objects in the cab and the driver's posture, and combines the improved Retinaface face detection algorithm with the Dlib feature-point algorithm to discriminate the fatigue driving state of the driver. The experimental results show that the accuracy of the three driver danger levels (R1, R2, and R3) obtained by the proposed system reaches 95.8%, 94.5%, and 96.3%, respectively. The experimental results of this system have a specific practical significance in driver-distracted driving warnings.

A Systematic Review of Time Series Classification Techniques Used in Biomedical Applications

Background: Digital clinical measures collected via various digital sensing technologies such as smartphones, smartwatches, wearables, and ingestible and implantable sensors are increasingly used by individuals and clinicians to capture the health outcomes or behavioral and physiological characteristics of individuals. Time series classification (TSC) is very commonly used for modeling digital clinical measures. While deep learning models for TSC are very common and powerful, there exist some fundamental challenges. This review presents the non-deep learning models that are commonly used for time series classification in biomedical applications that can achieve high performance. Objective: We performed a systematic review to characterize the techniques that are used in time series classification of digital clinical measures throughout all the stages of data processing and model building. Methods: We conducted a literature search on PubMed, as well as the Institute of Electrical and Electronics Engineers (IEEE), Web of Science, and SCOPUS databases using a range of search terms to retrieve peer-reviewed articles that report on the academic research about digital clinical measures from a five-year period between June 2016 and June 2021. We identified and categorized the research studies based on the types of classification algorithms and sensor input types. Results: We found 452 papers in total from four different databases: PubMed, IEEE, Web of Science Database, and SCOPUS. After removing duplicates and irrelevant papers, 135 articles remained for detailed review and data extraction. Among these, engineered features using time series methods that were subsequently fed into widely used machine learning classifiers were the most commonly used technique, and also most frequently achieved the best performance metrics (77 out of 135 articles). Statistical modeling (24 out of 135 articles) algorithms were the second most common and also the second-best classification technique. Conclusions: In this review paper, summaries of the time series classification models and interpretation methods for biomedical applications are summarized and categorized. While high time series classification performance has been achieved in digital clinical, physiological, or biomedical measures, no standard benchmark datasets, modeling methods, or reporting methodology exist. There is no single widely used method for time series model development or feature interpretation, however many different methods have proven successful.

Drowsiness Detection with Wireless, User-Generic, Dry Electrode Ear EEG

Drowsiness monitoring can reduce workplace and driving accidents. To enable a discreet device for drowsiness monitoring and detection, this work presents a drowsiness user-study with an in-ear EEG system, which uses two user-generic, dry electrode earpieces and a wireless interface for streaming data. Twenty-one drowsiness trials were recorded across five human users and drowsiness detection was implemented with three classifier models: logistic regression, support vector machine (SVM), and random forest. To estimate drowsiness detection performance across usage scenarios, these classifiers were validated with user-specific, leave-one-trial-out, and leave-one-user-out training. To our knowledge, this is the first wireless, multi-channel, dry electrode in-ear EEG to be used for drowsiness monitoring. With user-specific training, a SVM achieved a detection accuracy of 95.9%. When evaluating a never-before-seen user, a similar SVM achieved a 94.5% accuracy, comparable to the best performing state-of-the-art wet electrode in-ear and scalp EEG systems.

Beyond Pathogen Filtration: Possibility of Smart Masks as Wearable Devices for Personal and Group Health and Safety Management

Face masks are an important way to combat the COVID-19 pandemic. However, the prolonged pandemic has revealed confounding problems with the current face masks, including not only the spread of the disease but also concurrent psychological, social, and economic complications. As face masks have been worn for a long time, people have been interested in expanding the purpose of masks from protection to comfort and health, leading to the release of various "smart" mask products around the world. To envision how the smart masks will be extended, this paper reviewed 25 smart masks (12 from commercial products and 13 from academic prototypes) that emerged after the pandemic. While most smart masks presented in the market focus on resolving problems with user breathing discomfort, which arise from prolonged use, academic prototypes were designed for not only sensing COVID-19 but also general health monitoring aspects. Further, we investigated several specific sensors that can be incorporated into the mask for expanding biophysical features. On a larger scale, we discussed the architecture and possible applications with the help of connected smart masks. Namely, beyond a personal sensing application, a group or community sensing application may share an aggregate version of information with the broader population. In addition, this kind of collaborative sensing will also address the challenges of individual sensing, such as reliability and coverage. Lastly, we identified possible service application fields and further considerations for actual use. Along with daily-life health monitoring, smart masks may function as a general respiratory health tool for sports training, in an emergency room or ambulatory setting, as protection for industry workers and firefighters, and for soldier safety and survivability. For further considerations, we investigated design aspects in terms of sensor reliability and reproducibility, ergonomic design for user acceptance, and privacy-aware data-handling. Overall, we aim to explore new possibilities by examining the latest research, sensor technologies, and application platform perspectives for smart masks as one of the promising wearable devices. By integrating biomarkers of respiration symptoms, a smart mask can be a truly cutting-edge device that expands further knowledge on health monitoring to reach the next level of wearables.

Signal Quality Index Based on Template Cross-Correlation in Multimodal Biosignal Chair for Smart Healthcare

We investigated the effects of a quality screening method on unconstrained measured signals, including electrocardiogram (ECG), photoplethysmogram (PPG), and ballistocardiogram (BCG) signals, in our collective chair system for smart healthcare. Such an investigation is necessary because unattached or unbound sensors have weaker connections to body parts than do conventional methods. Using the biosignal chair, the physiological signals collected during sessions included a virtual driving task, a physically powered wheelchair drive, and three types of body motions. The signal quality index was defined by the similarity between the observed signals and noise-free signals from the perspective of the cross-correlations of coefficients with appropriate individual templates. The goal of the index was to qualify signals without a reference signal to assess the practical use of the chair in daily life. As expected, motion artifacts have adverse effects on the stability of physiological signals. However, we were able to observe a supplementary relationship between sensors depending on each movement trait. Except for extreme movements, the signal quality and estimated heart rate (HR) remained within the range of criteria usable for status monitoring. By investigating the signal reliability, we were able to confirm the suitability of using the unconstrained biosignal chair to collect real-life measurements to improve safety and healthcare.

A Review of EEG Signal Features and their Application in Driver Drowsiness Detection Systems

Detecting drowsiness in drivers, especially multi-level drowsiness, is a difficult problem that is often approached using neurophysiological signals as the basis for building a reliable system. In this context, electroencephalogram (EEG) signals are the most important source of data to achieve successful detection. In this paper, we first review EEG signal features used in the literature for a variety of tasks, then we focus on reviewing the applications of EEG features and deep learning approaches in driver drowsiness detection, and finally we discuss the open challenges and opportunities in improving driver drowsiness detection based on EEG. We show that the number of studies on driver drowsiness detection systems has increased in recent years and that future systems need to consider the wide variety of EEG signal features and deep learning approaches to increase the accuracy of detection.

Drowsiness Detection Based on Intelligent Systems with Nonlinear Features for Optimal Placement of Encephalogram Electrodes on the Cerebral Area

Drowsiness while driving can lead to accidents that are related to the loss of perception during emergencies that harm the health. Among physiological signals, brain waves have been used as informative signals for the analyses of behavioral observations, steering information, and other biosignals during drowsiness. We inspected the machine learning methods for drowsiness detection based on brain signals with varying quantities of information. The results demonstrated that machine learning could be utilized to compensate for a lack of information and to account for individual differences. Cerebral area selection approaches to decide optimal measurement locations could be utilized to minimize the discomfort of participants. Although other statistics could provide additional information in further study, the optimized machine learning method could prevent the dangers of drowsiness while driving by considering a transitional state with nonlinear features. Because brain signals can be altered not only by mental fatigue but also by health status, the optimization analysis of the system hardware and software will be able to increase the power-efficiency and accessibility in acquiring brain waves for health enhancements in daily life.

Applications of EEG indices for the quantification of human cognitive performance: A systematic review and bibliometric analysis

BACKGROUND

Neuroergonomics combines neuroscience with ergonomics to study human performance using recorded brain signals. Such neural signatures of performance can be measured using a variety of neuroimaging techniques, including functional magnetic resonance imaging (fMRI), functional near-infrared spectroscopy (fNIRS), and electroencephalography (EEG). EEG has an excellent temporal resolution, and EEG indices are highly sensitive to human brain activity fluctuations.

OBJECTIVE

The focus of this systematic review was to explore the applications of EEG indices for quantifying human performance in a variety of cognitive tasks at the macro and micro scales. To identify trends and the state of the field, we examined global patterns among selected articles, such as journal contributions, highly cited papers, affiliations, and high-frequency keywords. Moreover, we discussed the most frequently used EEG indices and synthesized current knowledge regarding the EEG signatures of associated human performance measurements.

METHODS

In this systematic review, we analyzed articles published in English (from peer-reviewed journals, proceedings, and conference papers), Ph.D. dissertations, textbooks, and reference books. All articles reviewed herein included exclusively EEG-based experimental studies in healthy participants. We searched Web-of-Science and Scopus databases using specific sets of keywords.

RESULTS

Out of 143 papers, a considerable number of cognitive studies focused on quantifying human performance with respect to mental fatigue, mental workload, mental effort, visual fatigue, emotion, and stress. An increasing trend for publication in this area was observed, with the highest number of publications in 2017. Most studies applied linear methods (e.g., EEG power spectral density and the amplitude of event-related potentials) to evaluate human cognitive performance. A few papers utilized nonlinear methods, such as fractal dimension, largest Lyapunov exponent, and signal entropy. More than 50% of the studies focused on evaluating an individual's mental states while operating a vehicle. Several different methods of artifact removal have also been noted. Based on the reviewed articles, research gaps, trends, and potential directions for future research were explored.

CONCLUSION

This systematic review synthesized current knowledge regarding the application of EEG indices for quantifying human performance in a wide variety of cognitive tasks. This knowledge is useful for understanding the global patterns of applications of EEG indices for the analysis and design of cognitive tasks.

In-Ear EEG Based Attention State Classification Using Echo State Network

It is important to maintain attention when carrying out significant daily-life tasks that require high levels of safety and efficiency. Since degradation of attention can sometimes have dire consequences, various brain activity measurement devices such as electroencephalography (EEG) systems have been used to monitor attention states in individuals. However, conventional EEG instruments have limited utility in daily life because they are uncomfortable to wear. Thus, this study was designed to investigate the possibility of discriminating between the attentive and resting states using in-ear EEG signals for potential application via portable, convenient earphone-shaped EEG instruments. We recorded both on-scalp and in-ear EEG signals from 6 subjects in a state of attentiveness during the performance of a visual vigilance task. We have designed and developed in-ear EEG electrodes customized by modelling both the left and right ear canals of the subjects. We use an echo state network (ESN), a powerful type of machine learning algorithm, to discriminate attention states on the basis of in-ear EEGs. We have found that the maximum average accuracy of the ESN method in discriminating between attentive and resting states is approximately 81.16% with optimal network parameters. This study suggests that portable in-ear EEG devices and an ESN can be used to monitor attention states during significant tasks to enhance safety and efficiency.

Classification of Drowsiness Levels Based on a Deep Spatio-Temporal Convolutional Bidirectional LSTM Network Using Electroencephalography Signals

Non-invasive brain-computer interfaces (BCI) have been developed for recognizing human mental states with high accuracy and for decoding various types of mental conditions. In particular, accurately decoding a pilot's mental state is a critical issue as more than 70% of aviation accidents are caused by human factors, such as fatigue or drowsiness. In this study, we report the classification of not only two mental states (i.e., alert and drowsy states) but also five drowsiness levels from electroencephalogram (EEG) signals. To the best of our knowledge, this approach is the first to classify drowsiness levels in detail using only EEG signals. We acquired EEG data from ten pilots in a simulated night flight environment. For accurate detection, we proposed a deep spatio-temporal convolutional bidirectional long short-term memory network (DSTCLN) model. We evaluated the classification performance using Karolinska sleepiness scale (KSS) values for two mental states and five drowsiness levels. The grand-averaged classification accuracies were 0.87 (±0.01) and 0.69 (±0.02), respectively. Hence, we demonstrated the feasibility of classifying five drowsiness levels with high accuracy using deep learning.

Effects of Different Re-referencing Methods on Spontaneously Generated Ear-EEG

In recent years, electroencephalography (EEG) measured around the ears, called ear-EEG, has been introduced to develop unobtrusive and ambulatory EEG-based applications. When measuring ear-EEGs, the availability of a reference site is restricted due to the miniaturized device structure, and therefore a reference electrode is generally placed near the recording electrodes. As the electrical brain activity recorded at a reference electrode closely placed to recording electrodes may significantly cancel or influence the brain activity recorded by the recording electrodes, an appropriate re-referencing method is often required to mitigate the impact of the reference brain activity. In this study, therefore, we systematically investigated the impact of different re-referencing methods on ear-EEGs spontaneously generated from endogenous paradigms. To this end, we used two ear-EEG datasets recorded behind both ears while subjects performed an alpha modulation task [eyes-closed (EC) and eyes-open (EO)] and two mental tasks [mental arithmetic (MA) and mental singing (MS)]. The measured ear-EEGs were independently re-referenced using five different methods: (i) all-mean, (ii) contralateral-mean, (iii) ipsilateral-mean, (iv) contralateral-bipolar, and (v) ipsilateral-bipolar. We investigated the changes in alpha power during EO and EC tasks, as well as event-related (de) synchronization (ERD/ERS) during MA and MS. To evaluate the effects of re-referencing methods on ear-EEGs, we estimated the signal-to-noise ratios (SNRs) of the two ear-EEG datasets, and assessed the classification performance of the two mental tasks (MA vs. MS). Overall patterns of changes in alpha power and ERD/ERS were similar among the five re-referencing methods, but the contralateral-mean method showed statistically higher SNRs than did the other methods for both ear-EEG datasets, except in the contralateral-bipolar method for the two mental tasks. In concordance with the SNR results, classification performance was also statistically higher for the contralateral-mean method than it was for the other re-referencing methods. The results suggest that employing contralateral mean information can be an efficient way to re-reference spontaneously generated ear-EEGs, thereby maximizing the reliability of ear-EEG-based applications in endogenous paradigms.