A generative adaptive convolutional neural network with attention mechanism for driver fatigue detection with class-imbalanced and insufficient data

Over the past few years, fatigue driving has emerged as one of the main causes of traffic accidents, necessitating the development of driver fatigue detection systems. However, many existing methods involves tedious manual parameter tunings, a process that is both time-consuming and results in task-specific models. On the other hand, most of the researches on fatigue recognition are based on class-balanced and sufficient data, and effectively "mine" meaningful information from class-imbalanced and insufficient data for fatigue recognition is still a challenge. In this paper, we proposed two novel models, the attention-based residual adaptive multiscale fully convolutional network-long short term memory network (ARMFCN-LSTM), and the Generative ARMFCN-LSTM (GARMFCN-LSTM) aiming to address this issue. ARMFCN-LSTM excels at automatically extracting multiscale representations through adaptive multiscale temporal convolutions, while capturing temporal dependency features through LSTM. GARMFCN-LSTM integrates Wasserstein GAN with gradient penalty (WGAN-GP) into ARMFCN-LSTM to improve driver fatigue detection performance by alleviating data scarcity and addressing class imbalances. Experimental results show that ARMFCN-LSTM achieves the highest classification accuracy of 95.84% in driver fatigue detection on the class-balanced EEG dataset (binary classification), and GARMFCN-LSTM attained an improved classification accuracy of 84.70% on the class-imbalanced EOG dataset (triple classification), surpassing the competing methods. Therefore, the proposed models are promising for further implementations in online driver fatigue detection systems.

A novel deep-learning model based on τ-shaped convolutional network (τNet) with long short-term memory (LSTM) for physiological fatigue detection from EEG and EOG signals

In recent years, fatigue driving has become the main cause of traffic accidents, leading to increased attention towards fatigue detection systems. However, the pooling and strided convolutional operations in fatigue detection algorithm based on traditional deep learning methods may led to the loss of some useful information. This paper proposed a novel [Formula: see text]-shaped convolutional network ([Formula: see text]) aiming to address this issue. Unlike traditional network structures, [Formula: see text] incorporates the operations of upsampling features and concatenating high- and low-level features, enabling full utilization of useful information. Moreover, considering that the fatigue state is a mental state involving temporal evolution, we proposed the novel long short-term memory (LSTM)-[Formula: see text]-shaped convolutional network (LSTM-[Formula: see text]), a parallel structure composed of LSTM and [Formula: see text] for fatigue detection, where [Formula: see text] extracts time-invariant features with location information, and LSTM extracts long temporal dependencies. We compared LSTM-[Formula: see text] with six competing methods based on two datasets. Results showed that the proposed algorithm achieved higher classification accuracy than the other methods, with 94.25% on EEG data (binary classification) and 82.19% on EOG data (triple classification). Additionally, the proposed algorithm exhibits low computational cost, good training stability, and robustness against insufficient training. Therefore, it is promising for further implementation of fatigue online detection systems.

Physiological measurements for driving drowsiness: A comparative study of multi-modality feature fusion and selection

A large number of traffic accidents were caused by drowsiness while driving. In-vehicle alert system based on physiological signals was one of the most promising solutions to monitor driving fatigue. However, different physiological modalities can be used, and many relative studies compared different modalities without considering the implementation feasibility of portable or wearable devices. Moreover, evaluations of each modality in previous studies were based on inconsistent choices of fatigue label and signal features, making it hard to compare the results of different studies. Therefore, the modality comparison and fusion for continuous drowsiness estimation while driving was still unclear. This work sought to comprehensively compare widely-used physiological modalities, including forehead electroencephalogram (EEG), electrooculogram (EOG), R-R intervals (RRI) and breath, in a hardware setting feasible for portable or wearable devices to monitor driving fatigue. Moreover, a more general conclusion on modality comparison and fusion was reached based on the regression of features or their combinations and the awake-to-drowsy transition. Finally, the feature subset of fused modalities was produced by feature selection method, to select the optimal feature combination and reduce computation consumption. Considering practical feasibility, the most effective combination with the highest correlation coefficient was using forehead EEG or EOG, along with RRI and RRI-derived breath. If more comfort and convenience was required, the combination of RRI and RRI-derived breath was also promising.

Detecting slow eye movements using multi-scale one-dimensional convolutional neural network for driver sleepiness detection

BACKGROUND

Slow eye movements (SEMs), which occurs during eye-closed periods with high time coverage rate during simulated driving process, indicate drivers' sleep onset.

NEW METHOD

For the multi-scale characteristics of slow eye movement waveforms, we propose a multi-scale one-dimensional convolutional neural network (MS-1D-CNN) for classification. The MS-1D-CNN performs multiple down-sampling processing branches on the original signal and uses the local convolutional layer to extract the features for each branch.

RESULTS

We evaluate the classification performance of this model on ten subjects' standard train-test datasets and continuous test datasets by means of subject-subject evaluation and leave-one-subject-out cross validation, respectively. For the standard train-test datasets, the overall average classification accuracies are about 99.1% and 98.6%, in subject-subject evaluation and leave-one-subject-out cross validation, respectively. For the continuous test datasets, the overall average values of accuracy, precision, recall and F1-score are 99.3%, 98.9%, 99.5% and 99.1% in subject-subject evaluation, are 99.2%, 98.8%, 99.3% and 99.0% in leave-one-subject-out cross validation.

COMPARISON WITH EXISTING METHOD

Results of the standard train-test datasets show that the overall average classification accuracy of the MS-1D-CNN is quite higher than the baseline method based on hand-designed features by 3.5% and 3.5%, in subject-subject evaluation and leave-one-subject-out cross validation, respectively.

CONCLUSIONS

These results suggest that multi-scale transformation in the MS-1D-CNN model can enhance the representation ability of features, thereby improving classification accuracy. Experimental results verify the good performance of the MS-1D-CNN model, even in leave-one-subject-out cross validation, thus promoting the application of SEMs detection technology for driver sleepiness detection.

A novel approach to automatic sleep stage classification using forehead electrophysiological signals

Background

Sleep stage scoring is very important for the effective diagnosis and intervention of sleep disorders. However, the current automatic sleep staging methods generally have the problems of poor model generalization ability and non-portable acquisition equipment.

Method

In this paper, we propose a novel automatic sleep scoring system based on forehead electrophysiological signals that is more effective and convenient than other systems. We extract 3 channel signals from the forehead, named forehead electroencephalogram 1 (Fh1), forehead electroencephalogram 2 (Fh2), and forehead center electrooculogram (Fhz). Spectral features, statistical features, and entropy features are extracted using the discrete wavelet transform (DWT) method. Light gradient boosting machine (LGB), random forest (RF), and support vector machine (SVM) are employed to classify four stages: awake, light sleep (LS), deep sleep (DS), and rapid eye movement (REM).

Result

The performance of the proposed method is validated using databases of the Sleep-EDFX and our Own-data, which include polysomnograms (PSGs) and forehead signals of 28 subjects. The overall classification accuracy of using the combination of Fh1, Fh2, and Fhz can reach up to 90.25% accuracy with a kappa coefficient of 0.857.

Conclusions

The proposed method could provide state-of-art multichannel sleep stage scoring performance with higher portability. This will facilitate the application of long-term monitoring of sleep quality in the future.

A multi-modal assessment of sleep stages using adaptive Fourier decomposition and machine learning

Healthy sleep is essential for the rejuvenation of the body and helps in maintaining good health. Many people suffer from sleep disorders that are characterized by abnormal sleep patterns. Automated assessment of such disorders using biomedical signals has been an active subject of research. Electroencephalogram (EEG) is a popular diagnostic used in this regard. We consider a widely-used publicly available database and process the signals using the Fourier decomposition method (FDM) to obtain narrowband signal components. Statistical features extracted from these components are passed on to machine learning classifiers to identify different stages of sleep. A novel feature measuring the non-stationarity of the signal is also used to capture salient information. It is shown that classification results can be improved by using multi-channel EEG instead of single-channel EEG data. Simultaneous utilization of multiple modalities, such as Electromyogram (EMG), Electrooculogram (EOG) along with EEG data leads to further enhancement in the obtained results. The proposed method can be efficiently implemented in real-time using fast Fourier transform (FFT), and it provides better classification results than the other algorithms existing in the literature. It can assist in the development of low-cost sensor-based setups for continuous patient monitoring and feedback.

Sleep staging classification based on a new parallel fusion method of multiple sources signals

APPROACH

First, the heart rate variability (HRV) is extracted from EOG with the Weight Calculation Algorithm (WCA) and an "HYF" RR interval detection algorithm. Second, three feature sets were extracted from HRV segments and EOG segments: time-domain features, frequency domain features and nonlinear-domain features. The frequency domain features and nonlinear-domain features were extracted by using Discrete Wavelet Transform (DWT), Autoregressive (AR), and Power Spectral entropy (PSE), and Refined Composite Multiscale Dispersion Entropy (RCMDE). Third, a new "Parallel Fusion Method" (PFM) for sleep stage classification is proposed. Three kinds of feature sets from EOG and HRV segments are fused by using PFM. Fourth, Extreme Gradient Boosting (XGBoost) and Support Vector Machine (SVM) classification models is employed for sleep staging.

MAIN RESULTS

Our experimental results show significant performance improvement on automatic sleep staging on the target domains achieved with the new sleep staging approach. The performance of the proposed method is testedby evaluating the average accuracy, Kappa coefficient. The average accuracy of sleep classification results by using XGBoost classification model with PFM is 82.7% and the kappa coefficient is 0.711, also by using SVM classification model with the PFM is 83.7%, and the kappa coefficient is 0.724. Experimental results show that the performance of the proposed method is competitive with the most current methods and results, and the recognition rate of S1 stage is significantly improved. Significance： As a consequence, it would enable one to improve the quality of automatic sleep staging models when the EOG and HRV signals are fused, which can be beneficial for monitor sleep quality and keep abreast of health conditions. Besides, our study provides good research ideas and methods for scholars, doctors and individuals.

Best Vitelliform Macular Dystrophy (BVMD) is a phenocopy of North Carolina Macular Dystrophy (NCMD/MCDR1)

PURPOSE

North Carolina Macular Dystrophy (NCMD) and Best Vitelliform Macular Dystrophy (BVMD) are rare autosomal dominant macular dystrophies. Both BVMD and NCMD have markedly variable expressivity. In some individuals, it can be difficult to differentiate between the two disease entities.

METHODS

Clinical findings including fundus photography, fundus autofluorescence (FAF), and spectral domain optical coherence tomography (SD-OCT) were evaluated in 5 individuals with NCMD and 3 with BMD. Electrooculography (EOG) was performed in 2 NCMD subjects. Molecular diagnosis was performed using Sanger DNA sequencing. IRB approval was obtained.

RESULTS

Five NCMD subjects had clinical findings indistinguishable from three of our BVMD subjects. Molecular diagnosis was confirmed in all but one BVMD subject who had an abnormal EOG prior to discovery of the BEST1 gene. Two NCMD subjects had an abnormal EOG with a normal ERG, which has been considered a unique feature of BVMD. SD-OCT in one BVMD subject demonstrated a small lucency/excavation into the choroid similar to that in grade 3 lesions of NCMD. Two NCMD subjects had elevated sub-macular lesions giving a pseudo-vitelliform appearance on OCT similar to BVMD.

CONCLUSION

Best Vitelliform Macular Dystrophy can be a phenocopy of NCMD. There is considerable clinical overlap between NCMD and BVMD, which can cause diagnostic inaccuracies. Our new findings demonstrate that like BVMD, NCMD can also have an abnormal EOG with a normal ERG. The overlapping phenotypes of BVMD with NCMD may provide insights into the mechanisms of the macular changes.

DESIGN AND DEVELOPMENT OF HUMAN COMPUTER INTERFACE USING ELECTROOCULOGRAM WITH DEEP LEARNING

Today's life assistive devices were playing significant role in our life to communicate with others. In that modality Human Computer Interface (HCI) based Electrooculogram (EOG) playing vital part. By using this method we can able to overcome the conventional methods in terms of performance and accuracy. To overcome such problem we analyze the EOG signal from twenty subjects to design nine states EOG based HCI using five electrodes system to measure the horizontal and vertical eye movements. Signals were preprocessed to remove the artifacts and extract the valuable information from collected data by using band power and Hilbert Huang Transform (HHT) and trained with Pattern Recognition Neural Network (PRNN) to classify the tasks. The classification results of 92.17% and 91.85% were shown for band power and HHT features using PRNN architecture. Recognition accuracy was analyzed in offline to identify the possibilities of designing HCI. We compare the two feature extraction techniques with PRNN to analyze the best method for classifying the tasks and recognizing single trail tasks to design the HCI. Our experimental result confirms that for classifying as well as recognizing accuracy of the collected signals using band power with PRNN shows better accuracy compared to other network used in this study. We compared the male subjects performance with female subjects to identify the performance. Finally we compared the male as well as female subjects in age group wise to identify the performance of the system. From that we concluded that male performance was appreciable compared with female subjects as well as age group between 26 to 32 performance and recognizing accuracy were high compared with other age groups used in this study.

ISCEV guide to visual electrodiagnostic procedures

Clinical electrophysiological testing of the visual system incorporates a range of noninvasive tests and provides an objective indication of function relating to different locations and cell types within the visual system. This document developed by the International Society for Clinical Electrophysiology of Vision provides an introduction to standard visual electrodiagnostic procedures in widespread use including the full-field electroretinogram (ERG), the pattern electroretinogram (pattern ERG or PERG), the multifocal electroretinogram (multifocal ERG or mfERG), the electrooculogram (EOG) and the cortical-derived visual evoked potential (VEP). The guideline outlines the basic principles of testing. Common clinical presentations and symptoms are described with illustrative examples and suggested investigation strategies.

Development of an electrooculogram-based eye-computer interface for communication of individuals with amyotrophic lateral sclerosis

BACKGROUND

Electrooculogram (EOG) can be used to continuously track eye movements and can thus be considered as an alternative to conventional camera-based eye trackers. Although many EOG-based eye tracking systems have been studied with the ultimate goal of providing a new way of communication for individuals with amyotrophic lateral sclerosis (ALS), most of them were tested with healthy people only. In this paper, we investigated the feasibility of EOG-based eye-writing as a new mode of communication for individuals with ALS.

METHODS

We developed an EOG-based eye-writing system and tested this system with 18 healthy participants and three participants with ALS. We also applied a new method for removing crosstalk between horizontal and vertical EOG components. All study participants were asked to eye-write specially designed patterns of 10 Arabic numbers three times after a short practice session.

RESULTS

Our system achieved a mean recognition rates of 95.93% for healthy participants and showed recognition rates of 95.00%, 66.67%, and 93.33% for the three participants with ALS. The low recognition rates in one of the participants with ALS was mainly due to miswritten letters, the number of which decreased as the experiment proceeded.

CONCLUSION

Our proposed eye-writing system is a feasible human-computer interface (HCI) tool for enabling practical communication of individuals with ALS.

Detection of eye blink artifacts from single prefrontal channel electroencephalogram

Eye blinks are one of the most influential artifact sources in electroencephalogram (EEG) recorded from frontal channels, and thereby detecting and rejecting eye blink artifacts is regarded as an essential procedure for improving the quality of EEG data. In this paper, a novel method to detect eye blink artifacts from a single-channel frontal EEG signal was proposed by combining digital filters with a rule-based decision system, and its performance was validated using an EEG dataset recorded from 24 healthy participants. The proposed method has two main advantages over the conventional methods. First, it uses single-channel EEG data without the need for electrooculogram references. Therefore, this method could be particularly useful in brain-computer interface applications using headband-type wearable EEG devices with a few frontal EEG channels. Second, this method could estimate the ranges of eye blink artifacts accurately. Our experimental results demonstrated that the artifact range estimated using our method was more accurate than that from the conventional methods, and thus, the overall accuracy of detecting epochs contaminated by eye blink artifacts was markedly increased as compared to conventional methods. The MATLAB package of our library source codes and sample data, named Eyeblink Master, is open for free download.