Driving fatigue detection based on brain source activity and ARMA model

Fatigue among drivers is a significant issue in society, and according to organizational reports, it substantially contributes to accidents. So accurate fatigue detection in drivers plays a crucial role in reducing the number of people fatalities or injured resulting from accidents. Several methods are proposed for fatigue driver recognition among which electroencephalography (EEG) is one. This paper proposed a method for fatigue recognition by EEG signals with extracted features from source and sensor spaces. The proposed method starts with preprocessing by applying filtering and artifact rejection. Then source localization methods are applied to EEG signals for active source extraction. A multivariate autoregressive (MVAR) model is fitted to selected sources, and a dual Kalman filter is applied to estimate the source activity and their relationships. Then multivariate autoregressive moving average (ARMA) is fitted between EEG and source activity signals. Features are extracted from model parameters, source relationship matrix, and wavelet transform of EEG and source activity signals. The novelty of this approach is the use of ARMA model between source activities (as input) and EEG signals (as output) and feature extraction from source relations. Relevant features are selected using a combination of RelifF and neighborhood component analysis (NCA) methods. Three classifiers, namely k-nearest neighbor (KNN), support vector machine (SVM), and naive Bayesian (NB) classifiers, are employed to classify drivers. To improve performance, the final label for fatigue detection is calculated by combining these classifiers using the voting method. The results demonstrate that the proposed method accurately recognizes and classifies fatigued drivers with the ensemble classifiers in comparison with other methods.

EEG-Based Cross-Subject Driver Drowsiness Recognition With an Interpretable Convolutional Neural Network

In the context of electroencephalogram (EEG)-based driver drowsiness recognition, it is still challenging to design a calibration-free system, since EEG signals vary significantly among different subjects and recording sessions. Many efforts have been made to use deep learning methods for mental state recognition from EEG signals. However, existing work mostly treats deep learning models as black-box classifiers, while what have been learned by the models and to which extent they are affected by the noise in EEG data are still underexplored. In this article, we develop a novel convolutional neural network combined with an interpretation technique that allows sample-wise analysis of important features for classification. The network has a compact structure and takes advantage of separable convolutions to process the EEG signals in a spatial-temporal sequence. Results show that the model achieves an average accuracy of 78.35% on 11 subjects for leave-one-out cross-subject drowsiness recognition, which is higher than the conventional baseline methods of 53.40%-72.68% and state-of-the-art deep learning methods of 71.75%-75.19%. Interpretation results indicate the model has learned to recognize biologically meaningful features from EEG signals, e.g., alpha spindles, as strong indicators of drowsiness across different subjects. In addition, we also explore reasons behind some wrongly classified samples with the interpretation technique and discuss potential ways to improve the recognition accuracy. Our work illustrates a promising direction on using interpretable deep learning models to discover meaningful patterns related to different mental states from complex EEG signals.

Driver drowsiness detection methods using EEG signals: a systematic review

Electroencephalography (EEG) is a complex signal that may require several years of training, advanced signal processing, and feature extraction methodologies to interpret correctly. Recently, many methods have been used to extract and classify EEG data. This study reviews 62 papers that used EEG signals to detect driver drowsiness, published between January 2018 and 2022. We extract trends and highlight interesting approaches from this large body of literature to inform future research and formulate recommendations. To find relevant papers published in scientific journals, conferences, and electronic preprint repositories, researchers searched major databases covering the domains of science and engineering. For each investigation, many data items about (1) the data, (2) the channels used, (3) the extraction and classification procedure, and (4) the outcomes were extracted. These items were then analyzed one by one to uncover trends. Our analysis reveals that the amount of EEG data used across studies varies. We saw that more than half the studies used simulation driving experimental. About 21% of the studies used support vector machine (SVM), while 19% used convolutional neural networks (CNN). Overall, we can conclude that drowsiness and fatigue impair driving performance, resulting in drivers who are more exposed to risky situations.

Belief and Possibility Belief Interval-Valued N-Soft Set and Their Applications in Multi-Attribute Decision-Making Problems

In this research article, we motivate and introduce the concept of possibility belief interval-valued N-soft sets. It has a great significance for enhancing the performance of decision-making procedures in many theories of uncertainty. The N-soft set theory is arising as an effective mathematical tool for dealing with precision and uncertainties more than the soft set theory. In this regard, we extend the concept of belief interval-valued soft set to possibility belief interval-valued N-soft set (by accumulating possibility and belief interval with N-soft set), and we also explain its practical calculations. To this objective, we defined related theoretical notions, for example, belief interval-valued N-soft set, possibility belief interval-valued N-soft set, their algebraic operations, and examined some of their fundamental properties. Furthermore, we developed two algorithms by using max-AND and min-OR operations of possibility belief interval-valued N-soft set for decision-making problems and also justify its applicability with numerical examples.

Research on Driving Fatigue Alleviation Using Interesting Auditory Stimulation Based on VMD-MMSE

The accurate detection and alleviation of driving fatigue are of great significance to traffic safety. In this study, we tried to apply the modified multi-scale entropy (MMSE) approach, based on variational mode decomposition (VMD), to driving fatigue detection. Firstly, the VMD was used to decompose EEG into multiple intrinsic mode functions (IMFs), then the best IMFs and scale factors were selected using the least square method (LSM). Finally, the MMSE features were extracted. Compared with the traditional sample entropy (SampEn), the VMD-MMSE method can identify the characteristics of driving fatigue more effectively. The VMD-MMSE characteristics combined with a subjective questionnaire (SQ) were used to analyze the change trends of driving fatigue under two driving modes: normal driving mode and interesting auditory stimulation mode. The results show that the interesting auditory stimulation method adopted in this paper can effectively relieve driving fatigue. In addition, the interesting auditory stimulation method, which simply involves playing interesting auditory information on the vehicle-mounted player, can effectively relieve driving fatigue. Compared with traditional driving fatigue-relieving methods, such as sleeping and drinking coffee, this interesting auditory stimulation method can relieve fatigue in real-time when the driver is driving normally.

A CNN identified by reinforcement learning-based optimization framework for EEG-based state evaluation

Objective.Electroencephalogram (EEG) data, as a kind of complex time-series, is one of the most widely-used information measurements for evaluating human psychophysiological states. Recently, numerous works applied deep learning techniques, especially the convolutional neural network (CNN), into EEG-based research. The design of the hyper-parameters of the CNN model has a great influence on the performance of the model. Therefore, automatically designing these hyper-parameters can save the time and labor of experts. This leads to the appearance of the neural architecture search technique. In this paper, we propose a reinforcement learning (RL)-based step-by-step framework to efficiently search for CNN models.Approach.Specifically, the deep Q network in RL is first used to determine the depth of convolutional layers and the connection modes among layers. Then particle swarm optimization algorithm is used to fine-tune the number and size of convolution kernels. Through this step-by-step strategy, the search space can be narrowed in each step for saving the overall time cost. This framework is employed for both EEG-based sleep stage classification and driver drowsiness evaluation tasks.Main results.The results show that compared with state-of-the-art methods, the high-performance CNN models identified by the proposed optimization framework, can achieve high overall accuracy and better root mean squared error in the two tasks.Significance.Therefore, the proposed optimization framework has a great potential to provide high-performance results for other kinds of classification and prediction tasks. In this way, it can greatly save researchers' time cost and promote broader applications of CNNs.

Toward practical driving fatigue detection using three frontal EEG channels: a proof-of-concept study

Objective. Although various driving fatigue detection strategies have been introduced, the limited practicability is still an obstacle for the real application of these technologies. This study is based on the newly proposed non-hair-bearing (NHB) method to achieve practical driving fatigue detection with fewer channels from NHB areas and more efficient electroencephalogram (EEG) features.Approach. EEG data were recorded from 20 healthy subjects (15 males, age = 22.2 ± 3.2 years) in a 90 min simulated driving task using a remote wireless cap. Behaviorally, subjects demonstrated a salient fatigue effect, as reflected by a monotonic increase in reaction time. Using a sliding-window approach, we determined the vigilant and fatigued states at individual level to reduce the inter-subject differences in behavioral impairment and brain activity. Multiple EEG features, including power-spectrum density (PSD), functional connectivity (FC), and entropy, were estimated in a pairwise manner, which were set as input for fatigue classification.Main results. Intriguingly, this data-driven approach showed that the best classification performance was achieved using three EEG channel pairs located in the NHB area. The mixed features of the frontal NHB area lead to the high within-subject detection rate of driving fatigue (92.7% ± 0.92%) with satisfactory generalizability for fatigue classification across different subjects (77.13% ± 0.85%). Moreover, we found the most prominent contributing features were PSD of different frequency bands within the frontal NHB area and FC within the frontal NHB area and between frontal and parietal areas.Significance. In summary, the current work provided objective evidence to support the effectiveness of the NHB method and further improved the performance, thereby moving a step forward towards practical driving fatigue detection in real-world scenarios.

A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing EEG signals

Driver fatigue is the one of the main reasons of the traffic accidents. The human brain is a complex structure, whose function can be evaluated with electroencephalogram (EEG). Automated driver fatigue detection utilizing EEG decreases the incidence probability of related traffic accidents. Therefore, devising an appropriate feature extraction technique and selecting a competent classification method can be considered as the crucial part of the effective driver fatigue detection. Therefore, in this study, an EEG-based intelligent system was devised for driver fatigue detection. The proposed framework includes a new feature generation network, which is implemented by using texture descriptors, for fatigue detection. The proposed scheme contains pre-processing, feature generation, informative features selection and classification with shallow classifiers phases. In the pre-processing, discrete cosine transform and fast Fourier transform are used together. Moreover, dynamic center based binary pattern and multi threshold ternary pattern are utilized together to create a new feature generation network. To improve the detection performance, we utilized discrete wavelet transform as a pooling method, in which the functional brain network-based feature describing the relationship between fatigue and brain network organization. In the feature selection phase, a hybrid three layered feature selection method is presented, and benchmark classifiers are used in the classification phase to demonstrate the strength of the proposed method. In the experiments, the proposed framework achieved 97.29% classification accuracy for fatigue detection using EEG signals. This result reveals that the proposed framework can be utilized effectively for driver fatigue detection.

A Discriminative Multi-Output Gaussian Processes Scheme for Brain Electrical Activity Analysis

The study of brain electrical activity (BEA) from different cognitive conditions has attracted a lot of interest in the last decade due to the high number of possible applications that could be generated from it. In this work, a discriminative framework for BEA via electroencephalography (EEG) is proposed based on multi-output Gaussian Processes (MOGPs) with a specialized spectral kernel. First, a signal segmentation stage is executed, and the channels from the EEG are used as the model outputs. Then, a novel covariance function within the MOGP known as the multispectral mixture kernel (MOSM) allows us to find and quantify the relationships between different channels. Several MOGPs are trained from different conditions grouped in bi-class problems, and the discrimination is performed based on the likelihood score of the test signals against all the models. Finally, the mean likelihood is computed to predict the correspondence of new inputs with each class’s existing models. Results show that this framework allows us to model the EEG signals adequately using generative models and allows analyzing the relationships between channels of the EEG for a particular condition. At the same time, the set of trained MOGPs is well suited to discriminate new input data.

Three-dimensional mechanical parts reconstruction technology based on two-dimensional image

With the development of computer technology and three-dimensional reconstruction technology, three-dimensional reconstruction based on visual images has become one of the research hotspots in computer graphics. Three-dimensional reconstruction based on visual image can be divided into three-dimensional reconstruction based on single photo and video. As an indirect three-dimensional modeling technology, this method is widely used in the fields of film and television production, cultural relics restoration, mechanical manufacturing, and medical health. This article studies and designs a stereo vision system based on two-dimensional image modeling technology. The system can be divided into image processing, camera calibration, stereo matching, three-dimensional point reconstruction, and model reconstruction. In the part of image processing, common image processing methods, feature point extraction algorithm, and edge extraction algorithm are studied. On this basis, interactive local corner extraction algorithm and interactive local edge detection algorithm are proposed. It is found that the Harris algorithm can effectively remove the features of less information and easy to generate clustering phenomenon. At the same time, the method of limit constraints is used to match the feature points extracted from the image. This method has high matching accuracy and short time. The experimental research has achieved good matching results. Using the platform of binocular stereo vision system, each step in the process of three-dimensional reconstruction has achieved high accuracy, thus achieving the three-dimensional reconstruction of the target object. Finally, based on the research of three-dimensional reconstruction of mechanical parts and the designed binocular stereo vision system platform, the experimental results of edge detection, camera calibration, stereo matching, and three-dimensional model reconstruction in the process of three-dimensional reconstruction are obtained, and the full text is summarized, analyzed, and prospected.

Negation of Pythagorean Fuzzy Number Based on a New Uncertainty Measure Applied in a Service Supplier Selection System

The Pythagorean fuzzy number (PFN) consists of membership and non-membership as an extension of the intuitionistic fuzzy number. PFN has a larger ambiguity, and it has a stronger ability to express uncertainty. In the multi-criteria decision-making (MCDM) problem, it is also very difficult to measure the ambiguity degree of a set of PFN. A new entropy of PFN is proposed based on a technique for order of preference by similarity to ideal solution (Topsis) method of revised relative closeness index in this paper. To verify the new entropy with a good performance in uncertainty measure, a new Pythagorean fuzzy number negation approach is proposed. We develop the PFN negation and find the correlation of the uncertainty measure. Existing methods can only evaluate the ambiguity of a single PFN. The newly proposed method is suitable to systematically evaluate the uncertainty of PFN in Topsis. Nowadays, there are no uniform criteria for measuring service quality. It brings challenges to the future development of airlines. Therefore, grasping the future market trends leads to winning with advanced and high-quality services. Afterward, the applicability in the service supplier selection system with the new entropy is discussed to evaluate the service quality and measure uncertainty. Finally, the new PFN entropy is verified with a good ability in the last MCDM numerical example.

Forecasting stock prices with long-short term memory neural network based on attention mechanism

The stock market is known for its extreme complexity and volatility, and people are always looking for an accurate and effective way to guide stock trading. Long short-term memory (LSTM) neural networks are developed by recurrent neural networks (RNN) and have significant application value in many fields. In addition, LSTM avoids long-term dependence issues due to its unique storage unit structure, and it helps predict financial time series. Based on LSTM and an attention mechanism, a wavelet transform is used to denoise historical stock data, extract and train its features, and establish the prediction model of a stock price. We compared the results with the other three models, including the LSTM model, the LSTM model with wavelet denoising and the gated recurrent unit(GRU) neural network model on S&P 500, DJIA, HSI datasets. Results from experiments on the S&P 500 and DJIA datasets show that the coefficient of determination of the attention-based LSTM model is both higher than 0.94, and the mean square error of our model is both lower than 0.05.

Unsupervised Learning of Multi-Sense Embedding with Matrix Factorization and Sparse Soft Clustering

In the natural language environment, accurately inferring the meaning of a token according to its context is crucial to understanding a sophisticated expression. However, this is not easy for a machine. The traditional language models used to train distributed word vectors are often restricted by single-sense embedding. In this paper, we develop a model called MSCvec (Multi-sense Soft Clustering Vector) for word sense disambiguation of polysemy in context. We extract the features of individual words by the co-occurrence PPMI (Positive Pointwise Mutual Information) matrix, and decompose the matrix by NMF (Nonnegative Matrix Factorization) into low-rank representations of target words, which are used as the input of an unsupervised sparse soft clustering method called Sparse Fuzzy C-means (SFCM). We use SFCM to determine the global semantic space of words, and partition the subspaces of multiple senses of a polysemous word. We relabel candidate words by the negative average log likelihood, and train multi-sense embedding with extensional vocabulary by the fastText model. Compared with the traditional static embeddings, the result shows that NMF and SFCM design can improve the performance in word similarity and relatedness tasks as well as in text classification tasks of different types of text. Accurate semantic representation of MSCvec would be necessary to produce outstanding results.