Decoding Micro-Electrocorticographic Signals by Using Explainable 3D Convolutional Neural Network to Predict Finger Movements

BACKGROUND

Electroencephalography (EEG) and electrocorticography (ECoG) recordings have been used to decode finger movements by analyzing brain activity. Traditional methods focused on single bandpass power changes for movement decoding, utilizing machine learning models requiring manual feature extraction.

NEW METHOD

This study introduces a 3D convolutional neural network (3D-CNN) model to decode finger movements using ECoG data. The model employs adaptive, explainable AI (xAI) techniques to interpret the physiological relevance of brain signals. ECoG signals from epilepsy patients during awake craniotomy were processed to extract power spectral density across multiple frequency bands. These data formed a 3D matrix used to train the 3D-CNN to predict finger trajectories.

RESULTS

The 3D-CNN model showed significant accuracy in predicting finger movements, with root-mean-square error (RMSE) values of 0.26-0.38 for single finger movements and 0.20-0.24 for combined movements. Explainable AI techniques, Grad-CAM and SHAP, identified the high gamma (HG) band as crucial for movement prediction, showing specific cortical regions involved in different finger movements. These findings highlighted the physiological significance of the HG band in motor control.

COMPARISON WITH EXISTING METHODS

The 3D-CNN model outperformed traditional machine learning approaches by effectively capturing spatial and temporal patterns in ECoG data. The use of xAI techniques provided clearer insights into the model's decision-making process, unlike the "black box" nature of standard deep learning models.

CONCLUSIONS

The proposed 3D-CNN model, combined with xAI methods, enhances the decoding accuracy of finger movements from ECoG data. This approach offers a more efficient and interpretable solution for brain-computer interface (BCI) applications, emphasizing the HG band's role in motor control.

A real-time epilepsy seizure detection approach based on EEG using short-time Fourier transform and Google-Net convolutional neural network

Epilepsy is one of the most common brain disorders, and seizures of epilepsy have severe adverse effects on patients. Real-time epilepsy seizure detection using electroencephalography (EEG) signals is an important research area aimed at improving the diagnosis and treatment of epilepsy. This paper proposed a real-time approach based on EEG signal for detecting epilepsy seizures using the STFT and Google-net convolutional neural network (CNN). The CHB-MIT database was used to evaluate the performance, and received the results of 97.74 % in accuracy, 98.90 % in sensitivity, 1.94 % in false positive rate. Additionally, the proposed method was implemented in a real-time manner using the sliding window technique. The processing time of the proposed method just 0.02 s for every 2-s EEG episode and achieved average 9.85- second delay in each seizure onset.

Basketball technique action recognition using 3D convolutional neural networks

This research investigates the recognition of basketball techniques actions through the implementation of three-dimensional (3D) Convolutional Neural Networks (CNNs), aiming to enhance the accurate and automated identification of various actions in basketball games. Initially, basketball action sequences are extracted from publicly available basketball action datasets, followed by data preprocessing, including image sampling, data augmentation, and label processing. Subsequently, a novel action recognition model is proposed, combining 3D convolutions and Long Short-Term Memory (LSTM) networks to model temporal features and capture the spatiotemporal relationships and temporal information of actions. This facilitates the facilitating automatic learning of the spatiotemporal features associated with basketball actions. The model's performance and robustness are further improved through the adoption of optimization algorithms, such as adaptive learning rate adjustment and regularization. The efficacy of the proposed method is verified through experiments conducted on three publicly available basketball action datasets: NTURGB + D, Basketball-Action-Dataset, and B3D Dataset. The results indicate that this approach achieves outstanding performance in basketball technique action recognition tasks across different datasets compared to two common traditional methods. Specifically, when compared to the frame difference-based method, this model exhibits a significant accuracy improvement of 15.1%. When compared to the optical flow-based method, this model demonstrates a substantial accuracy improvement of 12.4%. Moreover, this method showcases strong robustness, accurately recognizing actions under diverse lighting conditions and scenes, achieving an average accuracy of 93.1%. The research demonstrates that the method reported here effectively captures the spatiotemporal relationships of basketball actions, thereby providing reliable technical assessment tools for basketball coaches and players.

Supervised and Unsupervised Deep Learning Approaches for EEG Seizure Prediction

Epilepsy affects more than 50 million people worldwide, making it one of the world's most prevalent neurological diseases. The main symptom of epilepsy is seizures, which occur abruptly and can cause serious injury or death. The ability to predict the occurrence of an epileptic seizure could alleviate many risks and stresses people with epilepsy face. We formulate the problem of detecting preictal (or pre-seizure) with reference to normal EEG as a precursor to incoming seizure. To this end, we developed several supervised deep learning approaches model to identify preictal EEG from normal EEG. We further develop novel unsupervised deep learning approaches to train the models on only normal EEG, and detecting pre-seizure EEG as an anomalous event. These deep learning models were trained and evaluated on two large EEG seizure datasets in a person-specific manner. We found that both supervised and unsupervised approaches are feasible; however, their performance varies depending on the patient, approach and architecture. This new line of research has the potential to develop therapeutic interventions and save human lives.

Parallel Dual-Branch Fusion Network for Epileptic Seizure Prediction

Epilepsy is a prevalent chronic disorder of the central nervous system. The timely and accurate seizure prediction using the scalp Electroencephalography (EEG) signal can make patients adopt reasonable preventive measures before seizures occur and thus reduce harm to patients. In recent years, deep learning-based methods have made significant progress in solving the problem of epileptic seizure prediction. However, most current methods mainly focus on modeling short- or long-term dependence in EEG, while neglecting to consider both. In this study, we propose a Parallel Dual-Branch Fusion Network (PDBFusNet) which aims to combine the complementary advantages of Convolutional Neural Network (CNN) and Transformer. Specifically, the features of the EEG signal are first extracted using Mel Frequency Cepstral Coefficients (MFCC). Then, the extracted features are delivered into the parallel dual-branches to simultaneously capture the short- and long-term dependencies of EEG signal. Further, regarding the Transformer branch, a novel feature fusion module is developed to enhance the ability of utilizing time, frequency, and channel information. To evaluate our proposal, we perform sufficient experiments on the public epileptic EEG dataset CHB-MIT, where the accuracy, sensitivity, specificity and precision are 95.76%, 95.81%, 95.71% and 95.71%, respectively. PDBFusNet shows superior performance compared to state-of-the-art competitors, which confirms the effectiveness of our proposal.

Seizure prediction based on improved vision transformer model for EEG channel optimization

Epileptic seizures are unpredictable events caused by abnormal discharges of a patient's brain cells. Extensive research has been conducted to develop seizure prediction algorithms based on long-term continuous electroencephalogram (EEG) signals. This paper describes a patient-specific seizure prediction method that can serve as a basis for the design of lightweight, wearable and effective seizure-prediction devices. We aim to achieve two objectives using this method. The first aim is to extract robust feature representations from multichannel EEG signals, and the second aim is to reduce the number of channels used for prediction by selecting an optimal set of channels from multichannel EEG signals while ensuring good prediction performance. We design a seizure-prediction algorithm based on a vision transformer (ViT) model. The algorithm selects channels that play a key role in seizure prediction from 22 channels of EEG signals. First, we perform a time-frequency analysis of processed time-series signals to obtain EEG spectrograms. We then segment the spectrograms of multiple channels into many non-overlapping patches of the same size, which are input into the channel selection layer of the proposed model, named Sel-JPM-ViT, enabling it to select channels. Application of the Sel-JPM-ViT model to the Boston Children's Hospital-Massachusetts Institute of Technology scalp EEG dataset yields results using only three to six channels of EEG signals that are slightly better that the results obtained using 22 channels of EEG signals. Overall, the Sel-JPM-ViT model exhibits an average classification accuracy of 93.65%, an average sensitivity of 94.70% and an average specificity of 92.78%.

A ResNet-LSTM hybrid model for predicting epileptic seizures using a pretrained model with supervised contrastive learning

In this paper, we propose a method for predicting epileptic seizures using a pre-trained model utilizing supervised contrastive learning and a hybrid model combining residual networks (ResNet) and long short-term memory (LSTM). The proposed training approach encompasses three key phases: pre-processing, pre-training as a pretext task, and training as a downstream task. In the pre-processing phase, the data is transformed into a spectrogram image using short time Fourier transform (STFT), which extracts both time and frequency information. This step compensates for the inherent complexity and irregularity of electroencephalography (EEG) data, which often hampers effective data analysis. During the pre-training phase, augmented data is generated from the original dataset using techniques such as band-stop filtering and temporal cutout. Subsequently, a ResNet model is pre-trained alongside a supervised contrastive loss model, learning the representation of the spectrogram image. In the training phase, a hybrid model is constructed by combining ResNet, initialized with weight values from the pre-trained model, and LSTM. This hybrid model extracts image features and time information to enhance prediction accuracy. The proposed method's effectiveness is validated using datasets from CHB-MIT and Seoul National University Hospital (SNUH). The method's generalization ability is confirmed through Leave-one-out cross-validation. From the experimental results measuring accuracy, sensitivity, and false positive rate (FPR), CHB-MIT was 91.90%, 89.64%, 0.058 and SNUH was 83.37%, 79.89%, and 0.131. The experimental results demonstrate that the proposed method outperforms the conventional methods.

Dynamic Multi-Graph Convolution-Based Channel-Weighted Transformer Feature Fusion Network for Epileptic Seizure Prediction

Electroencephalogram (EEG) based seizure prediction plays an important role in the closed-loop neuromodulation system. However, most existing seizure prediction methods based on graph convolution network only focused on constructing the static graph, ignoring multi-domain dynamic changes in deep graph structure. Moreover, the existing feature fusion strategies generally concatenated coarse-grained epileptic EEG features directly, leading to the suboptimal seizure prediction performance. To address these issues, we propose a novel multi-branch dynamic multi-graph convolution based channel-weighted transformer feature fusion network (MB-dMGC-CWTFFNet) for the patient-specific seizure prediction with the superior performance. Specifically, a multi-branch (MB) feature extractor is first applied to capture the temporal, spatial and spectral representations fromthe epileptic EEG jointly. Then, we design a point-wise dynamic multi-graph convolution network (dMGCN) to dynamically learn deep graph structures, which can effectively extract high-level features from the multi-domain graph. Finally, by integrating the local and global channel-weighted strategies with the multi-head self-attention mechanism, a channel-weighted transformer feature fusion network (CWTFFNet) is adopted to efficiently fuse the multi-domain graph features. The proposed MB-dMGC-CWTFFNet is evaluated on the public CHB-MIT EEG and a private intracranial sEEG datasets, and the experimental results demonstrate that our proposed method achieves outstanding prediction performance compared with the state-of-the-art methods, indicating an effective tool for patient-specific seizure warning. Our code will be available at: https://github.com/Rockingsnow/MB-dMGC-CWTFFNet.

Epileptic Seizure Prediction Using Attention Augmented Convolutional Network

Early seizure prediction is crucial for epilepsy patients to reduce accidental injuries and improve their quality of life. Identifying pre-ictal EEG from the inter-ictal state is particularly challenging due to their nonictal nature and remarkable similarities. In this study, a novel epileptic seizure prediction method is proposed based on multi-head attention (MHA) augmented convolutional neural network (CNN) to address the issue of CNN's limit of capturing global information of input signals. First, data enhancement is performed on original EEG recordings to balance the pre-ictal and inter-ictal EEG data, and the EEG recordings are sliced into 6-second-long EEG segments. Subsequently, EEG time-frequency distribution is obtained using Stockwell transform (ST), and the attention augmented convolutional network is employed for feature extraction and classification. Finally, post-processing is utilized to reduce the false prediction rate (FPR). The CHB-MIT EEG database was used to evaluate the system. The validation results showed a segment-based sensitivity of 98.24% and an event-based sensitivity of 94.78% with a FPR of 0.05/h were yielded, respectively. The satisfying results of the proposed method demonstrate its possible potential for clinical applications.

A robust seizure detection and prediction method with feature selection and spatio-temporal casual neural network model

Objective.Epilepsy is a fairly common condition that affects the brain and causes frequent seizures. The sudden and recurring epilepsy brings a series of safety hazards to patients, which seriously affects the quality of their life. Therefore, real-time diagnosis of electroencephalogram (EEG) in epilepsy patients is of great significance. However, the conventional methods take in a tremendous amount of features to train the models, resulting in high computation cost and low portability. Our objective is to propose an efficient, light and robust seizure detecting and predicting algorithm.Approach.The algorithm is based on an interpretative feature selection method and spatial-temporal causal neural network (STCNN). The feature selection method eliminates the interference factors between different features and reduces the model size and training difficulties. The STCNN model takes both temporal and spatial information to accurately and dynamically track and diagnose the changing of the features. Considering the differences between medical application scenarios and patients, leave-one-out cross validation (LOOCV) and cross-patient validation (CPV) methods are used to conduct experiments on the dataset collected at the Children's Hospital Boston (CHB-MIT), Siena and Kaggle competition datasets.Main results.In LOOCV-based method, the detection accuracy and prediction sensitivity have been improved. A significant improvement is also achieved in the CPV-based method.Significance.The experimental results show that our proposed algorithm exhibits superior performance and robustness in seizure detection and prediction, which indicates it has higher capability to deal with different and complicated clinical situations.

CLEP: Contrastive Learning for Epileptic Seizure Prediction Using a Spatio-Temporal-Spectral Network

Seizure prediction of epileptic preictal period through electroencephalogram (EEG) signals is important for clinical epilepsy diagnosis. However, recent deep learning-based methods commonly employ intra-subject training strategy and need sufficient data, which are laborious and time-consuming for a practical system and pose a great challenge for seizure predicting. Besides, multi-domain characterizations, including spatio-temporal-spectral dependencies in an epileptic brain are generally neglected or not considered simultaneously in current approaches, and this insufficiency commonly leads to suboptimal seizure prediction performance. To tackle the above issues, in this paper, we propose Contrastive Learning for Epileptic seizure Prediction (CLEP) using a Spatio-Temporal-Spectral Network (STS-Net). Specifically, the CLEP learns intrinsic epileptic EEG patterns across subjects by contrastive learning. The STS-Net extracts multi-scale temporal and spectral representations under different rhythms from raw EEG signals. Then, a novel triple attention layer (TAL) is employed to construct inter-dimensional interaction among multi-domain features. Moreover, a spatio dynamic graph convolution network (sdGCN) is proposed to dynamically model the spatial relationships between electrodes and aggregate spatial information. The proposed CLEP-STS-Net achieves a sensitivity of 96.7% and a false prediction rate of 0.072/h on the CHB-MIT scalp EEG database. We also validate the proposed method on clinical intracranial EEG (iEEG) database from our Xuanwu Hospital of Capital Medical University, and the predicting system yielded a sensitivity of 95%, a false prediction rate of 0.087/h. The experimental results outperform the state-of-the-art studies which validate the efficacy of our method. Our code is available at https://github.com/LianghuiGuo/CLEP-STS-Net.

A self-attention model for cross-subject seizure detection

Epilepsy is a neurological disorder characterized by recurring seizures, detected by electroencephalography (EEG). EEG signals can be detected by manual time-consuming analysis and recently by automatic detection. The latter poses a significant challenge due to the high dimensional and non-stationary nature of EEG signals. Recently, deep learning (DL) techniques have emerged as valuable tools for seizure detection. In this study, a novel data-driven model based on DL, incorporating a self-attention mechanism (SAT), is proposed. One notable advantage of the proposed method is its simplicity in application, as the raw signal data is directly fed into the suggested network without requiring expertise in signal processing. The model leverages a one-dimensional convolutional neural network (CNN) to extract relevant features from EEG signals. These features are then passed through a long short-term memory (LSTM) module to benefit from its memory capabilities, along with a SAT mechanism. The key contribution of this paper lies in the addition of the SAT layer to the LSTM encoder, enabling enhanced exploration of the latent mapping during the encoding step. Cross-subject experiments revealed good performance of this approach with F1-score of 97.8% and 92.7% for binary and five-class epileptic seizure recognition tasks, respectively, on the public UCI dataset, and 97.9% on the CHB-MIT database, surpassing state-of-the-art DL performance. Besides, the proposed method exhibits robustness to inter-subject variability.

Characteristic analysis of epileptic brain network based on attention mechanism

Constructing an efficient and accurate epilepsy detection system is an urgent research task. In this paper, we developed an EEG-based multi-frequency multilayer brain network (MMBN) and an attentional mechanism based convolutional neural network (AM-CNN) model to study epilepsy detection. Specifically, based on the multi-frequency characteristics of the brain, we first use wavelet packet decomposition and reconstruction methods to divide the original EEG signals into eight frequency bands, and then construct MMBN through correlation analysis between brain regions, where each layer corresponds to a specific frequency band. The time, frequency and channel related information of EEG signals are mapped into the multilayer network topology. On this basis, a multi-branch AM-CNN model is designed, which completely matches the multilayer structure of the proposed brain network. The experimental results on public CHB-MIT datasets show that eight frequency bands divided in this work are all helpful for epilepsy detection, and the fusion of multi-frequency information can effectively decode the epileptic brain state, achieving accurate detection of epilepsy with an average accuracy of 99.75%, sensitivity of 99.43%, and specificity of 99.83%. All of these provide reliable technical solutions for EEG-based neurological disease detection, especially for epilepsy detection.

An Epileptic Seizure Prediction Method Based on CBAM-3D CNN-LSTM Model

Epilepsy as a common disease of the nervous system, with high incidence, sudden and recurrent characteristics. Therefore, timely prediction of seizures and intervention treatment can significantly reduce the accidental injury of patients and protect the life and health of patients. Epilepsy seizures is the result of temporal and spatial evolution, Existing deep learning methods often ignore its spatial features, in order to make better use of the temporal and spatial characteristics of epileptic EEG signals. We propose a CBAM-3D CNN-LSTM model to predict epilepsy seizures. First, we apply short-time Fourier transform(STFT) to preprocess EEG signals. Secondly, the 3D CNN model was used to extract the features of preictal stage and interictal stage from the preprocessed signals. Thirdly, Bi-LSTM is connected to 3D CNN for classification. Finally CBAM is introduced into the model. Different attention is given to the data channel and space to extract key information, so that the model can accurately extract interictal and pre-ictal features. Our proposed approach achieved an accuracy of 97.95%, a sensitivity of 98.40%, and a false alarm rate of 0.017 h-1 on 11 patients from the public CHB-MIT scalp EEG dataset. Clinical and Translational Impact Statement-Timely prediction of epileptic seizures and intervention treatment can significantly reduce the accidental injury of patients and protect the life and health of patients.

General and patient-specific seizure classification using deep neural networks

Seizure prediction algorithms have been central in the field of data analysis for the improvement of epileptic patients’ lives. The most recent advancements of which include the use of deep neural networks to present an optimized, accurate seizure prediction system. This work puts forth deep learning methods to automate the process of epileptic seizure detection with electroencephalogram (EEG) signals as input; both a patient-specific and general approach are followed. EEG signals are time structure series motivating the use of sequence algorithms such as temporal convolutional neural networks (TCNNs), and long short-term memory networks. We then compare this methodology to other prior pre-implemented structures, including our previous work for seizure prediction using machine learning approaches support vector machine and random under-sampling boost. Moreover, patient-specific and general seizure prediction approaches are used to evaluate the performance of the best algorithms. Area under curve (AUC) is used to select the best performing algorithm to account for the imbalanced dataset. The presented TCNN model showed the best patient-specific results than that of the general approach with, AUC of 0.73, while ML model had the best results for general classification with AUC of 0.75.

Automatic prediction of epileptic seizure using hybrid deep ResNet-LSTM model

Numerous advanced data processing and machine learning techniques for identifying epileptic seizures have been developed in the last two decades. Nonetheless, many of these solutions need massive data sets and intricate computations. Our approach transforms electroencephalogram (EEG) data into the time-frequency domain by utilizing a short-time fourier transform (STFT) and the spectrogram (t-f) images as the input stage of the deep learning model. Using EEG data, we have constructed a hybrid model comprising of a Deep Convolution Network (ResNet50) and a Long Short-Term Memory (LSTM) for predicting epileptic seizures. Spectrogram images are used to train the proposed hybrid model for feature extraction and classification. We analyzed the CHB-MIT scalp EEG dataset. For each preictal period of 5, 15, and 30 minutes, experiments are conducted to evaluate the performance of the proposed model. The experimental results indicate that the proposed model produced the optimum performance with a 5-minute preictal duration. We achieved an average accuracy of 94.5%, the average sensitivity of 93.7%, the f1-score of 0.9376, and the average false positive rate (FPR) of 0.055. Our proposed technique surpassed the random predictor and other current algorithms used for seizure prediction for all patients’ data in the dataset. One can use the effectiveness of our proposed model to help in the early diagnosis of epilepsy and provide early treatment.

Epilepsy seizure prediction with few-shot learning method

Epileptic seizures prediction and timely alarms allow the patient to take effective and preventive actions. In this paper, a convolutional neural network (CNN) is proposed to diagnose the preictal period. Our goal is for those epileptic patients in whom seizures occur late and it is very challenging to record the preictal signal for them. In the previous works, generalized methods were inevitably used for this group of patients which were not very accurate. Our approach to solve this problem is to provide a few-shot learning method. This method, having the previous knowledge, is trained with only a small number of samples, learns new tasks and reduces the efforts to collect more data. Evaluation results for three patients from the CHB–MIT database, for a 10-min seizure prediction horizon (SPH) and a 20-min seizure occurrence period (SOP), averaged sensitivity of 95.70% and a false prediction rate (FPR) of 0.057/h and for the 5-min prediction horizon and the 25-min seizure occurrence period averaged sensitivity of 98.52% and a false prediction rate of (FPR) of 0.045/h. The proposed few-shot learning method, based on previous knowledge gained from the generalizable method, is regulated with a few new patient samples for the patient. Our results show that the accuracy obtained in this method is higher than the generalizable methods.

The performance evaluation of the state-of-the-art EEG-based seizure prediction models

The recurrent and unpredictable nature of seizures can lead to unintentional injuries and even death. The rapid development of electroencephalogram (EEG) and Artificial Intelligence (AI) technologies has made it possible to predict seizures in real-time through brain-machine interfaces (BCI), allowing advanced intervention. To date, there is still much room for improvement in predictive seizure models constructed by EEG using machine learning (ML) and deep learning (DL). But, the most critical issue is how to improve the performance and generalization of the model, which involves some confusing conceptual and methodological issues. This review focuses on analyzing several factors affecting the performance of seizure prediction models, focusing on the aspects of post-processing, seizure occurrence period (SOP), seizure prediction horizon (SPH), and algorithms. Furthermore, this study presents some new directions and suggestions for building high-performance prediction models in the future. We aimed to clarify the concept for future research in related fields and improve the performance of prediction models to provide a theoretical basis for future applications of wearable seizure detection devices.

Spatio-Temporal-Spectral Hierarchical Graph Convolutional Network With Semisupervised Active Learning for Patient-Specific Seizure Prediction

Graph theory analysis using electroencephalogram (EEG) signals is currently an advanced technique for seizure prediction. Recent deep learning approaches, which fail to fully explore both the characterizations in EEGs themselves and correlations among different electrodes simultaneously, generally neglect the spatial or temporal dependencies in an epileptic brain and, thus, produce suboptimal seizure prediction performance consequently. To tackle this issue, in this article, a patient-specific EEG seizure predictor is proposed by using a novel spatio-temporal-spectral hierarchical graph convolutional network with an active preictal interval learning scheme (STS-HGCN-AL). Specifically, since the epileptic activities in different brain regions may be of different frequencies, the proposed STS-HGCN-AL framework first infers a hierarchical graph to concurrently characterize an epileptic cortex under different rhythms, whose temporal dependencies and spatial couplings are extracted by a spectral-temporal convolutional neural network and a variant self-gating mechanism, respectively. Critical intrarhythm spatiotemporal properties are then captured and integrated jointly and further mapped to the final recognition results by using a hierarchical graph convolutional network. Particularly, since the preictal transition may be diverse from seconds to hours prior to a seizure onset among different patients, our STS-HGCN-AL scheme estimates an optimal preictal interval patient dependently via a semisupervised active learning strategy, which further enhances the robustness of the proposed patient-specific EEG seizure predictor. Competitive experimental results validate the efficacy of the proposed method in extracting critical preictal biomarkers, indicating its promising abilities in automatic seizure prediction.

A general sample-weighted framework for epileptic seizure prediction

OBJECTIVE

Effective epileptic seizure prediction can make the patients know the onset of the seizure in advance to take timely preventive measures. Many studies based on machine learning methods have been proposed to tackle this problem and achieve significant progress in recent years. However, most studies treat each EEG training sample's contribution to the model as equal, while different samples have different predictive effects on epileptic seizures (e.g., preictal samples from different times). To this end, in this paper, we propose a general sample-weighted framework for patient-specific epileptic seizure prediction.

METHODS

Specifically, we define the mapping from the sample weights of training sets to the performance of the validation sets as the fitness function to be optimized. Then, the genetic algorithm is employed to optimize this fitness function and obtain the optimal sample weights. Finally, we obtain the final model by using the training sets with optimized sample weights.

RESULTS

To evaluate the effectiveness of our framework, we conduct extensive experiments on both traditional machine learning methods and prevalent deep learning methods. Our framework can significantly improve performance based on these methods. Among them, our framework based on Transformer achieves an average sensitivity of 94.6%, an average false prediction rate of 0.06/h, and an average AUC of 0.939 in 12 pediatric patients from the CHB-MIT database with the leave-one-out method, which outperforms the state-of-the-art methods.

CONCLUSION

This study provides new insights into the field of epileptic seizure prediction by considering the discrepancies between EEG samples. Moreover, we develop a general sample-weighted framework, which applies to almost all classical classification methods and can significantly improve performance based on these methods.

An Intelligent Epileptic Prediction System Based on Synchrosqueezed Wavelet Transform and Multi-Level Feature CNN for Smart Healthcare IoT

Epilepsy is a common neurological disease worldwide, characterized by recurrent seizures. There is currently no cure for epilepsy. However, seizures can be controlled by drugs and surgeries in about 70% of epileptic patients. A timely and accurate prediction of seizures can prevent injuries during seizures and improve the patients' quality of life. In this paper, we proposed an intelligent epileptic prediction system based on Synchrosqueezed Wavelet Transform (SWT) and Multi-Level Feature Convolutional Neural Network (MLF-CNN) for smart healthcare IoT network. In this system, we used SWT to map EEG signals to the frequency domain, which was able to measure the energy changes in EEG signals caused by seizures within a well-defined Time-Frequency (TF) plane. MLF-CNN was then applied to extract multi-level features from the processed EEG signals and classify the different seizure segments. The performance of our proposed system was evaluated with the publicly available CHB-MIT dataset and our private ZJU4H dataset. The system achieved an accuracy of 96.99% and 94.25%, a sensitivity of 96.48% and 97.76%, a specificity of 97.46% and 94.07% and a false prediction rate (FPR/h) of 0.031 and 0.049 FPR/h on the CHB-MIT dataset and the ZJU4H dataset, respectively.

Classification of EEG Signals for Prediction of Epileptic Seizures

Epilepsy is a common brain disorder that causes patients to face multiple seizures in a single day. Around 65 million people are affected by epilepsy worldwide. Patients with focal epilepsy can be treated with surgery, whereas generalized epileptic seizures can be managed with medications. It has been noted that in more than 30% of cases, these medications fail to control epileptic seizures, resulting in accidents and limiting the patient’s life. Predicting epileptic seizures in such patients prior to the commencement of an oncoming seizure is critical so that the seizure can be treated with preventive medicines before it occurs. Electroencephalogram (EEG) signals of patients recorded to observe brain electrical activity during a seizure can be quite helpful in predicting seizures. Researchers have proposed methods that use machine and/or deep learning techniques to predict epileptic seizures using scalp EEG signals; however, prediction of seizures with increased accuracy is still a challenge. Therefore, we propose a three-step approach. It includes preprocessing of scalp EEG signals with PREP pipeline, which is a more sophisticated alternative to basic notch filtering. This method uses a regression-based technique to further enhance the SNR, with a combination of handcrafted, i.e., statistical features such as temporal mean, variance, and skewness, and automated features using CNN, followed by classification of interictal state and preictal state segments using LSTM to predict seizures. We train and validate our proposed technique on the CHB-MIT scalp EEG dataset and achieve accuracy of 94%, sensitivity of 93.8%, and 91.2% specificity. The proposed technique achieves better sensitivity and specificity than existing methods.

Multi-Channel Vision Transformer for Epileptic Seizure Prediction

Epilepsy is a neurological disorder that causes recurrent seizures and sometimes loss of awareness. Around 30% of epileptic patients continue to have seizures despite taking anti-seizure medication. The ability to predict the future occurrence of seizures would enable the patients to take precautions against probable injuries and administer timely treatment to abort or control impending seizures. In this study, we introduce a Transformer-based approach called Multi-channel Vision Transformer (MViT) for automated and simultaneous learning of the spatio-temporal-spectral features in multi-channel EEG data. Continuous wavelet transform, a simple yet efficient pre-processing approach, is first used for turning the time-series EEG signals into image-like time-frequency representations named Scalograms. Each scalogram is split into a sequence of fixed-size non-overlapping patches, which are then fed as inputs to the MViT for EEG classification. Extensive experiments on three benchmark EEG datasets demonstrate the superiority of the proposed MViT algorithm over the state-of-the-art seizure prediction methods, achieving an average prediction sensitivity of 99.80% for surface EEG and 90.28-91.15% for invasive EEG data.

EEG Identity Authentication in Multi-Domain Features: A Multi-Scale 3D-CNN Approach

Electroencephalogram (EEG) authentication has become a research hotspot in the field of information security due to its advantages of living, internal, and anti-stress. However, the performance of identity authentication system is limited by the inherent attributes of EEG, such as low SNR, low stability, and strong randomness. Researchers generally believe that the in-depth fusion of features can improve the performance of identity authentication and have explored among various feature domains. This experiment invited 70 subjects to participate in the EEG identity authentication task, and the experimental materials were visual stimuli of the self and non-self-names. This paper proposes an innovative EEG authentication framework, including efficient three-dimensional representation of EEG signals, multi-scale convolution structure, and the combination of multiple authentication strategies. In this work, individual EEG signals are converted into spatial–temporal–frequency domain three-dimensional forms to provide multi-angle mixed feature representation. Then, the individual identity features are extracted by the various convolution kernel of multi-scale vision, and the strategy of combining multiple convolution kernels is explored. The results show that the small-size and long-shape convolution kernel is suitable for ERP tasks, which can obtain better convergence and accuracy. The experimental results show that the classification performance of the proposed framework is excellent, and the multi-scale convolution method is effective to extract high-quality identity characteristics across feature domains. The results show that the branch number matches the EEG component number can obtain the excellent cost performance. In addition, this paper explores the network training performance for multi-scale module combination strategy and provides reference for deep network construction strategy of EEG signal processing.

Patient-Specific Seizure Prediction via Adder Network and Supervised Contrastive Learning

Deep learning (DL) methods have been widely used in the field of seizure prediction from electroencephalogram (EEG) in recent years. However, DL methods usually have numerous multiplication operations resulting in high computational complexity. In addtion, most of the current approaches in this field focus on designing models with special architectures to learn representations, ignoring the use of intrinsic patterns in the data. In this study, we propose a simple and effective end-to-end adder network and supervised contrastive learning (AddNet-SCL). The method uses addition instead of the massive multiplication in the convolution process to reduce the computational cost. Besides, contrastive learning is employed to effectively use label information, points of the same class are clustered together in the projection space, and points of different class are pushed apart at the same time. Moreover, the proposed model is trained by combining the supervised contrastive loss from the projection layer and the cross-entropy loss from the classification layer. Since the adder networks uses the ℓ1-norm distance as the similarity measure between the input feature and the filters, the gradient function of the network changes, an adaptive learning rate strategy is employed to ensure the convergence of AddNet-CL. Experimental results show that the proposed method achieves 94.9% sensitivity, an area under curve (AUC) of 94.2%, and a false positive rate of (FPR) 0.077/h on 19 patients in the CHB-MIT database and 89.1% sensitivity, an AUC of 83.1%, and an FPR of 0.120/h in the Kaggle database. Competitive results show that this method has broad prospects in clinical practice.

Designing and Validating a Robust Adaptive Neuromodulation Algorithm for Closed-Loop Control of Brain States

OBJECTIVE

Neuromodulation systems that use closed-loop brain stimulation to control brain states can provide new therapies for brain disorders. To date, closed-loop brain stimulation has largely used linear time-invariant controllers. However, nonlinear time-varying brain network dynamics and external disturbances can appear during real-time stimulation, collectively leading to real-time model uncertainty. Real-time model uncertainty can degrade the performance or even cause instability of time-invariant controllers. Three problems need to be resolved to enable accurate and stable control under model uncertainty. First, an adaptive controller is needed to track the model uncertainty. Second, the adaptive controller additionally needs to be robust to noise and disturbances. Third, theoretical analyses of stability and robustness are needed as prerequisites for stable operation of the controller in practical applications.

APPROACH

We develop a robust adaptive neuromodulation algorithm that solves the above three problems. First, we develop a state-space brain network model that explicitly includes nonlinear terms of real-time model uncertainty and design an adaptive controller to track and cancel the model uncertainty. Second, to improve the robustness of the adaptive controller, we design two linear filters to increase steady-state control accuracy and reduce sensitivity to high-frequency noise and disturbances. Third, we conduct theoretical analyses to prove the stability of the neuromodulation algorithm and establish a trade-off between stability and robustness, which we further use to optimize the algorithm design. Finally, we validate the algorithm using comprehensive Monte Carlo simulations that span a broad range of model nonlinearity, uncertainty, and complexity.

MAIN RESULTS

The robust adaptive neuromodulation algorithm accurately tracks various types of target brain state trajectories, enables stable and robust control, and significantly outperforms state-of-the-art neuromodulation algorithms.

SIGNIFICANCE

Our algorithm has implications for future designs of precise, stable, and robust closed-loop brain stimulation systems to treat brain disorders and facilitate brain functions.

Pediatric Seizure Prediction in Scalp EEG Using a Multi-Scale Neural Network With Dilated Convolutions

Objective: Epileptic seizure prediction based on scalp electroencephalogram (EEG) is of great significance for improving the quality of life of patients with epilepsy. In recent years, a number of studies based on deep learning methods have been proposed to address this issue and achieve excellent performance. However, most studies on epileptic seizure prediction by EEG fail to take full advantage of temporal-spatial multi-scale features of EEG signals, while EEG signals carry information in multiple temporal and spatial scales. To this end, in this study, we proposed an end-to-end framework by using a temporal-spatial multi-scale convolutional neural network with dilated convolutions for patient-specific seizure prediction. Methods: Specifically, the model divides the EEG processing pipeline into two stages: the temporal multi-scale stage and the spatial multi-scale stage. In each stage, we firstly extract the multi-scale features along the corresponding dimension. A dilated convolution block is then conducted on these features to expand our model's receptive fields further and systematically aggregate global information. Furthermore, we adopt a feature-weighted fusion strategy based on an attention mechanism to achieve better feature fusion and eliminate redundancy in the dilated convolution block. Results: The proposed model obtains an average sensitivity of 93.3%, an average false prediction rate of 0.007 per hour, and an average proportion of time-in-warning of 6.3% testing in 16 patients from the CHB-MIT dataset with the leave-one-out method. Conclusion: Our model achieves superior performance in comparison to state-of-the-art methods, providing a promising solution for EEG-based seizure prediction.

A novel consistency-based training strategy for seizure prediction

BACKGROUND

Early prediction of epilepsy seizures can warn the patients to take precautions and improve their lives significantly. In recent years, deep learning has become increasingly predominant in seizure prediction for its outstanding performance. With the aim of predicting unseen seizures, it is essential to guarantee the generalization ability of the model, especially considering the non-stationary nature of EEG and the scarcity of seizure events in EEG recordings. Stability training against extra perturbations is an intuitive and effective way to improve the model's ability to generalize. Though a great number of deep learning methods have been developed for seizure prediction, their strategies to increase generalization performance focus on improving the model's architecture itself, and few of them pay attention to the stability of the model against small perturbations.

NEW METHOD

In this study, we propose a novel consistency-based training strategy to address this issue. The proposed strategy underlines that a robust model should maintain consistent results for the same input under extra perturbations. Specifically, during training, we use stochastic augmentations to make the input vary from iteration to iteration and consider the output as a stochastic variable. Then a consistency constraint is constructed to penalize the difference between the current output and previous outputs. In this way, the generalization ability of the model will be fully enhanced.

RESULTS

To better verify the effectiveness of our proposed strategy, we implement it in two state-of-the-art models with public-available codes, including STFT CNN and Multi-view CNN. Notably, we compare with the first baseline on a scalp EEG dataset and the other on an intracranial EEG dataset. The results show that our strategy could improve the performance significantly for both of them.

COMPARISON WITH EXISTING METHODS

Our strategy has increased the sensitivity by 7.1% and reduced the false prediction rate by 0.12/h on the first baseline while improving the AUC by 0.020 on the second baseline.

CONCLUSIONS

This study is easy to implement, providing a new solution to enhance the performance of seizure prediction.

Deep Learning Models for Predicting Epileptic Seizures Using iEEG Signals

Epilepsy is a chronic neurological disease characterized by a large electrical explosion that is excessive and uncontrolled, as defined by the world health organization. It is an anomaly that affects people of all ages. An electroencephalogram (EEG) of the brain activity is a widely known method designed as a reference dedicated to study epileptic seizures and to record the changes in brain electrical activity. Therefore, the prediction and early detection of epilepsy is necessary to provide timely preventive interventions that allow patients to be relieved from the harmful consequences of epileptic seizures. Despite decades of research, the prediction of these seizures with accuracy remains an unresolved problem. In this article, we have proposed five deep learning models on intracranial electroencephalogram (iEEG) datasets with the aim of automatically predicting epileptic seizures. The proposed models are based on the Convolutional Neural Network (CNN) model, the fusion of the two CNNs (2-CNN), the fusion of the three CNNs (3-CNN), the fusion of the four CNNs (4-CNN), and transfer learning with ResNet50. The experimental results show that our proposed methods based on 3-CNN and 4-CNN gave the best values. They both achieve an accuracy value of 95%. Finally, our proposed methods are compared with previous studies, which confirm that seizure prediction performance was significantly improved.

Semisupervised Seizure Prediction in Scalp EEG Using Consistency Regularization

Early prediction of epilepsy seizures can warn the patients to take precautions and improve their lives significantly. In recent years, deep learning has become increasingly predominant in seizure prediction. However, existing deep learning-based approaches in this field require a great deal of labeled data to guarantee performance. At the same time, labeling EEG signals does require the expertise of an experienced pathologist and is incredibly time-consuming. To address this issue, we propose a novel Consistency-based Semisupervised Seizure Prediction Model (CSSPM), where only a fraction of training data is labeled. Our method is based on the principle of consistency regularization, which underlines that a robust model should maintain consistent results for the same input under extra perturbations. Specifically, by using stochastic augmentation and dropout, we consider the entire neural network as a stochastic model and apply a consistency constraint to penalize the difference between the current prediction and previous predictions. In this way, unlabeled data could be fully utilized to improve the decision boundary and enhance prediction performance. Compared with existing studies requiring all training data to be labeled, the proposed method only needs a small portion of data to be labeled while still achieving satisfactory results. Our method provides a promising solution to alleviate the labeling cost for real-world applications.

Seizure Prediction in EEG Signals Using STFT and Domain Adaptation

Epileptic seizure prediction is one of the most used therapeutic adjuvant strategies for drug-resistant epilepsy. Conventional approaches commonly collect training and testing samples from the same patient due to inter-individual variability. However, the challenging problem of domain shift between various subjects remains unsolved, resulting in a low conversion rate to the clinic. In this work, a domain adaptation (DA)-based model is proposed to circumvent this issue. The short-time Fourier transform (STFT) is employed to extract the time-frequency features from raw EEG data, and an autoencoder is developed to map these features into high-dimensional space. By minimizing the inter-domain distance in the embedding space, this model learns the domain-invariant information, such that the generalization ability is improved by distribution alignment. Besides, to increase the feasibility of its application, this work mimics the data distribution under the clinical sampling situation and tests the model under this condition, which is the first study that adopts the assessment strategy. Experimental results on both intracranial and scalp EEG databases demonstrate that this method can minimize the domain gap effectively compared with previous approaches.

Detection of preictal state in epileptic seizures using ensemble classifier

OBJECTIVE

Epilepsy affected patient experiences more than one frequency seizures which can not be treated with medication or surgical procedures in 30% of the cases. Therefore, an early prediction of these seizures is inevitable for these cases to control them with therapeutic interventions.

METHODS

In recent years, researchers have proposed multiple deep learning based methods for detection of preictal state in electroencephalogram (EEG) signals, however, accurate detection of start of preictal state remains a challenge. We propose a novel ensemble classifier based method that gets the comprehensive feature set as input and combines three different classifiers to detect the preictal state.

RESULTS

We have applied the proposed method on the publicly available scalp EEG dataset CHBMIT of 22 subjects. An average accuracy of 94.31% with sensitivity and specificity of 94.73% and 93.72% respectively has been achieved with the method proposed in this study.

CONCLUSIONS

Proposed study utilizes the preprocessing techniques for noise removal, combines deep learning based and handcrafted features and an ensemble classifier for detection of start of preictal state. Proposed method gives better results in terms of accuracy, sensitivity, and specificity.

An Overview of EEG-based Machine Learning Methods in Seizure Prediction and Opportunities for Neurologists in this Field

The unpredictability of epileptic seizures is one of the most problematic aspects of the field of epilepsy. Methods or devices capable of detecting seizures minutes before they occur may help prevent injury or even death and significantly improve the quality of life. Machine learning (ML) is an emerging technology that can markedly enhance algorithm performance by interpreting data. ML has gained increasing attention from medical researchers in recent years. Its epilepsy applications range from the localization of the epileptic region, predicting the medical or surgical outcome of epilepsy, and automated electroencephalography (EEG) analysis to seizure prediction. While ML has good prospects with regard to detecting epileptic seizures via EEG signals, many clinicians are still unfamiliar with this field. This work briefly summarizes the history and recent significant progress made in this field and clarifies the essential components of the automatic seizure detection system using ML methodologies for clinicians. This review also proposes how neurologists can actively contribute to ensure improvements in seizure prediction using EEG-based ML.

An Effective Dual Self-Attention Residual Network for Seizure Prediction

As one of the most challenging data analysis tasks in chronic brain diseases, epileptic seizure prediction has attracted extensive attention from many researchers. Seizure prediction, can greatly improve patients' quality of life in many ways, such as preventing accidents and reducing harm that may occur during epileptic seizures. This work aims to develop a general method for predicting seizures in specific patients through exploring the time-frequency correlation of features obtained from multi-channel EEG signals. We convert the original EEG signals into spectrograms that represent time-frequency characteristics by applying short-time Fourier transform (STFT) to the EEG signals. For the first time, we propose a dual self-attention residual network (RDANet) that combines a spectrum attention module integrating local features with global features, with a channel attention module mining the interdependence between channel mappings to achieve better forecasting performance. Our proposed approach achieved a sensitivity of 89.33%, a specificity of 93.02%, an AUC of 91.26% and an accuracy of 92.07% on 13 patients from the public CHB-MIT scalp EEG dataset. Our experiments show that different EEG signal prediction segment lengths are an important factor affecting prediction performance. Our proposed method is competitive and achieves good robustness without patient-specific engineering.

A Hybrid DenseNet-LSTM Model for Epileptic Seizure Prediction

The number of people diagnosed with epilepsy as a common brain disease accounts for about 1% of the world’s total population. Seizure prediction is an important study that can improve the lives of patients with epilepsy, and, in recent years, it has attracted more and more attention. In this paper, we propose a novel hybrid deep learning model that combines a Dense Convolutional Network (DenseNet) and Long Short-Term Memory (LSTM) for epileptic seizure prediction using EEG data. The proposed method first converts the EEG data into the time-frequency domain through Discrete Wavelet Transform (DWT) for use in the input of the model. Then, we train the previously transformed image through a hybrid model combining Densenet and LSTM. To evaluate the performance of the proposed method, experiments are conducted for each preictal length of 5, 10, and 15 min using the CHB-MIT scalp EEG dataset. As a result, we obtained a prediction accuracy of 93.28%, a sensitivity of 92.92%, a specificity of 93.65%, a false positive rate of 0.063 per hour, and an F1-score of 0.923 when the preictal length was 5 min. Finally, as the proposed method is compared to previous studies, it is confirmed that the seizure prediction performance was improved significantly.

Spatiotemporal evolution of epileptic seizure based on mutual information and dynamic brain network

BACKGROUND

Epilepsy was defined as an abnormal brain network model disease in the latest definition. From a microscopic perspective, it is also particularly important to observe the Mutual Information (MI) of the whole brain network based on different lead positions.

METHODS

In this study, we selected EEG data from representative temporal lobe and frontal lobe epilepsy patients. Based on Phase Space Reconstruction and the calculation of MI indicator, we used Complex Network technology to construct a dynamic brain network function model of epilepsy seizure. At the same time, about the analysis of our network, we described the index changes and propagation paths of epilepsy discharge in different periods, and spatially monitors the seizure change process based on the analysis of the parameter characteristics of the complex network.

RESULTS

Our model portrayed the functional synergy between the various regions of the brain and the state transition during the seizure process. We also characterized the EEG synchronous propagation path and core nodes during seizures. The results shown the full node change path and the distribution of important indicators during the seizure process, which makes the state change of the seizure process more clearly.

CONCLUSION

In this study, we have demonstrated that synchronization-based brain networks change with time and space. The EEG synchronous propagation path and core nodes during epileptic seizures can provide a reference for finding the focus area.

Research on epileptic EEG recognition based on improved residual networks of 1-D CNN and indRNN

Background

Epilepsy is one of the diseases of the nervous system, which has a large population in the world. Traditional diagnosis methods mostly depended on the professional neurologists’ reading of the electroencephalogram (EEG), which was time-consuming, inefficient, and subjective. In recent years, automatic epilepsy diagnosis of EEG by deep learning had attracted more and more attention. But the potential of deep neural networks in seizure detection had not been fully developed.

Methods

In this article, we used a one-dimensional convolutional neural network (1-D CNN) to replace the residual network architecture’s traditional convolutional neural network (CNN). Moreover, we combined the Independent recurrent neural network (indRNN) and CNN to form a new residual network architecture-independent convolutional recurrent neural network (RCNN). Our model can achieve an automatic diagnosis of epilepsy EEG. Firstly, the important features of EEG were learned by using the residual network architecture of 1-D CNN. Then the relationship between the sequences were learned by using the recurrent neural network. Finally, the model outputted the classification results.

Results

On the small sample data sets of Bonn University, our method was superior to the baseline methods and achieved 100% classification accuracy, 100% classification specificity. For the noisy real-world data, our method also exhibited powerful performance.

Conclusion

The model we proposed can quickly and accurately identify the different periods of EEG in an ideal condition and the real-world condition. The model can provide automatic detection capabilities for clinical epilepsy EEG detection. We hoped to provide a positive significance for the prediction of epileptic seizures EEG.