1
|
Lin N, Gao W, Li L, Chen J, Liang Z, Yuan G, Sun H, Liu Q, Chen J, Jin L, Huang Y, Zhou X, Zhang S, Hu P, Dai C, He H, Dong Y, Cui L, Lu Q. vEpiNet: A multimodal interictal epileptiform discharge detection method based on video and electroencephalogram data. Neural Netw 2024; 175:106319. [PMID: 38640698 DOI: 10.1016/j.neunet.2024.106319] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2024] [Revised: 03/08/2024] [Accepted: 04/11/2024] [Indexed: 04/21/2024]
Abstract
To enhance deep learning-based automated interictal epileptiform discharge (IED) detection, this study proposes a multimodal method, vEpiNet, that leverages video and electroencephalogram (EEG) data. Datasets comprise 24 931 IED (from 484 patients) and 166 094 non-IED 4-second video-EEG segments. The video data is processed by the proposed patient detection method, with frame difference and Simple Keypoints (SKPS) capturing patients' movements. EEG data is processed with EfficientNetV2. The video and EEG features are fused via a multilayer perceptron. We developed a comparative model, termed nEpiNet, to test the effectiveness of the video feature in vEpiNet. The 10-fold cross-validation was used for testing. The 10-fold cross-validation showed high areas under the receiver operating characteristic curve (AUROC) in both models, with a slightly superior AUROC (0.9902) in vEpiNet compared to nEpiNet (0.9878). Moreover, to test the model performance in real-world scenarios, we set a prospective test dataset, containing 215 h of raw video-EEG data from 50 patients. The result shows that the vEpiNet achieves an area under the precision-recall curve (AUPRC) of 0.8623, surpassing nEpiNet's 0.8316. Incorporating video data raises precision from 70% (95% CI, 69.8%-70.2%) to 76.6% (95% CI, 74.9%-78.2%) at 80% sensitivity and reduces false positives by nearly a third, with vEpiNet processing one-hour video-EEG data in 5.7 min on average. Our findings indicate that video data can significantly improve the performance and precision of IED detection, especially in prospective real clinic testing. It suggests that vEpiNet is a clinically viable and effective tool for IED analysis in real-world applications.
Collapse
Affiliation(s)
- Nan Lin
- Department of Neurology, Peking Union Medical College Hospital, Beijing, 100730, China
| | - Weifang Gao
- Department of Neurology, Peking Union Medical College Hospital, Beijing, 100730, China
| | - Lian Li
- NetEase Media Technology Co., Ltd., Beijing, 100084, China
| | - Junhui Chen
- NetEase Media Technology Co., Ltd., Beijing, 100084, China
| | - Zi Liang
- NetEase Media Technology Co., Ltd., Beijing, 100084, China
| | - Gonglin Yuan
- NetEase Media Technology Co., Ltd., Beijing, 100084, China
| | - Heyang Sun
- Department of Neurology, Peking Union Medical College Hospital, Beijing, 100730, China
| | - Qing Liu
- Department of Neurology, Peking Union Medical College Hospital, Beijing, 100730, China
| | - Jianhua Chen
- Department of Neurology, Peking Union Medical College Hospital, Beijing, 100730, China
| | - Liri Jin
- Department of Neurology, Peking Union Medical College Hospital, Beijing, 100730, China
| | - Yan Huang
- Department of Neurology, Peking Union Medical College Hospital, Beijing, 100730, China
| | - Xiangqin Zhou
- Department of Neurology, Peking Union Medical College Hospital, Beijing, 100730, China
| | - Shaobo Zhang
- NetEase Media Technology Co., Ltd., Beijing, 100084, China
| | - Peng Hu
- NetEase Media Technology Co., Ltd., Beijing, 100084, China
| | - Chaoyue Dai
- NetEase Media Technology Co., Ltd., Beijing, 100084, China
| | - Haibo He
- NetEase Media Technology Co., Ltd., Beijing, 100084, China
| | - Yisu Dong
- NetEase Media Technology Co., Ltd., Beijing, 100084, China
| | - Liying Cui
- Department of Neurology, Peking Union Medical College Hospital, Beijing, 100730, China.
| | - Qiang Lu
- Department of Neurology, Peking Union Medical College Hospital, Beijing, 100730, China.
| |
Collapse
|
2
|
Abdi-Sargezeh B, Shirani S, Sanei S, Took CC, Geman O, Alarcon G, Valentin A. A review of signal processing and machine learning techniques for interictal epileptiform discharge detection. Comput Biol Med 2024; 168:107782. [PMID: 38070202 DOI: 10.1016/j.compbiomed.2023.107782] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2023] [Revised: 11/15/2023] [Accepted: 11/28/2023] [Indexed: 01/10/2024]
Abstract
Brain interictal epileptiform discharges (IEDs), as one of the hallmarks of epileptic brain, are transient events captured by electroencephalogram (EEG). IEDs are generated by seizure networks, and they occur between seizures (interictal periods). The development of a robust method for IED detection could be highly informative for clinical treatment procedures and epileptic patient management. Since 1972, different machine learning techniques, from template matching to deep learning, have been developed to automatically detect IEDs from scalp EEG (scEEG) and intracranial EEG (iEEG). While the scEEG signals suffer from low information details and high attenuation of IEDs due to the high skull electrical impedance, the iEEG signals recorded using implanted electrodes enjoy higher details and are more suitable for identifying the IEDs. In this review paper, we group IED detection techniques into six categories: (1) template matching, (2) feature representation (mimetic, time-frequency, and nonlinear features), (3) matrix decomposition, (4) tensor factorization, (5) neural networks, and (6) estimation of the iEEG from the concurrent scEEG followed by detection and classification. The methods are compared quantitatively (e.g., in terms of accuracy, sensitivity, and specificity), and their general advantages and limitations are described. Finally, current limitations and possible future research paths related to this field are mentioned.
Collapse
Affiliation(s)
- Bahman Abdi-Sargezeh
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK; School of Science and Technology, Nottingham Trent University, Nottingham, UK.
| | - Sepehr Shirani
- School of Science and Technology, Nottingham Trent University, Nottingham, UK
| | - Saeid Sanei
- School of Science and Technology, Nottingham Trent University, Nottingham, UK
| | - Clive Cheong Took
- Department of Electronic Engineering, Royal Holloway, University of London, London, UK
| | - Oana Geman
- Computer, Electronics and Automation Department, University Stefan cel Mare, Suceava, Romania
| | - Gonzalo Alarcon
- Department of Clinical Neurophysiology, Royal Manchester Children's Hospital, Manchester, UK
| | - Antonio Valentin
- Department of Clinical Neuroscience, King's College London, London, UK
| |
Collapse
|
3
|
Wang Y, Cui W, Yu T, Li X, Liao X, Li Y. Dynamic Multi-Graph Convolution-Based Channel-Weighted Transformer Feature Fusion Network for Epileptic Seizure Prediction. IEEE Trans Neural Syst Rehabil Eng 2023; 31:4266-4277. [PMID: 37782584 DOI: 10.1109/tnsre.2023.3321414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/04/2023]
Abstract
Electroencephalogram (EEG) based seizure prediction plays an important role in the closed-loop neuromodulation system. However, most existing seizure prediction methods based on graph convolution network only focused on constructing the static graph, ignoring multi-domain dynamic changes in deep graph structure. Moreover, the existing feature fusion strategies generally concatenated coarse-grained epileptic EEG features directly, leading to the suboptimal seizure prediction performance. To address these issues, we propose a novel multi-branch dynamic multi-graph convolution based channel-weighted transformer feature fusion network (MB-dMGC-CWTFFNet) for the patient-specific seizure prediction with the superior performance. Specifically, a multi-branch (MB) feature extractor is first applied to capture the temporal, spatial and spectral representations fromthe epileptic EEG jointly. Then, we design a point-wise dynamic multi-graph convolution network (dMGCN) to dynamically learn deep graph structures, which can effectively extract high-level features from the multi-domain graph. Finally, by integrating the local and global channel-weighted strategies with the multi-head self-attention mechanism, a channel-weighted transformer feature fusion network (CWTFFNet) is adopted to efficiently fuse the multi-domain graph features. The proposed MB-dMGC-CWTFFNet is evaluated on the public CHB-MIT EEG and a private intracranial sEEG datasets, and the experimental results demonstrate that our proposed method achieves outstanding prediction performance compared with the state-of-the-art methods, indicating an effective tool for patient-specific seizure warning. Our code will be available at: https://github.com/Rockingsnow/MB-dMGC-CWTFFNet.
Collapse
|
4
|
Li S, Tang Z, Yang L, Li M, Shang Z. Application of deep reinforcement learning for spike sorting under multi-class imbalance. Comput Biol Med 2023; 164:107253. [PMID: 37536094 DOI: 10.1016/j.compbiomed.2023.107253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Revised: 06/27/2023] [Accepted: 07/07/2023] [Indexed: 08/05/2023]
Abstract
Spike sorting is the basis for analyzing spike firing patterns encoded in high-dimensional information spaces. With the fact that high-density microelectrode arrays record multiple neurons simultaneously, the data collected often suffers from two problems: a few overlapping spikes and different neuronal firing rates, which both belong to the multi-class imbalance problem. Since deep reinforcement learning (DRL) assign targeted attention to categories through reward functions, we propose ImbSorter to implement spike sorting under multi-class imbalance. We describe spike sorting as a Markov sequence decision and construct a dynamic reward function (DRF) to improve the sensitivity of the agent to minor classes based on the inter-class imbalance ratios. The agent is eventually guided by the optimal strategy to classify spikes. We consider the Wave_Clus dataset, which contains overlapping spikes and diverse noise levels, and the macaque dataset, which has a multi-scale imbalance. ImbSorter is compared with classical DRL architectures, traditional machine learning algorithms, and advanced overlapping spike sorting techniques on these two above datasets. ImbSorter obtained improved results on the Macro_F1. The results show ImbSorter has a promising ability to resist overlapping and noise interference. It has high stability and promising performance in processing spikes with different degrees of skewed distribution.
Collapse
Affiliation(s)
- Suchen Li
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou, 450001, China; Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou, 450001, China
| | - Zhuo Tang
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou, 450001, China; Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou, 450001, China
| | - Lifang Yang
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou, 450001, China; Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou, 450001, China
| | - Mengmeng Li
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou, 450001, China; Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou, 450001, China.
| | - Zhigang Shang
- School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou, 450001, China; Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou, 450001, China.
| |
Collapse
|
5
|
Lih OS, Jahmunah V, Palmer EE, Barua PD, Dogan S, Tuncer T, García S, Molinari F, Acharya UR. EpilepsyNet: Novel automated detection of epilepsy using transformer model with EEG signals from 121 patient population. Comput Biol Med 2023; 164:107312. [PMID: 37597408 DOI: 10.1016/j.compbiomed.2023.107312] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2023] [Revised: 08/02/2023] [Accepted: 08/04/2023] [Indexed: 08/21/2023]
Abstract
BACKGROUND Epilepsy is one of the most common neurological conditions globally, and the fourth most common in the United States. Recurrent non-provoked seizures characterize it and have huge impacts on the quality of life and financial impacts for affected individuals. A rapid and accurate diagnosis is essential in order to instigate and monitor optimal treatments. There is also a compelling need for the accurate interpretation of epilepsy due to the current scarcity in neurologist diagnosticians and a global inequity in access and outcomes. Furthermore, the existing clinical and traditional machine learning diagnostic methods exhibit limitations, warranting the need to create an automated system using deep learning model for epilepsy detection and monitoring using a huge database. METHOD The EEG signals from 35 channels were used to train the deep learning-based transformer model named (EpilepsyNet). For each training iteration, 1-min-long data were randomly sampled from each participant. Thereafter, each 5-s epoch was mapped to a matrix using the Pearson Correlation Coefficient (PCC), such that the bottom part of the triangle was discarded and only the upper triangle of the matrix was vectorized as input data. PCC is a reliable method used to measure the statistical relationship between two variables. Based on the 5 s of data, single embedding was performed thereafter to generate a 1-dimensional array of signals. In the final stage, a positional encoding with learnable parameters was added to each correlation coefficient's embedding before being fed to the developed EpilepsyNet as input data to epilepsy EEG signals. The ten-fold cross-validation technique was used to generate the model. RESULTS Our transformer-based model (EpilepsyNet) yielded high classification accuracy, sensitivity, specificity and positive predictive values of 85%, 82%, 87%, and 82%, respectively. CONCLUSION The proposed method is both accurate and robust since ten-fold cross-validation was employed to evaluate the performance of the model. Compared to the deep models used in existing studies for epilepsy diagnosis, our proposed method is simple and less computationally intensive. This is the earliest study to have uniquely employed the positional encoding with learnable parameters to each correlation coefficient's embedding together with the deep transformer model, using a huge database of 121 participants for epilepsy detection. With the training and validation of the model using a larger dataset, the same study approach can be extended for the detection of other neurological conditions, with a transformative impact on neurological diagnostics worldwide.
Collapse
Affiliation(s)
- Oh Shu Lih
- Cogninet Australia, Sydney, NSW, 2010, Australia
| | - V Jahmunah
- School of Engineering, Nanyang Polytechnic, Singapore
| | - Elizabeth Emma Palmer
- Centre of Clinical Genetics, Sydney Children's Hospitals Network, Randwick, 2031, Australia; School of Women's and Children's Health, University of New South Wales, Randwick, 2031, Australia
| | - Prabal D Barua
- School of Business (Information System), University of Southern Queensland, Australia
| | - Sengul Dogan
- Department of Digital Forensics Engineering, Technology Faculty, Firat University, Elazig, Turkey
| | - Turker Tuncer
- Department of Digital Forensics Engineering, Technology Faculty, Firat University, Elazig, Turkey
| | - Salvador García
- Andalusian Institute of Data Science and Computational Intelligence, Department of Computer Science and Artificial Intelligence, University of Granada, Spain
| | - Filippo Molinari
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia.
| |
Collapse
|
6
|
Nhu D, Janmohamed M, Shakhatreh L, Gonen O, Perucca P, Gilligan A, Kwan P, O'Brien TJ, Tan CW, Kuhlmann L. Automated Interictal Epileptiform Discharge Detection from Scalp EEG Using Scalable Time-series Classification Approaches. Int J Neural Syst 2023; 33:2350001. [PMID: 36599664 DOI: 10.1142/s0129065723500016] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
Deep learning for automated interictal epileptiform discharge (IED) detection has been topical with many published papers in recent years. All existing works viewed EEG signals as time-series and developed specific models for IED classification; however, general time-series classification (TSC) methods were not considered. Moreover, none of these methods were evaluated on any public datasets, making direct comparisons challenging. This paper explored two state-of-the-art convolutional-based TSC algorithms, InceptionTime and Minirocket, on IED detection. We fine-tuned and cross-evaluated them on a public (Temple University Events - TUEV) and two private datasets and provided ready metrics for benchmarking future work. We observed that the optimal parameters correlated with the clinical duration of an IED and achieved the best area under precision-recall curve (AUPRC) of 0.98 and F1 of 0.80 on the private datasets, respectively. The AUPRC and F1 on the TUEV dataset were 0.99 and 0.97, respectively. While algorithms trained on the private sets maintained their performance when tested on the TUEV data, those trained on TUEV could not generalize well to the private data. These results emerge from differences in the class distributions across datasets and indicate a need for public datasets with a better diversity of IED waveforms, background activities and artifacts to facilitate standardization and benchmarking of algorithms.
Collapse
Affiliation(s)
- D Nhu
- Department of Data Science and Artificial Intelligence, Faculty of Information Technology, Monash University, Melbourne, VIC, Australia
| | - M Janmohamed
- Department of Neuroscience, Central Clinical School, Monash University, Melbourne, VIC, Australia
| | - L Shakhatreh
- Department of Neuroscience, Central Clinical School, Monash University, Melbourne, VIC, Australia
| | - O Gonen
- Department of Neuroscience, Central Clinical School, Monash University, Melbourne, VIC, Australia
| | - P Perucca
- Department of Neuroscience, Central Clinical School, Monash University, Melbourne, VIC, Australia.,Bladin-Berkovic Comprehensive Epilepsy Program, Department of Neurology, Austin Health, Melbourne, VIC, Australia.,Epilepsy Research Center, Department of Medicine (Austin Health), The University of Melbourne, Melbourne, VIC, Australia
| | - A Gilligan
- Bladin-Berkovic Comprehensive Epilepsy Program, Department of Neurology, Austin Health, Melbourne, VIC, Australia.,Neurosciences Clinical Institute, Epworth Healthcare, Melbourne, VIC, Australia
| | - P Kwan
- Department of Neuroscience, Central Clinical School, Monash University, Melbourne, VIC, Australia
| | - T J O'Brien
- Department of Neuroscience, Central Clinical School, Monash University, Melbourne, VIC, Australia
| | - C W Tan
- Department of Data Science and Artificial Intelligence, Faculty of Information Technology, Monash University, Melbourne, VIC, Australia
| | - L Kuhlmann
- Department of Data Science and Artificial Intelligence, Faculty of Information Technology, Monash University, Melbourne, VIC, Australia
| |
Collapse
|
7
|
Fukumori K, Yoshida N, Sugano H, Nakajima M, Tanaka T. Satelight: Self-attention-based model for epileptic spike detection from multi-electrode EEG. J Neural Eng 2022; 19. [PMID: 36073896 DOI: 10.1088/1741-2552/ac9050] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2022] [Accepted: 09/05/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Because of the lack of highly skilled experts, automated technologies that support electroencephalogram (EEG)-based in epilepsy diagnosis are advancing. Deep convolutional neural network-based models have been used successfully for detecting epileptic spikes, one of the biomarkers, from EEG. However, a sizeable number of supervised EEG records are required for training. APPROACH This study introduces the Satelight model, which uses the self-attention (SA) mechanism. The model was trained using a clinical EEG dataset labeled by ve specialists, including 16,008 epileptic spikes and 15,478 artifacts from 50 children. The SA mechanism is expected to reduce the number of parameters and efficiently extract features from a small amount of EEG data. To validate the effectiveness, we compared various spike detection approaches with the clinical EEG data. MAIN RESULTS The experimental results showed that the proposed method detected epileptic spikes more effectively than other models (accuracy = 0:876 and false positive rate = 0:133). SIGNIFICANCE The proposed model had only one-tenth the number of parameters as the other effective model, despite having such a high detection performance. Further exploration of the hidden parameters revealed that the model automatically attended to the EEG's characteristic waveform locations of interest.
Collapse
Affiliation(s)
- Kosuke Fukumori
- Tokyo University of Agriculture and Technology, 2-24-16 Nakacho, Koganei-shi, Tokyo, 184-8588, JAPAN
| | - Noboru Yoshida
- Juntendo University Nerima Hospital, 3-1-10 Takanodai, Nerima-ku, Tokyo, 177-8521, JAPAN
| | - Hidenori Sugano
- Juntendo University, 2-1-1 Hongo, Bunkyo-ku, Tokyo, 113-8421, JAPAN
| | - Madoka Nakajima
- Juntendo University School of Medicine, 3-1-3 Hongo, Bunkyo-ku, Tokyo, 113-0033, JAPAN
| | - Toshihisa Tanaka
- Tokyo University of Agriculture and Technology, 2-24-16 Nakacho, Koganei-shi, Tokyo, 184-8588, JAPAN
| |
Collapse
|
8
|
Cheng C, Liu Y, You B, Zhou Y, Gao F, Yang L, Dai Y. Multilevel Feature Learning Method for Accurate Interictal Epileptiform Spike Detection. IEEE Trans Neural Syst Rehabil Eng 2022; 30:2506-2516. [PMID: 35877795 DOI: 10.1109/tnsre.2022.3193666] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Interictal epileptiform spike (referred to as spike) detected from electroencephalograms lasting only 20- to 200-ms can provide a reliable evidence-based indicator for clinical seizure type diagnosis. Recent feature representation approaches focus either on the concrete-level or on abstract-level information mining of the spike, thus demonstrating suboptimal detection performance. Additionally, existing abstract-level information mining methods of the spike based deep learning networks have not realized the effective feature representation of long-term dependent distinguished information within similar waveform cycles caused by morphological heterogeneity, which affects detection performance. Thus, a multilevel feature learning method for accurate spike detection was proposed in this study. Specifically, the spatio-temporal-frequency multidomain information in concrete-level first are inferred the common mimetic properties of the spike using the multidomain feature extractors. Then, the effective feature representation of long-term dependent distinguished information within similar waveform cycles caused by morphological heterogeneity is suitably captured using the temporal convolutional network. Finally, the spatio-temporal-frequency multidomain long-term dependent feature representation of spike is calculated using the element-wise manner to fuse the feature representation in concrete- and abstract-levels. The experimental results indicate that the proposed method can achieve an accuracy of 90.62±1.38%, sensitivity of 90.38±1.52%, specificity of 91.00±1.60%, precision of 90.33±4.71%, and the false detection rate per minute is 0.148±0.020m-1, which are higher than when using the feature representation in the concrete- or abstract-level alone. Additionally, the detection results indicate that the proposed method avoids the subjectivity and inefficiency of visual inspection, and it enables a highly accurate detection of the spike.
Collapse
|