Deep learning approaches for seizure video analysis: A review

Seizure events can manifest as transient disruptions in the control of movements which may be organized in distinct behavioral sequences, accompanied or not by other observable features such as altered facial expressions. The analysis of these clinical signs, referred to as semiology, is subject to observer variations when specialists evaluate video-recorded events in the clinical setting. To enhance the accuracy and consistency of evaluations, computer-aided video analysis of seizures has emerged as a natural avenue. In the field of medical applications, deep learning and computer vision approaches have driven substantial advancements. Historically, these approaches have been used for disease detection, classification, and prediction using diagnostic data; however, there has been limited exploration of their application in evaluating video-based motion detection in the clinical epileptology setting. While vision-based technologies do not aim to replace clinical expertise, they can significantly contribute to medical decision-making and patient care by providing quantitative evidence and decision support. Behavior monitoring tools offer several advantages such as providing objective information, detecting challenging-to-observe events, reducing documentation efforts, and extending assessment capabilities to areas with limited expertise. The main applications of these could be (1) improved seizure detection methods; (2) refined semiology analysis for predicting seizure type and cerebral localization. In this paper, we detail the foundation technologies used in vision-based systems in the analysis of seizure videos, highlighting their success in semiology detection and analysis, focusing on work published in the last 7 years. We systematically present these methods and indicate how the adoption of deep learning for the analysis of video recordings of seizures could be approached. Additionally, we illustrate how existing technologies can be interconnected through an integrated system for video-based semiology analysis. Each module can be customized and improved by adapting more accurate and robust deep learning approaches as these evolve. Finally, we discuss challenges and research directions for future studies.

Computer vision for automated seizure detection and classification: A systematic review

Computer vision (CV) shows increasing promise as an efficient, low-cost tool for video seizure detection and classification. Here, we provide an overview of the fundamental concepts needed to understand CV and summarize the structure and performance of various model architectures used in video seizure analysis. We conduct a systematic literature review of the PubMed, Embase, and Web of Science databases from January 1, 2000 to September 15, 2023, to identify the strengths and limitations of CV seizure analysis methods and discuss the utility of these models when applied to different clinical seizure phenotypes. Reviews, nonhuman studies, and those with insufficient or poor quality data are excluded from the review. Of the 1942 records identified, 45 meet inclusion criteria and are analyzed. We conclude that the field has shown tremendous growth over the past 2 decades, leading to several model architectures with impressive accuracy and efficiency. The rapid and scalable detection offered by CV models holds the potential to reduce sudden unexpected death in epilepsy and help alleviate resource limitations in epilepsy monitoring units. However, a lack of standardized, thorough validation measures and concerns about patient privacy remain important obstacles for widespread acceptance and adoption. Investigation into the performance of models across varied datasets from clinical and nonclinical environments is an essential area for further research.

Automated analysis and detection of epileptic seizures in video recordings using artificial intelligence

Introduction

Automated seizure detection promises to aid in the prevention of SUDEP and improve the quality of care by assisting in epilepsy diagnosis and treatment adjustment.

Methods

In this phase 2 exploratory study, the performance of a contactless, marker-free, video-based motor seizure detection system is assessed, considering video recordings of patients (age 0-80 years), in terms of sensitivity, specificity, and Receiver Operating Characteristic (ROC) curves, with respect to video-electroencephalographic monitoring (VEM) as the medical gold standard. Detection performances of five categories of motor epileptic seizures (tonic-clonic, hyperkinetic, tonic, unclassified motor, automatisms) and psychogenic non-epileptic seizures (PNES) with a motor behavioral component lasting for >10 s were assessed independently at different detection thresholds (rather than as a categorical classification problem). A total of 230 patients were recruited in the study, of which 334 in-scope (>10 s) motor seizures (out of 1,114 total seizures) were identified by VEM reported from 81 patients. We analyzed both daytime and nocturnal recordings. The control threshold was evaluated at a range of values to compare the sensitivity (n = 81 subjects with seizures) and false detection rate (FDR) (n = all 230 subjects).

Results

At optimal thresholds, the performance of seizure groups in terms of sensitivity (CI) and FDR/h (CI): tonic-clonic- 95.2% (82.4, 100%); 0.09 (0.077, 0.103), hyperkinetic- 92.9% (68.5, 98.7%); 0.64 (0.59, 0.69), tonic- 78.3% (64.4, 87.7%); 5.87 (5.51, 6.23), automatism- 86.7% (73.5, 97.7%); 3.34 (3.12, 3.58), unclassified motor seizures- 78% (65.4, 90.4%); 4.81 (4.50, 5.14), and PNES- 97.7% (97.7, 100%); 1.73 (1.61, 1.86). A generic threshold recommended for all motor seizures under study asserted 88% sensitivity and 6.48 FDR/h.

Discussion

These results indicate an achievable performance for major motor seizure detection that is clinically applicable for use as a seizure screening solution in diagnostic workflows.

Multimodal nocturnal seizure detection in children with epilepsy: A prospective, multicenter, long-term, in-home trial

OBJECTIVE

There is a pressing need for reliable automated seizure detection in epilepsy care. Performance evidence on ambulatory non-electroencephalography-based seizure detection devices is low, and evidence on their effect on caregiver's stress, sleep, and quality of life (QoL) is still lacking. We aimed to determine the performance of NightWatch, a wearable nocturnal seizure detection device, in children with epilepsy in the family home setting and to assess its impact on caregiver burden.

METHODS

We conducted a phase 4, multicenter, prospective, video-controlled, in-home NightWatch implementation study (NCT03909984). We included children aged 4-16 years, with ≥1 weekly nocturnal major motor seizure, living at home. We compared a 2-month baseline period with a 2-month NightWatch intervention. The primary outcome was the detection performance of NightWatch for major motor seizures (focal to bilateral or generalized tonic-clonic [TC] seizures, focal to bilateral or generalized tonic seizures lasting >30 s, hyperkinetic seizures, and a remainder category of focal to bilateral or generalized clonic seizures and "TC-like" seizures). Secondary outcomes included caregivers' stress (Caregiver Strain Index [CSI]), sleep (Pittsburgh Quality of Sleep Index), and QoL (EuroQol five-dimension five-level scale).

RESULTS

We included 53 children (55% male, mean age = 9.7 ± 3.6 years, 68% learning disability) and analyzed 2310 nights (28 173 h), including 552 major motor seizures. Nineteen participants did not experience any episode of interest during the trial. The median detection sensitivity per participant was 100% (range = 46%-100%), and the median individual false alarm rate was .04 per hour (range = 0-.53). Caregiver's stress decreased significantly (mean total CSI score = 8.0 vs. 7.1, p = .032), whereas caregiver's sleep and QoL did not change significantly during the trial.

SIGNIFICANCE

The NightWatch system demonstrated high sensitivity for detecting nocturnal major motor seizures in children in a family home setting and reduced caregiver stress.

Adaptive Remote Sensing Paradigm for Real-Time Alerting of Convulsive Epileptic Seizures

Epilepsy is a debilitating neurological condition characterized by intermittent paroxysmal states called fits or seizures. Especially, the major motor seizures of a convulsive nature, such as tonic-clonic seizures, can cause aggravating consequences. Timely alerting for these convulsive epileptic states can therefore prevent numerous complications, during, or following the fit. Based on our previous research, a non-contact method using automated video camera observation and optical flow analysis underwent field trials in clinical settings. Here, we propose a novel adaptive learning paradigm for optimization of the seizure detection algorithm in each individual application. The main objective of the study was to minimize the false detection rate while avoiding undetected seizures. The system continuously updated detection parameters retrospectively using the data from the generated alerts. The system can be used under supervision or, alternatively, through autonomous validation of the alerts. In the latter case, the system achieved self-adaptive, unsupervised learning functionality. The method showed improvement of the detector performance due to the learning algorithm. This functionality provided a personalized seizure alerting device that adapted to the specific patient and environment. The system can operate in a fully automated mode, still allowing human observer to monitor and override the decision process while the algorithm provides suggestions as an expert system.

Optical Flow Estimation Improves Automated Seizure Detection in Neonatal EEG

PURPOSE

Existing automated seizure detection algorithms report sensitivities between 43% and 77% and specificities between 56% and 90%. The algorithms suffer from false alarms when applied to neonatal EEG because of the high degree of nurse handling and rhythmic patting used to soothe neonates. Computer vision technology that quantifies movement in real time could distinguish artifactual motion and improve automated neonatal seizure detection algorithms.

METHODS

The authors used video EEG recordings from 43 neonates undergoing monitoring for seizures as part of the NEOLEV2 clinical trial. The Persyst neonatal automated seizure detection algorithm ran in real time during study EEG acquisitions. Computer vision algorithms were applied to extract detailed accounts of artifactual movement of the neonate or people near the neonate though dense optical flow estimation.

RESULTS

Using the methods mentioned above, 197 periods of patting activity were identified and quantified, of which 45 generated false-positive automated seizure detection events. A binary patting detection algorithm was trained with a subset of 470 event videos. This supervised detection algorithm was applied to a testing subset of 187 event videos with 8 false-positive events, which resulted in a 24% reduction in false-positive automated seizure detections and a 50% reduction in false-positive events caused by neonatal care patting, while maintaining 11 of 12 true-positive seizure detection events.

CONCLUSIONS

This work presents a novel approach to improving automated seizure detection algorithms used during neonatal video EEG monitoring. This artifact detection mechanism can improve the ability of a seizure detector algorithm to distinguish between artifact and true seizure activity.

Owner's Perception of Seizure Detection Devices in Idiopathic Epileptic Dogs

Accurate knowledge of seizure frequency is key to optimising treatment. New methods for detecting epileptic seizures are currently investigated in humans, which rely on changes in biomarkers, also called seizure detection devices. Critical to device development, is understanding user needs and requirements. No information on this subject has been published in veterinary medicine. Many dog health collars are currently on the market, but none has proved to be a promising seizure detector. An online survey was created and consisted of 27 open, closed, and scaled questions divided over two parts: part one focused on general questions related to signalment and seizure semiology, the second part focused specifically on the use of seizure detection devices. Two hundred and thirty-one participants caring for a dog with idiopathic epilepsy, were included in the study. Open questions were coded using descriptive coding by two of the authors independently. Data was analysed using descriptive statistics and binary logistic regression. Our results showed that the unpredictability of seizures plays a major part in the management of canine epilepsy and dog owners have a strong desire to know when a seizure occurs. Nearly all dog owners made changes in their daily life, mainly focusing on intensifying supervision. Owners believed seizure detection devices would improve their dog's seizure management, including a better accuracy of seizure frequency and the ability to administer emergency drugs more readily. Owners that were already keeping track of their dog's seizures were 4.2 times more likely to show confidence in using seizure detection devices to manage their pet's seizures, highlighting the need for better monitoring systems. Our results show that there is a receptive market for wearable technology as a new management strategy in canine epilepsy and this topic should be further explored.

Video-Based Detection of Generalized Tonic-Clonic Seizures Using Deep Learning

Timely detection of seizures is crucial to implement optimal interventions, and may help reduce the risk of sudden unexpected death in epilepsy (SUDEP) in patients with generalized tonic-clonic seizures (GTCSs). While video-based automated seizure detection systems may be able to provide seizure alarms in both in-hospital and at-home settings, earlier studies have primarily employed hand-designed features for such a task. In contrast, deep learning-based approaches do not rely on prior feature selection and have demonstrated outstanding performance in many data classification tasks. Despite these advantages, neural network-based video classification has rarely been attempted for seizure detection. We here assessed the feasibility and efficacy of automated GTCSs detection from videos using deep learning. We retrospectively identified 76 GTCS videos from 37 participants who underwent long-term video-EEG monitoring (LTM) along with interictal video data from the same patients, and 10 full-night seizure-free recordings from additional patients. Using a leave-one-subject-out cross-validation approach (LOSO-CV), we evaluated the performance to detect seizures based on individual video frames (convolutional neural networks, CNNs) or video sequences [CNN+long short-term memory (LSTM) networks]. CNN+LSTM networks based on video sequences outperformed GTCS detection based on individual frames yielding a mean sensitivity of 88% and mean specificity of 92% across patients. The average detection latency after presumed clinical seizure onset was 22 seconds. Detection performance increased as a function of training dataset size. Collectively, we demonstrated that automated video-based GTCS detection with deep learning is feasible and efficacious. Deep learning-based methods may be able to overcome some limitations associated with traditional approaches using hand-crafted features, serve as a benchmark for future methods and analyses, and improve further with larger datasets.

Assessing the feasibility of detecting epileptic seizures using non-cerebral sensor data

This paper investigates the feasibility of using non-cerebral, time-series data to detect epileptic seizures. Data were recorded from fifteen patients (7 male, 5 female, 3 not noted, mean age 36.17 yrs), five of whom had a total of seven seizures. Patients were monitored in an inpatient setting using standard video-electroencephalography (vEEG), while also wearing sensors monitoring electrocardiography, electrodermal activity, electromyography, accelerometry, and audio signals (vocalizations). A systematic and detailed study was conducted to identify the sensors and the features derived from the non-cerebral sensors that contribute most significantly to separability of data acquired during seizures from non-seizure data. Post-processing of the data using linear discriminant analysis (LDA) shows that seizure data are strongly separable from non-seizure data based on features derived from the signals recorded. The mean area under the receiver operator characteristic (ROC) curve for each individual patient that experienced a seizure during data collection, calculated using LDA, was 0.9682. The features that contribute most significantly to seizure detection differ for each patient. The results show that a multimodal approach to seizure detection using the specified sensor suite is promising in detecting seizures with both sensitivity and specificity. Moreover, the study provides a means to quantify the contribution of each sensor and feature to separability. Development of a non-electroencephalography (EEG) based seizure detection device would give doctors a more accurate seizure count outside of the clinical setting, improving treatment and the quality of life of epilepsy patients.

Automated video-based detection of nocturnal motor seizures in children

Seizure detection devices can improve epilepsy care, but wearables are not always tolerated. We previously demonstrated good performance of a real‐time video‐based algorithm for detection of nocturnal convulsive seizures in adults with learning disabilities. The algorithm calculates the relative frequency content based on the group velocity reconstruction from video‐sequence optical flow. We aim to validate the video algorithm on nocturnal motor seizures in a pediatric population. We retrospectively analyzed the algorithm performance on a database including 1661 full recorded nights of 22 children (age = 3‐17 years) with refractory epilepsy at home or in a residential care setting. The algorithm detected 118 of 125 convulsions (median sensitivity per participant = 100%, overall sensitivity = 94%, 95% confidence interval = 61%‐100%) and identified all 135 hyperkinetic seizures. Most children had no false alarms; 81 false alarms occurred in six children (median false alarm rate [FAR] per participant per night = 0 [range = 0‐0.47], overall FAR = 0.05 per night). Most false alarms (62%) were behavior‐related (eg, awake and playing in bed). Our noncontact detection algorithm reliably detects nocturnal epileptic events with only a limited number of false alarms and is suitable for real‐time use.

Non-electroencephalography-based seizure detection

PURPOSE OF REVIEW

There is need for automated seizure detection using mobile or wearable devices, for objective seizure documentation and decreasing morbidity and mortality associated with seizures. Due to technological development, a high number of articles have addressed non-electroencephalography (EEG)-based seizure detection. However, the quality of study-design and reporting is extremely heterogeneous. We aimed at giving the reader a clear picture on the current state of seizure detection, describing the level of evidence behind the various devices.

RECENT FINDINGS

Fifteen studies of phase-2 or above, demonstrated that non-EEG-based devices detected generalized tonic-clonic seizures (GTCS) with high sensitivity (≥90%) and low false alarm rate (FAR) (down to 0.2/day). We found limited evidence for detection of motor seizures other than GTCS, mostly from subgroups in larger studies, targeting GTCS. There is little evidence for non-EEG-based detection of nonmotor seizures: sensitivity is low (19-74%) with extremely high FAR (50-216/day).

SUMMARY

Detection of GTCS is reliable and there are several, validated devices on the market. However, detection of other seizure types needs further research.

Automated remote fall detection using impact features from video and audio

Elderly people and people with epilepsy may need assistance after falling, but may be unable to summon help due to injuries or impairment of consciousness. Several wearable fall detection devices have been developed, but these are not used by all people at risk. We present an automated analysis algorithm for remote detection of high impact falls, based on a physical model of a fall, aiming at universality and robustness. Candidate events are automatically detected and event features are used as classifier input. The algorithm uses vertical velocity and acceleration features from optical flow outputs, corrected for distance from the camera using moving object size estimation. A sound amplitude feature is used to increase detector specificity. We tested the performance and robustness of our trained algorithm using acted data from a public database and real life data with falls resulting from epilepsy and with daily life activities. Applying the trained algorithm to the acted dataset resulted in 90% sensitivity for detection of falls, with 92% specificity. In the real life data, six/nine falls were detected with a specificity of 99.7%; there is a plausible explanation for not detecting each of the falls missed. These results reflect the algorithm's robustness and confirms the feasibility of detecting falls using this algorithm.

Seizure detection: do current devices work? And when can they be useful?

PURPOSE OF REVIEW

The unpredictability and apparent randomness of epileptic seizures is one of the most vexing aspects of epilepsy. Methods or devices capable of detecting seizures may help prevent injury or even death and significantly improve quality of life. Here, we summarize and evaluate currently available, unimodal, or polymodal detection systems for epileptic seizures, mainly in the ambulatory setting.

RECENT FINDINGS

There are two broad categories of detection devices: EEG-based and non-EEG-based systems. Wireless wearable EEG devices are now available both in research and commercial arenas. Neuro-stimulation devices are currently evolving and initial experiences of these show potential promise. As for non-EEG devices, different detecting systems show different sensitivity according to the different patient and seizure types. Regardless, when used in combination, these modalities may complement each other to increase positive predictive value. Although some devices with high sensitivity are promising, practical widespread use of such detection systems is still some way away. More research and experience are needed to evaluate the most efficient and integrated systems, to allow for better approaches to detection and prediction of seizures. The concept of closed-loop systems and prompt intervention may substantially improve quality of life for patients and carers.

Multimodal, automated detection of nocturnal motor seizures at home: Is a reliable seizure detector feasible?

Objective

Automated seizure detection and alarming could improve quality of life and potentially prevent sudden, unexpected death in patients with severe epilepsy. As currently available systems focus on tonic–clonic seizures, we want to detect a broader range of seizure types, including tonic, hypermotor, and clusters of seizures.

Methods

In this multicenter, prospective cohort study, the nonelectroencephalographic (non‐EEG) signals heart rate and accelerometry were measured during the night in patients undergoing a diagnostic video‐EEG examination. Based on clinical video‐EEG data, seizures were classified and categorized as clinically urgent or not. Seizures included for analysis were tonic, tonic–clonic, hypermotor, and clusters of short myoclonic/tonic seizures. Features reflecting physiological changes in heart rate and movement were extracted. Detection algorithms were developed based on stepwise fulfillment of conditions during increases in either feature. A training set was used for development of algorithms, and an independent test set was used for assessing performance.

Results

Ninety‐five patients were included, but due to sensor failures, data from only 43 (of whom 23 patients had 86 seizures, representing 402 h of data) could be used for analysis. The algorithms yield acceptable sensitivities, especially for clinically urgent seizures (sensitivity = 71–87%), but produce high false alarm rates (2.3–5.7 per night, positive predictive value = 25–43%). There was a large variation in the number of false alarms per patient.

Significance

It seems feasible to develop a detector with high sensitivity, but false alarm rates are too high for use in clinical practice. For further optimization, personalization of algorithms may be necessary.

Dynamics of convulsive seizure termination and postictal generalized EEG suppression

It is not fully understood how seizures terminate and why some seizures are followed by a period of complete brain activity suppression, postictal generalized EEG suppression. This is clinically relevant as there is a potential association between postictal generalized EEG suppression, cardiorespiratory arrest and sudden death following a seizure. We combined human encephalographic seizure data with data of a computational model of seizures to elucidate the neuronal network dynamics underlying seizure termination and the postictal generalized EEG suppression state. A multi-unit computational neural mass model of epileptic seizure termination and postictal recovery was developed. The model provided three predictions that were validated in EEG recordings of 48 convulsive seizures from 48 subjects with refractory focal epilepsy (20 females, age range 15-61 years). The duration of ictal and postictal generalized EEG suppression periods in human EEG followed a gamma probability distribution indicative of a deterministic process (shape parameter 2.6 and 1.5, respectively) as predicted by the model. In the model and in humans, the time between two clonic bursts increased exponentially from the start of the clonic phase of the seizure. The terminal interclonic interval, calculated using the projected terminal value of the log-linear fit of the clonic frequency decrease was correlated with the presence and duration of postictal suppression. The projected terminal interclonic interval explained 41% of the variation in postictal generalized EEG suppression duration (P < 0.02). Conversely, postictal generalized EEG suppression duration explained 34% of the variation in the last interclonic interval duration. Our findings suggest that postictal generalized EEG suppression is a separate brain state and that seizure termination is a plastic and autonomous process, reflected in increased duration of interclonic intervals that determine the duration of postictal generalized EEG suppression.

Automated Video Detection of Epileptic Convulsion Slowing as a Precursor for Post-Seizure Neuronal Collapse

Automated monitoring and alerting for adverse events in people with epilepsy can provide higher security and quality of life for those who suffer from this debilitating condition. Recently, we found a relation between clonic slowing at the end of a convulsive seizure (CS) and the occurrence and duration of a subsequent period of postictal generalized EEG suppression (PGES). Prolonged periods of PGES can be predicted by the amount of progressive increase of interclonic intervals (ICIs) during the seizure. The purpose of the present study is to develop an automated, remote video sensing-based algorithm for real-time detection of significant clonic slowing that can be used to alert for PGES. This may help preventing sudden unexpected death in epilepsy (SUDEP). The technique is based on our previously published optical flow video sequence processing paradigm that was applied for automated detection of major motor seizures. Here, we introduce an integral Radon-like transformation on the time–frequency wavelet spectrum to detect log–linear frequency changes during the seizure. We validate the automated detection and quantification of the ICI increase by comparison to the results from manually processed electroencephalography (EEG) traces as “gold standard”. We studied 48 cases of convulsive seizures for which synchronized EEG-video recordings were available. In most cases, the spectral ridges obtained from Gabor-wavelet transformations of the optical flow group velocities were in close proximity to the ICI traces detected manually from EEG data during the seizure. The quantification of the slowing-down effect measured by the dominant angle in the Radon transformed spectrum was significantly correlated with the exponential ICI increase factors obtained from manual detection. If this effect is validated as a reliable precursor of PGES periods that lead to or increase the probability of SUDEP, the proposed method would provide an efficient alerting device.

Non-EEG based ambulatory seizure detection designed for home use: What is available and how will it influence epilepsy care?

OBJECTIVE

This study aimed to (1) evaluate available systems and algorithms for ambulatory automatic seizure detection and (2) discuss benefits and disadvantages of seizure detection in epilepsy care.

METHODS

PubMed and EMBASE were searched up to November 2014, using variations and synonyms of search terms "seizure prediction" OR "seizure detection" OR "seizures" AND "alarm".

RESULTS

Seventeen studies evaluated performance of devices and algorithms to detect seizures in a clinical setting. Algorithms detecting generalized tonic-clonic seizures (GTCSs) had varying sensitivities (11% to 100%) and false alarm rates (0.2-4/24 h). For other seizure types, detection rates were low, or devices produced many false alarms. Five studies externally validated the performance of four different devices for the detection of GTCSs. Two devices were promising in both children and adults: a mattress-based nocturnal seizure detector (sensitivity: 84.6% and 100%; false alarm rate: not reported) and a wrist-based detector (sensitivity: 89.7%; false alarm rate: 0.2/24 h).

SIGNIFICANCE

Detection of seizure types other than GTCSs is currently unreliable. Two detection devices for GTCSs provided promising results when externally validated in a clinical setting. However, these devices need to be evaluated in the home setting in order to establish their true value. Automatic seizure detection may help prevent sudden unexpected death in epilepsy or status epilepticus, provided the alarm is followed by an effective intervention. Accurate seizure detection may improve the quality of life (QoL) of subjects and caregivers by decreasing burden of seizure monitoring and may facilitate diagnostic monitoring in the home setting. Possible risks are occurrence of alarm fatigue and invasion of privacy. Moreover, an unexpectedly high seizure frequency might be detected for which there are no treatment options. We propose that future studies monitor benefits and disadvantages of seizure detection systems with particular emphasis on QoL, comfort, and privacy of subjects and impact of false alarms.