Interictal Epileptiform Discharge Detection in EEG in Different Practice Settings

Expert accuracy and inter-rater agreement of "must-know" EEG findings for adult and child neurology residents

OBJECTIVE

We published a list of "must-know" routine EEG (rEEG) findings for trainees based on expert opinion. Here, we studied the accuracy and inter-rater agreement (IRA) of these "must-know" rEEG findings among international experts.

METHODS

A previously validated online rEEG examination was disseminated to EEG experts. It consisted of a survey and 30 multiple-choice questions predicated on the previously published "must-know" rEEG findings divided into four domains: normal, abnormal, normal variants, and artifacts. Questions contained de-identified 10-20-s epochs of EEG that were considered unequivocal examples by five EEG experts.

RESULTS

The examination was completed by 258 international EEG experts. Overall mean accuracy and IRA (AC1) were 81% and substantial (0.632), respectively. The domain-specific mean accuracies and IRA were: 76%, moderate (0.558) (normal); 78%, moderate (0.575) (abnormal); 85%, substantial (0.678) (normal variants); 85%, substantial (0.740) (artifacts). Academic experts had a higher accuracy than private practice experts (82% vs. 77%; p = .035). Country-specific overall mean accuracies and IRA were: 92%, almost perfect (0.836) (U.S.); 86%, substantial (0.762) (Brazil); 79%, substantial (0.646) (Italy); and 72%, moderate (0.496) (India). In conclusion, collective expert accuracy and IRA of "must-know" rEEG findings are suboptimal and heterogeneous.

SIGNIFICANCE

We recommend the development and implementation of pragmatic, accessible, country-specific ways to measure and improve the expert accuracy and IRA.

A review of signal processing and machine learning techniques for interictal epileptiform discharge detection

Brain interictal epileptiform discharges (IEDs), as one of the hallmarks of epileptic brain, are transient events captured by electroencephalogram (EEG). IEDs are generated by seizure networks, and they occur between seizures (interictal periods). The development of a robust method for IED detection could be highly informative for clinical treatment procedures and epileptic patient management. Since 1972, different machine learning techniques, from template matching to deep learning, have been developed to automatically detect IEDs from scalp EEG (scEEG) and intracranial EEG (iEEG). While the scEEG signals suffer from low information details and high attenuation of IEDs due to the high skull electrical impedance, the iEEG signals recorded using implanted electrodes enjoy higher details and are more suitable for identifying the IEDs. In this review paper, we group IED detection techniques into six categories: (1) template matching, (2) feature representation (mimetic, time-frequency, and nonlinear features), (3) matrix decomposition, (4) tensor factorization, (5) neural networks, and (6) estimation of the iEEG from the concurrent scEEG followed by detection and classification. The methods are compared quantitatively (e.g., in terms of accuracy, sensitivity, and specificity), and their general advantages and limitations are described. Finally, current limitations and possible future research paths related to this field are mentioned.

On-demand EEG education through competition - A novel, app-based approach to learning to identify interictal epileptiform discharges

Objective

Misinterpretation of EEGs harms patients, yet few resources exist to help trainees practice interpreting EEGs. We therefore sought to evaluate a novel educational tool to teach trainees how to identify interictal epileptiform discharges (IEDs) on EEG.

Methods

We created a public EEG test within the iOS app DiagnosUs using a pool of 13,262 candidate IEDs. Users were shown a candidate IED on EEG and asked to rate it as epileptiform (IED) or not (non-IED). They were given immediate feedback based on a gold standard. Learning was analyzed using a parametric model. We additionally analyzed IED features that best correlated with expert ratings.

Results

Our analysis included 901 participants. Users achieved a mean improvement of 13% over 1,000 questions and an ending accuracy of 81%. Users and experts appeared to rely on a similar set of IED morphologic features when analyzing candidate IEDs. We additionally identified particular types of candidate EEGs that remained challenging for most users even after substantial practice.

Conclusions

Users improved in their ability to properly classify candidate IEDs through repeated exposure and immediate feedback.

Significance

This app-based learning activity has great potential to be an effective supplemental tool to teach neurology trainees how to accurately identify IEDs on EEG.

Moving the field forward: detection of epileptiform abnormalities on scalp electroencephalography using deep learning—clinical application perspectives

The application of deep learning approaches for the detection of interictal epileptiform discharges is a nascent field, with most studies published in the past 5 years. Although many recent models have been published demonstrating promising results, deficiencies in descriptions of data sets, unstandardized methods, variation in performance evaluation and lack of demonstrable generalizability have made it difficult for these algorithms to be compared and progress to clinical validity. A few recent publications have provided a detailed breakdown of data sets and relevant performance metrics to exemplify the potential of deep learning in epileptiform discharge detection. This review provides an overview of the field and equips computer and data scientists with a synopsis of EEG data sets, background and epileptiform variation, model evaluation parameters and an awareness of the performance metrics of high impact and interest to the trained clinical and neuroscientist EEG end user. The gold standard and inter-rater disagreements in defining epileptiform abnormalities remain a challenge in the field, and a hierarchical proposal for epileptiform discharge labelling options is recommended. Standardized descriptions of data sets and reporting metrics are a priority. Source code-sharing and accessibility to public EEG data sets will increase the rigour, quality and progress in the field and allow validation and real-world clinical translation.

Improved access to rapid electroencephalography at a community hospital reduces inter-hospital transfers for suspected non-convulsive seizures

OBJECTIVE

Patients with suspected non-convulsive seizures are optimally evaluated with EEG. However, limited EEG infrastructure at community hospitals often necessitates transfer for long-term EEG monitoring (LTM). Novel point-of-care EEG systems could expedite management of nonconvulsive seizures and reduce unnecessary transfers. We aimed to describe the impact of rapid access to EEG using a novel EEG device with remote expert interpretation (tele-EEG) on rates of transfer for LTM.

METHODS

We retrospectively identified a cohort of patients who underwent Rapid-EEG (Ceribell Inc., Mountain View, CA) monitoring as part of a new standard-of-care at a community hospital. Rapid-EEGs were initially reviewed on-site by a community hospital neurologist before transitioning to tele-EEG review by epileptologists at an affiliated academic hospital. We compared the rate of transfer for LTM after Rapid-EEG/tele-EEG implementation to the expected rate if rapid access to EEG was unavailable.

RESULTS

Seventy-four patients underwent a total of 118 Rapid-EEG studies (10 with seizure, 18 with highly epileptiform patterns, 90 with slow/normal activity). Eighty-one studies (69%), including 9 of 10 studies that detected seizures, occurred after-hours when EEG was previously unavailable. Based on historical practice patterns, we estimated that Rapid-EEG potentially obviated transfer for LTM in 31 of 33 patients (94%); both completed transfers occurred before the transition to tele-EEG review.

SIGNIFICANCE

Rapid access to EEG led to the detection of seizures that would otherwise have been missed and reduced inter-hospital transfers for LTM. We estimate that the reduction in inter-hospital transportation costs alone would be in excess of $39,000 ($1,274 per patient). Point-of-care EEG systems may support a hub-and-spoke model for managing non-convulsive seizures (similar to that utilized in this study and analogous to existing acute stroke infrastructures), with increased EEG capacity at community hospitals and tele-EEG interpretation by specialists at academic hospitals that can accept transfers for LTM.

Measuring expertise in identifying interictal epileptiform discharges

OBJECTIVE

Interictal epileptiform discharges on EEG are integral to diagnosing epilepsy. However, EEGs are interpreted by readers with and without specialty training, and there is no accepted method to assess skill in interpretation. We aimed to develop a test to quantify IED recognition skills.

METHODS

A total of 13,262 candidate IEDs were selected from EEGs and scored by eight fellowship-trained reviewers to establish a gold standard. An online test was developed to assess how well readers with different training levels could distinguish candidate waveforms. Sensitivity, false positive rate and calibration were calculated for each reader. A simple mathematical model was developed to estimate each reader's skill and threshold in identifying an IED, and to develop receiver operating characteristics curves for each reader. We investigated the number of IEDs needed to measure skill level with acceptable precision.

RESULTS

Twenty-nine raters completed the test; nine experts, seven experienced non-experts and thirteen novices. Median calibration errors for experts, experienced non-experts and novices were -0.056, 0.012, 0.046; median sensitivities were 0.800, 0.811, 0.715; and median false positive rates were 0.177, 0.272, 0.396, respectively. The number of test questions needed to measure those scores was 549. Our analysis identified that novices had a higher noise level (uncertainty) compared to experienced non-experts and experts. Using calculated noise and threshold levels, receiver operating curves were created, showing increasing median area under the curve from novices (0.735), to experienced non-experts (0.852) and experts (0.891).

SIGNIFICANCE

Expert and non-expert readers can be distinguished based on ability to identify IEDs. This type of assessment could also be used to identify and correct differences in thresholds in identifying IEDs.

Automated spike detection: Which software package?

PURPOSE

We assessed three commercial automated spike detection software packages (Persyst, Encevis and BESA) to see which had the best performance.

METHODS

Thirty prolonged EEG records from people aged at least 16 years were collected and 30-minute representative epochs were selected. Interictal epileptiform discharges (IEDs) were marked by three human experts and by all three software packages. For each 30-minutes selection and for each 10-second epoch we measured whether or not IEDs had occurred. We defined the gold standard as the combined detections of the experts. Kappa scores, sensitivity and specificity were estimated for each software package.

RESULTS

Sensitivity for Persyst in the default setting was 95% for 30-minute selections and 82% for 10-second epochs. Sensitivity for Encevis was 86% (30-minute selections) and 61% (10-second epochs). The specificity for both packages was 88% for 30-minute selections and 96%-99% for the 10-second epochs. Interrater agreement between Persyst and Encevis and the experts was similar than between experts (0.67-0.83 versus 0.63-0.67). Sensitivity for BESA was 40% and specificity 100%. Interrater agreement (0.25) was low.

CONCLUSIONS

IED detection by the Persyst automated software is better than the Encevis and BESA packages, and similar to human review, when reviewing 30-minute selections and 10-second epochs. This findings may help prospective users choose a software package.

[Automatic detection of epileptiform potentials and seizures in the EEG]

Automatic computer-based algorithms for the detection of epileptiform potentials and seizure patterns on EEG facilitate a time-saving, objective method of quantitative EEG interpretation which is available 7/24. For the automatic detection of interictal epileptiform potentials sensitivities range from 65 to 99% with false positive detections of 0,09 to 13,4 per minute. Recent studies documented equal or even better performance of automatic spike detection programs compared with experienced human EEG readers. The seizure detection problem-one of the major problems in clinical epileptology-consists of the fact that the majority of focal onset seizures with impaired awareness and of seizures arising out of sleep occur unnoticed by patients and their caregivers. Automatic seizure detection systems could facilitate objective seizure documentation and thus help to solve the seizure detection problem. Furthermore, seizure detection systems may help to prevent seizure-related injuries and sudden unexpected death in epilepsy (SUDEP), and could be an integral part of novel, seizure-triggered on-demand therapies in epilepsy. During long-term video-EEG monitoring seizure detection systems could improve patient safety, provide a time-saving objective and reproducible analysis of seizure patterns and facilitate automatic computer-based patient testing during seizures. Sensitivities of seizure detection systems range from 75 to 90% with extratemporal seizures being more difficult to detect than temporal seizures. The false positive alarm rate ranges from 0,1 to 5 per 24 hours. Finally, machine learning algorithms, especially deep learning approaches, open a new highly promising era in automatic spike and seizure detection.

Computer-assisted EEG diagnostic review for idiopathic generalized epilepsy

Epilepsy diagnosis can be costly, time-consuming, and not uncommonly inaccurate. The reference standard diagnostic monitoring is continuous video-electroencephalography (EEG) monitoring, ideally capturing all events or concordant interictal discharges. Automating EEG data review would save time and resources, thus enabling more people to receive reference standard monitoring and also potentially heralding a more quantitative approach to therapeutic outcomes. There is substantial research into the automated detection of seizures and epileptic activity from EEG. However, automated detection software is not widely used in the clinic, and despite numerous published algorithms, few methods have regulatory approval for detecting epileptic activity from EEG. This study reports on a deep learning algorithm for computer-assisted EEG review. Deep convolutional neural networks were trained to detect epileptic discharges using a preexisting dataset of over 6000 labelled events in a cohort of 103 patients with idiopathic generalized epilepsy (IGE). Patients underwent 24-hour ambulatory outpatient EEG, and all data were curated and confirmed independently by two epilepsy specialists (Seneviratne et al., 2016). The resulting automated detection algorithm was then used to review diagnostic scalp EEG for seven patients (four with IGE and three with events mimicking seizures) to validate performance in a clinical setting. The automated detection algorithm showed state-of-the-art performance for detecting epileptic activity from clinical EEG, with mean sensitivity of >95% and corresponding mean false positive rate of 1 detection per minute. Importantly, diagnostic case studies showed that the automated detection algorithm reduced human review time by 80%-99%, without compromising event detection or diagnostic accuracy. The presented results demonstrate that computer-assisted review can increase the speed and accuracy of EEG assessment and has the potential to greatly improve therapeutic outcomes. This article is part of the Special Issue "NEWroscience 2018".

Deep learning for robust detection of interictal epileptiform discharges

OBJECTIVE

Automatic detection of interictal epileptiform discharges (IEDs, short as ``spikes'') from an epileptic brain can help predict seizure recurrence and support the diagnosis of epilepsy. Developing fast, reliable and robust detection methods for IEDs based on scalp or intracortical EEG may facilitate online seizure monitoring and closed-loop neurostimulation.

APPROACH

We developed a new deep learning approach, which employs a long short-term memory (LSTM) network architecture (``IEDnet'') and an auxiliary classifier generative adversarial network (AC-GAN), to train on both expert-annotated and augmented spike events from intracranial electroencephalography (iEEG) recordings of epilepsy patients. We validated our IEDnet with two real-world iEEG datasets, and compared IEDnet with the support vector machine (SVM) and random forest (RF) classifiers on their detection performances.

MAIN RESULTS

IEDnet achieved excellent cross-validated detection performances in terms of both sensitivity and specificity, and outperformed SVM and RF. Synthetic spike samples augmented by AC-GAN further improved the detection performance. In addition, the performance of IEDnet was robust with respect to the sampling frequency and noise. Furthermore, we also demonstrated the cross-institutional generalization ability of IEDnet while testing between two datasets.

SIGNIFICANCE

IEDnet achieves excellent detection performances in identifying interictal spikes. AC-GAN can produce augmented iEEG samples to improve supervised deep learning.

Incorporating Uncertainty in Data Labeling into Automatic Detection of Interictal Epileptiform Discharges from Concurrent Scalp-EEG via Multi-way Analysis

Interictal epileptiform discharges (IEDs) are elicited from an epileptic brain, whereas they can also be due to other neurological abnormalities. The diversity in their morphologies, their strengths, and their sources within the brain cause a great deal of uncertainty in their labeling by clinicians. The aim of this study is therefore to exploit and incorporate this uncertainty (the probability of the waveform being an IED) in the IED detection system which combines spatial component analysis (SCA) with the IED probabilities referred to as SCA-IEDP-based method. For comparison, we also propose and study SCA-based method in which probability of the waveform being an IED is ignored. The proposed models are employed to detect IEDs in two different classification approaches: (1) subject-dependent and (2) subject-independent classification approaches. The proposed methods are compared with two other state-of-the-art methods namely, time-frequency features and tensor factorization methods. The proposed SCA-IEDP model has achieved superior performance in comparison with the traditional SCA and other competing methods. It achieved 79.9% and 63.4% accuracy values in subject-dependent and subject-independent classification approaches, respectively. This shows that considering the IED probabilities in designing an IED detection system can boost its performance.

Interrater Reliability of Experts in Identifying Interictal Epileptiform Discharges in Electroencephalograms

Importance

The validity of using electroencephalograms (EEGs) to diagnose epilepsy requires reliable detection of interictal epileptiform discharges (IEDs). Prior interrater reliability (IRR) studies are limited by small samples and selection bias.

Objective

To assess the reliability of experts in detecting IEDs in routine EEGs.

Design, Setting, and Participants

This prospective analysis conducted in 2 phases included as participants physicians with at least 1 year of subspecialty training in clinical neurophysiology. In phase 1, 9 experts independently identified candidate IEDs in 991 EEGs (1 expert per EEG) reported in the medical record to contain at least 1 IED, yielding 87 636 candidate IEDs. In phase 2, the candidate IEDs were clustered into groups with distinct morphological features, yielding 12 602 clusters, and a representative candidate IED was selected from each cluster. We added 660 waveforms (11 random samples each from 60 randomly selected EEGs reported as being free of IEDs) as negative controls. Eight experts independently scored all 13 262 candidates as IEDs or non-IEDs. The 1051 EEGs in the study were recorded at the Massachusetts General Hospital between 2012 and 2016.

Main Outcomes and Measures

Primary outcome measures were percentage of agreement (PA) and beyond-chance agreement (Gwet κ) for individual IEDs (IED-wise IRR) and for whether an EEG contained any IEDs (EEG-wise IRR). Secondary outcomes were the correlations between numbers of IEDs marked by experts across cases, calibration of expert scoring to group consensus, and receiver operating characteristic analysis of how well multivariate logistic regression models may account for differences in the IED scoring behavior between experts.

Results

Among the 1051 EEGs assessed in the study, 540 (51.4%) were those of females and 511 (48.6%) were those of males. In phase 1, 9 experts each marked potential IEDs in a median of 65 (interquartile range [IQR], 28-332) EEGs. The total number of IED candidates marked was 87 636. Expert IRR for the 13 262 individually annotated IED candidates was fair, with the mean PA being 72.4% (95% CI, 67.0%-77.8%) and mean κ being 48.7% (95% CI, 37.3%-60.1%). The EEG-wise IRR was substantial, with the mean PA being 80.9% (95% CI, 76.2%-85.7%) and mean κ being 69.4% (95% CI, 60.3%-78.5%). A statistical model based on waveform morphological features, when provided with individualized thresholds, explained the median binary scores of all experts with a high degree of accuracy of 80% (range, 73%-88%).

Conclusions and Relevance

This study's findings suggest that experts can identify whether EEGs contain IEDs with substantial reliability. Lower reliability regarding individual IEDs may be largely explained by various experts applying different thresholds to a common underlying statistical model.

Automatische Erkennung von epilepsietypischen Potenzialen und Anfällen im EEG

Die Elektroenzephalografie (EEG) ist der wichtigste apparative Eckpfeiler in der Diagnostik und Therapieführung bei Epilepsien. Die visuelle EEG-Befundung stellt dabei nach wie vor den Goldstandard dar. Automatische computerunterstützte Methoden zur Detektion und Quantifizierung von interiktalen epilepsietypischen Potenzialen und Anfällen unterstützen eine zeitsparende, objektive, rasch und jederzeit verfügbare quantitative EEG-Befundung.

Persistent abnormalities in Rolandic thalamocortical white matter circuits in childhood epilepsy with centrotemporal spikes

OBJECTIVE

Childhood epilepsy with centrotemporal spikes (CECTS) is a common, focal, transient, developmental epilepsy syndrome characterized by unilateral or bilateral, independent epileptiform spikes in the Rolandic regions of unknown etiology. Given that CECTS presents during a period of dramatic white matter maturation and thatspikes in CECTS are activated during non-rapid eye movement (REM) sleep, we hypothesized that children with CECTS would have aberrant development of white matter connectivity between the thalamus and the Rolandic cortex. We further tested whether Rolandic thalamocortical structural connectivity correlates with spike rate during non-REM sleep.

METHODS

Twenty-three children with CECTS (age = 8-15 years) and 19 controls (age = 7-15 years) underwent 3-T structural and diffusion-weighted magnetic resonance imaging and 72-electrode electroencephalographic recordings. Thalamocortical structural connectivity to Rolandic and non-Rolandic cortices was quantified using probabilistic tractography. Developmental changes in connectivity were compared between groups using bootstrap analyses. Longitudinal analysis was performed in four subjects with 1-year follow-up data. Spike rate was quantified during non-REM sleep using manual and automated techniques and compared to Rolandic connectivity using regression analyses.

RESULTS

Children with CECTS had aberrant development of thalamocortical connectivity to the Rolandic cortex compared to controls (P = .01), where the expected increase in connectivity with age was not observed in CECTS. There was no difference in the development of thalamocortical connectivity to non-Rolandic regions between CECTS subjects and controls (P = .19). Subjects with CECTS observed longitudinally had reductions in thalamocortical connectivity to the Rolandic cortex over time. No definite relationship was found between Rolandic connectivity and non-REM spike rate (P > .05).

SIGNIFICANCE

These data provide evidence that abnormal maturation of thalamocortical white matter circuits to the Rolandic cortex is a feature of CECTS. Our data further suggest that the abnormalities in these tracts do not recover, but are increasingly dysmature over time, implicating a permanent but potentially compensatory process contributing to disease resolution.

Automated Detection of Interictal Epileptiform Discharges from Scalp Electroencephalograms by Convolutional Neural Networks

Visual evaluation of electroencephalogram (EEG) for Interictal Epileptiform Discharges (IEDs) as distinctive biomarkers of epilepsy has various limitations, including time-consuming reviews, steep learning curves, interobserver variability, and the need for specialized experts. The development of an automated IED detector is necessary to provide a faster and reliable diagnosis of epilepsy. In this paper, we propose an automated IED detector based on Convolutional Neural Networks (CNNs). We have evaluated the proposed IED detector on a sizable database of 554 scalp EEG recordings (84 epileptic patients and 461 nonepileptic subjects) recorded at Massachusetts General Hospital (MGH), Boston. The proposed CNN IED detector has achieved superior performance in comparison with conventional methods with a mean cross-validation area under the precision-recall curve (AUPRC) of 0.838[Formula: see text]±[Formula: see text]0.040 and false detection rate of 0.2[Formula: see text]±[Formula: see text]0.11 per minute for a sensitivity of 80%. We demonstrated the proposed system to be noninferior to 30 neurologists on a dataset from the Medical University of South Carolina (MUSC). Further, we clinically validated the system at National University Hospital (NUH), Singapore, with an agreement accuracy of 81.41% with a clinical expert. Moreover, the proposed system can be applied to EEG recordings with any arbitrary number of channels.

Using sampled visual EEG review in combination with automated detection software at the EMU

PURPOSE

Complete visual review of prolonged video-EEG recordings at an EMU (Epilepsy Monitoring Unit) is time consuming and can cause problems in times of paucity of educated personnel. In this study we aimed to show non inferiority for electroclinical diagnosis using sampled review in combination with EEG analysis softreferware (P13 software, Persyst Corporation), in comparison to complete visual review.

METHOD

Fifty prolonged video-EEG recordings in adults were prospectively evaluated using sampled visual EEG review in combination with automated detection software of the complete EEG record. Visually assessed samples consisted of one hour during wakefulness, one hour during sleep, half an hour of wakefulness after wake-up and all clinical events marked by the individual and/or nurses. The final electro-clinical diagnosis of this new review approach was compared with the electro-clinical diagnosis after complete visual review as presently used.

RESULTS

The electro-clinical diagnosis based on sampled visual review combined with automated detection software did not differ from the diagnosis based on complete visual review. Furthermore, the detection software was able to detect all records containing epileptiform abnormalities and epileptic seizures.

CONCLUSION

Sampled visual review in combination with automated detection using Persyst 13 is non-inferior to complete visual review for electroclinical diagnosis of prolonged video-EEG at an EMU setting, which makes this approach promising.

Most Experts Agree … But What About Other EEG Readers?

[Box: see text]