External validation of a prognostic model for seizure recurrence following a first unprovoked seizure and implications for driving

Predicting seizure recurrence after an initial seizure-like episode from routine clinical notes using large language models: a retrospective cohort study

BACKGROUND

The evaluation and management of first-time seizure-like events in children can be difficult because these episodes are not always directly observed and might be epileptic seizures or other conditions (seizure mimics). We aimed to evaluate whether machine learning models using real-world data could predict seizure recurrence after an initial seizure-like event.

METHODS

This retrospective cohort study compared models trained and evaluated on two separate datasets between Jan 1, 2010, and Jan 1, 2020: electronic medical records (EMRs) at Boston Children's Hospital and de-identified, patient-level, administrative claims data from the IBM MarketScan research database. The study population comprised patients with an initial diagnosis of either epilepsy or convulsions before the age of 21 years, based on International Classification of Diseases, Clinical Modification (ICD-CM) codes. We compared machine learning-based predictive modelling using structured data (logistic regression and XGBoost) with emerging techniques in natural language processing by use of large language models.

FINDINGS

The primary cohort comprised 14 021 patients at Boston Children's Hospital matching inclusion criteria with an initial seizure-like event and the comparison cohort comprised 15 062 patients within the IBM MarketScan research database. Seizure recurrence based on a composite expert-derived definition occurred in 57% of patients at Boston Children's Hospital and 63% of patients within IBM MarketScan. Large language models with additional domain-specific and location-specific pre-training on patients excluded from the study (F1-score 0·826 [95% CI 0·817-0·835], AUC 0·897 [95% CI 0·875-0·913]) performed best. All large language models, including the base model without additional pre-training (F1-score 0·739 [95% CI 0·738-0·741], AUROC 0·846 [95% CI 0·826-0·861]) outperformed models trained with structured data. With structured data only, XGBoost outperformed logistic regression and XGBoost models trained with the Boston Children's Hospital EMR (logistic regression: F1-score 0·650 [95% CI 0·643-0·657], AUC 0·694 [95% CI 0·685-0·705], XGBoost: F1-score 0·679 [0·676-0·683], AUC 0·725 [0·717-0·734]) performed similarly to models trained on the IBM MarketScan database (logistic regression: F1-score 0·596 [0·590-0·601], AUC 0·670 [0·664-0·675], XGBoost: F1-score 0·678 [0·668-0·687], AUC 0·710 [0·703-0·714]).

INTERPRETATION

Physician's clinical notes about an initial seizure-like event include substantial signals for prediction of seizure recurrence, and additional domain-specific and location-specific pre-training can significantly improve the performance of clinical large language models, even for specialised cohorts.

FUNDING

UCB, National Institute of Neurological Disorders and Stroke (US National Institutes of Health).

Prognosis of adults and children following a first unprovoked seizure

BACKGROUND

Epilepsy is clinically defined as two or more unprovoked epileptic seizures more than 24 hours apart. Given that, a diagnosis of epilepsy can be associated with significant morbidity and mortality, it is imperative that clinicians (and people with seizures and their relatives) have access to accurate and reliable prognostic estimates, to guide clinical practice on the risks of developing further unprovoked seizures (and by definition, a diagnosis of epilepsy) following single unprovoked epileptic seizure.

OBJECTIVES

1. To provide an accurate estimate of the proportion of individuals going on to have further unprovoked seizures at subsequent time points following a single unprovoked epileptic seizure (or cluster of epileptic seizures within a 24-hour period, or a first episode of status epilepticus), of any seizure type (overall prognosis). 2. To evaluate the mortality rate following a first unprovoked epileptic seizure.

SEARCH METHODS

We searched the following databases on 19 September 2019 and again on 30 March 2021, with no language restrictions. The Cochrane Register of Studies (CRS Web), MEDLINE Ovid (1946 to March 29, 2021), SCOPUS (1823 onwards), ClinicalTrials.gov, the World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP). CRS Web includes randomized or quasi-randomized, controlled trials from PubMed, Embase, ClinicalTrials.gov, the World Health Organization International Clinical Trials Registry Platform (ICTRP), the Cochrane Central Register of Controlled Trials (CENTRAL), and the Specialized Registers of Cochrane Review Groups including Epilepsy. In MEDLINE (Ovid) the coverage end date always lags a few days behind the search date.

SELECTION CRITERIA

We included studies, both retrospective and prospective, of all age groups (except those in the neonatal period (< 1 month of age)), of people with a single unprovoked seizure, followed up for a minimum of six months, with no upper limit of follow-up, with the study end point being seizure recurrence, death, or loss to follow-up. To be included, studies must have included at least 30 participants. We excluded studies that involved people with seizures that occur as a result of an acute precipitant or provoking factor, or in close temporal proximity to an acute neurological insult, since these are not considered epileptic in aetiology (acute symptomatic seizures). We also excluded people with situational seizures, such as febrile convulsions.

DATA COLLECTION AND ANALYSIS

Two review authors conducted the initial screening of titles and abstracts identified through the electronic searches, and removed non-relevant articles. We obtained the full-text articles of all remaining potentially relevant studies, or those whose relevance could not be determined from the abstract alone and two authors independently assessed for eligibility. All disagreements were resolved through discussion with no need to defer to a third review author. We extracted data from included studies using a data extraction form based on the checklist for critical appraisal and data extraction for systematicreviews of prediction modelling studies (CHARMS). Two review authors then appraised the included studies, using a standardised approach based on the quality in prognostic studies (QUIPS) tool, which was adapted for overall prognosis (seizure recurrence). We conducted a meta-analysis using Review Manager 2014, with a random-effects generic inverse variance meta-analysis model, which accounted for any between-study heterogeneity in the prognostic effect. We then summarised the meta-analysis by the pooled estimate (the average prognostic factor effect), its 95% confidence interval (CI), the estimates of I² and Tau² (heterogeneity), and a 95% prediction interval for the prognostic effect in a single population at three various time points, 6 months, 12 months and 24 months. Subgroup analysis was performed according to the ages of the cohorts included; studies involving all ages, studies that recruited adult only and those that were purely paediatric.

MAIN RESULTS

Fifty-eight studies (involving 54 cohorts), with a total of 12,160 participants (median 147, range 31 to 1443), met the inclusion criteria for the review. Of the 58 studies, 26 studies were paediatric studies, 16 were adult and the remaining 16 studies were a combination of paediatric and adult populations. Most included studies had a cohort study design with two case-control studies and one nested case-control study. Thirty-two studies (29 cohorts) reported a prospective longitudinal design whilst 15 studies had a retrospective design whilst the remaining studies were randomised controlled trials. Nine of the studies included presented mortality data following a first unprovoked seizure. For a mortality study to be included, a proportional mortality ratio (PMR) or a standardised mortality ratio (SMR) had to be given at a specific time point following a first unprovoked seizure. To be included in the meta-analysis a study had to present clear seizure recurrence data at 6 months, 12 months or 24 months. Forty-six studies were included in the meta-analysis, of which 23 were paediatric, 13 were adult, and 10 were a combination of paediatric and adult populations. A meta-analysis was performed at three time points; six months, one year and two years for all ages combined, paediatric and adult studies, respectively. We found an estimated overall seizure recurrence of all included studies at six months of 27% (95% CI 24% to 31%), 36% (95% CI 33% to 40%) at one year and 43% (95% CI 37% to 44%) at two years, with slightly lower estimates for adult subgroup analysis and slightly higher estimates for paediatric subgroup analysis. It was not possible to provide a summary estimate of the risk of seizure recurrence beyond these time points as most of the included studies were of short follow-up and too few studies presented recurrence rates at a single time point beyond two years. The evidence presented was found to be of moderate certainty.

AUTHORS' CONCLUSIONS

Despite the limitations of the data (moderate-certainty of evidence), mainly relating to clinical and methodological heterogeneity we have provided summary estimates for the likely risk of seizure recurrence at six months, one year and two years for both children and adults. This provides information that is likely to be useful for the clinician counselling patients (or their parents) on the probable risk of further seizures in the short-term whilst acknowledging the paucity of long-term recurrence data, particularly beyond 10 years.

Identification and Characterization of Pure Sleep Epilepsy in a Cohort of Patients With a First Seizure

OBJECTIVES

To study the development of pure sleep epilepsy (PSE) after a first-ever seizure from sleep in adults.

METHODS

Prospective observational study of patients seen at a tertiary hospital-based first seizure clinic between 2000 and 2011. Adults with a first-ever unprovoked seizure from sleep were consecutively recruited. All patients were followed up at least once after the initial seizure and those not requiring regular clinical review were contacted every one to two years. The timing and pattern of subsequent seizures as well as potential predictors of future awake seizures were analyzed.

RESULTS

239 adults with a first-ever unprovoked seizure from sleep were identified, 61% male; mean age 43 years (range 14-88 years) and median follow-up of 8.8 years (range 2 months -18 years). Of the 174 patients who had recurrent seizures, 130 patients (75%) had their second seizure from sleep, and of these, 76 of 94 (81%) also had their third seizure from sleep. 89 patients (37%) developed awake seizures during follow up. In half of these patients, the awake seizure occurred within two years of the initial seizure. The risk of an awake seizure within 1 year of a first-ever seizure from sleep was 13.9% (95%CI 9.4-18.3), falling to 2.0-5.3% per year after 3 years. The risks of an awake seizure within 1 year of a second or third consecutive sleep seizure were 9.9% (95%CI 4.6-15.3) and 8.7% (95%CI 2.0-15.4) respectively, and similarly decreased with time.

CONCLUSION

Most initial seizure recurrences after a first-ever sleep seizure occur during sleep. Whilst over one third eventually had awake seizures, the annual risk of an awake seizure was ≤14% and decreased with time, albeit with a small ongoing risk of between 2 and 5% per year. These findings may be utilized in counselling patients with seizures from sleep and inform driving recommendations.

Risk of seizure recurrence in people with single seizures and early epilepsy - Model development and external validation

•

Model predicts risk of seizure recurrence after single fit or epilepsy diagnosis.

•

Model performs well in independent data.

•

Future work required to ensure the model is adopted in clinical practice.

•

Model can improve the lives of people with single seizures and early epilepsy.

Purpose

Following a single seizure, or recent epilepsy diagnosis, it is difficult to balance risk of medication side effects with the potential to prevent seizure recurrence. A prediction model was developed and validated enabling risk stratification which in turn informs treatment decisions and individualises counselling.

Methods

Data from a randomised controlled trial was used to develop a prediction model for risk of seizure recurrence following a first seizure or diagnosis of epilepsy. Time-to-event data was modelled via Cox's proportional hazards regression. Model validity was assessed via discrimination and calibration using the original dataset and also using three external datasets – National General Practice Survey of Epilepsy (NGPSE), Western Australian first seizure database (WA) and FIRST (Italian dataset of people with first tonic-clonic seizures).

Results

People with neurological deficit, focal seizures, abnormal EEG, not indicated for CT/MRI scan, or not immediately treated have a significantly higher risk of seizure recurrence. Discrimination was fair and consistent across the datasets (c-statistics: 0.555 (NGPSE); 0.558 (WA); 0.597 (FIRST)). Calibration plots showed good agreement between observed and predicted probabilities in NGPSE at one and three years. Plots for WA and FIRST showed poorer agreement with the model underpredicting risk in WA, and over-predicting in FIRST. This was resolved following model recalibration.

Conclusion

The model performs well in independent data especially when recalibrated. It should now be used in clinical practice as it can improve the lives of people with single seizures and early epilepsy by enabling targeted treatment choices and more informed patient counselling.

Driving eligibility: Implications of studies on seizure recurrence risk

BACKGROUND

Driving is one of the most important issues for patients with seizures. The 2009 European directive provides a framework for evaluating standard situations in assessing the ability to drive. Such a framework may not be sufficient for individual scenarios.

AIMS OF THE STUDY

To analyse current data on seizure recurrence risks (RcRs) focusing on their potential implications for car driving issues (group 1).

METHODS

We evaluated current studies and meta-analyses on RcR.

RESULTS

A meta-analysis of seizure-free patients who withdrew their medication (Lamberink et al Lancet Neurology 2017;16:523) created a nomogram and a web-based tool that allow estimating RcR in individual patients and thus to identify those in whom medication withdrawal is possible without the common driving ban during withdrawal. The 2-year prediction model of that meta-analysis has been recently externally tested and confirmed. A meta-analysis of patients with a first unprovoked seizure (Bonnett et al PloS ONE 2014;9:e99063) determined to which extent RcRs depend on established risk factors. The seizure-free period required to restart driving could be tailored according to the individual RcR.

CONCLUSION

These current studies allow estimating individual RcR more precisely and thus modifying periods of driving bans beyond the existing guidelines.

Quantitative analysis of phenotypic elements augments traditional electroclinical classification of common familial epilepsies

OBJECTIVE

Classification of epilepsy into types and subtypes is important for both clinical care and research into underlying disease mechanisms. A quantitative, data-driven approach may augment traditional electroclinical classification and shed new light on existing classification frameworks.

METHODS

We used latent class analysis, a statistical method that assigns subjects into groups called latent classes based on phenotypic elements, to classify individuals with common familial epilepsies from the Epi4K Multiplex Families study. Phenotypic elements included seizure types, seizure symptoms, and other elements of the medical history. We compared class assignments to traditional electroclinical classifications and assessed familial aggregation of latent classes.

RESULTS

A total of 1120 subjects with epilepsy were assigned to five latent classes. Classes 1 and 2 contained subjects with generalized epilepsy, largely reflecting the distinction between absence epilepsies and younger onset (class 1) versus myoclonic epilepsies and older onset (class 2). Classes 3 and 4 contained subjects with focal epilepsies, and in contrast to classes 1 and 2, these did not adhere as closely to clinically defined focal epilepsy subtypes. Class 5 contained nearly all subjects with febrile seizures plus or unknown epilepsy type, as well as a few subjects with generalized epilepsy and a few with focal epilepsy. Family concordance of latent classes was similar to or greater than concordance of clinically defined epilepsy types.

SIGNIFICANCE

Quantitative classification of epilepsy has the potential to augment traditional electroclinical classification by (1) combining some syndromes into a single class, (2) splitting some syndromes into different classes, (3) helping to classify subjects who could not be classified clinically, and (4) defining the boundaries of clinically defined classifications. This approach can guide future research, including molecular genetic studies, by identifying homogeneous sets of individuals that may share underlying disease mechanisms.

"Chance Takers Are Accident Makers": Are Patients With Epilepsy Really Taking a Chance When They Drive?

This review compiles scientific data about the real dangers faced by people with epilepsy (PWE) who drive. Those include risks of motor vehicle accidents (MVA) in PWE as compared with controls (individuals without epilepsy) and as compared with persons with other medical conditions that impact fitness to drive. Data regarding Accident rates as related to seizure free intervals (SFI), single vs. multiple seizure events, and/or antiseizure drug (ASD) taper and reintroduction are discussed. Variation in state, national, and international laws and guidance for non-commercial and commercial drivers is highlighted, along with some related reasons for driving restrictions. The review concludes by emphasizing the importance of physicians educating patients about local driving laws and about risks of ASD non-adherence. The need for a broader, multi-stakeholder re-examination of driving regulations for PWE is noted.

Identifying patients who will not reachieve remission after breakthrough seizures

Objective

We aim to identify people with epilepsy who are unlikely to reachieve a 12‐month remission within 2 years after experiencing a breakthrough seizure following an initial 12‐month remission.

Methods

We apply a novel longitudinal discriminant approach to data from the Standard and New Antiepileptic Drugs study to dynamically predict the risk of a patient not achieving a second remission after a breakthrough seizure by combining both baseline covariates (collected at the time of breakthrough seizure) and follow‐up data.

Results

The model classifies 83% of patients. Of these, 73% of patients (95% confidence interval [CI] = 58%‐88%) who did not achieve a second remission were correctly identified (sensitivity), and 84% of patients (95% CI = 69%‐96%) who achieved a second remission were correctly identified (specificity). The area under the curve from our model was 87% (95% CI = 80%‐94%). Patients who did not achieve a second remission were correctly identified on average after 10 months of observation postbreakthrough. Occurrence of seizures after breakthrough and the number of seizures experienced were the most informative longitudinal variables. These longitudinal profiles were influenced by the following baseline covariates: age at breakthrough seizure, presence of neurological insult, and number of antiepileptic drugs required to achieve first remission.

Significance

Using longitudinal data gathered during patient follow‐up allows more accurate predictions than using baseline covariates in a standard Cox model. The model developed in this paper is a useful first step in developing a tool for identifying patients who develop drug resistance after an initial remission.

Identification of patients who will not achieve seizure remission within 5 years on AEDs

OBJECTIVE

To identify people with epilepsy who will not achieve a 12-month seizure remission within 5 years of starting treatment.

METHODS

The Standard and New Antiepileptic Drug (SANAD) study is the largest prospective study in patients with epilepsy to date. We applied a recently developed multivariable approach to the SANAD dataset that takes into account not only baseline covariates describing a patient's history before diagnosis but also follow-up data as predictor variables.

RESULTS

Changes in number of seizures and treatment history were the most informative time-dependent predictors and were associated with history of neurologic insult, epilepsy type, age at start of treatment, sex, and having a first-degree relative with epilepsy. Our model classified 95% of patients. Of those classified, 95% of patients observed not to achieve remission at 5 years were correctly classified (95% confidence interval [CI] 89.5%-100%), with 51% identified by 3 years and 90% within 4 years of follow-up. Ninety-seven percent (95% CI 93.3%-98.8%) of patients observed to achieve a remission within 5 years were correctly classified. Of those predicted not to achieve remission, 76% (95% CI 58.5%-88.2%) truly did not achieve remission (positive predictive value). The predictive model achieved similar accuracy levels via external validation in 2 independent United Kingdom-based datasets.

CONCLUSION

Our approach generates up-to-date predictions of the patient's risk of not achieving seizure remission whenever new clinical information becomes available that could influence patient counseling and management decisions.

Cross-sectional study of the hospital management of adult patients with a suspected seizure (EPIC2)

OBJECTIVE

To determine the clinical characteristics, management and outcomes of patients taken to hospital by emergency ambulance after a suspected seizure.

DESIGN

Quantitative cross-sectional retrospective study of a consecutive series of patients.

SETTING

An acute hospital trust in a large city in England.

PARTICIPANTS

In 2012-2013, the regions' ambulance service managed 605 481 emergency incidents, 74 141/605 481 originated from Sheffield (a large city in the region), 2121/74 141 (2.9%) were suspected seizures and 178/2121 occurred in May 2012. We undertook detailed analysis of the medical records of the 91/178 patients who were transported to the city's acute hospital. After undertaking a retrospective review of the medical records, the best available aetiological explanation for the seizures was determined.

RESULTS

The best available aetiological explanation for 74.7% (68/91) of the incidents was an epileptic seizure, 11.0% (10/91) were psychogenic non-epileptic seizures and 9.9% (9/91) were cardiogenic events. The epileptic seizures fall into the following four categories: first epileptic seizure (13.2%, 12/91), epileptic seizure with a historical diagnosis of epilepsy (30.8%, 28/91), recurrent epileptic seizures without a historical diagnosis of epilepsy (20.9%, 19/91) and acute symptomatic seizures (9.9%, 9/91). Of those with seizures (excluding cardiogenic events), 2.4% (2/82) of patients were seizing on arrival in the Emergency Department (ED), 19.5% (16/82) were postictal and 69.5% (57/82) were alert. 63.4% (52/82) were discharged at the end of their ED attendance and 36.5% (19/52) of these had no referral or follow-up.

CONCLUSIONS

Most suspected seizures are epileptic seizures but this is a diagnostically heterogeneous group. Only a small minority of patients require emergency medical care but most are transported to hospital. Few patients receive expert review and many are discharged home without referral to a specialist leaving them at risk of further seizures and the associated morbidity, mortality and health services costs of poorly controlled epilepsy.