Changing the approach to treatment choice in epilepsy using big data

Can machine learning predict late seizures after intracerebral hemorrhages? Evidence from real-world data

INTRODUCTION

Intracerebral hemorrhage represents 15 % of all strokes and it is associated with a high risk of post-stroke epilepsy. However, there are no reliable methods to accurately predict those at higher risk for developing seizures despite their importance in planning treatments, allocating resources, and advancing post-stroke seizure research. Existing risk models have limitations and have not taken advantage of readily available real-world data and artificial intelligence. This study aims to evaluate the performance of Machine-learning-based models to predict post-stroke seizures at 1 year and 5 years after an intracerebral hemorrhage in unselected patients across multiple healthcare organizations.

DESIGN/METHODS

We identified patients with intracerebral hemorrhage (ICH) without a prior diagnosis of seizures from 2015 until inception (11/01/22) in the TriNetX Diamond Network, using the International Classification of Diseases, Tenth Revision (ICD-10) I61 (I61.0, I61.1, I61.2, I61.3, I61.4, I61.5, I61.6, I61.8, and I61.9). The outcome of interest was any ICD-10 diagnosis of seizures (G40/G41) at 1 year and 5 years following the first occurrence of the diagnosis of intracerebral hemorrhage. We applied a conventional logistic regression and a Light Gradient Boosted Machine (LGBM) algorithm, and the performance of the model was assessed using the area under the receiver operating characteristics (AUROC), the area under the precision-recall curve (AUPRC), the F1 statistic, model accuracy, balanced-accuracy, precision, and recall, with and without seizure medication use in the models.

RESULTS

A total of 85,679 patients had an ICD-10 code of intracerebral hemorrhage and no prior diagnosis of seizures, constituting our study cohort. Seizures were present in 4.57 % and 6.27 % of patients within 1 and 5 years after ICH, respectively. At 1-year, the AUROC, AUPRC, F1 statistic, accuracy, balanced-accuracy, precision, and recall were respectively 0.7051 (standard error: 0.0132), 0.1143 (0.0068), 0.1479 (0.0055), 0.6708 (0.0076), 0.6491 (0.0114), 0.0839 (0.0032), and 0.6253 (0.0216). Corresponding metrics at 5 years were 0.694 (0.009), 0.1431 (0.0039), 0.1859 (0.0064), 0.6603 (0.0059), 0.6408 (0.0119), 0.1094 (0.0037) and 0.6186 (0.0264). These numerical values indicate that the statistical models fit the data very well.

CONCLUSION

Machine learning models applied to electronic health records can improve the prediction of post-hemorrhagic stroke epilepsy, presenting a real opportunity to incorporate risk assessments into clinical decision-making in post-stroke care clinical care and improve patients' selection for post-stroke epilepsy research.

Comparing the efficacy of anti-seizure medications using matched cohorts on a large insurance claims database

Epilepsy is a severe chronic neurological disease affecting 60 million people worldwide. Primary treatment is with anti-seizure medicines (ASMs), but many patients continue to experience seizures. We used retrospective insurance claims data on 280,587 patients with uncontrolled epilepsy (UE), defined as status epilepticus, need for a rescue medicine, or admission or emergency visit for an epilepsy code. We conducted a computational risk ratio analysis between pairs of ASMs using a causal inference method, in order to match 1034 clinical factors and simulate randomization. Data was extracted from the MarketScan insurance claims Research Database records from 2011 to 2015. The cohort consisted of individuals over 18 years old with a diagnosis of epilepsy who took one of eight ASMs and had more than a year of history prior to the filling of the drug prescription. Seven ASM exposures were analyzed: topiramate, phenytoin, levetiracetam, gabapentin, lamotrigine, valproate, and carbamazepine or oxcarbazepine (treated as the same exposure). We calculated the risk ratio of UE between pairs of ASM after controlling for bias with inverse propensity weighting applied to 1034 factors, such as demographics, confounding illnesses, non-epileptic conditions treated by ASMs, etc. All ASMs exhibited a significant reduction in the prevalence of UE, but three drugs showed pair-wise differences compared to other ASMs. Topiramate consistently was associated with a lower risk of UE, with a mean risk ratio range of 0.68-0.93 (average 0.82, CI: 0.56-1.08). Phenytoin and levetiracetam were consistently associated with a higher risk of UE with mean risk ratio ranges of 1.11 to 1.47 (average 1.13, CI 0.98-1.65) and 1.15 to 1.43 (average 1.2, CI 0.72-1.69), respectively. Large-scale retrospective insurance claims data - combined with causal inference analysis - provides an opportunity to compare the effect of treatments in real-world data in populations 1,000-fold larger than those in typical randomized trials. Our causal analysis identified the clinically unexpected finding of topiramate as being associated with a lower risk of UE; and phenytoin and levetiracetam as associated with a higher risk of UE (compared to other studied drugs, not to baseline). However, we note that our data set for this study only used insurance claims events, which does not comprise actual seizure frequencies, nor a clear picture of side effects. Our results do not advocate for any change in practice but demonstrate that conclusions from large databases may differ from and supplement those of randomized trials and clinical practice and therefore may guide further investigation.

Retention Rates and Successful Treatment with Antiseizure Medications in Newly-Diagnosed Epilepsy Patients

PURPOSE

Treatment for epilepsy primarily involves antiseizure medications (ASMs), which can be characterized using the clinical data warehouse (CDW) database. In this study, we compared retention rates and time to successful treatment for various ASMs to reflect both efficacy and adverse effects in patients with newly diagnosed epilepsy.

MATERIALS AND METHODS

We identified newly diagnosed epilepsy patients with ASM treatment for more than 12 months using CDW of a tertiary referral hospital. Clinical characteristics were compared between groups with successful and unsuccessful treatment. Cox regression analysis was performed to evaluate independent variables of age, sex, comorbidities, and attributes of ASM regimens.

RESULTS

Of 2515 eligible participants, 46.2% were successfully treated with the first ASM regimen, and 74.7% with all ASM regimens with the median time-to-treatment success of 14 months. Participants with second-generation ASM as the first ASM were more likely to be successfully treated with the first regimen compared to those with first-generation ASM (51.6% vs. 42.3%, p<0.001) and more successfully treated [hazard ratio (HR)=1.26; 95% confidence interval (CI): 1.15-1.39]. Overall, valproic acid was the most common ASM across a wide range of ages under 65 years, while levetiracetam in patients aged over 65 years or lamotrigine in female adult patients. Clinical factors associated with less favorable treatment outcomes included renal disease (HR=0.78; 95% CI: 0.66-0.92), liver disease (HR=0.65; 95% CI: 0.52-0.81), depression (HR=0.70; 95% CI: 0.57-0.84), and mechanical ventilation (HR=0.58; 95% CI: 0.50-0.67).

CONCLUSION

Second-generation ASMs have the advantage of more successful treatment with fewer ASM regimen changes compared with first-generation drugs. Various comorbid conditions as well as age and sex should be considered when selecting ASMs.

Machine Learning to Predict the Need for Postmastectomy Radiotherapy after Immediate Breast Reconstruction

Background

Post mastectomy radiotherapy (PMRT) is an independent predictor of reconstructive complications. PMRT may alter the timing and type of reconstruction recommended. This study aimed to create a machine learning model to predict the probability of requiring PMRT after immediate breast reconstruction (IBR).

Methods

In this retrospective study, breast cancer patients who underwent IBR from January 2017 to December 2020 were reviewed and data were collected on 81 preoperative characteristics. Primary outcome was recommendation for PMRT. Four algorithms were compared to maximize performance and clinical utility: logistic regression, elastic net (EN), logistic lasso, and random forest (RF). The cohort was split into a development dataset (75% of cohort for training-validation) and 25% used for the test set. Model performance was evaluated using area under the receiver operating characteristic curve (AUC), precision-recall curves, and calibration plots.

Results

In a total of 800 patients, 325 (40.6%) patients were recommended to undergo PMRT. With the training-validation dataset (n = 600), model performance was logistic regression 0.73 AUC [95% confidence interval (CI) 0.65-0.80]; RF 0.77 AUC (95% CI, 0.74-0.81); EN 0.77 AUC (95% CI, 0.73-0.81); logistic lasso 0.76 AUC (95% CI, 0.72-0.80). Without significantly sacrificing performance, 81 predictive factors were reduced to 12 for prediction with the EN method. With the test dataset (n = 200), performance of the EN prediction model was confirmed [0.794 AUC (95% CI, 0.730-0.858)].

Conclusion

A parsimonious accurate machine learning model for predicting PMRT after IBR was developed, tested, and translated into a clinically applicable online calculator for providers and patients.

Role of machine learning in the management of epilepsy: a systematic review protocol

INTRODUCTION

Machine learning is a rapidly expanding field and is already incorporated into many aspects of medicine including diagnostics, prognostication and clinical decision-support tools. Epilepsy is a common and disabling neurological disorder, however, management remains challenging in many cases, despite expanding therapeutic options. We present a systematic review protocol to explore the role of machine learning in the management of epilepsy.

METHODS AND ANALYSIS

This protocol has been drafted with reference to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for Protocols. A literature search will be conducted in databases including MEDLINE, Embase, Scopus and Web of Science. A PRISMA flow chart will be constructed to summarise the study workflow. As the scope of this review is the clinical application of machine learning, the selection of papers will be focused on studies directly related to clinical decision-making in management of epilepsy, specifically the prediction of response to antiseizure medications, development of drug-resistant epilepsy, and epilepsy surgery and neuromodulation outcomes. Data will be extracted following the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies checklist. Prediction model Risk Of Bias ASsessment Tool will be used for the quality assessment of the included studies. Syntheses of quantitative data will be presented in narrative format.

ETHICS AND DISSEMINATION

As this study is a systematic review which does not involve patients or animals, ethics approval is not required. The results of the systematic review will be submitted to peer-review journals for publication and presented in academic conferences.

PROSPERO REGISTRATION NUMBER

CRD42023442156.

Early prediction of drug-resistant epilepsy using clinical and EEG features based on convolutional neural network

OBJECTIVE

Machine learning utilization in electroencephalogram (EEG) analysis and epilepsy care is fast evolving. Thus, we aim to develop and validate two one-dimensional convolutional neural network (CNN) algorithms for predicting drug-resistant epilepsy (DRE) in patients with newly-diagnosed epilepsy based on EEG and clinical features.

METHODS

We included a total of 1010 EEG signal epochs and 15 clinical features from 101 patients with epilepsy. Each patient had 10 epochs of EEG signal data, with each signal recorded for 90 s. The ratio of development set and validation set was 80:20, and ten-fold cross validation was performed. First, a CNN algorithm was used to extract EEG features automatically. Then, Two one-dimensional CNNs were crafted.. Accuracy, specificity, precision, sensitivity, F1-score, kappa statistics, mean square error (MSE) and area under the curve (AUC) were calculated to evaluate the classifiers performance.

RESULTS

The clinical-EEG model showed good performance and clinical practical value, with the accuracy, specificity, precision, sensitivity, F1-score, kappa statistics, best MSE and AUC in test set were 0.99, 0.72, 0.82, 0.96, 0.89, 0.83, 32.00, 0.81, respectively, and the accuracy in validation set was 0.84. In the EEG model, the accuracy, specificity, precision, sensitivity, F1-score, kappa statistics, best MSE and AUC in test set were 0.99, 0.59, 0.82, 0.90, 0.86, 0.72, 181.76, 0.76, respectively, and the accuracy in validation set was 0.81.

CONCLUSION

We constructed a clinical-EEG model showed good potential for predicting DRE in patients with newly-diagnosed epilepsy, which could help identify patients at high risk of developing DRE at earlier stages.

Real-world antiseizure medication treatment outcomes in drug-resistant focal epilepsy patients

OBJECTIVE

To gather real-world evidence on antiseizure medications (ASMs) treatment patterns and related outcomes in patients with drug-resistant focal epilepsy.

METHODS

Medical insurance claims from the start of 2014 till the end of 2019 were used. Patient selection criteria included International Classification of Diseases (ICD) codes followed by documented ASM use. Baseline patient demographics along with ASM and rescue medication use patterns and related patient outcome were documented for first (index) ASM regimen. Patients who failed the first regimen and then failed the second regimen were considered drug resistant. Multivariate analyses were performed to identify risks and other characteristics for positive or negative treatment outcomes.

RESULTS

Study cohort consisted of 46 474 patients with a mean age of 47.23 (SD: 16.94). Levetiracetam was the most first-encountered ASM (37.94%). At baseline, 87.14% were treated with ASMs prior to having study-confirmed diagnoses. Mental comorbidities were present in 37.86% of patients. After first-year ASM treatment, 34.61% of patients persisted on their index regimen and 5.91% were seizure-free. Patients failing first ASM regimen numbered 12 868 (27.69%). Drug-resistant patients who failed first and then second ASM regimens numbered 6335 (49.23%). Percentages of patients who had successful second treatment and seizure-free were 21.32 and 3.65, respectively. Initiating patients on lamotrigine or carbamazepine (relative to levetiracetam), baseline use of index ASM, rescue medications, and older age or male gender all lowered the risk for treatment failure. Having higher comorbidity, comorbid mental illness, headache, or neoplasty increased such a risk. Baseline use of index ASM, depressive episode, or anxiety disorder all entailed higher risk of failing second ASM treatment.

SIGNIFICANCE

Overall, reported findings indicated that patient history at baseline and the early selection of an ASM all influenced treatment outcomes. Findings pointed to the complex nature of ASM treatment in drug-resistant focal epilepsy patients calling for additional research to identify the optimal treatment to achieve beneficial patient outcomes.

Assessing treatment switch among patients with multiple sclerosis: A machine learning approach

BACKGROUND

Patients with multiple sclerosis (MS) frequently switch their Disease-Modifying Agents (DMA) for effectiveness and safety concerns. This study aimed to develop and compare the random forest (RF) machine learning (ML) model with the logistic regression (LR) model for predicting DMA switching among MS patients.

METHODS

This retrospective longitudinal study used the TriNetX data from a federated electronic medical records (EMR) network. Between September 2010 and May 2017, adults (aged ≥18) MS patients with ≥1 DMA prescription were identified, and the earliest DMA date was assigned as the index date. Patients prescribed any DMAs different from their index DMAs were considered as treatment switch. . The RF and LR models were built with 72 baseline characteristics and trained with 70% of the randomly split data after up-sampling. Area Under the Curves (AUC), accuracy, recall, G-measure, and F-1 score were used to evaluate the model performance.

RESULTS

In this study, 7258 MS patients with ≥1 DMA were identified. Within two years, 16% of MS patients switched to a different DMA. The RF model obtained significantly better discrimination than the LR model (AUC = 0.65 vs. 0.63, p < 0.0001); however, the RF model had a similar predictive performance to the LR model with respect to F- and G-measures (RF: 72% and 73% vs. LR: 72% and 73%, respectively). The most influential features identified from the RF model were age, type of index medication, and year of index.

CONCLUSIONS

Compared to the LR model, RF performed better in predicting DMA switch in MS patients based on AUC measures; however, judged by F- and G-measures, the RF model performed similarly to LR. Further research is needed to understand the role of ML techniques in predicting treatment outcomes for the decision-making process to achieve optimal treatment goals.

Prescribed antiseizure medication doses and their relation to defined daily doses for achieving seizure freedom in newly diagnosed patients with epilepsy

OBJECTIVES

To investigate the antiseizure medication (ASM) doses required to achieve seizure freedom and their correlation with the World Health Organization's defined daily doses (DDDs) in patients aged 16 years or older with newly diagnosed epilepsy.

METHODS

The study included 459 patients with a validated diagnosis of new-onset epilepsy. Patient records were retrospectively analyzed to determine the ASM doses in patients with or without seizure freedom during follow-up. The DDD of the relevant ASM was then retrieved.

RESULTS

The seizure-freedom rate with first and subsequent ASMs was 88% (404/459 patients) during the follow-up. The mean prescribed doses (PDDs) and PDD/DDD ratio of the most commonly used ASMs, ie, oxcarbazepine (OXC), carbamazepine (CBZ), and valproic acid (VPA), differed significantly between seizure-free and non-seizure-free status (992 mg and 0.99 vs 1132 mg and 1.13; 547 mg and 0.55 vs 659 mg and 0.66; and 953 mg and 0.64 vs 1260 mg and 0.84, respectively). The effect of the OXC dose as the first failed ASM on the possibility of achieving seizure freedom was significant (Fisher's exact test, p = 0.002). Thirty-four of 43 patients (79%) in which an OXC dose of ≤900 mg failed became seizure-free, as compared with 24 of 54 patients (44%) with a failed OXC dose >900 mg.

SIGNIFICANCE

The present study provides new insights into the doses of the commonly used ASMs such as OXC, CBZ, and VPA that can lead to seizure freedom as monotherapy or as combination therapy. The higher PDD/DDD ratio of OXC (0.99) than that of CBZ or VPA renders a generalized PDD/DDD comparison highly problematic.

Review of Machine Learning and Artificial Intelligence (ML/AI) for the Pediatric Neurologist

Artificial intelligence (AI) and a popular branch of AI known as machine learning (ML) are increasingly being utilized in medicine and to inform medical research. This review provides an overview of AI and ML (AI/ML), including definitions of common terms. We discuss the history of AI and provide instances of how AI/ML can be applied to pediatric neurology. Examples include imaging in neuro-oncology, autism diagnosis, diagnosis from charts, epilepsy, cerebral palsy, and neonatal neurology. Topics such as supervised learning, unsupervised learning, and reinforcement learning are discussed.

Deep Thoughts—Predicting Initial Treatment Response in Newly Diagnosed Epilepsy

[Box: see text]

A machine-learning guided method for predicting add-on and switch in secondary data sources: A case study on anti-seizure medications in Danish registries

Purpose: There is a lack of available evidence regarding the treatment pattern of switches and add-ons for individuals aged 65 years or older with epilepsy during the first years from the time they received their first anti-seizure medication because of the lack of valid methods. Therefore, this study aimed to develop an algorithm for identifying switches and add-ons using secondary data sources for anti-seizure medication users. Methods: Danish nationwide databases were used as data sources. Residents in Denmark between 1996 and 2018 who were diagnosed with epilepsy and redeemed their first prescription for anti-seizure medication after epilepsy diagnosis were followed up for 730 days until the end of the follow-up period, death, or emigration to assess switches and add-ons occurred during the follow-up period. The study outcomes were the overall accuracy of the classification of switch or add-on of the newly developed algorithm. Results: In total, 15870 individuals were included in the study population with a median age of 72.9 years, of whom 52.0% were male and 48.0% were female. A total of 988 of the 15879 patients from the study population were present during the 730-day follow-up period, and 988 individuals (6.2%) underwent a total of 1485 medication events with co-exposure to two or more anti-seizure medications. The newly developed algorithmic method correctly identified 9 out of 10 add-ons (overall accuracy 92%) and 9 out of 10 switches (overall accuracy 88%). Conclusion: The majority of switches and add-ons occurred early during the first 2 years of disease and according to clinical recommendations. The newly developed algorithm correctly identified 9 out of 10 switches/add-ons.

Development and Validation of a Deep Learning Model for Predicting Treatment Response in Patients With Newly Diagnosed Epilepsy

Importance

Selection of antiseizure medications (ASMs) for epilepsy remains largely a trial-and-error approach. Under this approach, many patients have to endure sequential trials of ineffective treatments until the "right drugs" are prescribed.

Objective

To develop and validate a deep learning model using readily available clinical information to predict treatment success with the first ASM for individual patients.

Design, Setting, and Participants

This cohort study developed and validated a prognostic model. Patients were treated between 1982 and 2020. All patients were followed up for a minimum of 1 year or until failure of the first ASM. A total of 2404 adults with epilepsy newly treated at specialist clinics in Scotland, Malaysia, Australia, and China between 1982 and 2020 were considered for inclusion, of whom 606 (25.2%) were excluded from the final cohort because of missing information in 1 or more variables.

Exposures

One of 7 antiseizure medications.

Main Outcomes and Measures

With the use of the transformer model architecture on 16 clinical factors and ASM information, this cohort study first pooled all cohorts for model training and testing. The model was trained again using the largest cohort and externally validated on the other 4 cohorts. The area under the receiver operating characteristic curve (AUROC), weighted balanced accuracy, sensitivity, and specificity of the model were all assessed for predicting treatment success based on the optimal probability cutoff. Treatment success was defined as complete seizure freedom for the first year of treatment while taking the first ASM. Performance of the transformer model was compared with other machine learning models.

Results

The final pooled cohort included 1798 adults (54.5% female; median age, 34 years [IQR, 24-50 years]). The transformer model that was trained using the pooled cohort had an AUROC of 0.65 (95% CI, 0.63-0.67) and a weighted balanced accuracy of 0.62 (95% CI, 0.60-0.64) on the test set. The model that was trained using the largest cohort only had AUROCs ranging from 0.52 to 0.60 and a weighted balanced accuracy ranging from 0.51 to 0.62 in the external validation cohorts. Number of pretreatment seizures, presence of psychiatric disorders, electroencephalography, and brain imaging findings were the most important clinical variables for predicted outcomes in both models. The transformer model that was developed using the pooled cohort outperformed 2 of the 5 other models tested in terms of AUROC.

Conclusions and Relevance

In this cohort study, a deep learning model showed the feasibility of personalized prediction of response to ASMs based on clinical information. With improvement of performance, such as by incorporating genetic and imaging data, this model may potentially assist clinicians in selecting the right drug at the first trial.

The interaction between circadian rhythm and epilepsy

Evidence about the interaction between circadian rhythms (CR) and epilepsy has been expanded with the application of advanced detection technology. An adequate understanding of how circadian system and epilepsy interact with each other could contribute to more accurate seizure prediction as well as rapid development of potential treatment timed to specific phases of CR. In this review, we present the reciprocal relationship between CR and epileptic activities from aspects of sleep effect, genetic modulation and brain biochemistry. It has been found that sleep-wake patterns, circadian timing systems and multidien rhythms have essential roles in seizure activities and interictal epileptiform discharge (IED). For instance, specific distribution patterns of seizures and IED have been reported, i.e., lighter non-rapid eye movement (NREM) sleep stage (stage 2) induces seizures while deeper NREM sleep stage (stage 3) activates IEDs. Furthermore, the epilepsy type, seizure type and seizure onset zone can significantly affect the rhythms of seizure occurrence. Apart from the common seizure types, several specific epilepsy syndromes also have a close correlation with sleep-wakefulness patterns. Sleep influences the epilepsy rhythm, and conversely, epilepsy alters the sleep rhythm through multiple pathways. Clock genes accompanied by two feedback loops of regulation have an important role in cortical excitability and seizure occurrence, which may be involved in the mTORopathy. The suprachiasmatic nuclei (SCN) has a rhythm of melatonin and cortisol secretion under the circadian pattern, and then these hormones can feed back into a central oscillator to affect the SCN-dependent rhythms, leading to variable but prominent influence on epilepsy. Furthermore, we discuss the precise predictive algorithms and chronotherapy strategies based on different temporal patterns of seizure occurrence for patients with epilepsy, which may offer a valuable indication for non-invasive closed-loop treatment system. Optimization of the time and dose of antiseizure medications, and resynchronization of disturbed CR (by hormone therapy, light exposure, ketogenic diet, novel small molecules) would be beneficial for epileptic patients in the future. Before formal clinical practice, future large-scale studies are urgently needed to assist prediction and treatment of circadian seizure activities and address unsolved restrictions.

Severity of Epilepsy and Response to Antiseizure Medications in Individuals with Multiple Sclerosis: Analysis of a Real-World Dataset

Background and objectives:

Epilepsy is an important comorbidity that affects outcomes for people with multiple sclerosis (MS). However, it is unclear if seizure severity among individuals with coexistence of multiple sclerosis and epilepsy (MS+E) is higher than in those with other focal epilepsies. Our goal was to compare the overall severity of epilepsy in individuals with MS+E versus those with focal epilepsy without MS (E-MS) as defined by seizure-related healthcare utilization, frequency and duration of status epilepticus, and frequency of antiseizure medication (ASM) regimen changes.

Methods:

In this hypothesis-generating study, we analyzed a U.S. commercial nationwide de-identified claims dataset with > 86 million individuals between 1/1/2008 and 8/31/2019. Using validated algorithms, we identified adults with E-MS and those with MS+E. We compared the number and length of seizure-related hospital admissions, the number of claims and unique days with claims for status epilepticus, and the rates of ASM regimen changes between the MS+E and the E-MS groups.

Results:

During the study period, 66,708 individuals with E-MS and 537 with MS+E had ≥ 2 years of coverage after their initial diagnosis of epilepsy. There was no difference between the MS+E and E-MS groups in the percentage of individuals admitted for seizures and/or status epilepticus. However, MS+E with seizure-related admissions had more admissions and longer hospital stays than those with E-MS. MS+E who experienced status epilepticus had more unique days with status epilepticus claims compared to E-MS. MS+E were more likely to have ASM regimen changes in Year 2 after the initial diagnosis of epilepsy and had more ASM changes during Year 2 compared to E-MS. Among individuals with MS+E, there were no differences in our measures of seizure severity for those treated with sodium channel blockers/modulators versus other ASM classes.

Discussion:

This study supports the notion that individuals with MS+E can have more severe epilepsy than those with E-MS. Seizure severity among individuals with MS+E treated with sodium-channel blockers/modulators versus other ASM classes shows no significant differences.

Classification of evidence:

This study provides Class III evidence that individuals with MS+E can have more severe epilepsy than those with E-MS.

Patient characteristics and antiseizure medication pathways in newly diagnosed epilepsy: Feasibility and pilot results using the common data model in a single-center electronic medical record database

INTRODUCTION

Efforts to characterize variability in epilepsy treatment pathways are limited by the large number of possible antiseizure medication (ASM) regimens and sequences, heterogeneity of patients, and challenges of measuring confounding variables and outcomes across institutions. The Observational Health Data Science and Informatics (OHDSI) collaborative is an international data network representing over 1 billion patient records using common data standards. However, few studies have applied OHDSI's Common Data Model (CDM) to the population with epilepsy and none have validated relevant concepts. The goals of this study were to demonstrate the feasibility of characterizing adult patients with epilepsy and ASM treatment pathways using the CDM in an electronic health record (EHR)-derived database.

METHODS

We validated a phenotype algorithm for epilepsy in adults using the CDM in an EHR-derived database (2001-2020) against source records and a prospectively maintained database of patients with confirmed epilepsy. We obtained the frequency of all antecedent conditions and procedures for patients meeting the epilepsy phenotype criteria and characterized ASM exposure sequences over time and by age and sex.

RESULTS

The phenotype algorithm identified epilepsy with 73.0-85.0% positive predictive value and 86.3% sensitivity. Many patients had neurologic conditions and diagnoses antecedent to meeting epilepsy criteria. Levetiracetam incrementally replaced phenytoin as the most common first-line agent, but significant heterogeneity remained, particularly in second-line and subsequent agents. Drug sequences included up to 8 unique ingredients and a total of 1,235 unique pathways were observed.

CONCLUSIONS

Despite the availability of additional ASMs in the last 2 decades and accumulated guidelines and evidence, ASM use varies significantly in practice, particularly for second-line and subsequent agents. Multi-center OHDSI studies have the potential to better characterize the full extent of variability and support observational comparative effectiveness research, but additional work is needed to validate covariates and outcomes.

Big data analysis of ASM retention rates and expert ASM algorithm: a comparative study

Objective

Only 50% of patients with new‐onset epilepsy achieve seizure freedom with their first antiseizure medication (ASM). A growing body of data illustrates the complexity of predicting ASM response and tolerability, which is influenced by age, sex, and comorbidities. Randomized data with sufficient resolution for personalized medicine are unlikely to emerge. Two potential facilitators of ASM selection are big data using real‐world retention rates or algorithms based on expert opinion. We asked how these methods compare in adult‐onset focal epilepsy.

Methods

ASM retention rates were determined by cross‐referencing data from comprehensive Swedish registers for 37 643 individuals, with identified comorbidities. Eight fictive cases were created and expert advice was collected from the algorithm Epipick. We compared Epipick suggestions in representative patient subgroups, and determined whether ranking based on retention rate reflected expert advice.

Results

The Epipick algorithm suggested six ASM alternatives for younger patients and three ASM alternatives for older patients. In the real‐world data, retention rates for the ASMs ranked as best options by Epipick were high; 65%–72% for young patients and 71%–84% for older patients. The lowest retention rate for Epipick suggestions was 42%–56% in younger cases, and 70%–80% in older cases. The ASM with the best retention rate was generally recommended by Epipick.

Significance

We found a large overlap between expert advice and real‐world retention rates. Notably, Epipick did suggest some ASMs with more modest retention rates. Conversely, clearly inappropriate ASMs (not recommended by Epipick) had high retention rates in some cases, showing that decision systems should not rely indiscriminately on retention rates alone. In future clinical decision support systems, expert opinion and real‐world retention rates could work synergistically.

Machine learning models for decision support in epilepsy management: A critical review

PURPOSE

There remain major challenges for the clinician in managing patients with epilepsy effectively. Choosing anti-seizure medications (ASMs) is subject to trial and error. About one-third of patients have drug-resistant epilepsy (DRE). Surgery may be considered for selected patients, but time from diagnosis to surgery averages 20 years. We reviewed the potential use of machine learning (ML) predictive models as clinical decision support tools to help address some of these issues.

METHODS

We conducted a comprehensive search of Medline and Embase of studies that investigated the application of ML in epilepsy management in terms of predicting ASM responsiveness, predicting DRE, identifying surgical candidates, and predicting epilepsy surgery outcomes. Original articles addressing these 4 areas published in English between 2000 and 2020 were included.

RESULTS

We identified 24 relevant articles: 6 on ASM responsiveness, 3 on DRE prediction, 2 on identifying surgical candidates, and 13 on predicting surgical outcomes. A variety of potential predictors were used including clinical, neuropsychological, imaging, electroencephalography, and health system claims data. A number of different ML algorithms and approaches were used for prediction, but only one study utilized deep learning methods. Some models show promising performance with areas under the curve above 0.9. However, most were single setting studies (18 of 24) with small sample sizes (median number of patients 55), with the exception of 3 studies that utilized large databases and 3 studies that performed external validation. There was a lack of standardization in reporting model performance. None of the models reviewed have been prospectively evaluated for their clinical benefits.

CONCLUSION

The utility of ML models for clinical decision support in epilepsy management remains to be determined. Future research should be directed toward conducting larger studies with external validation, standardization of reporting, and prospective evaluation of the ML model on patient outcomes.

Towards realizing the vision of precision medicine: AI based prediction of clinical drug response

Accurate and individualized prediction of response to therapies is central to precision medicine. However, because of the generally complex and multifaceted nature of clinical drug response, realizing this vision is highly challenging, requiring integrating different data types from the same individual into one prediction model. We used the anti-epileptic drug brivaracetam as a case study and combine a hybrid data/knowledge-driven feature extraction with machine learning to systematically integrate clinical and genetic data from a clinical discovery dataset (n = 235 patients). We constructed a model that successfully predicts clinical drug response [area under the curve (AUC) = 0.76] and show that even with limited sample size, integrating high-dimensional genetics data with clinical data can inform drug response prediction. After further validation on data collected from an independently conducted clinical study (AUC = 0.75), we extensively explore our model to gain insights into the determinants of drug response, and identify various clinical and genetic characteristics predisposing to poor response. Finally, we assess the potential impact of our model on clinical trial design and demonstrate that, by enriching for probable responders, significant reductions in clinical study sizes may be achieved. To our knowledge, our model represents the first retrospectively validated machine learning model linking drug mechanism of action and the genetic, clinical and demographic background in epilepsy patients to clinical drug response. Hence, it provides a blueprint for how machine learning-based multimodal data integration can act as a driver in achieving the goals of precision medicine in fields such as neurology.

Potential for improved retention rate by personalized antiseizure medication selection: A register-based analysis

OBJECTIVE

The first antiseizure medication (ASM) is ineffective or intolerable in 50% of epilepsy cases. Selection between more than 25 available ASMs is guided by epilepsy factors, but also age and comorbidities. Randomized evidence for particular patient subgroups is seldom available. We asked whether register data could be used for retention rate calculations based on demographics, comorbidities, and ASM history, and quantified the potential improvement in retention rates of the first ASM in several large epilepsy cohorts. We also describe retention rates in patients with epilepsy after traumatic brain injury and dementia, patient groups with little available evidence.

METHODS

We used medical, demographic, and drug prescription data from epilepsy cohorts from comprehensive Swedish registers, containing 6380 observations. By analyzing 381 840 prescriptions, we studied retention rates of first- and second-line ASMs for patients with epilepsy in multiple sclerosis (MS), brain infection, dementia, traumatic brain injury, or stroke. The rank of retention rates of ASMs was validated by comparison to published randomized control trials. We identified the optimal stratification for each brain disease, and quantified the potential improvement if all patients had received the optimal ASM.

RESULTS

Using optimal stratification for each brain disease, the potential improvement in retention rate (percentage points) was MS, 20%; brain infection, 21%; dementia, 14%; trauma, 21%; and stroke, 14%. In epilepsy after trauma, levetiracetam had the highest retention rate at 80% (95% confidence interval [CI] = 65-89), exceeding that of the most commonly prescribed ASM, carbamazepine (p = .04). In epilepsy after dementia, lamotrigine (77%, 95% CI = 68-84) and levetiracetam (74%, 95% CI = 68-79) had higher retention rates than carbamazepine (p = .006 and p = .01, respectively).

SIGNIFICANCE

We conclude that personalized ASM selection could improve retention rates and that national registers have potential as big data sources for personalized medicine in epilepsy.

Quadruple Decision Making for Parkinson's Disease Patients: Combining Expert Opinion, Patient Preferences, Scientific Evidence, and Big Data Approaches to Reach Precision Medicine

Clinical decision making for Parkinson’s disease patients is supported by a combination of three distinct information resources: best available scientific evidence, professional expertise, and the personal needs and preferences of patients. All three sources have clear value but also share several important limitations, mainly regarding subjectivity, generalizability and variability. For example, current scientific evidence, especially from controlled clinical trials, is often based on selected study populations, making it difficult to translate the outcome to the care for individual patients in everyday clinical practice. Big data, including data from real-life unselected Parkinson populations, can help to bridge this information gap. Fine-grained patient profiles created from big data have the potential to aid in identifying therapeutic approaches that will be most effective given each patient’s individual characteristics, which is particularly important for a disorder characterized by such tremendous interindividual variability as Parkinson’s disease. In this viewpoint, we argue that big data approaches should be acknowledged and harnessed, not to replace existing information resources, but rather as a fourth and complimentary source of information in clinical decision making, helping to represent the full complexity of individual patients. We introduce the ‘quadruple decision making’ model and illustrate its mode of action by showing how this can be used to pursue precision medicine for persons living with Parkinson’s disease.

Artificial Intelligence in Pharmacoepidemiology: A Systematic Review. Part 2-Comparison of the Performance of Artificial Intelligence and Traditional Pharmacoepidemiological Techniques

Aim: To summarize the evidence on the performance of artificial intelligence vs. traditional pharmacoepidemiological techniques.

Methods: Ovid MEDLINE (01/1950 to 05/2019) was searched to identify observational studies, meta-analyses, and clinical trials using artificial intelligence techniques having a drug as the exposure or the outcome of the study. Only studies with an available full text in the English language were evaluated.

Results: In all, 72 original articles and five reviews were identified via Ovid MEDLINE of which 19 (26.4%) compared the performance of artificial intelligence techniques with traditional pharmacoepidemiological methods. In total, 44 comparisons have been performed in articles that aimed at 1) predicting the needed dosage given the patient’s characteristics (31.8%), 2) predicting the clinical response following a pharmacological treatment (29.5%), 3) predicting the occurrence/severity of adverse drug reactions (20.5%), 4) predicting the propensity score (9.1%), 5) identifying subpopulation more at risk of drug inefficacy (4.5%), 6) predicting drug consumption (2.3%), and 7) predicting drug-induced lengths of stay in hospital (2.3%). In 22 out of 44 (50.0%) comparisons, artificial intelligence performed better than traditional pharmacoepidemiological techniques. Random forest (seven out of 11 comparisons; 63.6%) and artificial neural network (six out of 10 comparisons; 60.0%) were the techniques that in most of the comparisons outperformed traditional pharmacoepidemiological methods.

Conclusion: Only a small fraction of articles compared the performance of artificial intelligence techniques with traditional pharmacoepidemiological methods and not all artificial intelligence techniques have been compared in a Pharmacoepidemiological setting. However, in 50% of comparisons, artificial intelligence performed better than pharmacoepidemiological techniques.

Models for predicting treatment efficacy of antiepileptic drugs and prognosis of treatment withdrawal in epilepsy patients

Although antiepileptic drugs (AEDs) are the most effective treatment for epilepsy, 30–40% of patients with epilepsy would develop drug-refractory epilepsy. An accurate, preliminary prediction of the efficacy of AEDs has great clinical significance for patient treatment and prognosis. Some studies have developed statistical models and machine-learning algorithms (MLAs) to predict the efficacy of AEDs treatment and the progression of disease after treatment withdrawal, in order to provide assistance for making clinical decisions in the aim of precise, personalized treatment. The field of prediction models with statistical models and MLAs is attracting growing interest and is developing rapidly. What’s more, more and more studies focus on the external validation of the existing model. In this review, we will give a brief overview of recent developments in this discipline.

New era of personalised epilepsy management

The trial and error approach to epilepsy treatment has not changed for over a century but machine learning and patient derived stem cells promise a personalised and more effective strategy, argue Patrick Kwan and colleagues

Big data in epilepsy: Clinical and research considerations. Report from the Epilepsy Big Data Task Force of the International League Against Epilepsy

Epilepsy is a heterogeneous condition with disparate etiologies and phenotypic and genotypic characteristics. Clinical and research aspects are accordingly varied, ranging from epidemiological to molecular, spanning clinical trials and outcomes, gene and drug discovery, imaging, electroencephalography, pathology, epilepsy surgery, digital technologies, and numerous others. Epilepsy data are collected in the terabytes and petabytes, pushing the limits of current capabilities. Modern computing firepower and advances in machine and deep learning, pioneered in other diseases, open up exciting possibilities for epilepsy too. However, without carefully designed approaches to acquiring, standardizing, curating, and making available such data, there is a risk of failure. Thus, careful construction of relevant ontologies, with intimate stakeholder inputs, provides the requisite scaffolding for more ambitious big data undertakings, such as an epilepsy data commons. In this review, we assess the clinical and research epilepsy landscapes in the big data arena, current challenges, and future directions, and make the case for a systematic approach to epilepsy big data.

Machine learning and wearable devices of the future

Machine learning (ML) is increasingly recognized as a useful tool in healthcare applications, including epilepsy. One of the most important applications of ML in epilepsy is seizure detection and prediction, using wearable devices (WDs). However, not all currently available algorithms implemented in WDs are using ML. In this review, we summarize the state of the art of using WDs and ML in epilepsy, and we outline future development in these domains. There is published evidence for reliable detection of epileptic seizures using implanted electroencephalography (EEG) electrodes and wearable, non-EEG devices. Application of ML using the data recorded with WDs from a large number of patients could change radically the way we diagnose and manage patients with epilepsy.

Artificial Intelligence in Pharmacoepidemiology: A Systematic Review. Part 1-Overview of Knowledge Discovery Techniques in Artificial Intelligence

Aim

To perform a systematic review on the application of artificial intelligence (AI) based knowledge discovery techniques in pharmacoepidemiology.

Study Eligibility Criteria

Clinical trials, meta-analyses, narrative/systematic review, and observational studies using (or mentioning articles using) artificial intelligence techniques were eligible. Articles without a full text available in the English language were excluded.

Data Sources

Articles recorded from 1950/01/01 to 2019/05/06 in Ovid MEDLINE were screened.

Participants

Studies including humans (real or simulated) exposed to a drug.

Results

In total, 72 original articles and 5 reviews were identified via Ovid MEDLINE. Twenty different knowledge discovery methods were identified, mainly from the area of machine learning (66/72; 91.7%). Classification/regression (44/72; 61.1%), classification/regression + model optimization (13/72; 18.0%), and classification/regression + features selection (12/72; 16.7%) were the three most frequent tasks in reviewed literature that machine learning methods has been applied to solve. The top three used techniques were artificial neural networks, random forest, and support vector machines models.

Conclusions

The use of knowledge discovery techniques of artificial intelligence techniques has increased exponentially over the years covering numerous sub-topics of pharmacoepidemiology.

Systematic Review Registration

Systematic review registration number in PROSPERO: CRD42019136552.

Machine learning applications in epilepsy

Machine learning leverages statistical and computer science principles to develop algorithms capable of improving performance through interpretation of data rather than through explicit instructions. Alongside widespread use in image recognition, language processing, and data mining, machine learning techniques have received increasing attention in medical applications, ranging from automated imaging analysis to disease forecasting. This review examines the parallel progress made in epilepsy, highlighting applications in automated seizure detection from electroencephalography (EEG), video, and kinetic data, automated imaging analysis and pre-surgical planning, prediction of medication response, and prediction of medical and surgical outcomes using a wide variety of data sources. A brief overview of commonly used machine learning approaches, as well as challenges in further application of machine learning techniques in epilepsy, is also presented. With increasing computational capabilities, availability of effective machine learning algorithms, and accumulation of larger datasets, clinicians and researchers will increasingly benefit from familiarity with these techniques and the significant progress already made in their application in epilepsy.

Prediction of antiepileptic drug treatment outcomes of patients with newly diagnosed epilepsy by machine learning

OBJECTIVE

The objective of this study was to build a supervised machine learning-based classifier, which can accurately predict the outcomes of antiepileptic drug (AED) treatment of patients with newly diagnosed epilepsy.

METHODS

We collected information from 287 patients with newly diagnosed epilepsy between 2009 and 2017 at the Second Affiliated Hospital of Zhejiang University. Patients were prospectively followed up for at least 3 years. A number of features, including demographic features, medical history, and auxiliary examinations (electroencephalogram [EEG] and magnetic resonance imaging [MRI]) are selected to distinguish patients with different remission outcomes. Seizure outcomes classified as remission and never remission. In addition, remission is further divided into early remission and late remission. Five classical machine learning algorithms, i.e., Decision Tree, Random Forest, Support Vector Machine, XGBoost, and Logistic Regression, are selected and trained by our dataset to get classification models.

RESULTS

Our study shows that 1) compared with the other four algorithms, the XGBoost algorithm based machine learning model achieves the best prediction performance of the AED treatment outcomes between remission and never remission patients with an F1 score of 0.947 and an area under the curve (AUC) value of 0.979; 2) The best discriminative factor for remission and never remission patients is higher number of seizures before treatment (>3); 3) XGBoost-based machine learning model also offers the best prediction between early remission and later remission patients, with an F1 score of 0.836 and an AUC value of 0.918; 4) multiple seizure type has the highest dependence to the categories of early and late remission patients.

SIGNIFICANCES

Our XGBoost-based machine learning classifier accurately predicts the most probable AED treatment outcome of a patient after he/she finishes all the standard examinations for the epilepsy disease. The classifier's prediction result could help disease guide counseling and eventually improve treatment strategies.

Using predictive analytics to identify drug-resistant epilepsy patients

Epilepsy is one of the most common brain disorders that greatly affects patients’ quality of life and poses serious risks to their health. While the majority of the patients positively respond to the existing anti-epilepsy drugs, others who developed the refractory type of epilepsy show resistance against drug therapy and need to undergo advance treatments such as surgery. Given that identifying such patients is not a straightforward process and requires long courses of trial and error with anti-epilepsy drugs, this study aims at predicting those at-risk patients using clinical and demographic data obtained from electronic medical records. Specifically, the study employs several predictive analytics machine-learning methods, equipped with a novel approach for data balancing, to identify drug-resistant patients using their comorbidities and demographic information along with the initial epilepsy-related diagnosis made by their physician. The promising results we obtained highlight the potential use of machine-learning techniques in facilitating medical decisions and suggest the possibility of extending the proposed approach for developing a clinical decision support system for medical professionals.

Predicting drug-resistant epilepsy - A machine learning approach based on administrative claims data

Patients with drug-resistant epilepsy (DRE) are at high risk of morbidity and mortality, yet their referral to specialist care is frequently delayed. The ability to identify patients at high risk of DRE at the time of treatment initiation, and to subsequently steer their treatment pathway toward more personalized interventions, has high clinical utility. Here, we aim to demonstrate the feasibility of developing algorithms for predicting DRE using machine learning methods. Longitudinal, intersected data sourced from US pharmacy, medical, and adjudicated hospital claims from 1,376,756 patients from 2006 to 2015 were analyzed; 292,892 met inclusion criteria for epilepsy, and 38,382 were classified as having DRE using a proxy measure for drug resistance. Patients were characterized using 1270 features reflecting demographics, comorbidities, medications, procedures, epilepsy status, and payer status. Data from 175,735 randomly selected patients were used to train three algorithms and from the remainder to assess the trained models' predictive power. A model with only age and sex was used as a benchmark. The best model, random forest, achieved an area under the receiver operating characteristic curve (95% confidence interval [CI]) of 0.764 (0.759, 0.770), compared with 0.657 (0.651, 0.663) for the benchmark model. Moreover, predicted probabilities for DRE were well-calibrated with the observed frequencies in the data. The model predicted drug resistance approximately 2 years before patients in the test dataset had failed two antiepileptic drugs (AEDs). Machine learning models constructed using claims data predicted which patients are likely to fail ≥3 AEDs and are at risk of developing DRE at the time of the first AED prescription. The use of such models can ensure that patients with predicted DRE receive specialist care with potentially more aggressive therapeutic interventions from diagnosis, to help reduce the serious sequelae of DRE.

Caring electronically for young outpatients who have epilepsy

PURPOSE

The purpose of this study was to review electronic tools that might improve the delivery of epilepsy care, reduce medical care costs, and empower families to improve self-management capability.

METHOD

We reviewed the epilepsy-specific literature about self-management, electronic patient-reported or provider-reported outcomes, on-going remote surveillance, and alerting/warning systems.

CONCLUSIONS

The improved care delivery system that we envision includes self-management, electronic patient (or provider)-reported outcomes, on-going remote surveillance, and alerting/warning systems. This system and variants have the potential to reduce seizure burden through improved management, keep children out of the emergency department and hospital, and even reduce the number of outpatient visits.

Characteristics of large patient-reported outcomes: Where can one million seizures get us?

Objective

To analyze data from Seizure Tracker, a large electronic seizure diary, including comparison of seizure characteristics among different etiologies, temporal patterns in seizure fluctuations, and specific triggers.

Methods

Zero‐inflated negative binomial mixed‐effects models were used to evaluate temporal patterns of seizure events (during the day or week), as well as group differences in monthly seizure frequency between children and adults and between etiologies. The association of long seizures with seizure triggers was evaluated using a mixed‐effects logistic model with subject as the random effect. Incidence rate ratios (IRRs) and odds ratios were reported for analyses involving zero‐inflated negative binomial and logistic mixed‐effects models, respectively.

Results

A total of 1,037,909 seizures were logged by 10,186 subjects (56.7% children) from December 2007 to January 2016. Children had more frequent seizures than adults did (median monthly seizure frequency 3.5 vs. 2.7, IRR 1.26; p < 0.001). Seizures demonstrated a circadian pattern (higher frequency between 07:00 a.m. and 10:00 a.m. and lower overnight), and seizures were reported differentially across the week (seizure rates higher Monday through Friday than Saturday or Sunday). Longer seizures (>5 or >30 min) had a higher proportion of the following triggers when compared with shorter seizures: “Overtired or irregular sleep,” “Bright or flashing lights,” and “Emotional stress” (p < 0.004).

Significance

This study explored a large cohort of patients with self‐reported seizures; strengths and limitations of large seizure diary databases are discussed. The findings in this study are consistent with those of prior work in smaller validated cohorts, suggesting that patient‐recorded databases are a valuable resource for epilepsy research, capable of both replication of results and generation of novel hypotheses.

Machine Learning for Outcome Prediction in Electroencephalograph (EEG)-Monitored Children in the Intensive Care Unit

The aim of this study was to evaluate the performance of models predicting in-hospital mortality in critically ill children undergoing continuous electroencephalography (cEEG) in the intensive care unit (ICU). We evaluated the performance of machine learning algorithms for predicting mortality in a database of 414 critically ill children undergoing cEEG in the ICU. The area under the receiver operating characteristic curve (AUC) in the test subset was highest for stepwise selection/elimination models (AUC = 0.82) followed by least absolute shrinkage and selection operator (LASSO) and support vector machine with linear kernel (AUC = 0.79), and random forest (AUC = 0.71). The explanatory models had the poorest discriminative performance (AUC = 0.63 for the model without considering etiology and AUC = 0.45 for the model considering etiology). Using few variables and a relatively small number of patients, machine learning techniques added information to explanatory models for prediction of in-hospital mortality.

Can a collaborative healthcare network improve the care of people with epilepsy?

New opportunities are now available to improve care in ways not possible previously. Information contained in electronic medical records can now be shared without identifying patients. With network collaboration, large numbers of medical records can be searched to identify patients most like the one whose complex medical situation challenges the physician. The clinical effectiveness of different treatment strategies can be assessed rapidly to help the clinician decide on the best treatment for this patient. Other capabilities from different components of the network can prompt the recognition of what is the best available option and encourage the sharing of information about programs and electronic tools. Difficulties related to privacy, harmonization, integration, and costs are expected, but these are currently being addressed successfully by groups of organizations led by those who recognize the benefits.