Towards trustworthy seizure onset detection using workflow notes

A major barrier to deploying healthcare AI is trustworthiness. One form of trustworthiness is a model's robustness across subgroups: while models may exhibit expert-level performance on aggregate metrics, they often rely on non-causal features, leading to errors in hidden subgroups. To take a step closer towards trustworthy seizure onset detection from EEG, we propose to leverage annotations that are produced by healthcare personnel in routine clinical workflows-which we refer to as workflow notes-that include multiple event descriptions beyond seizures. Using workflow notes, we first show that by scaling training data to 68,920 EEG hours, seizure onset detection performance significantly improves by 12.3 AUROC (Area Under the Receiver Operating Characteristic) points compared to relying on smaller training sets with gold-standard labels. Second, we reveal that our binary seizure onset detection model underperforms on clinically relevant subgroups (e.g., up to a margin of 6.5 AUROC points between pediatrics and adults), while having significantly higher FPRs (False Positive Rates) on EEG clips showing non-epileptiform abnormalities (+19 FPR points). To improve model robustness to hidden subgroups, we train a multilabel model that classifies 26 attributes other than seizures (e.g., spikes and movement artifacts) and significantly improve overall performance (+5.9 AUROC points) while greatly improving performance among subgroups (up to +8.3 AUROC points) and decreasing false positives on non-epileptiform abnormalities (by 8 FPR points). Finally, we find that our multilabel model improves clinical utility (false positives per 24 EEG hours) by a factor of 2×.

Connectivity increases during spikes and spike-free periods in self-limited epilepsy with centrotemporal spikes

OBJECTIVE

To understand the impact of interictal spikes on brain connectivity in patients with Self-Limited Epilepsy with Centrotemporal Spikes (SeLECTS).

METHODS

Electroencephalograms from 56 consecutive SeLECTS patients were segmented into periods with and without spikes. Connectivity between electrodes was calculated using the weighted phase lag index. To determine if there are chronic alterations in connectivity in SeLECTS, we compared spike-free connectivity to connectivity in 65 matched controls. To understand the acute impact of spikes, we compared connectivity immediately before, during, and after spikes versus baseline, spike-free connectivity. We explored whether behavioral state, spike laterality, or antiseizure medications affected connectivity.

RESULTS

Children with SeLECTS had markedly higher connectivity than controls during sleep but not wakefulness, with greatest difference in the right hemisphere. During spikes, connectivity increased globally; before and after spikes, left frontal and bicentral connectivity increased. Right hemisphere connectivity increased more during right-sided than left-sided spikes; left hemisphere connectivity was equally affected by right and left spikes.

CONCLUSIONS

SeLECTS patient have persistent increased connectivity during sleep; connectivity is further elevated during the spike and perispike periods.

SIGNIFICANCE

Testing whether increased connectivity impacts cognition or seizure susceptibility in SeLECTS and more severe epilepsies could help determine if spikes should be treated.

Cross-Modal Data Programming Enables Rapid Medical Machine Learning

A major bottleneck in developing clinically impactful machine learning models is a lack of labeled training data for model supervision. Thus, medical researchers increasingly turn to weaker, noisier sources of supervision, such as leveraging extractions from unstructured text reports to supervise image classification. A key challenge in weak supervision is combining sources of information that may differ in quality and have correlated errors. Recently, a statistical theory of weak supervision called data programming has shown promise in addressing this challenge. Data programming now underpins many deployed machine-learning systems in the technology industry, even for critical applications. We propose a new technique for applying data programming to the problem of cross-modal weak supervision in medicine, wherein weak labels derived from an auxiliary modality (e.g., text) are used to train models over a different target modality (e.g., images). We evaluate our approach on diverse clinical tasks via direct comparison to institution-scale, hand-labeled datasets. We find that our supervision technique increases model performance by up to 6 points area under the receiver operating characteristic curve (ROC-AUC) over baseline methods by improving both coverage and quality of the weak labels. Our approach yields models that on average perform within 1.75 points ROC-AUC of those supervised with physician-years of hand labeling and outperform those supervised with physician-months of hand labeling by 10.25 points ROC-AUC, while using only person-days of developer time and clinician work—a time saving of 96%. Our results suggest that modern weak supervision techniques such as data programming may enable more rapid development and deployment of clinically useful machine-learning models.

•

We propose cross-modal data programming (XMDP) for machine learning (ML) in medicine

•

XMDP reduces labeling costs by 96% versus hand labeling on 4 diverse medical ML tasks

•

Days of XMDP and years of hand labeling often yield similarly performing models

•

XMDP performance continually improves as more unlabeled data becomes available

Machine learning can achieve record-breaking performance on many tasks, but machine learning development is often hindered by insufficient hand-labeled data for model training. This issue is particularly prohibitive in areas such as medical diagnostic analysis, where data are private and require expensive labeling by clinicians.

A promising approach to handle this bottleneck is weak supervision, where machine learning models are trained using cheaper, noisier labels. We extend a recent, theoretically grounded weak supervision paradigm—data programming—wherein subject matter expert users write labeling functions to label training data imprecisely rather than hand-labeling data points. We show that our approach allows us to train machine learning models using person-days of effort that previously required person-years of hand labeling. Our methods could enable researchers and practitioners to leverage machine learning models over high-dimensional data (e.g., images, time series) even when labeled training sets are unavailable.

Weak supervision as an efficient approach for automated seizure detection in electroencephalography

Automated seizure detection from electroencephalography (EEG) would improve the quality of patient care while reducing medical costs, but achieving reliably high performance across patients has proven difficult. Convolutional Neural Networks (CNNs) show promise in addressing this problem, but they are limited by a lack of large labeled training datasets. We propose using imperfect but plentiful archived annotations to train CNNs for automated, real-time EEG seizure detection across patients. While these weak annotations indicate possible seizures with precision scores as low as 0.37, they are commonly produced in large volumes within existing clinical workflows by a mixed group of technicians, fellows, students, and board-certified epileptologists. We find that CNNs trained using such weak annotations achieve Area Under the Receiver Operating Characteristic curve (AUROC) values of 0.93 and 0.94 for pediatric and adult seizure onset detection, respectively. Compared to currently deployed clinical software, our model provides a 31% increase (18 points) in F1-score for pediatric patients and a 17% increase (11 points) for adult patients. These results demonstrate that weak annotations, which are sustainably collected via existing clinical workflows, can be leveraged to produce clinically useful seizure detection models.

iEEG-BIDS, extending the Brain Imaging Data Structure specification to human intracranial electrophysiology

The Brain Imaging Data Structure (BIDS) is a community-driven specification for organizing neuroscience data and metadata with the aim to make datasets more transparent, reusable, and reproducible. Intracranial electroencephalography (iEEG) data offer a unique combination of high spatial and temporal resolution measurements of the living human brain. To improve internal (re)use and external sharing of these unique data, we present a specification for storing and sharing iEEG data: iEEG-BIDS.

Clinical Transcriptome Sequencing Confirms Activation of a Cryptic Splice Site in Suspected SYNGAP1-Related Disorder

SYNGAP1 encodes a brain-specific Ras GTPase activating protein (GAP) that regulates synaptic strength in glutamatergic neurons. Pathogenic variants in this gene are associated with a neurodevelopmental disorder characterized by intellectual and developmental disabilities, generalized epilepsy, hypotonia, and autism spectrum disorders. We describe a young male with suspected SYNGAP1-related disorder given clinical overlap and identification of an intronic variant of uncertain significance; clinical transcriptome analysis demonstrated activation of a cryptic acceptor splice site resulting in frameshift and introduction of a stop codon. This report highlights the utility of functional studies newly available to clinical practice in confirming a suspected genetic diagnosis, which can directly impact medical management and preclude the need for additional diagnostic testing.

Optogenetic Stimulation of Neural Grafts Enhances Neurotransmission and Downregulates the Inflammatory Response in Experimental Stroke Model

Compelling evidence suggests that transplantation of neural stem cells (NSCs) from multiple sources ameliorates motor deficits after stroke. However, it is currently unknown to what extent the electrophysiological activity of grafted NSC progeny participates in the improvement of motor deficits and whether excitatory phenotypes of the grafted cells are beneficial or deleterious to sensorimotor performances. To address this question, we used optogenetic tools to drive the excitatory outputs of the grafted NSCs and assess the impact on local circuitry and sensorimotor performance. We genetically engineered NSCs to express the Channelrhodopsin-2 (ChR2), a light-gated cation channel that evokes neuronal depolarization and initiation of action potentials with precise temporal control to light stimulation. To test the function of these cells in a stroke model, rats were subjected to an ischemic stroke and grafted with ChR2-NSCs. The grafted NSCs identified with a human-specific nuclear marker survived in the peri-infarct tissue and coexpressed the ChR2 transgene with the neuronal markers TuJ1 and NeuN. Gene expression analysis in stimulated versus vehicle-treated animals showed a differential upregulation of transcripts involved in neurotransmission, neuronal differentiation, regeneration, axonal guidance, and synaptic plasticity. Interestingly, genes involved in the inflammatory response were significantly downregulated. Behavioral analysis demonstrated that chronic optogenetic stimulation of the ChR2-NSCs enhanced forelimb use on the stroke-affected side and motor activity in an open field test. Together these data suggest that excitatory stimulation of grafted NSCs elicits beneficial effects in experimental stroke model through cell replacement and non-cell replacement, anti-inflammatory/neurotrophic effects.

Prolonged neuropsychiatric effects following management of chloroquine intoxication with psychotropic polypharmacy

Susceptibility to quinoline antimalarial intoxication may reflect individual genetic and drug-induced variation in neuropharmacokinetics. In this report, we describe a case of chloroquine intoxication that appeared to be prolonged by subsequent use of multiple psychotropic medications. This case highlights important new considerations for the management of quinoline antimalarial intoxication.

Clinical and molecular heterogeneity in patients with the cblD inborn error of cobalamin metabolism

OBJECTIVES

To describe 3 patients with the cblD disorder, a rare inborn error of cobalamin metabolism caused by mutations in the MMADHC gene that can result in isolated homocystinuria, isolated methylmalonic aciduria, or combined homocystinuria and methylmalonic aciduria.

STUDY DESIGN

Patient clinical records were reviewed. Biochemical and somatic cell genetic studies were performed on cultured fibroblasts. Sequence analysis of the MMADHC gene was performed on patient DNA.

RESULTS

Patient 1 presented with isolated methylmalonic aciduria, patient 3 with isolated homocystinuria, and patient 2 with combined methylmalonic aciduria and homocystinuria. Studies of cultured fibroblasts confirmed decreased synthesis of adenosylcobalamin in patient 1, decreased synthesis of methylcobalamin in patient 3, and decreased synthesis of both cobalamin derivatives in patient 2. The diagnosis of cblD was established in each patient by complementation analysis. Mutations in the MMADHC gene were identified in all patients.

CONCLUSIONS

The results emphasize the heterogeneous clinical, cellular and molecular phenotype of the cblD disorder. The results of molecular analysis of the MMADHC gene are consistent with the hypothesis that mutations affecting the N terminus of the MMADHC protein are associated with methylmalonic aciduria, and mutations affecting the C terminus are associated with homocystinuria.