Transforming epilepsy research: A systematic review on natural language processing applications

Annotation of epilepsy clinic letters for natural language processing

BACKGROUND

Natural language processing (NLP) is increasingly being used to extract structured information from unstructured text to assist clinical decision-making and aid healthcare research. The availability of expert-annotated documents for the development and validation of NLP applications is limited. We created synthetic clinical documents to address this, and to validate the Extraction of Epilepsy Clinical Text version 2 (ExECTv2) NLP pipeline.

METHODS

We created 200 synthetic clinic letters based on hospital outpatient consultations with epilepsy specialists. The letters were double annotated by trained clinicians and researchers according to agreed guidelines. We used the annotation tool, Markup, with an epilepsy concept list based on the Unified Medical Language System ontology. All annotations were reviewed, and a gold standard set of annotations was agreed and used to validate the performance of ExECTv2.

RESULTS

The overall inter-annotator agreement (IAA) between the two sets of annotations produced a per item F1 score of 0.73. Validating ExECTv2 using the gold standard gave an overall F1 score of 0.87 per item, and 0.90 per letter.

CONCLUSION

The synthetic letters, annotations, and annotation guidelines have been made freely available. To our knowledge, this is the first publicly available set of annotated epilepsy clinic letters and guidelines that can be used for NLP researchers with minimum epilepsy knowledge. The IAA results show that clinical text annotation tasks are difficult and require a gold standard to be arranged by researcher consensus. The results for ExECTv2, our automated epilepsy NLP pipeline, extracted detailed epilepsy information from unstructured epilepsy letters with more accuracy than human annotators, further confirming the utility of NLP for clinical and research applications.

Natural Language Processing Applied to Spontaneous Recall of Famous Faces Reveals Memory Dysfunction in Temporal Lobe Epilepsy Patients

Objective and Background

Epilepsy patients rank memory problems as their most significant cognitive comorbidity. Current clinical assessments are laborious to administer and score and may not always detect subtle memory decline. The Famous Faces Task (FF) has robustly demonstrated that left temporal lobe epilepsy (LTLE) patients remember fewer names and biographical details compared to right TLE (RTLE) patients and healthy controls (HCs). We adapted the FF task to capture subjects' entire spontaneous spoken recall, then scored responses using manual and natural language processing (NLP) methods. We expected to replicate previous group level differences using spontaneous speech and semi-automated analysis.

Methods

Seventy-three (N=73) adults (28 LTLE, 18 RTLE, and 27 HCs) were included in a case-control prospective study design. Twenty FF in politics, sports, and entertainment (active 2008-2017) were shown to subjects, who were asked if they could recognize and spontaneously recall as much biographical detail as possible. We created human-generated and automatically-generated keyword dictionaries for each celebrity, based on a randomly selected training set of half of the HC transcripts. To control for speech output, we measured the speech duration, total word count and content word count for the FF task and a Cookie Theft Control Task (CTT), in which subjects were merely asked to describe a visual scene. Subjects' responses to FF and CTT tasks were recorded, transcribed, and analyzed in a blinded manner with a combination of manual and automated NLP approaches.

Results

Famous face recognition accuracy was similar between groups. LTLE patients recalled fewer biographical details compared to HCs and RTLEs using both the gold-standard human-generated dictionary (24%±12% vs. 31%±12% and 30%±12%, p=0.007) and the automated dictionary (24%±12% vs. 31%±12% and 32%±13%, p=0.007). There were no group level differences in speech duration, total word count, or content word count for either the FF and CTT to explain difference in recall performance. There was a positive, statistically significant relationship between MOCA score and FF recall performance as scored by the human-generated (ρ= .327, p= .029) and automatically-generated dictionaries (ρ= .422, p= .004) for TLE subjects, but not HCs, an effect that was driven by LTLE subjects.

Discussion

LTLE patients remember fewer details of famous people than HCs or RTLE patients, as discovered by NLP analysis of spontaneous recall. Decreased biographical memory was not due to decreased speech output and correlated with lower MOCA scores. NLP analysis of spontaneous recall can detect memory dysfunction in clinical populations in a semi-automated, objective, and sensitive manner.

Clinical signatures of genetic epilepsies precede diagnosis in electronic medical records of 32,000 individuals

PURPOSE

An early genetic diagnosis can guide the time-sensitive treatment of individuals with genetic epilepsies. However, most genetic diagnoses occur long after disease onset. We aimed to identify early clinical features suggestive of genetic diagnoses in individuals with epilepsy through large-scale analysis of full-text electronic medical records.

METHODS

We extracted 89 million time-stamped standardized clinical annotations using Natural Language Processing from 4,572,783 clinical notes from 32,112 individuals with childhood epilepsy, including 1925 individuals with known or presumed genetic epilepsies. We applied these features to train random forest models to predict SCN1A-related disorders and any genetic diagnosis.

RESULTS

We identified 47,774 age-dependent associations of clinical features with genetic etiologies a median of 3.6 years before molecular diagnosis. Across all 710 genetic etiologies identified in our cohort, neurodevelopmental differences between 6 to 9 months increased the likelihood of a later molecular diagnosis 5-fold (P < .0001, 95% CI = 3.55-7.42). A later diagnosis of SCN1A-related disorders (area under the curve [AUC] = 0.91) or an overall positive genetic diagnosis (AUC = 0.82) could be reliably predicted using random forest models.

CONCLUSION

Clinical features predictive of genetic epilepsies precede molecular diagnoses by up to several years in conditions with known precision treatments. An earlier diagnosis facilitated by automated electronic medical records analysis has the potential for earlier targeted therapeutic strategies in the genetic epilepsies.

The potential of large language model chatbots for application to epilepsy: Let's talk about physical exercise

In this paper, we discuss how artificial intelligence chatbots based on large-scale language models (LLMs) can be used to disseminate information about the benefits of physical exercise for individuals with epilepsy. LLMs have demonstrated the ability to generate increasingly detailed text and allow structured dialogs. These can be useful tools, providing guidance and advice to people with epilepsy on different forms of treatment as well as physical exercise. We also examine the limitations of LLMs, which include the need for human supervision and the risk of providing imprecise and unreliable information regarding specific or controversial aspects of the topic. Despite these challenges, LLM chatbots have demonstrated the potential to support the management of epilepsy and break down barriers to information access, particularly information on physical exercise.

Artificial intelligence in epilepsy - applications and pathways to the clinic

Artificial intelligence (AI) is rapidly transforming health care, and its applications in epilepsy have increased exponentially over the past decade. Integration of AI into epilepsy management promises to revolutionize the diagnosis and treatment of this complex disorder. However, translation of AI into neurology clinical practice has not yet been successful, emphasizing the need to consider progress to date and assess challenges and limitations of AI. In this Review, we provide an overview of AI applications that have been developed in epilepsy using a variety of data modalities: neuroimaging, electroencephalography, electronic health records, medical devices and multimodal data integration. For each, we consider potential applications, including seizure detection and prediction, seizure lateralization, localization of the seizure-onset zone and assessment for surgical or neurostimulation interventions, and review the performance of AI tools developed to date. We also discuss methodological considerations and challenges that must be addressed to successfully integrate AI into clinical practice. Our goal is to provide an overview of the current state of the field and provide guidance for leveraging AI in future to improve management of epilepsy.

Potential merits and flaws of large language models in epilepsy care: A critical review

The current pace of development and applications of large language models (LLMs) is unprecedented and will impact future medical care significantly. In this critical review, we provide the background to better understand these novel artificial intelligence (AI) models and how LLMs can be of future use in the daily care of people with epilepsy. Considering the importance of clinical history taking in diagnosing and monitoring epilepsy-combined with the established use of electronic health records-a great potential exists to integrate LLMs in epilepsy care. We present the current available LLM studies in epilepsy. Furthermore, we highlight and compare the most commonly used LLMs and elaborate on how these models can be applied in epilepsy. We further discuss important drawbacks and risks of LLMs, and we provide recommendations for overcoming these limitations.

Automated Category and Trend Analysis of Scientific Articles on Ophthalmology Using Large Language Models: Development and Usability Study

BACKGROUND

In this paper, we present an automated method for article classification, leveraging the power of large language models (LLMs).

OBJECTIVE

The aim of this study is to evaluate the applicability of various LLMs based on textual content of scientific ophthalmology papers.

METHODS

We developed a model based on natural language processing techniques, including advanced LLMs, to process and analyze the textual content of scientific papers. Specifically, we used zero-shot learning LLMs and compared Bidirectional and Auto-Regressive Transformers (BART) and its variants with Bidirectional Encoder Representations from Transformers (BERT) and its variants, such as distilBERT, SciBERT, PubmedBERT, and BioBERT. To evaluate the LLMs, we compiled a data set (retinal diseases [RenD] ) of 1000 ocular disease-related articles, which were expertly annotated by a panel of 6 specialists into 19 distinct categories. In addition to the classification of articles, we also performed analysis on different classified groups to find the patterns and trends in the field.

RESULTS

The classification results demonstrate the effectiveness of LLMs in categorizing a large number of ophthalmology papers without human intervention. The model achieved a mean accuracy of 0.86 and a mean F1-score of 0.85 based on the RenD data set.

CONCLUSIONS

The proposed framework achieves notable improvements in both accuracy and efficiency. Its application in the domain of ophthalmology showcases its potential for knowledge organization and retrieval. We performed a trend analysis that enables researchers and clinicians to easily categorize and retrieve relevant papers, saving time and effort in literature review and information gathering as well as identification of emerging scientific trends within different disciplines. Moreover, the extendibility of the model to other scientific fields broadens its impact in facilitating research and trend analysis across diverse disciplines.

Natural history of rare diseases using natural language processing of narrative unstructured electronic health records: The example of Dravet syndrome

OBJECTIVE

The increasing implementation of electronic health records allows the use of advanced text-mining methods for establishing new patient phenotypes and stratification, and for revealing outcome correlations. In this study, we aimed to explore the electronic narrative clinical reports of a cohort of patients with Dravet syndrome (DS) longitudinally followed at our center, to identify the capacity of this methodology to retrace natural history of DS during the early years.

METHODS

We used a document-based clinical data warehouse employing natural language processing to recognize the phenotype concepts in the narrative medical reports. We included patients with DS who have a medical report produced before the age of 2 years and a follow-up after the age of 3 years ("DS cohort," 56 individuals). We selected two control populations, a "general control cohort" (275 individuals) and a "neurological control cohort" (281 individuals), with similar characteristics in terms of gender, number of reports, and age at last report. To find concepts specifically associated with DS, we performed a phenome-wide association study using Cox regression, comparing the reports of the three cohorts. We then performed a qualitative analysis of the surviving concepts based on their median age at first appearance.

RESULTS

A total of 76 concepts were prevalent in the reports of children with DS. Concepts appearing during the first 2 years were mostly related with the epilepsy features at the onset of DS (convulsive and prolonged seizures triggered by fever, often requiring in-hospital care). Subsequently, concepts related to new types of seizures and to drug resistance appeared. A series of non-seizure-related concepts emerged after the age of 2-3 years, referring to the nonseizure comorbidities classically associated with DS.

SIGNIFICANCE

The extraction of clinical terms by narrative reports of children with DS allows outlining the known natural history of this rare disease in early childhood. This original model of "longitudinal phenotyping" could be applied to other rare and very rare conditions with poor natural history description.

NLP-based tools for localization of the epileptogenic zone in patients with drug-resistant focal epilepsy

Epilepsy surgery is an option for people with focal onset drug-resistant (DR) seizures but a delayed or incorrect diagnosis of epileptogenic zone (EZ) location limits its efficacy. Seizure semiological manifestations and their chronological appearance contain valuable information on the putative EZ location but their interpretation relies on extensive experience. The aim of our work is to support the localization of EZ in DR patients automatically analyzing the semiological description of seizures contained in video-EEG reports. Our sample is composed of 536 descriptions of seizures extracted from Electronic Medical Records of 122 patients. We devised numerical representations of anamnestic records and seizures descriptions, exploiting Natural Language Processing (NLP) techniques, and used them to feed Machine Learning (ML) models. We performed three binary classification tasks: localizing the EZ in the right or left hemisphere, temporal or extra-temporal, and frontal or posterior regions. Our computational pipeline reached performances above 70% in all tasks. These results show that NLP-based numerical representation combined with ML-based classification models may help in localizing the origin of the seizures relying only on seizures-related semiological text data alone. Accurate early recognition of EZ could enable a more appropriate patient management and a faster access to epilepsy surgery to potential candidates.

Large Language Models in Neurology Research and Future Practice

Recent advancements in generative artificial intelligence, particularly using large language models (LLMs), are gaining increased public attention. We provide a perspective on the potential of LLMs to analyze enormous amounts of data from medical records and gain insights on specific topics in neurology. In addition, we explore use cases for LLMs, such as early diagnosis, supporting patient and caregivers, and acting as an assistant for clinicians. We point to the potential ethical and technical challenges raised by LLMs, such as concerns about privacy and data security, potential biases in the data for model training, and the need for careful validation of results. Researchers must consider these challenges and take steps to address them to ensure that their work is conducted in a safe and responsible manner. Despite these challenges, LLMs offer promising opportunities for improving care and treatment of various neurologic disorders.

Machine Learning and Artificial Intelligence Applications to Epilepsy: a Review for the Practicing Epileptologist

PURPOSE OF REVIEW

Machine Learning (ML) and Artificial Intelligence (AI) are data-driven techniques to translate raw data into applicable and interpretable insights that can assist in clinical decision making. Some of these tools have extremely promising initial results, earning both great excitement and creating hype. This non-technical article reviews recent developments in ML/AI in epilepsy to assist the current practicing epileptologist in understanding both the benefits and limitations of integrating ML/AI tools into their clinical practice.

RECENT FINDINGS

ML/AI tools have been developed to assist clinicians in almost every clinical decision including (1) predicting future epilepsy in people at risk, (2) detecting and monitoring for seizures, (3) differentiating epilepsy from mimics, (4) using data to improve neuroanatomic localization and lateralization, and (5) tracking and predicting response to medical and surgical treatments. We also discuss practical, ethical, and equity considerations in the development and application of ML/AI tools including chatbots based on Large Language Models (e.g., ChatGPT). ML/AI tools will change how clinical medicine is practiced, but, with rare exceptions, the transferability to other centers, effectiveness, and safety of these approaches have not yet been established rigorously. In the future, ML/AI will not replace epileptologists, but epileptologists with ML/AI will replace epileptologists without ML/AI.

Natural language processing for identification of refractory status epilepticus in children

OBJECTIVE

Pediatric status epilepticus is one of the most frequent pediatric emergencies, with high mortality and morbidity. Utilizing electronic health records (EHRs) permits analysis of care approaches and disease outcomes at a lower cost than prospective research. However, reviewing EHR manually is time intensive. We aimed to compare refractory status epilepticus (rSE) cases identified by human EHR review with a natural language processing (NLP)-assisted rSE screen followed by a manual review.

METHODS

We used the NLP screening tool Document Review Tool (DrT) to generate regular expressions, trained a bag-of-words NLP classifier on EHRs from 2017 to 2019, and then tested our algorithm on data from February to December 2012. We compared results from manual review to NLP-assisted search followed by manual review.

RESULTS

Our algorithm identified 1528 notes in the test set. After removing notes pertaining to the same event by DrT, the user reviewed a total number of 400 notes to find patients with rSE. Within these 400 notes, we identified 31 rSE cases, including 12 new cases not found in manual review, and 19 of the 20 previously identified cases. The NLP-assisted model found 31 of 32 cases, with a sensitivity of 96.88% (95% CI = 82%-99.84%), whereas manual review identified 20 of 32 cases, with a sensitivity of 62.5% (95% CI = 43.75%-78.34%).

SIGNIFICANCE

DrT provided a highly sensitive model compared to human review and an increase in patient identification through EHRs. The use of DrT is a suitable application of NLP for identifying patients with a history of recent rSE, which ultimately contributes to the implementation of monitoring techniques and treatments in near real time.

Artificial intelligence in epilepsy phenotyping

Artificial intelligence (AI) allows data analysis and integration at an unprecedented granularity and scale. Here we review the technological advances, challenges, and future perspectives of using AI for electro-clinical phenotyping of animal models and patients with epilepsy. In translational research, AI models accurately identify behavioral states in animal models of epilepsy, allowing identification of correlations between neural activity and interictal and ictal behavior. Clinical applications of AI-based automated and semi-automated analysis of audio and video recordings of people with epilepsy, allow significant data reduction and reliable detection and classification of major motor seizures. AI models can accurately identify electrographic biomarkers of epilepsy, such as spikes, high-frequency oscillations, and seizure patterns. Integrating AI analysis of electroencephalographic, clinical, and behavioral data will contribute to optimizing therapy for patients with epilepsy.

Opportunities and challenges of 5G network technology toward precision medicine

Moving away from traditional "one-size-fits-all" treatment to precision-based medicine has tremendously improved disease prognosis, accuracy of diagnosis, disease progression prediction, and targeted-treatment. The current cutting-edge of 5G network technology is enabling a growing trend in precision medicine to extend its utility and value to the smart healthcare system. The 5G network technology will bring together big data, artificial intelligence, and machine learning to provide essential levels of connectivity to enable a new health ecosystem toward precision medicine. In the 5G-enabled health ecosystem, its applications involve predictive and preventative measurements which enable advances in patient personalization. This review aims to discuss the opportunities, challenges, and prospects posed to 5G network technology in moving forward to deliver personalized treatments and patient-centric care via a precision medicine approach.

Using artificial intelligence to improve public health: a narrative review

Artificial intelligence (AI) is a rapidly evolving tool revolutionizing many aspects of healthcare. AI has been predominantly employed in medicine and healthcare administration. However, in public health, the widespread employment of AI only began recently, with the advent of COVID-19. This review examines the advances of AI in public health and the potential challenges that lie ahead. Some of the ways AI has aided public health delivery are via spatial modeling, risk prediction, misinformation control, public health surveillance, disease forecasting, pandemic/epidemic modeling, and health diagnosis. However, the implementation of AI in public health is not universal due to factors including limited infrastructure, lack of technical understanding, data paucity, and ethical/privacy issues.

Text mining of electronic health records can validate a register-based diagnosis of epilepsy and subgroup into focal and generalized epilepsy

OBJECTIVE

Combining population-based health registries and electronic health records offers the opportunity to create large, phenotypically detailed patient cohorts of high quality. In this study, we used text mining of clinical notes to confirm International Classification of Diseases, 10th Revision (ICD-10)-registered epilepsy diagnoses and classify patients according to focal and generalized epilepsy types.

METHODS

Using the Danish National Patient Registry, we identified patients who between 2006 and 2016 received an ICD-10 diagnosis of epilepsy. To validate the epilepsy diagnosis and stratify patients into focal and generalized epilepsy types, we constructed dictionaries for text mining-based extraction of clinical notes. Two physicians manually reviewed the clinical notes for a total of 527 patients and assigned epilepsy diagnoses, which were compared with the text-mined diagnoses.

RESULTS

We identified 23 632 patients with an ICD-10 diagnosis of epilepsy, of whom 50% were registered with an unspecified epilepsy diagnosis. In total, 11 211 patients were considered likely to have epilepsy by text mining, with an F1 measure ranging from 82% to 90%. Manual review of the electronic health records for 310 patients revealed a false discovery rate of 29%. This rate was decreased to 4% by the text mining algorithm. The weighted average F1 measure for text mining-assigned epilepsy types was 79% (82% for focal and 76% for generalized epilepsy). Text mining successfully assigned a focal or generalized epilepsy type to 92% of the text mining-eligible patients registered with unspecified epilepsy.

SIGNIFICANCE

Text mining of electronic health records can be used to establish a patient cohort with much higher likelihood of having a diagnosis of epilepsy and a focal or generalized epilepsy type compared to the cohort created from ICD-10 epilepsy codes alone. We believe the concept will be essential for future genome-wide and phenome-wide association studies and subsequently the development of precision medicine for epilepsy patients.

Long-term epilepsy outcome dynamics revealed by natural language processing of clinic notes

OBJECTIVE

Electronic medical records allow for retrospective clinical research with large patient cohorts. However, epilepsy outcomes are often contained in free text notes that are difficult to mine. We recently developed and validated novel natural language processing (NLP) algorithms to automatically extract key epilepsy outcome measures from clinic notes. In this study, we assessed the feasibility of extracting these measures to study the natural history of epilepsy at our center.

METHODS

We applied our previously validated NLP algorithms to extract seizure freedom, seizure frequency, and date of most recent seizure from outpatient visits at our epilepsy center from 2010 to 2022. We examined the dynamics of seizure outcomes over time using Markov model-based probability and Kaplan-Meier analyses.

RESULTS

Performance of our algorithms on classifying seizure freedom was comparable to that of human reviewers (algorithm F1 = .88 vs. human annotatorκ = .86). We extracted seizure outcome data from 55 630 clinic notes from 9510 unique patients written by 53 unique authors. Of these, 30% were classified as seizure-free since the last visit, 48% of non-seizure-free visits contained a quantifiable seizure frequency, and 47% of all visits contained the date of most recent seizure occurrence. Among patients with at least five visits, the probabilities of seizure freedom at the next visit ranged from 12% to 80% in patients having seizures or seizure-free at the prior three visits, respectively. Only 25% of patients who were seizure-free for 6 months remained seizure-free after 10 years.

SIGNIFICANCE

Our findings demonstrate that epilepsy outcome measures can be extracted accurately from unstructured clinical note text using NLP. At our tertiary center, the disease course often followed a remitting and relapsing pattern. This method represents a powerful new tool for clinical research with many potential uses and extensions to other clinical questions.