Feasibility of using an automated analysis of formulation effort in patients' spoken seizure descriptions in the differential diagnosis of epileptic and nonepileptic seizures

Extracting seizure control metrics from clinic notes of patients with epilepsy: A natural language processing approach

OBJECTIVES

Monitoring seizure control metrics is key to clinical care of patients with epilepsy. Manually abstracting these metrics from unstructured text in electronic health records (EHR) is laborious. We aimed to abstract the date of last seizure and seizure frequency from clinical notes of patients with epilepsy using natural language processing (NLP).

METHODS

We extracted seizure control metrics from notes of patients seen in epilepsy clinics from two hospitals in Boston. Extraction was performed with the pretrained model RoBERTa_for_seizureFrequency_QA, for both date of last seizure and seizure frequency, combined with regular expressions. We designed the algorithm to categorize the timing of last seizure ("today", "1-6 days ago", "1-4 weeks ago", "more than 1-3 months ago", "more than 3-6 months ago", "more than 6-12 months ago", "more than 1-2 years ago", "more than 2 years ago") and seizure frequency ("innumerable", "multiple", "daily", "weekly", "monthly", "once per year", "less than once per year"). Our ground truth consisted of structured questionnaires filled out by physicians. Model performance was measured using the areas under the receiving operating characteristic curve (AUROC) and precision recall curve (AUPRC) for categorical labels, and median absolute error (MAE) for ordinal labels, with 95 % confidence intervals (CI) estimated via bootstrapping.

RESULTS

Our cohort included 1773 adult patients with a total of 5658 visits with reported seizure control metrics, seen in epilepsy clinics between December 2018 and May 2022. The cohort average age was 42 years old, the majority were female (57 %), White (81 %) and non-Hispanic (85 %). The models achieved an MAE (95 % CI) for date of last seizure of 4 (4.00-4.86) weeks, and for seizure frequency of 0.02 (0.02-0.02) seizures per day.

CONCLUSIONS

Our NLP approach demonstrates that the extraction of seizure control metrics from EHR is feasible allowing for large-scale EHR research.

Natural Language Processing Applied to Spontaneous Recall of Famous Faces Reveals Memory Dysfunction in Temporal Lobe Epilepsy Patients

Objective and Background

Epilepsy patients rank memory problems as their most significant cognitive comorbidity. Current clinical assessments are laborious to administer and score and may not always detect subtle memory decline. The Famous Faces Task (FF) has robustly demonstrated that left temporal lobe epilepsy (LTLE) patients remember fewer names and biographical details compared to right TLE (RTLE) patients and healthy controls (HCs). We adapted the FF task to capture subjects' entire spontaneous spoken recall, then scored responses using manual and natural language processing (NLP) methods. We expected to replicate previous group level differences using spontaneous speech and semi-automated analysis.

Methods

Seventy-three (N=73) adults (28 LTLE, 18 RTLE, and 27 HCs) were included in a case-control prospective study design. Twenty FF in politics, sports, and entertainment (active 2008-2017) were shown to subjects, who were asked if they could recognize and spontaneously recall as much biographical detail as possible. We created human-generated and automatically-generated keyword dictionaries for each celebrity, based on a randomly selected training set of half of the HC transcripts. To control for speech output, we measured the speech duration, total word count and content word count for the FF task and a Cookie Theft Control Task (CTT), in which subjects were merely asked to describe a visual scene. Subjects' responses to FF and CTT tasks were recorded, transcribed, and analyzed in a blinded manner with a combination of manual and automated NLP approaches.

Results

Famous face recognition accuracy was similar between groups. LTLE patients recalled fewer biographical details compared to HCs and RTLEs using both the gold-standard human-generated dictionary (24%±12% vs. 31%±12% and 30%±12%, p=0.007) and the automated dictionary (24%±12% vs. 31%±12% and 32%±13%, p=0.007). There were no group level differences in speech duration, total word count, or content word count for either the FF and CTT to explain difference in recall performance. There was a positive, statistically significant relationship between MOCA score and FF recall performance as scored by the human-generated (ρ= .327, p= .029) and automatically-generated dictionaries (ρ= .422, p= .004) for TLE subjects, but not HCs, an effect that was driven by LTLE subjects.

Discussion

LTLE patients remember fewer details of famous people than HCs or RTLE patients, as discovered by NLP analysis of spontaneous recall. Decreased biographical memory was not due to decreased speech output and correlated with lower MOCA scores. NLP analysis of spontaneous recall can detect memory dysfunction in clinical populations in a semi-automated, objective, and sensitive manner.

Potential merits and flaws of large language models in epilepsy care: A critical review

The current pace of development and applications of large language models (LLMs) is unprecedented and will impact future medical care significantly. In this critical review, we provide the background to better understand these novel artificial intelligence (AI) models and how LLMs can be of future use in the daily care of people with epilepsy. Considering the importance of clinical history taking in diagnosing and monitoring epilepsy-combined with the established use of electronic health records-a great potential exists to integrate LLMs in epilepsy care. We present the current available LLM studies in epilepsy. Furthermore, we highlight and compare the most commonly used LLMs and elaborate on how these models can be applied in epilepsy. We further discuss important drawbacks and risks of LLMs, and we provide recommendations for overcoming these limitations.

A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis

Disease diagnosis represents a critical and arduous endeavor within the medical field. Artificial intelligence (AI) techniques, spanning from machine learning and deep learning to large model paradigms, stand poised to significantly augment physicians in rendering more evidence-based decisions, thus presenting a pioneering solution for clinical practice. Traditionally, the amalgamation of diverse medical data modalities (e.g., image, text, speech, genetic data, physiological signals) is imperative to facilitate a comprehensive disease analysis, a topic of burgeoning interest among both researchers and clinicians in recent times. Hence, there exists a pressing need to synthesize the latest strides in multi-modal data and AI technologies in the realm of medical diagnosis. In this paper, we narrow our focus to five specific disorders (Alzheimer's disease, breast cancer, depression, heart disease, epilepsy), elucidating advanced endeavors in their diagnosis and treatment through the lens of artificial intelligence. Our survey not only delineates detailed diagnostic methodologies across varying modalities but also underscores commonly utilized public datasets, the intricacies of feature engineering, prevalent classification models, and envisaged challenges for future endeavors. In essence, our research endeavors to contribute to the advancement of diagnostic methodologies, furnishing invaluable insights for clinical decision making.

Predicting the cause of seizures using features extracted from interactions with a virtual agent

OBJECTIVE

A clinical decision tool for Transient Loss of Consciousness (TLOC) could reduce currently high misdiagnosis rates and waiting times for specialist assessments. Most clinical decision tools based on patient-reported symptom inventories only distinguish between two of the three most common causes of TLOC (epilepsy, functional /dissociative seizures, and syncope) or struggle with the particularly challenging differentiation between epilepsy and FDS. Based on previous research describing differences in spoken accounts of epileptic seizures and FDS seizures, this study explored the feasibility of predicting the cause of TLOC by combining the automated analysis of patient-reported symptoms and spoken TLOC descriptions.

METHOD

Participants completed an online web application that consisted of a 34-item medical history and symptom questionnaire (iPEP) and spoken interaction with a virtual agent (VA) that asked eight questions about the most recent experience of TLOC. Support Vector Machines (SVM) were trained using different combinations of features and nested leave-one-out cross validation. The iPEP provided a baseline performance. Inspired by previous qualitative research three spoken language based feature sets were designed to assess: (1) formulation effort, (2) the proportion of words from different semantic categories, and (3) verb, adverb, and adjective usage.

RESULTS

76 participants completed the application (Epilepsy = 24, FDS = 36, syncope = 16). Only 61 participants also completed the VA interaction (Epilepsy = 20, FDS = 29, syncope = 12). The iPEP model accurately predicted 65.8 % of all diagnoses, but the inclusion of the language features increased the accuracy to 85.5 % by improving the differential diagnosis between epilepsy and FDS.

CONCLUSION

These findings suggest that an automated analysis of TLOC descriptions collected using an online web application and VA could improve the accuracy of current clinical decisions tools for TLOC and facilitate clinical stratification processes (such as ensuring appropriate referral to cardiological versus neurological investigation and management pathways).

The promises and pitfalls of seizure phenomenology

The typical adult patient presenting with a first seizure has a normal clinical examination, uninformative investigations, and often has no witness to their episode. The assessing clinician, therefore, has one primary source of information to guide their assessment; the patient's experience. However, seizure phenomenology - the subjective seizure experience - has received relatively less attention by researchers than objective semiology or investigations. This essay reviews the clinical importance of seizure phenomenology, and the challenges clinicians face in eliciting accurate and clinically relevant descriptions of ictal experience. I conclude by discussing tools that clinicians may use to support the clinical application of seizure phenomenology, and exploring the subjectivity of epilepsy more broadly.

Differentiating between epileptic and functional/dissociative seizures using semantic content analysis of transcripts of routine clinic consultations

The common causes of Transient Loss of Consciousness (TLOC) are syncope, epilepsy, and functional/dissociative seizures (FDS). Simple, questionnaire-based decision-making tools for non-specialists who may have to deal with TLOC (such as clinicians working in primary or emergency care) reliably differentiate between patients who have experienced syncope and those who have had one or more seizures but are more limited in their ability to differentiate between epileptic seizures and FDS. Previous conversation analysis research has demonstrated that qualitative expert analysis of how people talk to clinicians about their seizures can help distinguish between these two TLOC causes. This paper investigates whether automated language analysis - using semantic categories measured by the Linguistic Inquiry and Word Count (LIWC) toolkit - can contribute to the distinction between epilepsy and FDS. Using patient-only talk manually transcribed from recordings of 58 routine doctor-patient clinic interactions, we compared the word frequencies for 21 semantic categories and explored the predictive performance of these categories using 5 different machine learning algorithms. Machine learning algorithms trained using the chosen semantic categories and leave-one-out cross-validation were able to predict the diagnosis with an accuracy of up to 81%. The results of this proof of principle study suggest that the analysis of semantic variables in seizure descriptions could improve clinical decision tools for patients presenting with TLOC.

Voices from the past: The pioneering use of the phonograph in neurology

Since its discovery by the American inventor and industrialist Thomas Alva Edison (1847-1931) in 1877, the phonograph attracted much interest in the field of medicine. This article describes the earliest pioneering examples of the use of the phonograph in neurology. In France, the use of the phonograph for obtaining audio recordings of delusions and speech or language disturbances was first proposed by Victor Maurice Dupont (1857-1910) in 1889 and in Italy by the physician Gaetano Rummo (1853-1917), who had studied at La Salpêtrière under Jean-Martin Charcot (1825-1893). The applicability of the phonograph to the record of speech disturbances was illustrated in England by John Hughlings Jackson (1835-1911) and William Halse Rivers (1864-1922), and by William Hale White (1857-1949) and Cuthbert Hilton Golding-Bird (1848-1939) in 1891. Since then, audio recordings have been used rarely in neurology, a branch of medicine where the visual aspects dominate, to the extent that inspection can be enough to reach a definite clinical diagnosis. In the mid-20th century, the advent of audio and video recordings supplanted audio recordings alone, relegating them to a very marginal role.

Transforming epilepsy research: A systematic review on natural language processing applications

Despite improved ancillary investigations in epilepsy care, patients' narratives remain indispensable for diagnosing and treatment monitoring. This wealth of information is typically stored in electronic health records and accumulated in medical journals in an unstructured manner, thereby restricting complete utilization in clinical decision-making. To this end, clinical researchers increasing apply natural language processing (NLP)-a branch of artificial intelligence-as it removes ambiguity, derives context, and imbues standardized meaning from free-narrative clinical texts. This systematic review presents an overview of the current NLP applications in epilepsy and discusses the opportunities and drawbacks of NLP alongside its future implications. We searched the PubMed and Embase databases with a "natural language processing" and "epilepsy" query (March 4, 2022) and included original research articles describing the application of NLP techniques for textual analysis in epilepsy. Twenty-six studies were included. Fifty-eight percent of these studies used NLP to classify clinical records into predefined categories, improving patient identification and treatment decisions. Other applications of NLP had structured clinical information retrieval from electronic health records, scientific papers, and online posts of patients. Challenges and opportunities of NLP applications for enhancing epilepsy care and research are discussed. The field could further benefit from NLP by replicating successes in other health care domains, such as NLP-aided quality evaluation for clinical decision-making, outcome prediction, and clinical record summarization.

The hermeneutics of symptoms

The clinical encounter begins with presentation of an illness experience; but throughout that encounter, something else is constructed from it - a symptom. The symptom is a particular interpretation of that experience, useful for certain purposes in particular contexts. The hermeneutics of medicine - the study of the interpretation of human experience in medical terms - has largely taken the process of symptom-construction to be transparent, focussing instead on how constellations of symptoms are interpreted as representative of particular conditions. This paper examines the hermeneutical activity of symptom-construction more closely. I propose a fourfold account of the clinical function of symptoms: as theoretical entities; as tools for communication; as guides to palliative intervention; and as candidates for medical explanation or intervention. I also highlight roles they might play in illness experience. I use this framework to discuss four potential failures of symptom-interpretation: failure of symptom-type and symptom-token recognition; loss of the complete picture of illness experience through overwhelming emphasis on its symptomatic interpretation; and intersubjective feedback effects of symptom description altering the ill person's own perceptions of their phenomenal experience. I conclude with some suggestions of potential remedies for failures in the process of symptom-construction.

An MRI-Based Clinical-Perfusion Model Predicts Pathological Subtypes of Prevascular Mediastinal Tumors

This study aimed to build machine learning prediction models for predicting pathological subtypes of prevascular mediastinal tumors (PMTs). The candidate predictors were clinical variables and dynamic contrast–enhanced MRI (DCE-MRI)–derived perfusion parameters. The clinical data and preoperative DCE–MRI images of 62 PMT patients, including 17 patients with lymphoma, 31 with thymoma, and 14 with thymic carcinoma, were retrospectively analyzed. Six perfusion parameters were calculated as candidate predictors. Univariate receiver-operating-characteristic curve analysis was performed to evaluate the performance of the prediction models. A predictive model was built based on multi-class classification, which detected lymphoma, thymoma, and thymic carcinoma with sensitivity of 52.9%, 74.2%, and 92.8%, respectively. In addition, two predictive models were built based on binary classification for distinguishing Hodgkin from non-Hodgkin lymphoma and for distinguishing invasive from noninvasive thymoma, with sensitivity of 75% and 71.4%, respectively. In addition to two perfusion parameters (efflux rate constant from tissue extravascular extracellular space into the blood plasma, and extravascular extracellular space volume per unit volume of tissue), age and tumor volume were also essential parameters for predicting PMT subtypes. In conclusion, our machine learning–based predictive model, constructed with clinical data and perfusion parameters, may represent a useful tool for differential diagnosis of PMT subtypes.