Leveraging electronic health records data to predict multiple sclerosis disease activity

Clinical and genetic contributions to medical comorbidity in bipolar disorder: a study using electronic health records-linked biobank data

Bipolar disorder is a chronic and complex polygenic disease with high rates of comorbidity. However, the independent contribution of either diagnosis or genetic risk of bipolar disorder to the medical comorbidity profile of individuals with the disease remains unresolved. Here, we conducted a multi-step phenome-wide association study (PheWAS) of bipolar disorder using phenomes derived from the electronic health records of participants enrolled in the Mayo Clinic Biobank and the Mayo Clinic Bipolar Disorder Biobank. First, we explored the conditions associated with a diagnosis of bipolar disorder by conducting a phenotype-based PheWAS followed by LASSO-penalized regression to account for correlations within the phenome. Then, we explored the conditions associated with bipolar disorder polygenic risk score (BD-PRS) using a PRS-based PheWAS with a sequential exclusion approach to account for the possibility that diagnosis, instead of genetic risk, may drive such associations. 53,386 participants (58.7% women) with a mean age at analysis of 67.8 years (SD = 15.6) were included. A bipolar disorder diagnosis (n = 1479) was associated with higher rates of psychiatric conditions, injuries and poisonings, endocrine/metabolic and neurological conditions, viral hepatitis C, and asthma. BD-PRS was associated with psychiatric comorbidities but, in contrast, had no positive associations with general medical conditions. While our findings warrant confirmation with longitudinal-prospective studies, the limited associations between bipolar disorder genetics and medical conditions suggest that shared environmental effects or environmental consequences of diagnosis may have a greater impact on the general medical comorbidity profile of individuals with bipolar disorder than its genetic risk.

Examining predictors for 6-month mortality and healthcare utilization for patients admitted for heart failure in the acute care setting

BACKGROUND

Acute heart failure (AHF) is a leading cause of mortality and hospitalization. Past studies reported increased healthcare spending in the last year of life in high-income countries, and this has been characterized as inappropriate healthcare resource utilization. The study aimed to examine potentially (in)appropriate healthcare utilization by comparing healthcare utilization patterns across predicted and observed 6-month mortality among patients admitted for HF.

METHODS

We conducted a retrospective cohort study among patients presenting at the emergency department (ED) of a tertiary hospital with HF as primary diagnosis and admitted after their ED discharge. We used LASSO Cox proportional hazards models to predict 6-month mortality, and estimated healthcare utilization patterns of predicted and observed mortality across inpatient healthcare services.

RESULTS

3946 patients were admitted into the emergency department with a primary diagnosis of HF. From 57 candidate variables, 17 were retained in the final 6- month mortality model (C-statistic 0.66). Patients who died within 6-months of ED admission had longer length of stay (LOS) and less inpatient surgeries than those who survived. Patients with a greater predicted mortality risk were admitted to the ICU more often and had a longer LOS than those with a lower predicted mortality risk.

CONCLUSIONS

There were significant differences in healthcare resource utilization in patients admitted for AHF across predicted versus actual mortality. Lack of information on patients' preferences prevents the estimation of (in)appropriateness. Future studies should account for these considerations to estimate inappropriate healthcare utilization among these patients.

Association between serum multi-protein biomarker profile and real-world disability in multiple sclerosis

Few studies examined blood biomarkers informative of patient-reported outcome (PRO) of disability in people with multiple sclerosis (MS). We examined the associations between serum multi-protein biomarker profiles and patient-reported MS disability. In this cross-sectional study (2017-2020), adults with diagnosis of MS (or precursors) from two independent clinic-based cohorts were divided into a training and test set. For predictors, we examined seven clinical factors (age at sample collection, sex, race/ethnicity, disease subtype, disease duration, disease-modifying therapy [DMT], and time interval between sample collection and closest PRO assessment) and 19 serum protein biomarkers potentially associated with MS disease activity endpoints identified from prior studies. We trained machine learning (ML) models (Least Absolute Shrinkage and Selection Operator regression [LASSO], Random Forest, Extreme Gradient Boosting, Support Vector Machines, stacking ensemble learning, and stacking classification) for predicting Patient Determined Disease Steps (PDDS) score as the primary endpoint and reported model performance using the held-out test set. The study included 431 participants (mean age 49 years, 81% women, 94% non-Hispanic White). For binary PDDS score, combined feature input of routine clinical factors and the 19 proteins consistently outperformed base models (comprising clinical features alone or clinical features plus one single protein at a time) in predicting severe (PDDS ≥ 4) versus mild/moderate (PDDS < 4) disability across multiple machine learning approaches, with LASSO achieving the best area under the curve (AUCPDDS = 0.91) and other metrics. For ordinal PDDS score, LASSO model comprising combined clinical factors and 19 proteins as feature input (R2PDDS = 0.31) again outperformed base models. The two best-performing LASSO models (i.e., binary and ordinal PDDS score) shared six clinical features (age, sex, race/ethnicity, disease subtype, disease duration, DMT efficacy) and nine proteins (cluster of differentiation 6, CUB-domain-containing protein 1, contactin-2, interleukin-12 subunit-beta, neurofilament light chain [NfL], protogenin, serpin family A member 9, tumor necrosis factor superfamily member 13B, versican). By comparison, LASSO models with clinical features plus one single protein at a time as feature input did not select either NfL or glial fibrillary acidic protein (GFAP) as a final feature. Forcing either NfL or GFAP as a single protein feature into models did not improve performance beyond clinical features alone. Stacking classification model using five functional pathways to represent multiple proteins as meta-features implicated those involved in neuroaxonal integrity as significant contributors to predictive performance. Thus, serum multi-protein biomarker profiles improve the prediction of real-world MS disability status beyond clinical profile alone or clinical profile plus single protein biomarker, reaching clinically actionable performance.

Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis

BACKGROUND

Multiple sclerosis (MS) is a chronic inflammatory disease of the central nervous system that affects millions of people worldwide. The disease course varies greatly across individuals and many disease-modifying treatments with different safety and efficacy profiles have been developed recently. Prognostic models evaluated and shown to be valid in different settings have the potential to support people with MS and their physicians during the decision-making process for treatment or disease/life management, allow stratified and more precise interpretation of interventional trials, and provide insights into disease mechanisms. Many researchers have turned to prognostic models to help predict clinical outcomes in people with MS; however, to our knowledge, no widely accepted prognostic model for MS is being used in clinical practice yet.

OBJECTIVES

To identify and summarise multivariable prognostic models, and their validation studies for quantifying the risk of clinical disease progression, worsening, and activity in adults with MS.

SEARCH METHODS

We searched MEDLINE, Embase, and the Cochrane Database of Systematic Reviews from January 1996 until July 2021. We also screened the reference lists of included studies and relevant reviews, and references citing the included studies.

SELECTION CRITERIA

We included all statistically developed multivariable prognostic models aiming to predict clinical disease progression, worsening, and activity, as measured by disability, relapse, conversion to definite MS, conversion to progressive MS, or a composite of these in adult individuals with MS. We also included any studies evaluating the performance of (i.e. validating) these models. There were no restrictions based on language, data source, timing of prognostication, or timing of outcome.

DATA COLLECTION AND ANALYSIS

Pairs of review authors independently screened titles/abstracts and full texts, extracted data using a piloted form based on the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS), assessed risk of bias using the Prediction Model Risk Of Bias Assessment Tool (PROBAST), and assessed reporting deficiencies based on the checklist items in Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD). The characteristics of the included models and their validations are described narratively. We planned to meta-analyse the discrimination and calibration of models with at least three external validations outside the model development study but no model met this criterion. We summarised between-study heterogeneity narratively but again could not perform the planned meta-regression.

MAIN RESULTS

We included 57 studies, from which we identified 75 model developments, 15 external validations corresponding to only 12 (16%) of the models, and six author-reported validations. Only two models were externally validated multiple times. None of the identified external validations were performed by researchers independent of those that developed the model. The outcome was related to disease progression in 39 (41%), relapses in 8 (8%), conversion to definite MS in 17 (18%), and conversion to progressive MS in 27 (28%) of the 96 models or validations. The disease and treatment-related characteristics of included participants, and definitions of considered predictors and outcome, were highly heterogeneous amongst the studies. Based on the publication year, we observed an increase in the percent of participants on treatment, diversification of the diagnostic criteria used, an increase in consideration of biomarkers or treatment as predictors, and increased use of machine learning methods over time. Usability and reproducibility All identified models contained at least one predictor requiring the skills of a medical specialist for measurement or assessment. Most of the models (44; 59%) contained predictors that require specialist equipment likely to be absent from primary care or standard hospital settings. Over half (52%) of the developed models were not accompanied by model coefficients, tools, or instructions, which hinders their application, independent validation or reproduction. The data used in model developments were made publicly available or reported to be available on request only in a few studies (two and six, respectively). Risk of bias We rated all but one of the model developments or validations as having high overall risk of bias. The main reason for this was the statistical methods used for the development or evaluation of prognostic models; we rated all but two of the included model developments or validations as having high risk of bias in the analysis domain. None of the model developments that were externally validated or these models' external validations had low risk of bias. There were concerns related to applicability of the models to our research question in over one-third (38%) of the models or their validations. Reporting deficiencies Reporting was poor overall and there was no observable increase in the quality of reporting over time. The items that were unclearly reported or not reported at all for most of the included models or validations were related to sample size justification, blinding of outcome assessors, details of the full model or how to obtain predictions from it, amount of missing data, and treatments received by the participants. Reporting of preferred model performance measures of discrimination and calibration was suboptimal.

AUTHORS' CONCLUSIONS

The current evidence is not sufficient for recommending the use of any of the published prognostic prediction models for people with MS in clinical routine today due to lack of independent external validations. The MS prognostic research community should adhere to the current reporting and methodological guidelines and conduct many more state-of-the-art external validation studies for the existing or newly developed models.

Chinese medical event detection based on event frequency distribution ratio and document consistency

Structured information especially medical events extracted from electronic medical records has extremely practical application value and play a basic role in various intelligent diagnosis and treatment systems. Fine-grained Chinese medical event detection is crucial in the process of structuring Chinese Electronic Medical Record (EMR). The current methods for detecting fine-grained Chinese medical events primarily rely on statistical machine learning and deep learning. However, they have two shortcomings: 1) they neglect to take into account the distribution characteristics of these fine-grained medical events. 2) they overlook the consistency in the distribution of medical events within each individual document. Therefore, this paper presents a fine-grained Chinese medical event detection method, which is based on event frequency distribution ratio and document consistency. To start with, a significant number of Chinese EMR texts are used to adapt the Chinese pre-training model BERT to the domain. Second, based on the fundamental features, the Event Frequency - Event Distribution Ratio (EF-DR) is devised to select distinct event information as supplementary features, taking into account the distribution of events within the EMR. Finally, using EMR document consistency within the model improves the outcome of event detection. Our experiments demonstrate that the proposed method significantly outperforms the baseline model.

Integrating Rehabilomics into the Multi-Omics Approach in the Management of Multiple Sclerosis: The Way for Precision Medicine?

Over recent years, significant improvements have been made in the understanding of (epi)genetics and neuropathophysiological mechanisms driving the different forms of multiple sclerosis (MS). For example, the role and importance of the bidirectional communications between the brain and the gut-also referred to as the gut-brain axis-in the pathogenesis of MS is receiving increasing interest in recent years and is probably one of the most promising areas of research for the management of people with MS. However, despite these important advances, it must be noted that these data are not-yet-used in rehabilitation. Neurorehabilitation is a cornerstone of MS patient management, and there are many techniques available to clinicians and patients, including technology-supported rehabilitation. In this paper, we will discuss how new findings on the gut microbiome could help us to better understand how rehabilitation can improve motor and cognitive functions. We will also see how the data gathered during the rehabilitation can help to get a better diagnosis of the patients. Finally, we will discuss how these new techniques can better guide rehabilitation to lead to precision rehabilitation and ultimately increase the quality of patient care.

Validation of a machine learning approach to estimate expanded disability status scale scores for multiple sclerosis

Background

Disability assessment using the Expanded Disability Status Scale (EDSS) is important to inform treatment decisions and monitor the progression of multiple sclerosis. Yet, EDSS scores are documented infrequently in electronic medical records.

Objective

To validate a machine learning model to estimate EDSS scores for multiple sclerosis patients using clinical notes from neurologists.

Methods

A machine learning model was developed to estimate EDSS scores on specific encounter dates using clinical notes from neurologist visits. The OM1 MS Registry data were used to create a training cohort of 2632 encounters and a separate validation cohort of 857 encounters, all with clinician-recorded EDSS scores. Model performance was assessed using the area under the receiver-operating-characteristic curve (AUC), positive predictive value (PPV), and negative predictive value (NPV), calculated using a binarized version of the outcome. The Spearman R and Pearson R values were calculated. The model was then applied to encounters without clinician-recorded EDSS scores in the MS Registry.

Results

The model had a PPV of 0.85, NPV of 0.85, and AUC of 0.91. The model had a Spearman R value of 0.75 and Pearson R value of 0.74 when evaluating performance using the continuous estimated EDSS and clinician-recorded EDSS scores. Application of the model to eligible encounters resulted in the generation of eEDSS scores for an additional 190,282 encounters from 13,249 patients.

Conclusion

EDSS scores can be estimated with very good performance using a machine learning model applied to clinical notes, thus increasing the utility of real-world data sources for research purposes.

Clinical and Research Applications of the Electronic Medical Record in Multiple Sclerosis: A Narrative Review of Current Uses and Future Applications

BACKGROUND

The electronic medical record (EMR) has revolutionized health care workflow and delivery. It has evolved from a clinical adjunct to a multifaceted tool, with uses relevant to patient care and research.

METHODS

A MEDLINE literature review was conducted to identify data regarding the use of EMR for multiple sclerosis (MS) clinical care and research.

RESULTS

Of 282 relevant articles identified, 29 were included. A variety of EMR integrated platforms with features specific to MS have been designed, with options for documenting disease course, disability status, and treatment. Research efforts have focused on early diagnosis identification, relapse prediction, and surrogates for disability status.

CONCLUSIONS

The available platforms and associated research support the utility of harnessing EMR for MS care. The adoption of a core set of discrete EMR elements should be considered to support future research efforts and the ability to harmonize data across institutions.

Toward attention-based learning to predict the risk of brain degeneration with multimodal medical data

Introduction

Brain degeneration is commonly caused by some chronic diseases, such as Alzheimer's disease (AD) and diabetes mellitus (DM). The risk prediction of brain degeneration aims to forecast the situation of disease progression of patients in the near future based on their historical health records. It is beneficial for patients to make an accurate clinical diagnosis and early prevention of disease. Current risk predictions of brain degeneration mainly rely on single-modality medical data, such as Electronic Health Records (EHR) or magnetic resonance imaging (MRI). However, only leveraging EHR or MRI data for the pertinent and accurate prediction is insufficient because of single-modality information (e.g., pixel or volume information of image data or clinical context information of non-image data).

Methods

Several deep learning-based methods have used multimodal data to predict the risks of specified diseases. However, most of them simply integrate different modalities in an early, intermediate, or late fusion structure and do not care about the intra-modal and intermodal dependencies. A lack of these dependencies would lead to sub-optimal prediction performance. Thus, we propose an encoder-decoder framework for better risk prediction of brain degeneration by using MRI and EHR. An encoder module is one of the key components and mainly focuses on feature extraction of input data. Specifically, we introduce an encoder module, which integrates intra-modal and inter-modal dependencies with the spatial-temporal attention and cross-attention mechanism. The corresponding decoder module is another key component and mainly parses the features from the encoder. In the decoder module, a disease-oriented module is used to extract the most relevant disease representation features. We take advantage of a multi-head attention module followed by a fully connected layer to produce the predicted results.

Results

As different types of AD and DM influence the nature and severity of brain degeneration, we evaluate the proposed method for three-class prediction of AD and three-class prediction of DM. Our results show that the proposed method with integrated MRI and EHR data achieves an accuracy of 0.859 and 0.899 for the risk prediction of AD and DM, respectively.

Discussion

The prediction performance is significantly better than the benchmarks, including MRI-only, EHR-only, and state-of-the-art multimodal fusion methods.

Temporal trends of multiple sclerosis disease activity: Electronic health records indicators

BACKGROUND

Long-term data on multiple sclerosis (MS) inflammatory disease activity are limited. We examined electronic health records (EHR) indicators of disease activity in people with MS.

METHODS

We analyzed prospectively collected research registry data and linked EHR data in a clinic-based cohort from 2000 to 2016. We used the trend of the yearly incident relapse rate from the registry data as benchmark. We then calculated the temporal trends of potentially relevant EHR measures, including mean count of the MS diagnostic code, mentions of MS-related concepts, MS-related health utilizations and selected prescriptions.

RESULTS

1,555 MS patients had both registry and EHR data. Between 2000 and 2016, the registry data showed a declining trend in the yearly incident relapse rate, parallel to an increasing trend of DMT usage. Among the EHR measures, covariate-adjusted frequency of diagnostic code of MS, procedure codes of MS-related imaging studies and emergency room visits, and electronic prescription for steroids declined over time, mirroring the temporal trend of the benchmark yearly incident relapse rate.

CONCLUSION

This study highlights EHR indicators of MS relapse that could enable large-scale examination of long-term disease activities or inform individual patient monitoring in clinical settings where EHR data are available.

Natural language processing in clinical neuroscience and psychiatry: A review

Natural language processing (NLP) is rapidly becoming an important topic in the medical community. The ability to automatically analyze any type of medical document could be the key factor to fully exploit the data it contains. Cutting-edge artificial intelligence (AI) architectures, particularly machine learning and deep learning, have begun to be applied to this topic and have yielded promising results. We conducted a literature search for 1,024 papers that used NLP technology in neuroscience and psychiatry from 2010 to early 2022. After a selection process, 115 papers were evaluated. Each publication was classified into one of three categories: information extraction, classification, and data inference. Automated understanding of clinical reports in electronic health records has the potential to improve healthcare delivery. Overall, the performance of NLP applications is high, with an average F1-score and AUC above 85%. We also derived a composite measure in the form of Z-scores to better compare the performance of NLP models and their different classes as a whole. No statistical differences were found in the unbiased comparison. Strong asymmetry between English and non-English models, difficulty in obtaining high-quality annotated data, and train biases causing low generalizability are the main limitations. This review suggests that NLP could be an effective tool to help clinicians gain insights from medical reports, clinical research forms, and more, making NLP an effective tool to improve the quality of healthcare services.

Comparison of Dimethyl Fumarate vs Fingolimod and Rituximab vs Natalizumab for Treatment of Multiple Sclerosis

Importance

As disease-modifying treatment options for multiple sclerosis increase, comparisons of the options based on real-world evidence may guide clinical decision-making.

Objective

To compare the relapse outcomes between 2 pairs of disease-modifying treatments: dimethyl fumarate vs fingolimod and natalizumab vs rituximab.

Design, Setting, and Participants

This comparative effectiveness study integrated data from a clinic-based multiple sclerosis research registry and its linked electronic health records (EHR) system between January 1, 2006, and December 31, 2016, and built treatment groups for each pairwise disease-modifying treatment comparison according to both registry records and electronic prescriptions. Parallel analyses were conducted from October 11, 2019, to July 7, 2021.

Main Outcomes and Measures

The main outcomes were the 1-year and 2-year relapse rates as well as the time to relapse. To compare relapse outcomes, the study adjusted for covariates from 2 sources (registry and EHR) and corrected for confounding biases among the covariates by the doubly robust estimation.

Results

The study included 4 treatment groups: dimethyl fumarate (n = 260; 198 women [76.2%]; 227 non-Hispanic White individuals [87.3%]; mean [SD] age at diagnosis, 41.7 [10.4] years), fingolimod (n = 267; 190 women [71.2%]; 222 non-Hispanic White individuals [83.1%]; mean [SD] age at diagnosis, 37.9 [9.9] years), natalizumab (n = 204; 160 women [78.4%]; 172 non-Hispanic White individuals [84.3%]; mean [SD] age at diagnosis, 37.2 [10.6] years), and rituximab (n = 115; 83 women [72.2%]; 99 non-Hispanic White individuals [86.1%]; mean [SD] age at diagnosis, 44.1 [11.1] years). No significant differences were found in the relapse outcomes between dimethyl fumarate and fingolimod after correcting for confounding biases and multiple testing (difference in 1-year relapse rate, 0.028 [95% CI, -0.031 to 0.084]; difference in 2-year relapse rate, 0.071 [95% CI, 0.008-0.128]; relative risk of 2-year non-relapse, 0.957 [95% CI, 0.884-1.035] with dimethyl fumarate as reference). When compared with rituximab, natalizumab was associated with a higher relapse rate for all 3 outcomes after bias correction and multiple testing (difference in 1-year relapse rate, 0.080 [95% CI, 0.013-0.137]; difference in 2-year relapse rate, 0.132 [95% CI, 0.043-0.189]; relative risk of 2-year non-relapse, 0.903 [95% CI, 0.822-0.944]). Confounders were identified from EHR data not recorded in the registry data through data-driven feature selection.

Conclusions and Relevance

This study reports real-world evidence of equivalent relapse outcomes between dimethyl fumarate and fingolimod and relapse reduction in favor of rituximab relative to natalizumab. This approach illustrates the value of incorporating EHR data as high-dimensional covariates in real-world treatment comparison.

The Application of Projection Word Embeddings on Medical Records Scoring System

Medical records scoring is important in a health care system. Artificial intelligence (AI) with projection word embeddings has been validated in its performance disease coding tasks, which maintain the vocabulary diversity of open internet databases and the medical terminology understanding of electronic health records (EHRs). We considered that an AI-enhanced system might be also applied to automatically score medical records. This study aimed to develop a series of deep learning models (DLMs) and validated their performance in medical records scoring task. We also analyzed the practical value of the best model. We used the admission medical records from the Tri-Services General Hospital during January 2016 to May 2020, which were scored by our visiting staffs with different levels from different departments. The medical records were scored ranged 0 to 10. All samples were divided into a training set (n = 74,959) and testing set (n = 152,730) based on time, which were used to train and validate the DLMs, respectively. The mean absolute error (MAE) was used to evaluate each DLM performance. In original AI medical record scoring, the predicted score by BERT architecture is closer to the actual reviewer score than the projection word embedding and LSTM architecture. The original MAE is 0.84 ± 0.27 using the BERT model, and the MAE is 1.00 ± 0.32 using the LSTM model. Linear mixed model can be used to improve the model performance, and the adjusted predicted score was closer compared to the original score. However, the project word embedding with the LSTM model (0.66 ± 0.39) provided better performance compared to BERT (0.70 ± 0.33) after linear mixed model enhancement (p < 0.001). In addition to comparing different architectures to score the medical records, this study further uses a mixed linear model to successfully adjust the AI medical record score to make it closer to the actual physician's score.