Drug safety surveillance using de-identified EMR and claims data: issues and challenges

Enhancing Precision in Detecting Severe Immune-Related Adverse Events: Comparative Analysis of Large Language Models and International Classification of Disease Codes in Patient Records

PURPOSE

Current approaches to accurately identify immune-related adverse events (irAEs) in large retrospective studies are limited. Large language models (LLMs) offer a potential solution to this challenge, given their high performance in natural language comprehension tasks. Therefore, we investigated the use of an LLM to identify irAEs among hospitalized patients, comparing its performance with manual adjudication and International Classification of Disease (ICD) codes.

METHODS

Hospital admissions of patients receiving immune checkpoint inhibitor (ICI) therapy at a single institution from February 5, 2011, to September 5, 2023, were individually reviewed and adjudicated for the presence of irAEs. ICD codes and an LLM with retrieval-augmented generation were applied to detect frequent irAEs (ICI-induced colitis, hepatitis, and pneumonitis) and the most fatal irAE (ICI-myocarditis) from electronic health records. The performance between ICD codes and LLM was compared via sensitivity and specificity with an α = .05, relative to the gold standard of manual adjudication. External validation was performed using a data set of hospital admissions from June 1, 2018, to May 31, 2019, from a second institution.

RESULTS

Of the 7,555 admissions for patients on ICI therapy in the initial cohort, 2.0% were adjudicated to be due to ICI-colitis, 1.1% ICI-hepatitis, 0.7% ICI-pneumonitis, and 0.8% ICI-myocarditis. The LLM demonstrated higher sensitivity than ICD codes (94.7% v 68.7%), achieving significance for ICI-hepatitis (P < .001), myocarditis (P < .001), and pneumonitis (P = .003) while yielding similar specificities (93.7% v 92.4%). The LLM spent an average of 9.53 seconds/chart in comparison with an estimated 15 minutes for adjudication. In the validation cohort (N = 1,270), the mean LLM sensitivity and specificity were 98.1% and 95.7%, respectively.

CONCLUSION

LLMs are a useful tool for the detection of irAEs, outperforming ICD codes in sensitivity and adjudication in efficiency.

A Comparison of Active Pharmacovigilance Strategies Used to Monitor Adverse Events to Antiviral Agents: A Systematic Review

INTRODUCTION

The safety of antiviral agents in real-world clinical settings is crucial, as pre-marketing studies often do not capture all adverse events (AE). Active pharmacovigilance strategies are essential for detecting and characterising these AE comprehensively.

OBJECTIVE

The aim of this study was to identify and characterise active pharmacovigilance strategies used in real-world clinical settings for patients under systemic antiviral agents, focusing on the frequency of AE and the clinical data sources used.

METHODS

We conducted a systematic review by searching three electronic bibliographic databases targeting observational prospective active pharmacovigilance studies, phase IV clinical trials for post-marketing safety surveillance, and interventional studies assessing active pharmacovigilance strategies, focusing on individuals exposed to systemic antiviral agents.

RESULTS

We included 36 primary studies, predominantly using Drug Event Monitoring (DEM), with a minority employing sentinel sites and registries. Human immunodeficiency virus (HIV) was the most common condition, with the majority using DEM. Within the DEM, there was a wide range of incidences of patients experiencing at least one AE, and most of these studies used one or two data sources. Sentinel site studies were less common, with two on hepatitis C virus (HCV) and one on HIV, each relying on one or two data sources. The single study using a registry focusing on HIV therapy reported using just one data source. Patient interviews were the most common data source, followed by medical records and laboratory tests. The quality of the studies was considered 'good' in 18/36, 'fair' in 1/36, and 'poor' in 17/36 studies.

CONCLUSION

DEM was the predominant pharmacovigilance strategy, employing multiple data sources, and appears to increase the likelihood of detecting higher AE incidence. Establishing such a framework would facilitate a more detailed and consistent approach across different studies and settings.

Adverse drug reactions in hospitals: population estimates for Portugal and the ICD-9-CM to ICD-10-CM crosswalk

BACKGROUND

Adverse drug reactions (ADR), both preventable and non-preventable, are frequent and pose a significant burden. This study aimed to produce up-to-date estimates for ADR rates in hospitals, in Portugal, from 2010 to 2018. In addition, it explores possible pitfalls when crosswalking between ICD-9-CM and ICD-10-CM code sets for ADR identification.

METHODS

The Portuguese Hospital Morbidity Database was used to identify hospital episodes (outpatient or inpatient) with at least one ICD code of ADR. Since the study period spanned from 2010 to 2018, both ICD-9-CM and ICD-10-CM codes based on previously published studies were used to define episodes. This was an exploratory study, and descriptive statistics were used to provide ADR rates and summarise episode features for the full period (2010-2018) as well as for the ICD-9-CM (2010-2016) and ICD -10-CM (2017-2018) eras.

RESULTS

Between 2010 and 2018, ADR occurred in 162,985 hospital episodes, corresponding to 1.00% of the total number of episodes during the same period. Higher rates were seen in the oldest age groups. In the same period, the mean annual rate of episodes related to ADR was 174.2/100,000 population. The episode rate (per 100,000 population) was generally higher in males, except in young adults (aged '15-20', '25-30' and '30-35' years), although the overall frequency of ADR in hospital episodes was higher in females.

CONCLUSIONS

Despite the ICD-10-CM transition, administrative health data in Portugal remain a feasible source for producing up-to-date estimates on ADR in hospitals. There is a need for future research to identify target recipients for preventive interventions and improve medication safety practices in Portugal.

Sustained Effect of Clinical Decision Support for Heart Failure: A Natural Experiment Using Implementation Science

OBJECTIVES

In a randomized controlled trial, we found that applying implementation science (IS) methods and best practices in clinical decision support (CDS) design to create a locally customized, "enhanced" CDS significantly improved evidence-based prescribing of β blockers (BB) for heart failure compared with an unmodified commercially available CDS. At trial conclusion, the enhanced CDS was expanded to all sites. The purpose of this study was to evaluate the real-world sustained effect of the enhanced CDS compared with the commercial CDS.

METHODS

In this natural experiment of 28 primary care clinics, we compared clinics exposed to the commercial CDS (preperiod) to clinics exposed to the enhanced CDS (both periods). The primary effectiveness outcome was the proportion of alerts resulting in a BB prescription. Secondary outcomes included patient reach and clinician adoption (dismissals).

RESULTS

There were 367 alerts for 183 unique patients and 171 unique clinicians (pre: March 2019-August 2019; post: October 2019-March 2020). The enhanced CDS increased prescribing by 26.1% compared with the commercial (95% confidence interval [CI]: 17.0-35.1%), which is consistent with the 24% increase in the previous study. The odds of adopting the enhanced CDS was 81% compared with 29% with the commercial (odds ratio: 4.17, 95% CI: 1.96-8.85). The enhanced CDS adoption and effectiveness rates were 62 and 14% in the preperiod and 92 and 10% in the postperiod.

CONCLUSION

Applying IS methods with CDS best practices was associated with improved and sustained clinician adoption and effectiveness compared with a commercially available CDS tool.

Explainable detection of adverse drug reaction with imbalanced data distribution

Analysis of health-related texts can be used to detect adverse drug reactions (ADR). The greatest challenge for ADR detection lies in imbalanced data distributions where words related to ADR symptoms are often minority classes. As a result, trained models tend to converge to a point that strongly biases towards the majority class and then ignores the minority class. Since the most used cross-entropy criteria is an approximation to accuracy, the model focuses more readily on the majority class to achieve high accuracy. To address this issue, existing methods apply either oversampling or down-sampling strategies to balance the data distribution and exploit the most difficult samples of the minority class. However, increasing or reducing the number of individual tokens alone in sequence labeling tasks will result in the loss of the syntactic relations of the sentence. This paper proposes a weighted variant of conditional random field (CRF) for data-imbalanced sequence labeling tasks. Such a weighting strategy can alleviate data distribution imbalances between majority and minority classes. Instead of using softmax in the output layer, the CRF can capture the relationship of labels between tokens. The locally interpretable model-agnostic explanations (LIME) algorithm was applied to investigate performance differences between models with and without the weighted loss function. Experimental results on two different ADR tasks show that the proposed model outperforms previously proposed sequence labeling methods.

A weakly supervised model for the automated detection of adverse events using clinical notes

With clinical trials unable to detect all potential adverse reactions to drugs and medical devices prior to their release into the market, accurate post-market surveillance is critical to ensure their safety and efficacy. Electronic health records (EHR) contain rich observational patient data, making them a valuable source to actively monitor the safety of drugs and devices. While structured EHR data and spontaneous reporting systems often underreport the complexities of patient encounters and outcomes, free-text clinical notes offer greater detail about a patient's status. Previous studies have proposed machine learning methods to detect adverse events from clinical notes, but suffer from manually extracted features, reliance on costly hand-labeled data, and lack of validation on external datasets. To address these challenges, we develop a weakly-supervised machine learning framework for adverse event detection from unstructured clinical notes and evaluate it on insulin pump failure as a test case. Our model accurately detected cases of pump failure with 0.842 PR AUC on the holdout test set and 0.815 PR AUC when validated on an external dataset. Our approach allowed us to leverage a large dataset with far less hand-labeled data and can be easily transferred to additional adverse events for scalable post-market surveillance.

Comparative assessment of manual chart review and ICD claims data in evaluating immunotherapy-related adverse events

BACKGROUND

The aim of this retrospective study was to demonstrate that irAEs, specifically gastrointestinal and pulmonary, examined through International Classification of Disease (ICD) data leads to underrepresentation of true irAEs and overrepresentation of false irAEs, thereby concluding that ICD claims data are a poor approach to electronic health record (EHR) data mining for irAEs in immunotherapy clinical research.

METHODS

This retrospective analysis was conducted in 1,063 cancer patients who received ICIs between 2011 and 2017. We identified irAEs by manual review of medical records to determine the incidence of each of our endpoints, namely colitis, hepatitis, pneumonitis, other irAE, or no irAE. We then performed a secondary analysis utilizing ICD claims data alone using a broad range of symptom and disease-specific ICD codes representative of irAEs.

RESULTS

16% (n = 174/1,063) of the total study population was initially found to have either pneumonitis 3% (n = 37), colitis 7% (n = 81) or hepatitis 5% (n = 56) on manual review. Of these patients, 46% (n = 80/174) did not have ICD code evidence in the EHR reflecting their irAE. Of the total patients not found to have any irAEs during manual review, 61% (n = 459/748) of patients had ICD codes suggestive of possible irAE, yet were not identified as having an irAE during manual review.

DISCUSSION

Examining gastrointestinal and pulmonary irAEs through the International Classification of Disease (ICD) data leads to underrepresentation of true irAEs and overrepresentation of false irAEs.

Real-world data-based adverse drug reactions detection from the Korea Adverse Event Reporting System databases with electronic health records-based detection algorithm

Pharmacovigilance involves monitoring of drugs and their adverse drug reactions (ADRs) and is essential for their safety post-marketing. Because of the different types and structures of medical databases, several previous surveillance studies have analyzed only one database. In the present study, we extracted potential drug-ADR pairs from electronic health record (EHR) data using the MetaNurse algorithm and analyzed them using the Korean Adverse Event Reporting System (KAERS) database for systematic validation. The Medical Dictionary for Regulatory Activities (MedDRA) and World Health Organization (WHO) Adverse Reactions Terminology (WHO-ART) were mapped for signal detection. We used the Side Effect Resource (SIDER) database to select 2663 drug-ADR pairs to investigate unknown drug-induced ADRs. The reporting odds ratio (ROR) value was calculated for the drug-exposed and non-exposed groups of drug-ADR pairs, and 19 potential pairs showed significant signals. Appropriate terminology systems and criteria are needed to handle diverse medical databases.

Adverse drug event rates in pediatric pulmonary hypertension: a comparison of real-world data sources

Objective

Real-world data (RWD) are increasingly used for pharmacoepidemiology and regulatory innovation. Our objective was to compare adverse drug event (ADE) rates determined from two RWD sources, electronic health records and administrative claims data, among children treated with drugs for pulmonary hypertension.

Materials and Methods

Textual mentions of medications and signs/symptoms that may represent ADEs were identified in clinical notes using natural language processing. Diagnostic codes for the same signs/symptoms were identified in our electronic data warehouse for the patients with textual evidence of taking pulmonary hypertension-targeted drugs. We compared rates of ADEs identified in clinical notes to those identified from diagnostic code data. In addition, we compared putative ADE rates from clinical notes to those from a healthcare claims dataset from a large, national insurer.

Results

Analysis of clinical notes identified up to 7-fold higher ADE rates than those ascertained from diagnostic codes. However, certain ADEs (eg, hearing loss) were more often identified in diagnostic code data. Similar results were found when ADE rates ascertained from clinical notes and national claims data were compared.

Discussion

While administrative claims and clinical notes are both increasingly used for RWD-based pharmacovigilance, ADE rates substantially differ depending on data source.

Conclusion

Pharmacovigilance based on RWD may lead to discrepant results depending on the data source analyzed. Further work is needed to confirm the validity of identified ADEs, to distinguish them from disease effects, and to understand tradeoffs in sensitivity and specificity between data sources.

A study of deep learning approaches for medication and adverse drug event extraction from clinical text

OBJECTIVE

This article presents our approaches to extraction of medications and associated adverse drug events (ADEs) from clinical documents, which is the second track of the 2018 National NLP Clinical Challenges (n2c2) shared task.

MATERIALS AND METHODS

The clinical corpus used in this study was from the MIMIC-III database and the organizers annotated 303 documents for training and 202 for testing. Our system consists of 2 components: a named entity recognition (NER) and a relation classification (RC) component. For each component, we implemented deep learning-based approaches (eg, BI-LSTM-CRF) and compared them with traditional machine learning approaches, namely, conditional random fields for NER and support vector machines for RC, respectively. In addition, we developed a deep learning-based joint model that recognizes ADEs and their relations to medications in 1 step using a sequence labeling approach. To further improve the performance, we also investigated different ensemble approaches to generating optimal performance by combining outputs from multiple approaches.

RESULTS

Our best-performing systems achieved F1 scores of 93.45% for NER, 96.30% for RC, and 89.05% for end-to-end evaluation, which ranked #2, #1, and #1 among all participants, respectively. Additional evaluations show that the deep learning-based approaches did outperform traditional machine learning algorithms in both NER and RC. The joint model that simultaneously recognizes ADEs and their relations to medications also achieved the best performance on RC, indicating its promise for relation extraction.

CONCLUSION

In this study, we developed deep learning approaches for extracting medications and their attributes such as ADEs, and demonstrated its superior performance compared with traditional machine learning algorithms, indicating its uses in broader NER and RC tasks in the medical domain.

Estimation of sodium-glucose cotransporter 2 inhibitor-related genital and urinary tract infections via electronic medical record-based common data model

WHAT IS KNOWN AND OBJECTIVES

In Korea, the side effects of sodium-glucose cotransporter 2 inhibitors (SGLT2i) have not been clearly reported, aside from voluntary reporting. We aimed to develop detection algorithms for SGLT2i-related genital tract infections (GTIs) and urinary tract infections (UTIs) via a common data model (CDM), an electronic medical record-based database for supporting multi-hospital clinical research. We estimated the occurrence of GTIs and UTIs and-by assessing the status of each step of the algorithm-we also aimed to determine how clinicians responded to the SGLT2i-related GTIs and UTIs.

METHODS

We targeted all patients who were prescribed SGLT2i at Catholic University Seoul St. Mary's Hospital and Hallym University Dongtan Sacred Heart Hospital from January 2014 to August 2018. We developed algorithms for detection of SGLT2i-related GTIs or UTIs that divided patients into "most likely," "possibly" or "less likely" categories of GTIs or UTIs. The numbers of patients at each step were extracted.

RESULTS AND DISCUSSION

A total of 4253 patients received their first prescription of SGLT2i. According to the algorithm used in this study, the proportions of "most likely GTI" and "possibly GTI" were 0.9% (37 out of 4253) and 19.4% (826 out of 4253 patients), respectively. Similarly, the proportions of "most likely UTI" and "possibly UTI" were 0.9% (38 out of 4253) and 20.2% (858 out of 4253 patients), respectively. Compared to the various existing prospective studies, both GTIs and UTIs showed lower occurrence among patients who met "most likely" criteria and higher occurrence among those who met "possibly" criteria. When a GTI or UTI occurred or was suspected, the overall rate of discontinuing SGLT2i was 51.8% (1721 out of 3323). Despite a confirmed or suspected GTI and an UTI, 62.8% (1460 out of 2323) and 14.2% (142 out of 1000) of patients continued to take SGLT2i, respectively. The discontinuation rate for suspected GTIs was significantly lower than that for suspected UTIs (37.2% vs. 85.8%, p < 0.001).

WHAT IS NEW AND CONCLUSION

In this study, although the GTIs appeared to have a similar occurrence as UTIs, however, the discontinuation rate of SGLT2i for suspected GTIs was relatively lower. Our study is novel in that we identified how the physicians approached SGLT2i-related GTIs or UTIs at each step in a real-world clinical practice setting. Although we could estimate SGLT2i-related GTIs and UTIs via CDM, we were limited in our ability to accurately detect mild drug side effects via CDM, which lacked data for operational definition.

Development of an algorithm to link electronic health record prescriptions with pharmacy dispense claims

Objective

Medication adherence is an important aspect of chronic disease management. Electronic health record (EHR) data are often not linked to dispensing data, limiting clinicians' understanding of which of their patients fill their medications, and how to tailor care appropriately. We aimed to develop an algorithm to link EHR prescribing to claims-based dispensing data and use the results to quantify how often patients with diabetes filled prescribed chronic disease medications.

Materials and Methods

We developed an algorithm linking EHR prescribing data (RxNorm terminology) to claims-based dispensing data (NDC terminology), within sample of adult (19-64) community health center (CHC) patients with diabetes from a network of CHCs across 12 states. We demonstrate an application of the method by calculating dispense rates for a set of commonly prescribed diabetes and cardio-protective medications. To further inform clinical care, we computed adjusted odds ratios of dispense by patient-, encounter-, and clinic-level characteristics.

Results

Seventy-six percent of cardio-protective medication prescriptions and 74% of diabetes medications were linked to a dispensing record. Age, income, ethnicity, insurance, assigned primary care provider, comorbidity, time on EHR, and clinic size were significantly associated with odds of dispensing.

Discussion

EHR prescriptions and pharmacy dispense data can be linked at the record level across different terminologies. Dispensing rates in this low-income population with diabetes were similar to other populations.

Conclusion

Record linkage resulted in the finding that CHC patients with diabetes largely had their chronic disease medications dispensed. Understanding factors associated with dispensing rates highlight barriers and opportunities for optimal disease management.

Extracting Adverse Drug Event Information with Minimal Engineering

In this paper we describe an evaluation of the potential of classical information extraction methods to extract drug-related attributes, including adverse drug events, and compare to more recently developed neural methods. We use the 2018 N2C2 shared task data as our gold standard data set for training. We train support vector machine classifiers to detect drug and drug attribute spans, and pair these detected entities as training instances for an SVM relation classifier, with both systems using standard features. We compare to baseline neural methods that use standard contextualized embedding representations for entity and relation extraction. The SVM-based system and a neural system obtain comparable results, with the SVM system doing better on concepts and the neural system performing better on relation extraction tasks. The neural system obtains surprisingly strong results compared to the system based on years of research in developing features for information extraction.

A Machine-Learning-Based Drug Repurposing Approach Using Baseline Regularization

We present the baseline regularization model for computational drug repurposing using electronic health records (EHRs). In EHRs, drug prescriptions of various drugs are recorded throughout time for various patients. In the same time, numeric physical measurements (e.g., fasting blood glucose level) are also recorded. Baseline regularization uses statistical relationships between the occurrences of prescriptions of some particular drugs and the increase or the decrease in the values of some particular numeric physical measurements to identify potential repurposing opportunities.

Detecting Adverse Drug Events with Rapidly Trained Classification Models

INTRODUCTION

Identifying occurrences of medication side effects and adverse drug events (ADEs) is an important and challenging task because they are frequently only mentioned in clinical narrative and are not formally reported.

METHODS

We developed a natural language processing (NLP) system that aims to identify mentions of symptoms and drugs in clinical notes and label the relationship between the mentions as indications or ADEs. The system leverages an existing word embeddings model with induced word clusters for dimensionality reduction. It employs a conditional random field (CRF) model for named entity recognition (NER) and a random forest model for relation extraction (RE).

RESULTS

Final performance of each model was evaluated separately and then combined on a manually annotated evaluation set. The micro-averaged F1 score was 80.9% for NER, 88.1% for RE, and 61.2% for the integrated systems. Outputs from our systems were submitted to the NLP Challenges for Detecting Medication and Adverse Drug Events from Electronic Health Records (MADE 1.0) competition (Yu et al. in http://bio-nlp.org/index.php/projects/39-nlp-challenges , 2018). System performance was evaluated in three tasks (NER, RE, and complete system) with multiple teams submitting output from their systems for each task. Our RE system placed first in Task 2 of the challenge and our integrated system achieved third place in Task 3.

CONCLUSION

Adding to the growing number of publications that utilize NLP to detect occurrences of ADEs, our study illustrates the benefits of employing innovative feature engineering.

Deep learning for pharmacovigilance: recurrent neural network architectures for labeling adverse drug reactions in Twitter posts

Objective

Social media is an important pharmacovigilance data source for adverse drug reaction (ADR) identification. Human review of social media data is infeasible due to data quantity, thus natural language processing techniques are necessary. Social media includes informal vocabulary and irregular grammar, which challenge natural language processing methods. Our objective is to develop a scalable, deep-learning approach that exceeds state-of-the-art ADR detection performance in social media.

Materials and Methods

We developed a recurrent neural network (RNN) model that labels words in an input sequence with ADR membership tags. The only input features are word-embedding vectors, which can be formed through task-independent pretraining or during ADR detection training.

Results

Our best-performing RNN model used pretrained word embeddings created from a large, non-domain-specific Twitter dataset. It achieved an approximate match F-measure of 0.755 for ADR identification on the dataset, compared to 0.631 for a baseline lexicon system and 0.65 for the state-of-the-art conditional random field model. Feature analysis indicated that semantic information in pretrained word embeddings boosted sensitivity and, combined with contextual awareness captured in the RNN, precision.

Discussion

Our model required no task-specific feature engineering, suggesting generalizability to additional sequence-labeling tasks. Learning curve analysis showed that our model reached optimal performance with fewer training examples than the other models.

Conclusion

ADR detection performance in social media is significantly improved by using a contextually aware model and word embeddings formed from large, unlabeled datasets. The approach reduces manual data-labeling requirements and is scalable to large social media datasets.

Data Mining Approach to Estimate the Duration of Drug Therapy from Longitudinal Electronic Medical Records

Background:

Electronic Medical Records (EMRs) from primary/ ambulatory care systems present a new and promising source of information for conducting clinical and translational research.

Objectives:

To address the methodological and computational challenges in order to extract reliable medication information from raw data which is often complex, incomplete and erroneous. To assess whether the use of specific chaining fields of medication information may additionally improve the data quality.

Methods:

Guided by a range of challenges associated with missing and internally inconsistent data, we introduce two methods for the robust extraction of patient-level medication data. First method relies on chaining fields to estimate duration of treatment (“chaining”), while second disregards chaining fields and relies on the chronology of records (“continuous”). Centricity EMR database was used to estimate treatment duration with both methods for two widely prescribed drugs among type 2 diabetes patients: insulin and glucagon-like peptide-1 receptor agonists.

Results:

At individual patient level the “chaining” approach could identify the treatment alterations longitudinally and produced more robust estimates of treatment duration for individual drugs, while the “continuous” method was unable to capture that dynamics. At population level, both methods produced similar estimates of average treatment duration, however, notable differences were observed at individual-patient level.

Conclusion:

The proposed algorithms explicitly identify and handle longitudinal erroneous or missing entries and estimate treatment duration with specific drug(s) of interest, which makes them a valuable tool for future EMR based clinical and pharmaco-epidemiological studies. To improve accuracy of real-world based studies, implementing chaining fields of medication information is recommended.

Feasibility of Prioritizing Drug-Drug-Event Associations Found in Electronic Health Records

Background and Objective

Several studies have demonstrated the ability to detect adverse events potentially related to multiple drug exposure via data mining. However, the number of putative associations produced by such computational approaches is typically large, making experimental validation difficult. We theorized that those potential associations for which there is evidence from multiple complementary sources are more likely to be true, and explored this idea using a published database of drug–drug-adverse event associations derived from electronic health records (EHRs).

Methods

We prioritized drug–drug-event associations derived from EHRs using four sources of information: (1) public databases, (2) sources of spontaneous reports, (3) literature, and (4) non-EHR drug–drug interaction (DDI) prediction methods. After pre-filtering the associations by removing those found in public databases, we devised a ranking for associations based on the support from the remaining sources, and evaluated the results of this rank-based prioritization.

Results

We collected information for 5983 putative EHR-derived drug–drug-event associations involving 345 drugs and ten adverse events from four data sources and four prediction methods. Only seven drug–drug-event associations (<0.5 %) had support from the majority of evidence sources, and about one third (1777) had support from at least one of the evidence sources.

Conclusions

Our proof-of-concept method for scoring putative drug–drug-event associations from EHRs offers a systematic and reproducible way of prioritizing associations for further study. Our findings also quantify the agreement (or lack thereof) among complementary sources of evidence for drug–drug-event associations and highlight the challenges of developing a robust approach for prioritizing signals of these associations.

Electronic supplementary material

The online version of this article (doi:10.1007/s40264-015-0352-2) contains supplementary material, which is available to authorized users.

Complex temporal topic evolution modelling using the Kullback-Leibler divergence and the Bhattacharyya distance

The rapidly expanding corpus of medical research literature presents major challenges in the understanding of previous work, the extraction of maximum information from collected data, and the identification of promising research directions. We present a case for the use of advanced machine learning techniques as an aide in this task and introduce a novel methodology that is shown to be capable of extracting meaningful information from large longitudinal corpora and of tracking complex temporal changes within it. Our framework is based on (i) the discretization of time into epochs, (ii) epoch-wise topic discovery using a hierarchical Dirichlet process-based model, and (iii) a temporal similarity graph which allows for the modelling of complex topic changes. More specifically, this is the first work that discusses and distinguishes between two groups of particularly challenging topic evolution phenomena: topic splitting and speciation and topic convergence and merging, in addition to the more widely recognized emergence and disappearance and gradual evolution. The proposed framework is evaluated on a public medical literature corpus.

Computational Drug Repositioning Using Continuous Self-Controlled Case Series

Computational Drug Repositioning (CDR) is the task of discovering potential new indications for existing drugs by mining large-scale heterogeneous drug-related data sources. Leveraging the patient-level temporal ordering information between numeric physiological measurements and various drug prescriptions provided in Electronic Health Records (EHRs), we propose a Continuous Self-controlled Case Series (CSCCS) model for CDR. As an initial evaluation, we look for drugs that can control Fasting Blood Glucose (FBG) level in our experiments. Applying CSCCS to the Marshfield Clinic EHR, well-known drugs that are indicated for controlling blood glucose level are rediscovered. Furthermore, some drugs with recent literature support for the potential effect of blood glucose level control are also identified.

Generalized enrichment analysis improves the detection of adverse drug events from the biomedical literature

Background

Identification of associations between marketed drugs and adverse events from the biomedical literature assists drug safety monitoring efforts. Assessing the significance of such literature-derived associations and determining the granularity at which they should be captured remains a challenge. Here, we assess how defining a selection of adverse event terms from MeSH, based on information content, can improve the detection of adverse events for drugs and drug classes.

Results

We analyze a set of 105,354 candidate drug adverse event pairs extracted from article indexes in MEDLINE. First, we harmonize extracted adverse event terms by aggregating them into higher-level MeSH terms based on the terms’ information content. Then, we determine statistical enrichment of adverse events associated with drug and drug classes using a conditional hypergeometric test that adjusts for dependencies among associated terms. We compare our results with methods based on disproportionality analysis (proportional reporting ratio, PRR) and quantify the improvement in signal detection with our generalized enrichment analysis (GEA) approach using a gold standard of drug-adverse event associations spanning 174 drugs and four events. For single drugs, the best GEA method (Precision: .92/Recall: .71/F1-measure: .80) outperforms the best PRR based method (.69/.69/.69) on all four adverse event outcomes in our gold standard. For drug classes, our GEA performs similarly (.85/.69/.74) when increasing the level of abstraction for adverse event terms. Finally, on examining the 1609 individual drugs in our MEDLINE set, which map to chemical substances in ATC, we find signals for 1379 drugs (10,122 unique adverse event associations) on applying GEA with p < 0.005.

Conclusions

We present an approach based on generalized enrichment analysis that can be used to detect associations between drugs, drug classes and adverse events at a given level of granularity, at the same time correcting for known dependencies among events. Our study demonstrates the use of GEA, and the importance of choosing appropriate abstraction levels to complement current drug safety methods. We provide an R package for exploration of alternative abstraction levels of adverse event terms based on information content.

Electronic supplementary material

The online version of this article (doi:10.1186/s12859-016-1080-z) contains supplementary material, which is available to authorized users.

Text mining for adverse drug events: the promise, challenges, and state of the art

Text mining is the computational process of extracting meaningful information from large amounts of unstructured text. It is emerging as a tool to leverage underutilized data sources that can improve pharmacovigilance, including the objective of adverse drug event (ADE) detection and assessment. This article provides an overview of recent advances in pharmacovigilance driven by the application of text mining, and discusses several data sources-such as biomedical literature, clinical narratives, product labeling, social media, and Web search logs-that are amenable to text mining for pharmacovigilance. Given the state of the art, it appears text mining can be applied to extract useful ADE-related information from multiple textual sources. Nonetheless, further research is required to address remaining technical challenges associated with the text mining methodologies, and to conclusively determine the relative contribution of each textual source to improving pharmacovigilance.

Pharmacovigilance using clinical notes

With increasing adoption of electronic health records (EHRs), there is an opportunity to use the free-text portion of EHRs for pharmacovigilance. We present novel methods that annotate the unstructured clinical notes and transform them into a deidentified patient-feature matrix encoded using medical terminologies. We demonstrate the use of the resulting high-throughput data for detecting drug-adverse event associations and adverse events associated with drug-drug interactions. We show that these methods flag adverse events early (in most cases before an official alert), allow filtering of spurious signals by adjusting for potential confounding, and compile prevalence information. We argue that analyzing large volumes of free-text clinical notes enables drug safety surveillance using a yet untapped data source. Such data mining can be used for hypothesis generation and for rapid analysis of suspected adverse event risk.

ADEpedia 2.0: Integration of Normalized Adverse Drug Events (ADEs) Knowledge from the UMLS

A standardized Adverse Drug Events (ADEs) knowledge base that encodes known ADE knowledge can be very useful in improving ADE detection for drug safety surveillance. In our previous study, we developed the ADEpedia that is a standardized knowledge base of ADEs based on drug product labels. The objectives of the present study are 1) to integrate normalized ADE knowledge from the Unified Medical Language System (UMLS) into the ADEpedia; and 2) to enrich the knowledge base with the drug-disorder co-occurrence data from a 51-million-document electronic medical records (EMRs) system. We extracted 266,832 drug-disorder concept pairs from the UMLS, covering 14,256 (1.69%) distinct drug concepts and 19,006 (3.53%) distinct disorder concepts. Of them, 71,626 (26.8%) concept pairs from UMLS co-occurred in the EMRs. We performed a preliminary evaluation on the utility of the UMLS ADE data. In conclusion, we have built an ADEpedia 2.0 framework that intends to integrate known ADE knowledge from disparate sources. The UMLS is a useful source for providing standardized ADE knowledge relevant to indications, contraindications and adverse effects, and complementary to the ADE data from drug product labels. The statistics from EMRs would enable the meaningful use of ADE data for drug safety surveillance.

Harmonization process for the identification of medical events in eight European healthcare databases: the experience from the EU-ADR project

OBJECTIVE

Data from electronic healthcare records (EHR) can be used to monitor drug safety, but in order to compare and pool data from different EHR databases, the extraction of potential adverse events must be harmonized. In this paper, we describe the procedure used for harmonizing the extraction from eight European EHR databases of five events of interest deemed to be important in pharmacovigilance: acute myocardial infarction (AMI); acute renal failure (ARF); anaphylactic shock (AS); bullous eruption (BE); and rhabdomyolysis (RHABD).

DESIGN

The participating databases comprise general practitioners' medical records and claims for hospitalization and other healthcare services. Clinical information is collected using four different disease terminologies and free text in two different languages. The Unified Medical Language System was used to identify concepts and corresponding codes in each terminology. A common database model was used to share and pool data and verify the semantic basis of the event extraction queries. Feedback from the database holders was obtained at various stages to refine the extraction queries.

MEASUREMENTS

Standardized and age specific incidence rates (IRs) were calculated to facilitate benchmarking and harmonization of event data extraction across the databases. This was an iterative process.

RESULTS

The study population comprised overall 19 647 445 individuals with a follow-up of 59 929 690 person-years (PYs). Age adjusted IRs for the five events of interest across the databases were as follows: (1) AMI: 60-148/100 000 PYs; (2) ARF: 3-49/100 000 PYs; (3) AS: 2-12/100 000 PYs; (4) BE: 2-17/100 000 PYs; and (5) RHABD: 0.1-8/100 000 PYs.

CONCLUSIONS

The iterative harmonization process enabled a more homogeneous identification of events across differently structured databases using different coding based algorithms. This workflow can facilitate transparent and reproducible event extractions and understanding of differences between databases.

Cross-terminology mapping challenges: a demonstration using medication terminological systems

Standardized terminological systems for biomedical information have provided considerable benefits to biomedical applications and research. However, practical use of this information often requires mapping across terminological systems-a complex and time-consuming process. This paper demonstrates the complexity and challenges of mapping across terminological systems in the context of medication information. It provides a review of medication terminological systems and their linkages, then describes a case study in which we mapped proprietary medication codes from an electronic health record to SNOMED CT and the UMLS Metathesaurus. The goal was to create a polyhierarchical classification system for querying an i2b2 clinical data warehouse. We found that three methods were required to accurately map the majority of actively prescribed medications. Only 62.5% of source medication codes could be mapped automatically. The remaining codes were mapped using a combination of semi-automated string comparison with expert selection, and a completely manual approach. Compound drugs were especially difficult to map: only 7.5% could be mapped using the automatic method. General challenges to mapping across terminological systems include (1) the availability of up-to-date information to assess the suitability of a given terminological system for a particular use case, and to assess the quality and completeness of cross-terminology links; (2) the difficulty of correctly using complex, rapidly evolving, modern terminologies; (3) the time and effort required to complete and evaluate the mapping; (4) the need to address differences in granularity between the source and target terminologies; and (5) the need to continuously update the mapping as terminological systems evolve.

Annotation Analysis for Testing Drug Safety Signals using Unstructured Clinical Notes

Background

The electronic surveillance for adverse drug events is largely based upon the analysis of coded data from reporting systems. Yet, the vast majority of electronic health data lies embedded within the free text of clinical notes and is not gathered into centralized repositories. With the increasing access to large volumes of electronic medical data—in particular the clinical notes—it may be possible to computationally encode and to test drug safety signals in an active manner.

Results

We describe the application of simple annotation tools on clinical text and the mining of the resulting annotations to compute the risk of getting a myocardial infarction for patients with rheumatoid arthritis that take Vioxx. Our analysis clearly reveals elevated risks for myocardial infarction in rheumatoid arthritis patients taking Vioxx (odds ratio 2.06) before 2005.

Conclusions

Our results show that it is possible to apply annotation analysis methods for testing hypotheses about drug safety using electronic medical records.

Using temporal patterns in medical records to discern adverse drug events from indications

Researchers estimate that electronic health record systems record roughly 2-million ambulatory adverse drug events and that patients suffer from adverse drug events in roughly 30% of hospital stays. Some have used structured databases of patient medical records and health insurance claims recently-going beyond the current paradigm of using spontaneous reporting systems like AERS-to detect drug-safety signals. However, most efforts do not use the free-text from clinical notes in monitoring for drug-safety signals. We hypothesize that drug-disease co-occurrences, extracted from ontology-based annotations of the clinical notes, can be examined for statistical enrichment and used for drug safety surveillance. When analyzing such co-occurrences of drugs and diseases, one major challenge is to differentiate whether the disease in a drug-disease pair represents an indication or an adverse event. We demonstrate that it is possible to make this distinction by combining the frequency distribution of the drug, the disease, and the drug-disease pair as well as the temporal ordering of the drugs and diseases in each pair across more than one million patients.

Translational bioinformatics embraces big data

We review the latest trends and major developments in translational bioinformatics in the year 2011-2012. Our emphasis is on highlighting the key events in the field and pointing at promising research areas for the future. The key take-home points are: • Translational informatics is ready to revolutionize human health and healthcare using large-scale measurements on individuals. • Data-centric approaches that compute on massive amounts of data (often called "Big Data") to discover patterns and to make clinically relevant predictions will gain adoption. • Research that bridges the latest multimodal measurement technologies with large amounts of electronic healthcare data is increasing; and is where new breakthroughs will occur.

Rates of 5 common antidepressant side effects among new adult and adolescent cases of depression: a retrospective US claims study

BACKGROUND

Antidepressants are the first-line treatment for depression, yet medication-related side effects may be associated with antidepressant discontinuation before reaching a period of exposure believed to result in effectiveness. There is a gap in knowledge of the prevalence of side effects across commonly prescribed antidepressants and the effect of the type of antidepressant on the likelihood of side effects in real-world clinical practice.

OBJECTIVE

The aim of this study was to estimate and compare the prevalence of headaches, nausea or vomiting, agitation, sedation, and sexual dysfunction among patients diagnosed with depression who initiated monotherapy across different classes of antidepressants and to estimate the effect of the type of antidepressant on the likelihood of each of the 5 side effects.

METHODS

A retrospective cohort of patients aged ≥13 who were newly diagnosed with depression and began antidepressant monotherapy was created using LifeLink managed care claims from 1998 to 2008. Antidepressant groups included selective serotonin reuptake inhibitors (SSRIs), serotonin-norepinephrine reuptake inhibitors (SNRIs), tricyclic antidepressants (TCAs), monoamine oxidase inhibitors (MAOIs), bupropion, phenylpiperazine, and tetracyclic antidepressants. Prevalence of headache, nausea or vomiting, agitation, sedation, and sexual dysfunction were compared across antidepressant groups. Propensity-adjusted Cox proportional hazards regression was used to estimate the likelihood of each of the 5 side effects for each antidepressant group compared with SSRIs, adjusted for demographic, clinical, and treatment characteristics.

RESULTS

The study cohort included 40,017 patients (3617 adolescents, aged 13-18 years, and 36,400 adults, aged ≥19 years; mean age = 45 years; 67% female) with a new episode of depression who were initiated on antidepressant monotherapy within 30 days of diagnosis (SSRI [66%], bupropion [14%], SNRI [12%], other [8%]). The most common side effects were headache (up to 17/1000 person-months of therapy in adults and adolescents) and nausea (up to 7.2/1000 in adults, 9.3/1000 in adolescents). Relative to adults receiving SSRIs, adults receiving SNRIs had a higher risk of nausea (hazard ratio [HR] = 1.26; 95%CI,1.05-1.51). Adults (HR = 0.78; 95% CI, 0.62-0.96) and adolescents (HR = 0.43; 95% CI, 0.21-0.87) taking bupropion were less likely to experience headaches compared with adults and adolescents, respectively, taking an SSRI. Adolescents receiving a tetracyclic were more likely to experience headaches than adolescents receiving an SSRI (HR = 3.16; 95%CI, 1.13-8.84).

CONCLUSIONS

Prevalence and risk of the 5 side effects varied across types of antidepressants for both adults and adolescents. Results from this study were consistent with prior clinical trials, suggesting that variation in side effect profiles exists in a more generalized managed care population.

Use of electronic medical records (EMR) for oncology outcomes research: assessing the comparability of EMR information to patient registry and health claims data

Electronic medical records (EMRs) are used increasingly for research in clinical oncology, epidemiology, and comparative effectiveness research (CER).

OBJECTIVE

To assess the utility of using EMR data in population-based cancer research by comparing a database of EMRs from community oncology clinics against Surveillance Epidemiology and End Results (SEER) cancer registry data and two claims databases (Medicare and commercial claims).

STUDY DESIGN AND SETTING

DEMOGRAPHIC, CLINICAL, AND TREATMENT PATTERNS IN THE EMR, SEER, MEDICARE, AND COMMERCIAL CLAIMS DATA WERE COMPARED USING SIX TUMOR SITES: breast, lung/bronchus, head/neck, colorectal, prostate, and non-Hodgkin's lymphoma (NHL). We identified various challenges in data standardization and selection of appropriate statistical procedures. We describe the patient and clinic inclusion criteria, treatment definitions, and consideration of the administrative and clinical purposes of the EMR, registry, and claims data to address these challenges.

RESULTS

Sex and 10-year age distributions of patient populations for each tumor site were generally similar across the data sets. We observed several differences in racial composition and treatment patterns, and modest differences in distribution of tumor site.

CONCLUSION

Our experience with an oncology EMR database identified several factors that must be considered when using EMRs for research purposes or generalizing results to the US cancer population. These factors were related primarily to evaluation of treatment patterns, including evaluation of stage, geographic location, race, and specialization of the medical facilities. While many specialty EMRs may not provide the breadth of data on medical care, as found in comprehensive claims databases and EMR systems, they can provide detailed clinical data not found in claims that are extremely important in conducting epidemiologic and outcomes research.