Methods for drug safety signal detection using routinely collected observational electronic health care data: A systematic review

Implementation of a Taxonomy-Based Framework for the Selection of Appropriate Drugs and Outcomes for Real-World Data Signal Detection Studies

INTRODUCTION

For signal detection studies investigating either drug safety or method evaluation, the choice of drug-outcome pairs needs to be tailored to the planned study design and vice versa. While this is well understood in hypothesis-testing epidemiology, it should be as important in signal detection, but this has not widely been considered. There is a need for a taxonomy framework to provide guidance and a systematic reproducible approach to the selection of appropriate drugs and outcomes for signal detection studies either investigating drug safety or assessing method performance using real-world data.

OBJECTIVE

The aim was to design a general framework for the selection of appropriate drugs and outcomes for signal detection studies given a study design of interest. As a motivating example, we illustrate how the framework is applied to build a reference set for a study aiming to assess the performance of the self-controlled case series with active comparators.

METHODS

We reviewed criteria presented in two published studies which aimed to provide practical advice for choosing the appropriate signal evaluation methodology, and assessed their relevance for signal detection. Further characteristics specific to signal detection were added. The final framework is based on: the application of study design requirements, the database(s) of interest, and the clinical importance of the drug(s) and outcome(s) under consideration. This structure was applied by selecting drug-outcome pairs as a reference set (i.e. list of drug-outcome pairs classified as positive or negative controls) for which the method is expected to work well for a signal detection study aiming to assess the performance of self-controlled case series. Eight criteria were used, related to the application of self-controlled case series assumptions, choice of active comparators, coverage in the database of interest and clinical importance of the outcomes.

RESULTS

After application of the framework, two classes of antibiotics (seven drugs) were selected for the study, and 28 outcomes from all organ classes were chosen from the drug labels, out of the 273 investigated. In total, this corresponds to 104 positive controls (drug-outcome pairs) and 58 negative controls.

CONCLUSIONS

We proposed and applied a framework for the selection of drugs and outcomes for both drug safety signal detection and method assessment used in signal detection to optimise their performance given a study design. This framework will eliminate part of the bias relating to drugs and outcomes not being suited to the method or database. The main difficulty lies in the choice of the criteria and their application to ensure systematic selection, especially as some information remains unknown in signal detection, and clinical judgement was needed on occasions. The same framework could be adapted for other methods.

High-Throughput Computing to Automate Population-Based Studies to Detect the 30-Day Risk of Adverse Outcomes After New Outpatient Medication Use in Older Adults with Chronic Kidney Disease: A Clinical Research Protocol

Background

Safety issues are detected in about one third of prescription drugs in the years following regulatory agency approval. Older adults, especially those with chronic kidney disease, are at particular risk of adverse reactions to prescription drugs. This protocol describes a new approach that may identify credible drug-safety signals more efficiently using administrative health care data.

Objective

To use high-throughput computing and automation to conduct 700+ drug-safety cohort studies in older adults in Ontario, Canada. Each study will compare 74 acute (30-day) outcomes in patients who start a new prescription drug (new users) to a group of nonusers with similar baseline health characteristics. Risks will be assessed within strata of baseline kidney function.

Design and setting

The studies will be population-based, new-user cohort studies conducted using linked administrative health care databases in Ontario, Canada (January 1, 2008, to March 1, 2020). The source population for these studies will be residents of Ontario aged 66 years or older who filled at least one outpatient prescription through the Ontario Drug Benefit (ODB) program during the study period (all residents have universal health care, and those aged 65+ have universal prescription drug coverage through the ODB).

Patients

We identified 3.2 million older adults in the source population during the study period and built 700+ initial medication cohorts, each containing mutually exclusive groups of new users and nonusers. Nonusers were randomly assigned cohort entry dates that followed the same distribution of prescription start dates as new users. Eligibility criteria included a baseline estimated glomerular filtration rate (eGFR) measurement within 12 months before the cohort entry date (median time was 71 days before cohort entry in the new user group), no prior receipt of maintenance dialysis or a kidney transplant, and no prior prescriptions for drugs in the same subclass as the study drug. New users and nonusers will be balanced on ~400 baseline health characteristics using inverse probability of treatment weighting on propensity scores within 3 strata of baseline eGFR: ≥60, 45 to <60, <45 mL/min per 1.73 m2.

Outcomes

We will compare new user and nonuser groups on 74 clinically relevant outcomes (17 composites and 57 individual outcomes) in the 30 days after cohort entry. We used a prespecified approach to identify these 74 outcomes.

Statistical analysis plan

In each cohort, we will obtain eGFR-stratum-specific weighted risk ratios and risk differences using modified Poisson regression and binomial regression, respectively. Additive and multiplicative interaction by eGFR category will be examined. Drug-outcome associations that meet prespecified criteria (identified signals) will be further examined in additional analyses (including survival, negative-control exposure, and E-value analyses) and visualizations.

Results

The initial medication cohorts had a median of 6120 new users per cohort (interquartile range: 1469-38 839) and a median of 1 088 301 nonusers (interquartile range: 751 697-1 267 009). Medications with the largest number of new users were amoxicillin trihydrate (n = 1 000 032), cephalexin (n = 571 566), prescription acetaminophen (n = 571 563), and ciprofloxacin (n = 504,374); 19% to 29% of new users in these cohorts had an eGFR <60 mL/min per 1.73 m2.

Limitations

Despite our use of robust techniques to balance baseline indicators and to control for confounding by indication, residual confounding will remain a possibility. Only acute (30-day) outcomes will be examined. Our data sources do not include nonprescription (over-the-counter) drugs or drugs prescribed in hospitals and do not include outpatient prescription drug use in children or adults <65 years.

Conclusion

This accelerated approach to conducting postmarket drug-safety studies has the potential to more efficiently detect drug-safety signals in a vulnerable population. The results of this protocol may ultimately help improve medication safety.

Epidemiological surveillance of drug safety using cumulative sequential analysis in electronic healthcare data

BACKGROUND

Methods for safety signal detection in electronic healthcare data analysing data sequentially are being developed to meet the limitations of spontaneous reporting systems.

OBJECTIVES

This study aims to provide an overview of the literature on sequential analysis of electronic healthcare data and describe the development and testing of a novel epidemiological surveillance system.

METHODS

We searched Medline, Embase, PubMed, Scopus, Web of Science, and the Cochrane Library applying similar in- and exclusion criteria as those of a previous systematic review. The proposed system consisted of repeated cohort studies and was tested in an emulated prospective setting. Two signal evaluations were performed with several sensitivity analyses and a target trial emulation.

FINDINGS

In the literature, 11 studies analysed the data sequentially of which two applied traditional epidemiological methods. Epidemiological surveillance of several exposures and outcomes can be successfully conducted with the newly proposed sequential analysis of electronic healthcare data. Signal evaluation studies confirmed the results of the system.

CONCLUSIONS

Very few studies in the literature analysed data at multiple time points, although this seems to be a prerequisite for testing the methods in a realistic setting. We demonstrated the feasibility of a sequential surveillance system using electronic healthcare data.

Supporting Pharmacovigilance Signal Validation and Prioritization with Analyses of Routinely Collected Health Data: Lessons Learned from an EHDEN Network Study

INTRODUCTION

Individual case reports are the main asset in pharmacovigilance signal management. Signal validation is the first stage after signal detection and aims to determine if there is sufficient evidence to justify further assessment. Throughout signal management, a prioritization of signals is continually made. Routinely collected health data can provide relevant contextual information but are primarily used at a later stage in pharmacoepidemiological studies to assess communicated signals.

OBJECTIVE

The aim of this study was to examine the feasibility and utility of analysing routine health data from a multinational distributed network to support signal validation and prioritization and to reflect on key user requirements for these analyses to become an integral part of this process.

METHODS

Statistical signal detection was performed in VigiBase, the WHO global database of individual case safety reports, targeting generic manufacturer drugs and 16 prespecified adverse events. During a 5-day study-a-thon, signal validation and prioritization were performed using information from VigiBase, regulatory documents and the scientific literature alongside descriptive analyses of routine health data from 10 partners of the European Health Data and Evidence Network (EHDEN). Databases included in the study were from the UK, Spain, Norway, the Netherlands and Serbia, capturing records from primary care and/or hospitals.

RESULTS

Ninety-five statistical signals were subjected to signal validation, of which eight were considered for descriptive analyses in the routine health data. Design, execution and interpretation of results from these analyses took up to a few hours for each signal (of which 15-60 minutes were for execution) and informed decisions for five out of eight signals. The impact of insights from the routine health data varied and included possible alternative explanations, potential public health and clinical impact and feasibility of follow-up pharmacoepidemiological studies. Three signals were selected for signal assessment, two of these decisions were supported by insights from the routine health data. Standardization of analytical code, availability of adverse event phenotypes including bridges between different source vocabularies, and governance around the access and use of routine health data were identified as important aspects for future development.

CONCLUSIONS

Analyses of routine health data from a distributed network to support signal validation and prioritization are feasible in the given time limits and can inform decision making. The cost-benefit of integrating these analyses at this stage of signal management requires further research.

Design, implementation, and inferential issues associated with clinical trials that rely on data in electronic medical records: a narrative review

Real world evidence is now accepted by authorities charged with assessing the benefits and harms of new therapies. Clinical trials based on real world evidence are much less expensive than randomized clinical trials that do not rely on "real world evidence" such as contained in electronic health records (EHR). Consequently, we can expect an increase in the number of reports of these types of trials, which we identify here as 'EHR-sourced trials.' 'In this selected literature review, we discuss the various designs and the ethical issues they raise. EHR-sourced trials have the potential to improve/increase common data elements and other aspects of the EHR and related systems. Caution is advised, however, in drawing causal inferences about the relationships among EHR variables. Nevertheless, we anticipate that EHR-CTs will play a central role in answering research and regulatory questions.