Successful Comparison of US Food and Drug Administration Sentinel Analysis Tools to Traditional Approaches in Quantifying a Known Drug-Adverse Event Association

Application of the U.S. Food and Drug Administration's Sentinel Routine Querying Tools to the Taiwan Sentinel Data Model-formatted National Health Insurance Research Database

The U.S. Food and Drug Administration's Sentinel System is a leading distributed data network for drug safety surveillance in the world. The National Health Insurance Research Database (NHIRD) in Taiwan was converted into the Taiwan Sentinel Data Model (TSDM) based on the Sentinel Common Data Model (SCDM) version 6.0.2. The goal of this study was to investigate the feasibility of applying the same study designs, analytic choices, and analytic tools as used by the U.S. Sentinel System to examine the same drug-outcome associations in the TSDM-formatted NHIRD. Four known drug-outcome associations previously examined by the U.S. Sentinel System were selected as the use cases: (1) use of angiotensin-converting enzyme inhibitors (ACEIs) and risk of angioedema, (2) use of warfarin and risk of gastrointestinal bleeding, (3) use of oral clindamycin and risk of Clostridioides difficile infection (CDI), and (4) use of glyburide and risk of serious hypoglycemia. We followed the same study designs and analytic choices used by the U.S. Sentinel System and applied the Sentinel Routine Querying Tools to answer the same study questions within the TSDM-formatted NHIRD. The results showed that ACEIs were associated with a non-significant increase in risk of angioedema compared to beta-blockers (hazard ratio [HR]: 1.21; 95% confidence interval [CI]: 0.89-1.64); warfarin was associated with a higher risk of gastrointestinal bleeding compared to statins (HR: 1.72; 1.50-1.98); glyburide was associated with an increased risk of hypoglycemia compared to glipizide (HR: 1.61, 1.30-2.00). We were unable to evaluate the association between oral clindamycin and risk of CDI due to the low event number. Our study demonstrated that it was feasible to directly apply the publicly available Sentinel Routine Querying Tools within the TSDM-formatted NHIRD. However, sources of heterogeneity other than design and analytic differences should be carefully considered when comparing the results between the two systems.

A new analytic tool for assessing the impact of the US Food and Drug Administration regulatory actions

PURPOSE

Develop and test a flexible, scalable tool using interrupted time series (ITS) analysis to assess the impact of Food and Drug Administration (FDA) regulatory actions on drug use.

METHODS

We applied the tool in the Sentinel Distributed Database to assess the impact of FDA's 2010 drug safety communications (DSC) concerning the safety of long-acting beta2-agonists (LABA) in adult asthma patients. We evaluated changes in LABA use by measuring the initiation of LABA alone and concomitant use of LABA and asthma controller medications (ACM) after the DSCs. The tool generated ITS graphs and used segmented regression to estimate baseline slope, level change, slope change, and absolute and relative changes at up to two user-specified time point (s) after the intervention. We tested the tool and compared our results against prior analyses that used similar measures.

RESULTS

Initiation of LABA alone declined among asthma patients aged 18-45 years before FDA DSCs (-0.10% per quarter; 95%CI: -0.11% to -0.09%) and the downward trend continued after. Concomitant use of LABA and ACM was stable before FDA DSCs. After FDA DSCs, there was a small trend decrease of 0.006% per quarter (95% CI, -0.008% to -0.003%). We found similar results among those aged 46-64 years and patients with poorly-controlled asthma. Our results were consistent with previous studies, confirming the performance of the new tool.

CONCLUSIONS

We developed and tested a reusable ITS tool in real-world databases formatted to the Sentinel Common Data Model that can assess the impact of regulatory actions on drug use.

Assessing medical product safety during pregnancy using parameterizable tools in the sentinel distributed database

PURPOSE

The US Food and Drug Administration established the Sentinel System to monitor the safety of medical products. A component of this system includes parameterizable analytic tools to identify mother-infant pairs and evaluate infant outcomes to enable the routine monitoring of the utilization and safety of drugs used in pregnancy. We assessed the feasibility of using the data and tools in the Sentinel System by assessing a known association between topiramate use during pregnancy and oral clefts in the infant.

METHODS

We identified mother-infant pairs using the mother-infant linkage table from six data partners contributing to the Sentinel Distributed Database from January 1, 2000, to September 30, 2015. We compared mother-infant pairs with first-trimester exposure to topiramate to mother-infant pairs that were topiramate-unexposed or lamotrigine-exposed and used a validated algorithm to identify oral clefts in the infant. We estimated adjusted risk ratios through propensity score stratification.

RESULTS

There were 2007 topiramate-exposed and 1 066 086 unexposed mother-infant pairs in the main comparison. In the active-comparator analysis, there were 1996 topiramate-exposed and 2859 lamotrigine-exposed mother-infant pairs. After propensity score stratification, the odds ratio for oral clefts was 2.92 (95% CI: 1.43, 5.93) comparing the topiramate-exposed to unexposed groups and 2.72 (95% CI: 0.75, 9.93) comparing the topiramate-exposed to lamotrigine-exposed groups.

CONCLUSIONS

We found an increased risk of oral clefts after topiramate exposure in the first trimester in the Sentinel database. These results are similar to prior published observational study results and demonstrate the ability of Sentinel's data and analytic tools to assess medical product safety in cohorts of mother-infant pairs in a timely manner.

Comparative Risk of Angioedema With Sacubitril-Valsartan vs Renin-Angiotensin-Aldosterone Inhibitors

BACKGROUND

Data on angioedema risk among sacubitril-valsartan (SV) users in real-world settings are limited.

OBJECTIVES

We sought to evaluate the risk of angioedema among SV new users compared with angiotensin-converting enzyme (ACE) inhibitor and angiotensin-receptor-blocker (ARB) new users separately.

METHODS

We conducted a propensity score-matched cohort study, comparing SV new users (no use of SV, ACE inhibitor, ARB 6 months before) and SV new users with prior use (within 183 or 14 days) of ACE inhibitor or ARB (ACE inhibitor-SV and ARB-SV users; recent ACE inhibitor-SV and recent ARB-SV users, respectively) vs ACE inhibitor and ARB new users separately.

RESULTS

Compared with ACE inhibitor, SV new (HR: 0.18; 95% CI: 0.11-0.29) and ACE inhibitor-SV users (HR: 0.31; 95% CI: 0.23-0.43) showed lower risk of angioedema. On the other hand, there was no difference in angioedema risk when SV new users (HR: 0.59; 95% CI: 0.35-1.01) or ARB-SV users (HR: 0.85; 95% CI: 0.58-1.26) were compared with ARB new users. Compared with SV new users, ACE inhibitor-SV users (HR: 1.62; 95% CI: 0.91-2.89) trended toward higher angioedema risk, which intensified when the ACE inhibitor to SV switch occurred within 14 days (recent ACE inhibitor-SV) (HR: 1.98; 95% CI: 1.11-3.53). Similarly, ARB-SV users (HR: 2.03; 95% CI: 1.16-3.54) experienced an increased risk compared with SV new users, which intensified for the more recent switchers (recent ARB-SV) (HR: 2.45; 95% CI: 1.36-4.43).

CONCLUSIONS

We did not observe an increased risk of angioedema among SV new users compared with ACE inhibitor or ARB users. However, there was an increased risk of angioedema among SV users who recently switched from ACE inhibitor or ARB compared with SV new users.

Revealing Unknown Benefits of Existing Medications to Aid the Discovery of New Treatments for Post‐Traumatic Stress Disorder

Objective

To systematically identify novel pharmacological strategies for preventing or treating post‐traumatic stress disorder (PTSD) by leveraging large‐scale analysis of real‐world observational data.

Methods

Using a self‐controlled study design, the association between 1399 medications and the incidence of PTSD across four US insurance claims databases covering commercially insured, Medicare eligible, and Medicaid patients was examined. A validated algorithm for identifying PTSD in claims data was used, and medications were identified by their RxNorm ingredient. Medications used to treat PTSD or its symptoms (e.g., antidepressants, antipsychotics) were excluded. Medications associated with ≥30% reduction in risk of PTSD in ≥2 databases were identified.

Results

A total of 137,182,179 individuals were included in the analysis. Fifteen medications met the threshold criteria for a potential protective effect on PTSD; six were categorized as “primary signals” while the remaining nine were considered “potential signals”. The primary signals include a beta blocker that has been previously studied for PTSD, and five medications used to treat attention‐deficit/hyperactivity disorder. The potential signals include four medications used to treat substance use disorders and five medications used to treat sleep disorders.

Discussion

The medications identified in this analysis provide targets for further research in studies that are designed to examine specific hypotheses regarding these medications and the incidence of PTSD. This work may aid in discovering novel therapeutic approaches to treat PTSD, wherein new and effective treatments are badly needed.

Four large US‐based administrative claims databases were used to analyze the association between all marketed prescription medications and the outcome of incident post‐traumatic stress disorder (PTSD)

Of the 1399 medications examined, there were 15 that met the strict filtering criteria for showing consistent, moderate‐to‐strong, protective effects against the outcome

Medications fell into four main classes: (1) a beta blocker (propranolol), (2) five medications used to treat attention‐deficit/hyperactivity disorder (ADHD), (3) four medications used to treat substance use disorders and (4) five medications used to treat sleep disorders

These findings identify rational starting points for future hypothesis‐driven research to explore these associations in greater detail

Conducting Real-world Evidence Studies on the Clinical Outcomes of Diabetes Treatments

Real-world evidence (RWE), the understanding of treatment effectiveness in clinical practice generated from longitudinal patient-level data from the routine operation of the healthcare system, is thought to complement evidence on the efficacy of medications from randomized controlled trials (RCTs). RWE studies follow a structured approach. (1) A design layer decides on the study design, which is driven by the study question and refined by a medically informed target population, patient-informed outcomes, and biologically informed effect windows. Imagining the randomized trial we would ideally perform before designing an RWE study in its likeness reduces bias; the new-user active comparator cohort design has proven useful in many RWE studies of diabetes treatments. (2) A measurement layer transforms the longitudinal patient-level data stream into variables that identify the study population, the pre-exposure patient characteristics, the treatment, and the treatment-emergent outcomes. Working with secondary data increases the measurement complexity compared to primary data collection that we find in most RCTs. (3) An analysis layer focuses on the causal treatment effect estimation. Propensity score analyses have gained in popularity to minimize confounding in healthcare database analyses. Well-understood investigator errors, like immortal time bias, adjustment for causal intermediates, or reverse causation, should be avoided. To increase reproducibility of RWE findings, studies require full implementation transparency. This article integrates state-of-the-art knowledge on how to conduct and review RWE studies on diabetes treatments to maximize study validity and ultimately increased confidence in RWE-based decision making.

Risk of Nonmelanoma Skin Cancer in Association With Use of Hydrochlorothiazide-Containing Products in the United States

Background

European studies reported an increased risk of nonmelanoma skin cancer associated with hydrochlorothiazide (HCTZ)-containing products. We examined the risks of basal cell carcinoma (BCC) and squamous cell carcinoma (SCC) associated with HCTZ compared with angiotensin-converting enzyme inhibitors (ACEIs) in a US population.

Methods

We conducted a retrospective cohort study in the US Food and Drug Administration's Sentinel System. From the date of HCTZ or ACEI dispensing, patients were followed until a SCC or BCC diagnosis requiring excision or topical chemotherapy treatment on or within 30 days after the diagnosis date or a censoring event. Using Cox proportional hazards regression models, we estimated the hazard ratios (HRs), overall and separately by age, sex, and race. We also examined site- and age-adjusted incidence rate ratios (IRRs) by cumulative HCTZ dose within the matched cohort.

Results

Among 5.2 million propensity-score matched HCTZ and ACEI users, the incidence rate (per 1000 person-years) of BCC was 2.78 and 2.82, respectively, and 1.66 and 1.60 for SCC. Overall, there was no difference in risk between HCTZ and ACEIs for BCC (HR = 0.99, 95% confidence interval [CI] = 0.97 to 1.00), but there was an increased risk for SCC (HR = 1.04, 95% CI = 1.02 to 1.06). HCTZ use was associated with higher risks of BCC (HR = 1.09, 95% CI = 1.07 to 1.11) and SCC (HR = 1.15, 95% CI = 1.12 to 1.17) among Caucasians. Cumulative HCTZ dose of 50 000 mg or more was associated with an increased risk of SCC in the overall population (IRR = 1.19, 95% CI = 1.05 to 1.35) and among Caucasians (IRR = 1.27, 95% CI = 1.10 to 1.47).

Conclusions

Among Caucasians, we identified small increased risks of BCC and SCC with HCTZ compared with ACEI. Appropriate risk mitigation strategies should be taken while using HCTZ.

Automation in signal management in pharmacovigilance-an insight

Drugs are the imperial part of modern society, but along with their therapeutic effects, drugs can also cause adverse effects, which can be mild to morbid. Pharmacovigilance is the process of collection, detection, assessment, monitoring and prevention of adverse drug events in both clinical trials as well as in the post-marketing phase. The recent trends in increasing unknown adverse events, known as signals, have raised the need to develop an ideal system for monitoring and detecting the potential signals timely. The process of signal management comprises of techniques to identify individual case safety reports systematically. Automated signal detection is highly based upon the data mining of the spontaneous reporting system such as reports from health care professional, observational studies, medical literature or from social media. If a signal is not managed properly, it can become an identical risk associated with the drug which can be hazardous for the patient safety and may have fatal outcomes which may impact health care system adversely. Once a signal is detected quantitatively, it can be further processed by the signal management team for the qualitative analysis and further evaluations. The main components of automated signal detection are data extraction, data acquisition, data selection, and data analysis and data evaluation. This system must be developed in the correct format and context, which eventually emphasizes the quality of data collected and leads to the optimal decision-making based upon the scientific evaluation.

Leveraging the Capabilities of the FDA's Sentinel System To Improve Kidney Care

The Sentinel System is a national electronic postmarketing resource established by the US Food and Drug Administration to support assessment of the safety and effectiveness of marketed medical products. It has built a large, multi-institutional, distributed data network that contains comprehensive electronic health data, covering about 700 million person-years of longitudinal observation time nationwide. With its sophisticated infrastructure and a large selection of flexible analytic tools, the Sentinel System permits rapid and secure analyses, while preserving patient privacy and health-system autonomy. The Sentinel System also offers enhanced capabilities, including accessing full-text medical records, supporting randomized clinical trials embedded in healthcare delivery systems, and facilitating effective collection of patient-reported data using mobile devices, among many other research programs. The nephrology research community can use the infrastructure, tools, and data that this national resource offers for evidence generation. This review summarizes the Sentinel System and its ability to rapidly generate high-quality, real-world evidence; discusses the program's experience in, and potential for, addressing gaps in kidney care; and outlines avenues for conducting research, leveraging this national resource in collaboration with Sentinel investigators.

Evaluation of the US Food and Drug Administration Sentinel Analysis Tools Using a Comparator with a Different Indication: Comparing the Rates of Gastrointestinal Bleeding in Warfarin and Statin Users

BACKGROUND

The US Food and Drug Administration's Sentinel System was established to monitor safety of regulated medical products. Sentinel investigators identified known associations between drugs and adverse events to test reusable analytic tools developed for Sentinel. This test case used a comparator with a different indication.

OBJECTIVE

We tested the ability of Sentinel's reusable analytic tools to identify the known association between warfarin and gastrointestinal bleeding (GIB). Statins, expected to have no effect on GIB, were the comparator. We further explored the impact of analytic features, including matching ratio and stratifying Cox regression analyses, on matched pairs.

METHODS

This evaluation included data from 14 Sentinel Data Partners. New users of warfarin and statins, aged 18 years and older, who had not received other anticoagulants or had recent GIB were matched on propensity score using 1:1 and 1:n variable ratio matching, matching statin users with warfarin users to estimate the average treatment effect in warfarin-treated patients. We compared the risk of GIB using Cox proportional hazards regression, following patients for the duration of their observed continuous treatment or until a GIB. For the 1:1 matched cohort, we conducted analyses with and without stratification on matched pair. The variable ratio matched cohort analysis was stratified on the matched set.

RESULTS

We identified 141,398 new users of warfarin and 2,275,694 new users of statins. In analyses stratified on matched pair/set, the hazard ratios (HR) for GIB in warfarin users compared with statin users were 2.78 (95% confidence interval [CI] 2.36-3.28) in the 1:1 matched cohort and 3.10 (95% CI 2.76-3.49) in the variable ratio matched cohort. The HR was lower in the analysis of the 1:1 matched cohort not stratified by matched pair (2.22, 95% CI 1.97-2.49), and highest early in treatment. Follow-up for warfarin users tended to be shorter than for statin users.

CONCLUSIONS

This study identified the expected GIB risk with warfarin compared with statins using an analytic tool developed for Sentinel. Our findings suggest that comparators with different indications may be useful in surveillance in select circumstances. Finally, in the presence of differential censoring, stratification by matched pair may reduce the potential for bias in Cox regression analyses.

Aiding the discovery of new treatments for dementia by uncovering unknown benefits of existing medications

Introduction

There is a significant need for disease-modifying therapies to treat and prevent dementia, including Alzheimer's disease. Availability of real-world observational information and new analytic techniques to analyze large volumes of data can provide a path to aid drug discovery.

Methods

Using a self-controlled study design, we examined the association between 2181 medications and incidence of dementia across four US insurance claims databases. Medications associated with ≥50% reduction in risk of dementia in ≥2 databases were examined.

Results

A total of 117,015,066 individuals were included in the analysis. Seventeen medications met our threshold criteria for a potential protective effect on dementia and fell into five classes: catecholamine modulators, anticonvulsants, antibiotics/antivirals, anticoagulants, and a miscellaneous group.

Discussion

The biological pathways of the medications identified in this analysis may be targets for further research and may aid in discovering novel therapeutic approaches to treat dementia. These data show association not causality.

Reproducing Protocol-Based Studies Using Parameterizable Tools-Comparison of Analytic Approaches Used by Two Medical Product Surveillance Networks

The US Sentinel System and the Canadian Network for Observational Drug Effect Studies (CNODES) are two medical product safety surveillance networks. Using Sentinel's preprogrammed, parameterizable analytic tools, we reproduced two protocol-based studies conducted by CNODES to assess the risks of acute pancreatitis and heart failure (HF) associated with the use of incretin-based drugs, compared with use of ≥ 2 oral hypoglycemic agents. Results from the replication new-user cohort analyses aligned with those from the CNODES nested case-control studies. The adjusted hazard ratios were 0.95 (0.81-1.12; vs. 1.03 (0.87-1.22) in CNODES) for acute pancreatitis and 0.91 (0.84-1.00; vs. 0.82 (0.67-1.00) in CNODES) for HF among patients without HF history. The CNODES's common protocol approach allows studies tailored to specific safety questions, whereas the Sentinel's common data model plus pretested program approach enables more rapid analysis. Despite these differences, it is possible to obtain comparable results using both approaches.

Choosing Among Common Data Models for Real-World Data Analyses Fit for Making Decisions About the Effectiveness of Medical Products

Many real-world data analyses use common data models (CDMs) to standardize terminologies for medication use, medical events and procedures, data structures, and interpretations of data to facilitate analyses across data sources. For decision makers, key aspects that influence the choice of a CDM may include (i) adaptability to a specific question; (ii) transparency to reproduce findings, assess validity, and instill confidence in findings; and (iii) ease and speed of use. Organizing CDMs preserve the original information from a data source and have maximum adaptability. Full mapping data models, or preconfigured rules systems, are easy to use, since all raw codes are mapped to medical constructs. Adaptive rule systems grow libraries of reusable measures that can easily adjust to preserve adaptability, expedite analyses, and ensure study-specific transparency.

Evaluation of Switching Patterns in FDA's Sentinel System: A New Tool to Assess Generic Drugs

INTRODUCTION

Nearly 90% of drugs dispensed in the US are generic products.

OBJECTIVE

The aim of this study was to develop and implement a tool for analyzing manufacturer-level drug utilization and switching patterns within the US Food and Drug Administration's Sentinel system.

METHODS

A descriptive tool was designed to analyze data in the Sentinel common data model and was tested with two case studies-metoprolol extended release (ER) and lamotrigine ER-using claims data from four Sentinel data partners. We plotted initiators of each brand and generic product over time. For metoprolol ER, we evaluated rates of switching from generics around the time of manufacturing issues. For lamotrigine ER, we examined rates of switching back to the brand among those who switched from brand to generic.

RESULTS

We identified 1,651,285 initiators of metoprolol ER products between July 2008 and September 2015. We observed a large decrease in monthly metoprolol ER initiators (from 25,465 in December 2008 to 13,128 in February 2009), corresponding to recalls by generic manufacturers. We observed simultaneous increases in utilization of the authorized generic and brand products. We identified 4266 initiators of lamotrigine ER with an epilepsy diagnosis between January 2012 and September 2015. Among those who switched from brand to generic, the cumulative incidence of switching back was close to 20% at 2 years. Switchback rates were higher for the first available generic products.

CONCLUSIONS

This developed tool was able to elucidate novel utilization and switching patterns in two case studies. Such information can be used to support surveillance of generic drugs and biosimilars.

The Devil's in the details: Reports on reproducibility in pharmacoepidemiologic studies

PURPOSE

The U.S. Food and Drug Administration's Sentinel Initiative "modular programs" have been shown to replicate findings from conventional protocol-driven, custom-programmed studies. One such parallel assessment-dabigatran and warfarin and selected outcomes-produced concordant findings for three of four study outcomes. The effect estimates and confidence intervals for the fourth-acute myocardial infarction-had more variability as compared with other outcomes. This paper evaluates the potential sources of that variability that led to unexpected divergence in findings.

METHODS

We systematically compared the two studies and evaluated programming differences and their potential impact using a different dataset that allowed more granular data access for investigation. We reviewed the output at each of five main processing steps common in both study programs: cohort identification, propensity score estimation, propensity score matching, patient follow-up, and risk estimation.

RESULTS

Our findings point to several design features that warrant greater investigator attention when performing observational database studies: (a) treatment of recorded events (eg, diagnoses, procedures, and dispensings) co-occurring on the index date of study drug dispensing in cohort eligibility criteria and propensity score estimation and (b) construction of treatment episodes for study drugs of interest that have more complex dispensing patterns.

CONCLUSIONS

More precise and unambiguous operational definitions of all study parameters will increase transparency and reproducibility in observational database studies.

Data Mining for Adverse Drug Events With a Propensity Score-matched Tree-based Scan Statistic

The tree-based scan statistic is a statistical data mining tool that has been used for signal detection with a self-controlled design in vaccine safety studies. This disproportionality statistic adjusts for multiple testing in evaluation of thousands of potential adverse events. However, many drug safety questions are not well suited for self-controlled analysis. We propose a method that combines tree-based scan statistics with propensity score-matched analysis of new initiator cohorts, a robust design for investigations of drug safety. We conducted plasmode simulations to evaluate performance. In multiple realistic scenarios, tree-based scan statistics in cohorts that were propensity score matched to adjust for confounding outperformed tree-based scan statistics in unmatched cohorts. In scenarios where confounding moved point estimates away from the null, adjusted analyses recovered the prespecified type 1 error while unadjusted analyses inflated type 1 error. In scenarios where confounding moved point estimates toward the null, adjusted analyses preserved power, whereas unadjusted analyses greatly reduced power. Although complete adjustment of true confounders had the best performance, matching on a moderately mis-specified propensity score substantially improved type 1 error and power compared with no adjustment. When there was true elevation in risk of an adverse event, there were often co-occurring signals for clinically related concepts. TreeScan with propensity score matching shows promise as a method for screening and prioritization of potential adverse events. It should be followed by clinical review and safety studies specifically designed to quantify the magnitude of effect, with confounding control targeted to the outcome of interest.

Association of Risk for Venous Thromboembolism With Use of Low-Dose Extended- and Continuous-Cycle Combined Oral Contraceptives: A Safety Study Using the Sentinel Distributed Database

IMPORTANCE

Continuous/extended cyclic estrogen use (84/7 or 365/0 days cycles) in combined oral contraceptives (COCs) could potentially expose women to an increased cumulative dose of estrogen, compared with traditional cyclic regimens (21/7 days cycle), and may increase the risk for venous thromboembolism (VTE).

OBJECTIVE

To determine, while holding the progestogen type constant, whether the risk for VTE is higher with use of continuous/extended COCs than with cyclic COCs among women who initiated a COC containing ethinyl estradiol and levonorgestrel.

DESIGN, SETTING, AND PARTICIPANTS

Incident user retrospective cohort study of primarily commercially insured US population identified from the Sentinel Distributed Database. Participants were women aged 18 to 50 years at the time of initiating a study COC between May 2007 and September 2015. Using a propensity score approach and Cox proportional hazards regression models, we estimated the hazard ratios of VTE overall and separately by ethinyl estradiol dose and age groups.

EXPOSURES

Initiation of continuous/extended or traditional cyclic COCs containing ethinyl estradiol or levonorgestrel of any dose.

MAIN OUTCOMES AND MEASURES

First VTE hospitalization that occurred during the study follow-up, identified by an inpatient International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis code of 415.1, 415.1x, 453, 453.x, or 453.xx.

RESULTS

We identified 210 691 initiators of continuous/extended COCs (mean [SD] age, 30.4 [8.6] years) and 522 316 initiators of cyclic COCs (mean [SD] age, 28.8 [8.3] years), with a mean of 0.7 person-years at risk among continuous/extended and cyclic users. Baseline cardiovascular and metabolic conditions (7.2% vs 4.7%), gynecological conditions (39.7% vs 32.3%), and health services utilization were slightly higher among continuous/extended cyclic than cyclic COC users. Propensity score matching decreased the hazard ratio estimates from 1.84 (95% CI, 1.53-2.21) to 1.32 (95% CI, 1.07-1.64) for continuous/extended use compared with cyclic COC use. The absolute risk difference (0.27 per 1000 persons) and the incidence rate difference (0.35 cases per 1000 person-years [1.44 vs 1.09 cases per 1000 person-years]) between the 2 propensity score-matched cohorts remained low, which may not translate into a clinically significant risk differences between cyclic and noncyclic estrogen use.

CONCLUSIONS AND RELEVANCE

Holding the progestogen type constant (levonorgestrel), we observed a slightly elevated VTE risk in association with continuous/extended COC use when compared with cyclic COC use. However, due to the small absolute risk difference and potential residual confounding, our findings did not show strong evidence supporting a VTE risk difference between continuous/extended and cyclic COC use.

Relative Performance of Propensity Score Matching Strategies for Subgroup Analyses

Postapproval drug safety studies often use propensity scores (PSs) to adjust for a large number of baseline confounders. These studies may involve examining whether treatment safety varies across subgroups. There are many ways a PS could be used to adjust for confounding in subgroup analyses. These methods have trade-offs that are not well understood. We conducted a plasmode simulation to compare relative performance of 5 methods involving PS matching for subgroup analysis, including methods frequently used in applied literature whose performance has not been previously directly compared. These methods varied as to whether the overall PS, subgroup-specific PS, or no rematching was used in subgroup analysis as well as whether subgroups were fully nested within the main analytical cohort. The evaluated PS subgroup matching methods performed similarly in terms of balance, bias, and precision in 12 simulated scenarios varying size of the cohort, prevalence of exposure and outcome, strength of relationships between baseline covariates and exposure, the true effect within subgroups, and the degree of confounding within subgroups. Each had strengths and limitations with respect to other performance metrics that could inform choice of method.

Automated data-adaptive analytics for electronic healthcare data to study causal treatment effects

BACKGROUND

Decision makers in health care increasingly rely on nonrandomized database analyses to assess the effectiveness, safety, and value of medical products. Health care data scientists use data-adaptive approaches that automatically optimize confounding control to study causal treatment effects. This article summarizes relevant experiences and extensions.

METHODS

The literature was reviewed on the uses of high-dimensional propensity score (HDPS) and related approaches for health care database analyses, including methodological articles on their performance and improvement. Articles were grouped into applications, comparative performance studies, and statistical simulation experiments.

RESULTS

The HDPS algorithm has been referenced frequently with a variety of clinical applications and data sources from around the world. The appeal of HDPS for database research rests in 1) its superior performance in situations of unobserved confounding through proxy adjustment, 2) its predictable efficiency in extracting confounding information from a given data source, 3) its ability to automate estimation of causal treatment effects to the extent achievable in a given data source, and 4) its independence of data source and coding system. Extensions of the HDPS approach have focused on improving variable selection when exposure is sparse, using free text information and time-varying confounding adjustment.

CONCLUSION

Semiautomated and optimized confounding adjustment in health care database analyses has proven successful across a wide range of settings. Machine-learning extensions further automate its use in estimating causal treatment effects across a range of data scenarios.

Assessing the impact of the new ICD-10-CM coding system on pharmacoepidemiologic studies-An application to the known association between angiotensin-converting enzyme inhibitors and angioedema

PURPOSE

To replicate the well-established association between angiotensin-converting enzyme inhibitors versus beta blockers and angioedema in the International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) era.

METHODS

We conducted a retrospective, inception cohort study in a large insurance database formatted to the Sentinel Common Data Model. We defined study periods spanning the ICD-9-CM era only, ICD-10-CM era only, and ICD-9-CM and ICD-10-CM era and conducted simple-forward mapping (SFM), simple-backward mapping (SBM), and forward-backward mapping (FBM) referencing the General Equivalence Mappings to translate the outcome (angioedema) and covariates from ICD-9-CM to ICD-10-CM. We performed propensity score (PS)-matched and PS-stratified Cox proportional hazards regression to estimate hazard ratios (HRs) and 95% confidence intervals (CIs).

RESULTS

In the ICD-9-CM and ICD-10-CM eras spanning April 1 to September 30 of 2015 and 2016, there were 152 017 and 145 232 angiotensin-converting enzyme inhibitor initiators and 115 073 and 116 652 beta-blocker initiators, respectively. The PS-matched HR was 4.19 (95% CI, 2.82-6.23) in the ICD-9-CM era, 4.37 (2.92-6.52) in the ICD-10-CM era using SFM, and 4.64 (3.05-7.07) in the ICD-10-CM era using SBM and FBM. The PS-matched HRs from the mixed ICD-9-CM and ICD-10-CM eras ranged from 3.91 (2.69-5.68) to 4.35 (3.33-5.70).

CONCLUSION

The adjusted HRs across different diagnostic coding eras and the use of SFM versus SBM and FBM produced numerically different but clinically similar results. Additional investigations as ICD-10-CM data accumulate are warranted.

A Query Workflow Design to Perform Automatable Distributed Regression Analysis in Large Distributed Data Networks

Introduction:

Patient privacy and data security concerns often limit the feasibility of pooling patient-level data from multiple sources for analysis. Distributed data networks (DDNs) that employ privacy-protecting analytical methods, such as distributed regression analysis (DRA), can mitigate these concerns. However, DRA is not routinely implemented in large DDNs.

Objective:

We describe the design and implementation of a process framework and query workflow that allow automatable DRA in real-world DDNs that use PopMedNet™, an open-source distributed networking software platform.

Methods:

We surveyed and catalogued existing hardware and software configurations at all data partners in the Sentinel System, a PopMedNet-driven DDN. Key guiding principles for the design included minimal disruptions to the current PopMedNet query workflow and minimal modifications to data partners’ hardware configurations and software requirements.

Results:

We developed and implemented a three-step process framework and PopMedNet query workflow that enables automatable DRA: 1) assembling a de-identified patient-level dataset at each data partner, 2) distributing a DRA package to data partners for local iterative analysis, and 3) iteratively transferring intermediate files between data partners and analysis center. The DRA query workflow is agnostic to statistical software, accommodates different regression models, and allows different levels of user-specified automation.

Discussion:

The process framework can be generalized to and the query workflow can be adopted by other PopMedNet-based DDNs.

Conclusion:

DRA has great potential to change the paradigm of data analysis in DDNs. Successful implementation of DRA in Sentinel will facilitate adoption of the analytic approach in other DDNs.

Evaluation of the US Food and Drug Administration sentinel analysis tools in confirming previously observed drug-outcome associations: The case of clindamycin and Clostridium difficile infection

PURPOSE

The Food and Drug Administration's Sentinel System developed parameterized, reusable analytic programs for evaluation of medical product safety. Research on outpatient antibiotic exposures, and Clostridium difficile infection (CDI) with non-user reference groups led us to expect a higher rate of CDI among outpatient clindamycin users vs penicillin users. We evaluated the ability of the Cohort Identification and Descriptive Analysis and Propensity Score Matching tools to identify a higher rate of CDI among clindamycin users.

METHODS

We matched new users of outpatient dispensings of oral clindamycin or penicillin from 13 Data Partners 1:1 on propensity score and followed them for up to 60 days for development of CDI. We used Cox proportional hazards regression stratified by Data Partner and matched pair to compare CDI incidence.

RESULTS

Propensity score models at 3 Data Partners had convergence warnings and a limited range of predicted values. We excluded these Data Partners despite adequate covariate balance after matching. From the 10 Data Partners where these models converged without warnings, we identified 807 919 new clindamycin users and 8 815 441 new penicillin users eligible for the analysis. The stratified analysis of 807 769 matched pairs included 840 events among clindamycin users and 290 among penicillin users (hazard ratio 2.90, 95% confidence interval 2.53, 3.31).

CONCLUSIONS

This evaluation produced an expected result and identified several potential enhancements to the Propensity Score Matching tool. This study has important limitations. CDI risk may have been related to factors other than the inherent properties of the drugs, such as duration of use or subsequent exposures.

Development and application of two semi-automated tools for targeted medical product surveillance in a distributed data network

PURPOSE OF REVIEW

An important component of the Food and Drug Administration's Sentinel Initiative is the active post-market risk identification and analysis (ARIA) system, which utilizes semi-automated, parameterized computer programs to implement propensity-score adjusted and self-controlled risk interval designs to conduct targeted surveillance of medical products in the Sentinel Distributed Database. In this manuscript, we review literature relevant to the development of these programs and describe their application within the Sentinel Initiative.

RECENT FINDINGS

These quality-checked and publicly available tools have been successfully used to conduct rapid, replicable, and targeted safety analyses of several medical products. In addition to speed and reproducibility, use of semi-automated tools allows investigators to focus on decisions regarding key methodological parameters. We also identified challenges associated with the use of these methods in distributed and prospective datasets like the Sentinel Distributed Database, namely uncertainty regarding the optimal approach to estimating propensity scores in dynamic data among data partners of heterogeneous size.

SUMMARY

Future research should focus on the methodological challenges raised by these applications as well as developing new modular programs for targeted surveillance of medical products.

Sentinel Modular Program for Propensity Score-Matched Cohort Analyses: Application to Glyburide, Glipizide, and Serious Hypoglycemia

Sentinel is a program sponsored by the US Food and Drug Administration to monitor the safety of medical products. We conducted a cohort assessment to evaluate the ability of the Sentinel Propensity Score Matching Tool to reproduce in an expedited fashion the known association between glyburide (vs. glipizide) and serious hypoglycemia. Thirteen data partners who contribute to the Sentinel Distributed Database participated in this analysis. A pretested and customizable analytic program was run at each individual site. De-identified summary results from each data partner were returned and aggregated at the Sentinel Operations Center. We identified a total of 198,550 and 379,507 new users of glyburide and glipizide, respectively. The incidence of emergency department visits and hospital admissions for serious hypoglycemia was 19 per 1000 person-years (95% confidence interval = 17.9, 19.7) for glyburide users and 22 (21.6, 22.7) for glipizide users. In cohorts matched by propensity score based on predefined variables, the hazard ratio (HR) for glyburide was 1.36 (1.24, 1.49) versus glipizide. In cohorts matched on a high-dimensional propensity score based on empirically selected variables, for which the program ran to completion in five data partners, the HR was 1.49 (1.31, 1.70). In cohorts matched on propensity scores based on both predefined and empirically selected variables via the high-dimensional propensity score algorithm (the same five data partners), the HR was 1.51 (1.32, 1.71). These findings are consistent with the literature, and demonstrate the ability of the Sentinel Propensity Score Matching Tool to reproduce this known association in an expedited fashion.See video abstract at, http://links.lww.com/EDE/B275.