Using negative control outcomes to assess the comparability of treatment groups among women with osteoporosis in the United States

Evaluating the comparability of osteoporosis treatments using propensity score and negative control outcome methods in UK and Denmark electronic health record databases

Evidence on the comparative effectiveness of osteoporosis treatments is heterogeneous. This may be attributed to different populations and clinical practice, but also to differing methodologies ensuring comparability of treatment groups before treatment effect estimation and the amount of residual confounding by indication. This study assessed the comparability of denosumab vs oral bisphosphonate (OBP) groups using propensity score (PS) methods and negative control outcome (NCO) analysis. A total of 280 288 women aged ≥50 yr initiating denosumab or OBP in 2011-2018 were included from the UK Clinical Practice Research Datalink (CPRD) and the Danish National Registries (DNR). Balance of observed covariates was assessed using absolute standardized mean difference (ASMD) before and after PS weighting, matching, and stratification, with ASMD >0.1 indicating imbalance. Residual confounding was assessed using NCOs with ≥100 events. Hazard ratio (HR) and 95%CI between treatment and NCO were estimated using Cox models. Presence of residual confounding was evaluated with 2 approaches (1) >5% of NCOs with 95% CI excluding 1, (2) >5% of NCOs with an upper CI <0.75 or lower CI >1.3. The number of imbalanced covariates before adjustment (CPRD 22/87; DNR 18/83) decreased, with 2%-11% imbalance remaining after weighting, matching, or stratification. Using approach 1, residual confounding was present for all PS methods in both databases (≥8% of NCOs), except for stratification in DNR (3.8%). Using approach 2, residual confounding was present in CPRD with PS matching (5.3%) and stratification (6.4%), but not with weighting (4.3%). Within DNR, no NCOs had HR estimates with upper or lower CI limits beyond the specified bounds indicating residual confounding for any PS method. Achievement of covariate balance and determination of residual bias were dependent upon several factors including the population under study, PS method, prevalence of NCO, and the threshold indicating residual confounding.

Comparative effectiveness of denosumab vs alendronate among postmenopausal women with osteoporosis

Although clinical trials have shown that denosumab significantly increases bone mineral density at key skeletal sites more than oral bisphosphonates, evidence is lacking from head-to-head randomized trials evaluating fracture outcomes. This retrospective cohort study uses administrative claims data from Medicare fee-for service beneficiaries to evaluate the comparative effectiveness of denosumab vs alendronate in reducing fracture risk among women with PMO in the US. Women with PMO ≥ 66 yr of age with no prior history of osteoporosis treatment, who initiated denosumab (n = 89 115) or alendronate (n = 389 536) from 2012 to 2018, were followed from treatment initiation until the first of a specific fracture outcome, treatment discontinuation or switch, end of study (December 31, 2019), or other censoring criteria. A doubly robust inverse-probability of treatment and censoring weighted function was used to estimate the risk ratio associated with the use of denosumab compared with alendronate for hip, nonvertebral (NV; includes hip, humerus, pelvis, radius/ulna, other femur), non-hip nonvertebral (NHNV), hospitalized vertebral (HV), and major osteoporotic (MOP; consisting of NV and HV) fractures. Overall, denosumab reduced the risk of MOP by 39%, hip by 36%, NV by 43%, NHNV by 50%, and HV fractures by 30% compared with alendronate. Denosumab reduced the risk of MOP fractures by 9% at year 1, 12% at year 2, 18% at year 3, and 31% at year 5. An increase in the magnitude of fracture risk reduction with increasing duration of exposure was also observed for other NV fracture outcomes. In this cohort of almost half-a-million treatment-naive women with PMO, we observed clinically significant reductions in the risk of MOP, hip, NV, NHNV, and HV fractures for patients on denosumab compared with alendronate. Patients who remained on denosumab for longer periods of time experienced greater reductions in fracture risk.

Causal association between long-term exposure to air pollution and incident Parkinson's disease

Epidemiological evidence for long-term air pollution exposure and Parkinson's disease (PD) is controversial, and analysis of causality is limited. We identified 293,888 participants who were free of PD at baseline in the UK Biobank (2006-2010). Time-varying air pollution [fine particulate (PM2.5) and ozone (O3)] exposures were estimated using spatio-temporal models. Incident cases of PD were identified using validated algorithms. Four methods were used to investigate the associations between air pollution and PD, including (1) standard time-varying Cox proportional-hazard model; (2) Cox models weighted by generalized propensity score (GPS) and inverse-probability weights (IPW); (3) instrumental variable (IV) analysis; and (4) negative control outcome analysis. During a median of 11.6 years of follow-up, 1822 incident PD cases were identified. Based on standard Cox regression, the hazard ratios (95% confidence interval) for a 1 µg/m3 or ppb increase in PM2.5 and O3 were 1.23 (1.17, 1.30) and 1.02 (0.98, 1.05), respectively. Consistent results were found in models weighted by GPS and IPW, and in IV analysis. There were no significant associations between air pollution and negative control outcomes. This study provides evidence to support a causal association between PM2.5 exposure and PD. Mitigation of air pollution could be a protective measure against PD.

Staging and clean room: Constructs designed to facilitate transparency and reduce bias in comparative analyses of real-world data

PURPOSE

We describe constructs designed to protect the integrity of the results from comparative analyses using real-world data (RWD): staging and clean room.

METHODS

Staging involves performing sequential preliminary analyses and evaluating the population size available and potential bias before conducting comparative analyses. A clean room involves restricted access to data and preliminary results, policies governing exploratory analyses and protocol deviations, and audit trail. These constructs are intended to allow decisions about protocol deviations, such as changes to design or model specification, to be made without knowledge of how they might affect subsequent analyses. We describe an example for implementing staging with a clean room.

RESULTS

Stage 1 may involve selecting a data source, developing and registering a protocol, establishing a clean room, and applying inclusion/exclusion criteria. Stage 2 may involve attempting to achieve covariate balance, often through propensity score models. Stage 3 may involve evaluating the presence of residual confounding using negative control outcomes. After each stage, check points may be implemented when a team of statisticians, epidemiologists and clinicians masked to how their decisions may affect study outcomes, reviews the results. This review team may be tasked with making recommendations for protocol deviations to address study precision or bias. They may recommend proceeding to the next stage, conducting additional analyses to address bias, or terminating the study. Stage 4 may involve conducting the comparative analyses.

CONCLUSIONS

The staging and clean room constructs are intended to protect the integrity and enhance confidence in the results of analyses of RWD.

The State of Use and Utility of Negative Controls in Pharmacoepidemiologic Studies

Uses of real-world data in drug safety and effectiveness studies are often challenged by various sources of bias. We undertook a systematic search of the published literature through September 2020 to evaluate the state of use and utility of negative controls to address bias in pharmacoepidemiologic studies. Two reviewers independently evaluated study eligibility and abstracted data. Our search identified 184 eligible studies for inclusion. Cohort studies (115, 63%) and administrative data (114, 62%) were, respectively, the most common study design and data type used. Most studies used negative control outcomes (91, 50%), and for most studies the target source of bias was unmeasured confounding (93, 51%). We identified 4 utility domains of negative controls: 1) bias detection (149, 81%), 2) bias correction (16, 9%), 3) P-value calibration (8, 4%), and 4) performance assessment of different methods used in drug safety studies (31, 17%). The most popular methodologies used were the 95% confidence interval and P-value calibration. In addition, we identified 2 reference sets with structured steps to check the causality assumption of the negative control. While negative controls are powerful tools in bias detection, we found many studies lacked checking the underlying assumptions. This article is part of a Special Collection on Pharmacoepidemiology.

Cardiovascular Safety in Postmenopausal Women and Men With Osteoporosis Treated With Denosumab and Zoledronic Acid: A Post-Authorization Safety Study

Osteoporosis and cardiovascular disease are common in older adults. Treatment of osteoporosis reduces the burden of debilitating fractures; however, it is important to understand the benefit versus risk of treatment. This study evaluates the risk of stroke (ischemic or hemorrhagic) and myocardial infarction (MI) among postmenopausal women and men initiating osteoporosis treatment with denosumab (receptor activator of nuclear factor κB ligand [RANKL] inhibitor) or zoledronic acid (bisphosphonate) between October 2010 and June 2019. A retrospective cohort study employing the new user/active comparator design was conducted. Analyses were conducted separately in two national US commercial databases, MarketScan® and Optum® for reproducibility. Inverse probability of treatment and censoring weighting was employed to control for confounding and informative censoring. Cumulative risks at 6-month, 12-month, and 36-month time points were calculated and adjusted risk ratios and differences (with 95% confidence intervals [CIs]) were estimated. In MarketScan® and Optum® databases, 96,611 and 73,127 patients met all study eligibility criteria, respectively. At 36 months, the risk ratio estimates (zoledronic acid referent group) were 1.22 (95% CI, 0.77-1.66) and 0.97 (95% CI, 0.63-1.32) for MI and 1.00 (95% CI, 0.61-1.40) and 0.87 (95% CI, 0.56-1.17) for stroke in MarketScan and Optum, respectively. Most of the treatment associations across the other time periods and outcomes also had 95% CIs including the null value. In these large samples of real-world US patients, no increased risk in MI and stroke were identified for up to 36 months of treatment in denosumab users compared with zoledronic acid users. © 2023 Amgen. JBMR Plus published by Wiley Periodicals LLC on behalf of American Society for Bone and Mineral Research.

External Comparator Groups Derived from Real-world Data Used in Support of Regulatory Decision Making: Use Cases and Challenges

Real-world data (RWD) from electronic health records (EHRs) and administrative claims databases are used increasingly to generate real-world evidence (RWE). RWE is used to support clinical evidence packages for medicines that inform decision-makers. In this review of current issues in the use of RWD-derived external comparator groups to support regulatory filings, we assess a series of topics that generally apply across many disease indications. However, most of the examples and illustrations focus on the oncology clinical research setting. The topics include an overview of current uses of RWD in drug development, a discussion of regulatory filings using RWD-derived external comparators, a brief overview of guidance documents and white papers pertaining to external comparators, a summary of some limitations and methodological issues in the use of external comparator groups and finally, a look at the future of this area and recommendations.

Comparative risk of thrombosis with thrombocytopenia syndrome or thromboembolic events associated with different covid-19 vaccines: international network cohort study from five European countries and the US

OBJECTIVE

To quantify the comparative risk of thrombosis with thrombocytopenia syndrome or thromboembolic events associated with use of adenovirus based covid-19 vaccines versus mRNA based covid-19 vaccines.

DESIGN

International network cohort study.

SETTING

Routinely collected health data from contributing datasets in France, Germany, the Netherlands, Spain, the UK, and the US.

PARTICIPANTS

Adults (age ≥18 years) registered at any contributing database and who received at least one dose of a covid-19 vaccine (ChAdOx1-S (Oxford-AstraZeneca), BNT162b2 (Pfizer-BioNTech), mRNA-1273 (Moderna), or Ad26.COV2.S (Janssen/Johnson & Johnson)), from December 2020 to mid-2021.

MAIN OUTCOME MEASURES

Thrombosis with thrombocytopenia syndrome or venous or arterial thromboembolic events within the 28 days after covid-19 vaccination. Incidence rate ratios were estimated after propensity scores matching and were calibrated using negative control outcomes. Estimates specific to the database were pooled by use of random effects meta-analyses.

RESULTS

Overall, 1 332 719 of 3 829 822 first dose ChAdOx1-S recipients were matched to 2 124 339 of 2 149 679 BNT162b2 recipients from Germany and the UK. Additionally, 762 517 of 772 678 people receiving Ad26.COV2.S were matched to 2 851 976 of 7 606 693 receiving BNT162b2 in Germany, Spain, and the US. All 628 164 Ad26.COV2.S recipients from the US were matched to 2 230 157 of 3 923 371 mRNA-1273 recipients. A total of 862 thrombocytopenia events were observed in the matched first dose ChAdOx1-S recipients from Germany and the UK, and 520 events after a first dose of BNT162b2. Comparing ChAdOx1-S with a first dose of BNT162b2 revealed an increased risk of thrombocytopenia (pooled calibrated incidence rate ratio 1.33 (95% confidence interval 1.18 to 1.50) and calibrated incidence rate difference of 1.18 (0.57 to 1.8) per 1000 person years). Additionally, a pooled calibrated incidence rate ratio of 2.26 (0.93 to 5.52) for venous thrombosis with thrombocytopenia syndrome was seen with Ad26.COV2.S compared with BNT162b2.

CONCLUSIONS

In this multinational study, a pooled 30% increased risk of thrombocytopenia after a first dose of the ChAdOx1-S vaccine was observed, as was a trend towards an increased risk of venous thrombosis with thrombocytopenia syndrome after Ad26.COV2.S compared with BNT162b2. Although rare, the observed risks after adenovirus based vaccines should be considered when planning further immunisation campaigns and future vaccine development.

Real-world effectiveness of osteoporosis treatments in Germany

This observational study assessed the impact on the fracture incidence of osteoporosis medications in postmenopausal women in Germany. Continued treatment with osteoporosis medications was associated with reductions of fracture rates in a real-world setting.

PURPOSE

The efficacy of osteoporosis medications has been demonstrated in clinical trials, but a lack of evidence exists of their real-world effectiveness. This real-world study assessed the treatment patterns and impact on the fracture incidence of osteoporosis medications in postmenopausal women in Germany.

METHODS

This cohort study used data from the WIG2 benchmark database, a German anonymised healthcare claims database. All women ≥ 50 years of age with ≥ 1 prescription for osteoporosis medication between 1 January 2013 and 31 December 2017 were included. The primary outcome was treatment effectiveness, evaluated as the change in fracture incidence after initiating treatment. Fracture types included all fractures, clinical vertebral, hip and wrist/forearm. Fracture incidence was assessed during the early-treatment period (0-3 months) and the on-treatment period (4-12, 13-24, 25-36 and 37-48 months).

RESULTS

Baseline covariates and treatment patterns were determined for 41,861 patients. The median duration of therapy was longer with denosumab (587 days) than with intravenous ibandronate (451 days), intravenous zoledronate (389 days) or oral bisphosphonates (258 days). The baseline incidence rate of all fractures was higher in patients receiving denosumab than in those receiving other treatments (87.6, 78.2, 56.6 and 66.0 per 1000 person-years for denosumab, oral bisphosphonates, intravenous ibandronate and intravenous zoledronate, respectively). Rates of all fractures declined with continued denosumab (by 38%, 50%, 56% and 67% at 12, 24, 36 and 48 months, respectively) and oral bisphosphonates (by 39%, 44%, 49% and 42%, respectively) treatment.

CONCLUSION

Continued treatment with osteoporosis medications was associated with reductions of fracture rates in a real-world setting.

Use of negative control outcomes to assess the comparability of patients initiating lipid-lowering therapies

PURPOSE

Clinical trials have demonstrated efficacy of proprotein convertase subtilisin/kexin type 9 inhibitors (PCSK9i) in reducing risk of cardiovascular disease events, but effectiveness in routine clinical care has not been well-studied. We used negative control outcomes to assess potential confounding in an observational study of PCSK9i versus ezetimibe or high-intensity statin.

METHODS

Using commercial claims, we identified U.S. adults initiating PCSK9i, ezetimibe, or high-intensity statin in 2015-2018, with other lipid-lowering therapy (LLT) use in the year prior (LLT cohort) or atherosclerotic cardiovascular disease (ASCVD) in the past 90 days (ASCVD cohort). We compared initiators of PCSK9i to ezetimibe and high-intensity statin by estimating one-year risks of negative control outcomes influenced by frailty or health-seeking behaviors. Inverse probability of treatment and censoring weighted estimators of risk differences (RDs) were used to evaluate residual confounding after controlling for covariates.

RESULTS

PCSK9i initiators had lower one-year risks of negative control outcomes associated with frailty, such as decubitus ulcer in the ASCVD cohort (PCSK9i vs. high-intensity statin RD = -3.5%, 95% confidence interval (CI): -4.6%, -2.5%; PCSK9i vs. ezetimibe RD = -1.3%, 95% CI: -2.1%, -0.6%), with similar but attenuated associations in the LLT cohort. Lower risks of accidents and fractures were also observed for PCSK9i, varying by cohort. Risks were similar for outcomes associated with health-seeking behaviors, although trended higher for PCSK9i in the ASCVD cohort.

CONCLUSIONS

Observed associations suggest lower frailty and potentially greater health-seeking behaviors among PCSK9i initiators, particularly those with a recent ASCVD diagnosis, with the potential to bias real-world analyses of treatment effectiveness.