Bias-variance trade-off in pharmacoepidemiological studies using physician-preference-based instrumental variables: a simulation study

Assessing the performance of physician's prescribing preference as an instrumental variable in comparative effectiveness research with moderate and small sample sizes: a simulation study

Aim: This simulation study is to assess the utility of physician's prescribing preference (PPP) as an instrumental variable for moderate and smaller sample sizes. Materials & methods: We designed a simulation study to imitate a comparative effectiveness research under different sample sizes. We compare the performance of instrumental variable (IV) and non-IV approaches using two-stage least squares (2SLS) and ordinary least squares (OLS) methods, respectively. Further, we test the performance of different forms of proxies for PPP as an IV. Results: The percent bias of 2SLS is around approximately 20%, while the percent bias of OLS is close to 60%. The sample size is not associated with the level of bias for the PPP IV approach. Conclusion: Irrespective of sample size, the PPP IV approach leads to less biased estimates of treatment effectiveness than OLS adjusting for known confounding only. Particularly for smaller sample sizes, we recommend constructing PPP from long prescribing histories to improve statistical power.

Comparative Analysis of Instrumental Variables on the Assignment of Buprenorphine/Naloxone or Methadone for the Treatment of Opioid Use Disorder

BACKGROUND

Instrumental variable (IV) analysis provides an alternative set of identification assumptions in the presence of uncontrolled confounding when attempting to estimate causal effects. Our objective was to evaluate the suitability of measures of prescriber preference and calendar time as potential IVs to evaluate the comparative effectiveness of buprenorphine/naloxone versus methadone for treatment of opioid use disorder (OUD).

METHODS

Using linked population-level health administrative data, we constructed five IVs: prescribing preference at the individual, facility, and region levels (continuous and categorical variables), calendar time, and a binary prescriber's preference IV in analyzing the treatment assignment-treatment discontinuation association using both incident-user and prevalent-new-user designs. Using published guidelines, we assessed and compared each IV according to the four assumptions for IVs, employing both empirical assessment and content expertise. We evaluated the robustness of results using sensitivity analyses.

RESULTS

The study sample included 35,904 incident users (43.3% on buprenorphine/naloxone) initiated on opioid agonist treatment by 1585 prescribers during the study period. While all candidate IVs were strong (A1) according to conventional criteria, by expert opinion, we found no evidence against assumptions of exclusion (A2), independence (A3), monotonicity (A4a), and homogeneity (A4b) for prescribing preference-based IV. Some criteria were violated for the calendar time-based IV. We determined that preference in provider-level prescribing, measured on a continuous scale, was the most suitable IV for comparative effectiveness of buprenorphine/naloxone and methadone for the treatment of OUD.

CONCLUSIONS

Our results suggest that prescriber's preference measures are suitable IVs in comparative effectiveness studies of treatment for OUD.

Dealing with confounding in observational studies: A scoping review of methods evaluated in simulation studies with single-point exposure

The aim of this article was to perform a scoping review of methods available for dealing with confounding when analyzing the effect of health care treatments with single-point exposure in observational data. We aim to provide an overview of methods and their performance assessed by simulation studies indexed in PubMed. We searched PubMed for simulation studies published until January 2021. Our search was restricted to studies evaluating binary treatments and binary and/or continuous outcomes. Information was extracted on the methods' assumptions, performance, and technical properties. Of 28,548 identified references, 127 studies were eligible for inclusion. Of them, 84 assessed 14 different methods (ie, groups of estimators that share assumptions and implementation) for dealing with measured confounding, and 43 assessed 10 different methods for dealing with unmeasured confounding. Results suggest that there are large differences in performance between methods and that the performance of a specific method is highly dependent on the estimator. Furthermore, the methods' assumptions regarding the specific data features also substantially influence the methods' performance. Finally, the methods result in different estimands (ie, target of inference), which can even vary within methods. In conclusion, when choosing a method to adjust for measured or unmeasured confounding it is important to choose the most appropriate estimand, while considering the population of interest, data structure, and whether the plausibility of the methods' required assumptions hold.

Adapting SIMEX to correct for bias due to interval-censored outcomes in survival analysis with time-varying exposure

Many clinical and epidemiological applications of survival analysis focus on interval-censored events that can be ascertained only at discrete times of clinic visits. This implies that the values of time-varying covariates are not correctly aligned with the true, unknown event times, inducing a bias in the estimated associations. To address this issue, we adapted the simulation-extrapolation (SIMEX) methodology, based on assessing how the estimates change with the artificially increased time between clinic visits. We propose diagnostics to choose the extrapolating function. In simulations, the SIMEX-corrected estimates reduced considerably the bias to the null and generally yielded a better bias/variance trade-off than conventional estimates. In a real-life pharmacoepidemiological application, the proposed method increased by 27% the excess hazard of the estimated association between a time-varying exposure, representing the 2-year cumulative duration of past use of a hypertensive medication, and the hazard of nonmelanoma skin cancer (interval-censored events). These simulation-based and real-life results suggest that the proposed SIMEX-based correction may help improve the accuracy of estimated associations between time-varying exposures and the hazard of interval-censored events in large cohort studies where the events are recorded only at relatively sparse times of clinic visits/assessments. However, these advantages may be less certain for smaller studies and/or weak associations.

Low- versus high-concentration intravenous immunoglobulin for children with Kawasaki disease in the acute phase

PURPOSE

Few studies have compared the effects of low-concentration (5%) and high-concentration (10%) intravenous immunoglobulin (IVIG) preparations for patients with Kawasaki disease (KD) in the acute phase. The purpose of this study was to compare outcomes between low- and high-concentration IVIG preparations in children with KD, using a national inpatient database in Japan.

METHOD

We used the Diagnostic Procedure Combination database to identify patients with KD treated with IVIG from April 2012 to March 2020. We identified those receiving high- and low-concentration IVIG preparations as an initial treatment. The outcomes included the proportions of patients with coronary artery abnormalities (CAAs) and IVIG resistance, length of stay, and medical costs. Propensity score-matched analyses were conducted to compare the outcomes between the 2 groups. Instrumental variable analyses were performed to confirm the results.

RESULT

We identified 48 046 patients with KD and created 4:1 propensity score-matched pairs between the low- and high-concentration IVIG groups. There was a significant difference in the percentage with IVIG resistance between the 2 groups (20.6% vs 24.1%; risk difference, 3.5% [95% confidence interval, 2.3-4.7]; P < .001). However, there was no significant difference in CAAs (1.6% vs 1.6%; risk difference, 0.013% [95% confidence interval, -0.34 to 0.37]; P = .953). The instrumental variable analyses showed similar results.

CONCLUSIONS

The proportion of CAAs did not differ significantly between those receiving low- and high-concentration IVIG. To confirm the results of this study, prospective studies adjusting for duration of IVIG administration and duration of observation are needed.

A theoretical exploration of therapeutic monomania as a physician-based instrumental variable

PURPOSE

To explore the utility of physician prescribing preference as an instrumental variable.

METHODS

Expert (non-systematic) review of relevant literature on the appropriate selection of instrumental variables and theoretical exploration of individual physician and physician group prescriber preference.

RESULTS

An instrumental variable must satisfy three criteria: (1) It must predict the treatment received (strength of the instrument); (2) it cannot influence the outcome other that through the treatment received (exclusion restriction); and (3) it cannot be influenced by any factor that also influences the outcome (independence assumption). Arguments in favor of prescriber preference as an instrumental variable and suggestions for how to approach specific scenarios that may be encountered are offered.

CONCLUSIONS

Prescriber preference, be it of individual physicians or groups of physicians, may, under the right conditions, be powerful instrumental variables. Empiric experimental data are required to determine the appropriateness of combining propensity matching and instrumental variable analysis.

Mortality and Morbidity After Hartmann's Procedure Versus Primary Anastomosis Without a Diverting Stoma for Colorectal Perforation: A Nationwide Observational Study

BACKGROUND

The benefit of primary anastomosis (PA) without a diverting stoma over Hartmann's procedure (HP) for colorectal perforation remains controversial. We compared postoperative mortality and morbidity between HP and PA without a diverting stoma for colorectal perforation of various etiologies.

METHODS

Using the Japanese Diagnosis Procedure Combination database, we extracted data on patients who underwent emergency open laparotomy for colorectal perforation of various etiologies from July 1, 2010 to March 31, 2014. We compared 30-day mortality, postoperative complication rates, and postoperative critical care interventions between HP and PA groups using propensity score matching, inverse probability of treatment weighting, and instrumental variable analyses to adjust for measured and unmeasured confounding factors.

RESULTS

We identified 8500 eligible patients (5455 HP and 3045 PA). In the propensity score-matched model, a significant difference between the HP and PA groups was detected in 30-day mortality (7.7% vs. 9.6%; risk difference, 1.9%; 95% confidence interval [CI], 0.5-3.4). The inverse probability of treatment weighting showed similar results (8.8% vs. 10.7%; risk difference, 1.9%; 95% CI, 1.0-2.8). In the instrumental variable analysis, the point estimate suggested similar direction to that of the propensity score analyses (risk difference, 4.4%; 95% CI, -3.3 to 12.1). The PA group had significantly higher rates of secondary surgery for complications (4.6% vs. 8.4%; risk difference, 3.8%; 95% CI, 2.5-4.1) and slightly longer duration of postoperative critical care interventions.

CONCLUSIONS

This study revealed a significant difference in 30-day mortality between HP and PA without a diverting stoma.

Association of Osteoporosis Medication Use After Hip Fracture With Prevention of Subsequent Nonvertebral Fractures: An Instrumental Variable Analysis

IMPORTANCE

Osteoporosis medication treatment is recommended after hip fracture, yet contemporary estimates of rates of initiation and clinical benefit in the patient population receiving routine care are not well documented.

OBJECTIVES

To report osteoporosis treatment initiation rates between January 1, 2004, and September 30, 2015, and to estimate the risk reduction in subsequent nonvertebral fractures associated with treatment initiation in patients with hip fracture.

DESIGN, SETTING, AND PARTICIPANTS

In this cohort study, data from a commercial insurance claims database from the United States were analyzed. Patients 50 years and older who had a hip fracture and were not receiving treatment with osteoporosis medications before their fracture were included.

EXPOSURE

Prescription dispensing of an osteoporosis medication within 180 days of a hip fracture hospitalization.

MAIN OUTCOMES AND MEASURES

Each initiation episode was matched with 10 nonuse episodes on person-time after the index hip fracture event to preclude immortal time bias and followed up for the outcome of nonvertebral fracture until change in exposure or a censoring event. An instrumental variable analysis using 2-stage residual inclusion method was conducted using calendar year, specialist access, geographical variation in prescribing patterns, and hospital preference.

RESULTS

Among 97 169 patients with a hip fracture identified, the mean (SD) age was 80.2 (10.8) years, and 64 164 (66.0%) were women. A continuous decline over the study years was observed in osteoporosis medication initiation rates from 9.8% (95% CI, 9.0%-10.6%) in 2004 to 3.3% (95% CI, 2.9%-3.8%) in 2015. In the effectiveness analyses, the hospital preference instrumental variable had a stronger association with treatment (pseudo R2 = 0.20) than the other 3 instrumental variables (specialist access: pseudo R2 = 0.04; calendar year: pseudo R2 = 0.05; and geographic variation: pseudo R2 = 0.07). Instrumental variable analysis with hospital preference suggested a rate difference of 4.2 events (95% CI, 1.1-7.3) per 100 person-years in subsequent fractures associated with osteoporosis treatment initiation compared with nonuse in an additive hazard model.

CONCLUSIONS AND RELEVANCE

Low rates of osteoporosis treatment initiation after a hip fracture in recent years were observed. Clinically meaningful reduction in subsequent nonvertebral fracture rates associated with treatment suggests that improving prescriber adherence to guidelines and patient adherence to prescribed regimens may result in notable public health benefit.

A tutorial on the use of instrumental variables in pharmacoepidemiology

PURPOSE

Instrumental variable (IV) methods are used increasingly in pharmacoepidemiology to address unmeasured confounding. In this tutorial, we review the steps used in IV analyses and the underlying assumptions. We also present methods to assess the validity of those assumptions and describe sensitivity analysis to examine the effects of possible violations of those assumptions.

METHODS

Observational studies based on regression or propensity score analyses rely on the untestable assumption that there are no unmeasured confounders. IV analysis is a tool that removes the bias caused by unmeasured confounding provided that key assumptions (some of which are also untestable) are met.

RESULTS

When instruments are valid, IV methods provided unbiased treatment effect estimation in the presence of unmeasured confounders. However, the standard error of the IV estimate is higher than the standard error of non-IV estimates, e.g., regression and propensity score methods. Sensitivity analyses provided insight about the robustness of the IV results to the plausible degrees of violation of assumptions.

CONCLUSIONS

IV analysis should be used cautiously because the validity of IV estimates relies on assumptions that are, in general, untestable and difficult to be certain about. Thus, assessing the sensitivity of the estimate to violations of these assumptions is important and can better inform the causal inferences that can be drawn from the study. Copyright © 2017 John Wiley & Sons, Ltd.

Martingale residual-based method to control for confounders measured only in a validation sample in time-to-event analysis

Unmeasured confounding remains an important problem in observational studies, including pharmacoepidemiological studies of large administrative databases. Several recently developed methods utilize smaller validation samples, with information on additional confounders, to control for confounders unmeasured in the main, larger database. However, up-to-date applications of these methods to survival analyses seem to be limited to propensity score calibration, which relies on a strong surrogacy assumption. We propose a new method, specifically designed for time-to-event analyses, which uses martingale residuals, in addition to measured covariates, to enhance imputation of the unmeasured confounders in the main database. The method is applicable for analyses with both time-invariant data and time-varying exposure/confounders. In simulations, our method consistently eliminated bias because of unmeasured confounding, regardless of surrogacy violation and other relevant design parameters, and almost always yielded lower mean squared errors than other methods applicable for survival analyses, outperforming propensity score calibration in several scenarios. We apply the method to a real-life pharmacoepidemiological database study of the association between glucocorticoid therapy and risk of type II diabetes mellitus in patients with rheumatoid arthritis, with additional potential confounders available in an external validation sample. Compared with conventional analyses, which adjust only for confounders measured in the main database, our estimates suggest a considerably weaker association. Copyright © 2016 John Wiley & Sons, Ltd.

The missing cause approach to unmeasured confounding in pharmacoepidemiology

Unmeasured confounding is a major threat to the validity of pharmacoepidemiological studies of medication safety and effectiveness. We propose a new method for detecting and reducing the impact of unobserved confounding in large observational database studies. The method uses assumptions similar to the prescribing preference-based instrumental variable (IV) approach. Our method relies on the new 'missing cause' principle, according to which the impact of unmeasured confounding by (contra-)indication may be detected by assessing discrepancies between the following: (i) treatment actually received by individual patients and (ii) treatment that they would be expected to receive based on the observed data. Specifically, we use the treatment-by-discrepancy interaction to test for the presence of unmeasured confounding and correct the treatment effect estimate for the resulting bias. Under standard IV assumptions, we first proved that unmeasured confounding induces a spurious treatment-by-discrepancy interaction in risk difference models for binary outcomes and then simulated large pharmacoepidemiological studies with unmeasured confounding. In simulations, our estimates had four to six times smaller bias than conventional treatment effect estimates, adjusted only for measured confounders, and much smaller variance inflation than unbiased but very unstable IV estimates, resulting in uniformly lowest root mean square errors. The much lower variance of our estimates, relative to IV estimates, was also observed in an application comparing gastrointestinal safety of two classes of anti-inflammatory drugs. In conclusion, our missing cause-based method may complement other methods and enhance accuracy of analyses of large pharmacoepidemiological studies.

The proportion of all previous patients was a potential instrument for patients' actual prescriptions of nonsteroidal anti-inflammatory drugs

OBJECTIVES

To investigate whether physician's prescribing preference is a valid instrumental variable (IV) for patients' actual prescription of selective cyclooxygenase-2 (COX-2) inhibitors in the German Pharmacoepidemiological Research Database (GePaRD).

STUDY DESIGN AND SETTING

We compared the effect of COX-2 inhibitors vs. traditional nonsteroidal anti-inflammatory drugs (tNSAIDs) on the risk of gastrointestinal complications using physician's preference as IV. We used different definitions of physician's preference for COX-2 inhibitors. A retrospective cohort of new users was built which was further restricted to subcohorts. We compared IV-based risk difference estimates, using a two-stage approach, to estimates from conventional multivariate models.

RESULTS

We observed only a small proportion of COX-inhibitor users (3.2%) in our study. All instruments, in the full cohort and in the subcohorts, reduced the imbalance in most of the covariates. However, the IV treatment effect estimates had a highly inflated variance. Compared to the most recent prescription, the proportion of previous patients was a stronger instrument and reduced the variance of the estimates.

CONCLUSION

The proportion of all previous patients is a potential IV for comparing COX-2 inhibitors vs. tNSAIDs in GePaRD. Our study demonstrates that valid instruments in one health care system may not be directly applicable to others.

Evaluating possible confounding by prescriber in comparative effectiveness research

In nonrandomized studies of comparative effectiveness of medications, the prescriber may be the most important determinant of treatment assignment, yet the majority of analyses ignore the prescriber. Via Monte Carlo simulation, we evaluated the bias of 3 approaches that utilize the prescriber in analysis compared against the default approach that ignores the prescriber. Prescriber preference instrumental variable (IV) analyses were unbiased when IV criteria were met, which required no clustering of unmeasured patient characteristics within prescriber. In all other scenarios, IV analyses were highly biased, and stratification on the prescriber reduced confounding bias at the patient or prescriber levels. Including a prescriber random intercept in the propensity score model reversed the direction of confounding from measured patient factors and resulted in unpredictable changes in bias. Therefore, we recommend caution when using the IV approach, particularly when the instrument is weak. Stratification on the prescriber may be more robust; this approach warrants additional research.

Instrumental variable applications using nursing home prescribing preferences in comparative effectiveness research

PURPOSE

Nursing home residents are of particular interest for comparative effectiveness research given their susceptibility to adverse treatment effects and systematic exclusion from trials. However, the risk of residual confounding because of unmeasured markers of declining health using conventional analytic methods is high. We evaluated the validity of instrumental variable (IV) methods based on nursing home prescribing preference to mitigate such confounding, using psychotropic medications to manage behavioral problems in dementia as a case study.

METHODS

A cohort using linked data from Medicaid, Medicare, Minimum Data Set, and Online Survey, Certification and Reporting for 2001-2004 was established. Dual-eligible patients ≥65 years who initiated psychotropic medication use after admission were selected. Nursing home prescribing preference was characterized using mixed-effects logistic regression models. The plausibility of IV assumptions was explored, and the association between psychotropic medication class and 180-day mortality was estimated.

RESULTS

High-prescribing and low-prescribing nursing homes differed by a factor of 2. Each preference-based IV measure described a substantial proportion of variation in psychotropic medication choice (β(IV → treatment): 0.22-0.36). Measured patient characteristics were well balanced across patient groups based on instrument status (52% average reduction in Mahalanobis distance). There was no evidence that instrument status was associated with markers of nursing home quality of care.

CONCLUSION

Findings indicate that IV analyses using nursing home prescribing preference may be a useful approach in comparative effectiveness studies, and should extend naturally to analyses including untreated comparison groups, which are of great scientific interest but subject to even stronger confounding.

Impact of physician preferences for homeopathic or conventional medicines on patients with musculoskeletal disorders: results from the EPI3-MSD cohort

OBJECTIVE

The objective of this study was to assess the effect of physician practicing preferences (PPP) in primary care for homeopathy (Ho), CAM (Complementary and alternative medicines) with conventional medicine (Mx) or exclusively conventional medicine (CM) on patients with musculoskeletal disorders (MSDs), with reference to clinical progression, drug consumption, side effects and loss of therapeutic opportunity.

METHODS

The EPI3-MSD study was a nationwide observational cohort of a representative sample of general practitioners (GP) and their patients in France. Recruitment of GP was stratified by PPP, which was self-declared. Diagnoses and comorbidities were recorded by GP at inclusion. Patients completed a standardized telephone interview at inclusion, one, three and twelve months, including MSD-functional scales and medication consumption.

RESULTS

1153 MSD patients were included in the three PPP groups. Patients did not differ between groups except for chronicity of MSDs (>12 weeks), which was higher in the Ho group (62.1%) than in the CM (48.6%) and Mx groups (50.3%). The twelve-month development of specific functional scores was identical across the three groups after controlling for baseline score (p > 0.05). After adjusting for propensity scores, NSAID use over 12 months was almost half in the Ho group (OR, 0.54; 95%CI, 0.38-0.78) as compared to the CM group; no difference was found in the Mx group (OR, 0.81; 95% CI: 0.59-1.15).

CONCLUSION

MSD patients seen by homeopathic physicians showed a similar clinical progression when less exposed to NSAID in comparison to patients seen in CM practice, with fewer NSAID-related adverse events and no loss of therapeutic opportunity.

Reducing the variance of the prescribing preference-based instrumental variable estimates of the treatment effect

Instrumental variable (IV) methods based on the physician's prescribing preference may remove bias due to unobserved confounding in pharmacoepidemiologic studies. However, IV estimates, originally defined as the treatment prescribed for a single previous patient of a given physician, show important variance inflation. The authors proposed and validated in simulations a new method to reduce the variance of IV estimates even when physicians' preferences change over time. First, a potential "change-time," after which the physician's preference has changed, was estimated for each physician. Next, all patients of a given physician were divided into 2 homogeneous subsets: those treated before the change-time versus those treated after the change-time. The new IV was defined as the proportion of all previous patients in a corresponding homogeneous subset who were prescribed a specific drug. In simulations, all alternative IV estimators avoided strong bias of the conventional estimates. The change-time method reduced the standard deviation of the estimates by approximately 30% relative to the original previous patient-based IV. In an empirical example, the proposed IV correlated better with the actual treatment and yielded smaller standard errors than alternative IV estimators. Therefore, the new method improved the overall accuracy of IV estimates in studies with unobserved confounding and time-varying prescribing preferences.