On the use of propensity scores in case of rare exposure

Medications for Opioid Use Disorder and Mortality and Hospitalization Among People With Opioid Use-related Infections

BACKGROUND

Severe skin and soft tissue infections related to injection drug use have increased in concordance with a shift to heroin and illicitly manufactured fentanyl. Opioid agonist therapy medications (methadone and buprenorphine) may improve long-term outcomes by reducing injection drug use. We aimed to examine the association of medication use with mortality among people with opioid use-related skin or soft tissue infections.

METHODS

An observational cohort study of Medicaid enrollees aged 18 years or older following their first documented medical encounters for opioid use-related skin or soft tissue infections during 2007-2018 in North Carolina. The exposure was documented medication use (methadone or buprenorphine claim) in the first 30 days following initial infection compared with no medication claim. Using Kaplan-Meier estimators, we examined the difference in 3-year incidence of mortality by medication use, weighted for year, age, comorbidities, and length of hospital stay.

RESULTS

In this sample, there were 13,286 people with opioid use-related skin or soft tissue infections. The median age was 37 years, 68% were women, and 78% were white. In Kaplan-Meier curves for the total study population, 12 of every 100 patients died during the first 3 years. In weighted models, for every 100 people who used medications, there were four fewer deaths over 3 years (95% confidence interval = 2, 6).

CONCLUSION

In this study, people with opioid use-related skin and soft tissue infections had a high risk of mortality following their initial healthcare visit for infections. Methadone or buprenorphine use was associated with reductions in mortality.

Directed Acyclic Graph Assisted Method For Estimating Average Treatment Effect

Observational data, such as electronic clinical records and claims data, can prove invaluable for evaluating the Average Treatment Effect (ATE) and supporting decision-making, provided they are employed correctly. The Inverse Probability of Treatment Weighting (IPTW) method, based on propensity scores, has demonstrated remarkable efficacy in estimating ATE, assuming that the assumptions of exchangeability, consistency, and positivity are met. Directed Acyclic Graphs (DAGs) offer a practical approach to assess the exchangeability assumption, which asserts that treatment assignment and potential outcomes are independent given a set of confounding variables that block all backdoor paths from treatment assignment to potential outcomes. To ensure a consistent ATE estimator, one can adjust for a minimally sufficient adjustment set of confounding variables that block all backdoor paths from treatment assignment to the outcome. To enhance the efficiency of ATE estimators, our proposal involves incorporating both the minimally sufficient adjustment set of confounding variables and predictors into the propensity score model. Extensive simulations were conducted to evaluate the performance of propensity score-based IPTW methods in estimating ATE when different sets of covariates were included in the propensity score models. The simulation results underscored the significance of including the minimally sufficient adjustment set of confounding variables along with predictors in the propensity score models to obtain a consistent and efficient ATE estimator. We applied this proposed method to investigate whether tracheostomy was causally associated with in-hospital infant mortality, utilizing the 2016 Healthcare Cost and Utilization Project Kids' Inpatient Database. The estimated ATE was found to be approximately 2.30%-2.46% with p-value >0.05.

Effects of a mastitis J5 bacterin vaccination on the productive performance of dairy cows: An observational study using propensity score matching techniques

Inferring causal effects between variables when utilizing observational data is challenging due to confounding factors not controlled through a randomized experiment. Propensity score matching can decrease confounding in observational studies and offers insights about potential causal effects of prophylactic management interventions such as vaccinations. The objective of this study was to determine potential causality and impact of vaccination with an Escherichia coli J5 bacterin on the productive performance of dairy cows applying propensity score matching techniques to farm-recorded (e.g., observational) data. Traits of interest included 305-d milk yield (MY305), 305-d fat yield (FY305), 305-d protein yield (PY305), and somatic cell score (SCS). Records from 6,418 lactations generated by 5,121 animals were available for the analysis. Vaccination status of each animal was obtained from producer-recorded information. Confounding variables considered were herd-year-season groups (56 levels), parity (5 levels: 1, 2, 3, 4, and ≥5), and genetic quartile groups (4 levels: top 25% through bottom 25%) derived from genetic predictions for MY305, FY305, PY305, and SCS, as well as for the genetic susceptibility to mastitis. A logistic regression model was applied to estimate the propensity score (PS) for each cow. Subsequently, PS values were used to form pairs of animals (1 vaccinated with 1 unvaccinated control), depending on their PS similarities (difference in PS values of cows within a match required to be <20% of 1 standard deviation of the logit of PS). After the matching process, 2,091 pairs of animals (4,182 records) remained available to infer the causal effects of vaccinating dairy cows with the E. coli J5 bacterin. Causal effects estimation was performed using 2 approaches: simple matching and a bias-corrected matching. According to the PS methodology, causal effects of vaccinating dairy cows with a J5 bacterin on their productive performance were identified for MY305. The simple matched estimator suggested that vaccinated cows produced 163.89 kg more milk over an entire lactation when compared with nonvaccinated counterparts, whereas the bias-corrected estimator suggested that such increment in milk production was of 150.48 kg. Conversely, no causal effects of immunizing dairy cows with a J5 bacterin were identified for FY305, PY305, or SCS. In conclusion, the utilization of PS matching techniques applied to farm-recorded data was feasible and allowed us to identify that vaccination with an E. coli J5 bacterin relates to an overall milk production increment without compromising milk quality.

Causal inference in case of near-violation of positivity: comparison of methods

In causal studies, the near-violation of the positivity may occur by chance, because of sample-to-sample fluctuation despite the theoretical veracity of the positivity assumption in the population. It may mostly happen when the exposure prevalence is low or when the sample size is small. We aimed to compare the robustness of g-computation (GC), inverse probability weighting (IPW), truncated IPW, targeted maximum likelihood estimation (TMLE), and truncated TMLE in this situation, using simulations and one real application. We also tested different extrapolation situations for the sub-group with a positivity violation. The results illustrated that the near-violation of the positivity impacted all methods. We demonstrated the robustness of GC and TMLE-based methods. Truncation helped in limiting the bias in near-violation situations, but at the cost of bias in normal conditions. The application illustrated the variability of the results between the methods and the importance of choosing the most appropriate one. In conclusion, compared to propensity score-based methods, methods based on outcome regression should be preferred when suspecting near-violation of the positivity assumption.

G-computation and doubly robust standardisation for continuous-time data: A comparison with inverse probability weighting

In time-to-event settings, g-computation and doubly robust estimators are based on discrete-time data. However, many biological processes are evolving continuously over time. In this paper, we extend the g-computation and the doubly robust standardisation procedures to a continuous-time context. We compare their performance to the well-known inverse-probability-weighting estimator for the estimation of the hazard ratio and restricted mean survival times difference, using a simulation study. Under a correct model specification, all methods are unbiased, but g-computation and the doubly robust standardisation are more efficient than inverse-probability-weighting. We also analyse two real-world datasets to illustrate the practical implementation of these approaches. We have updated the R package RISCA to facilitate the use of these methods and their dissemination.

Standardizing Discrete-Time Hazard Ratios With a Disease Risk Score

The disease risk score (DRS) is a summary score that is a function of a potentially large set of covariates. The DRS can be used to control for confounding by the covariates that went into estimation of the DRS and obtain a standardized estimate of an exposure's effect on disease. However, to date, literature on the DRS has not addressed analyses that focus on estimation of survival or hazard functions, which are common in epidemiologic analyses of cohort data. Here, we propose a method for standardization of hazard ratios using the DRS in longitudinal analyses of the association between a binary exposure and an outcome. This approach to handling a potentially large set of covariates through a model-based approach to standardization may provide a useful tool for cohort analyses of hazard ratios and may be particularly well-suited to settings where an exposure propensity score is difficult to model. Simulations are used in this paper to illustrate the approach, and an empirical example is provided.

G-computation, propensity score-based methods, and targeted maximum likelihood estimator for causal inference with different covariates sets: a comparative simulation study

Controlling for confounding bias is crucial in causal inference. Distinct methods are currently employed to mitigate the effects of confounding bias. Each requires the introduction of a set of covariates, which remains difficult to choose, especially regarding the different methods. We conduct a simulation study to compare the relative performance results obtained by using four different sets of covariates (those causing the outcome, those causing the treatment allocation, those causing both the outcome and the treatment allocation, and all the covariates) and four methods: g-computation, inverse probability of treatment weighting, full matching and targeted maximum likelihood estimator. Our simulations are in the context of a binary treatment, a binary outcome and baseline confounders. The simulations suggest that considering all the covariates causing the outcome led to the lowest bias and variance, particularly for g-computation. The consideration of all the covariates did not decrease the bias but significantly reduced the power. We apply these methods to two real-world examples that have clinical relevance, thereby illustrating the real-world importance of using these methods. We propose an R package RISCA to encourage the use of g-computation in causal inference.

Performance of matching methods in studies of rare diseases: a simulation study

Matching is a common method of adjusting for confounding in observational studies. Studies in rare diseases usually include small numbers of exposed subjects, but the performance of matching methods in such cases has not been evaluated thoroughly. In this study, we compare the performance of several matching methods when number of exposed subjects is small. We used Monte Carlo simulations to compare the following methods: Propensity score matching (PSM) with greedy or optimal algorithm, Mahalanobis distance matching, and mixture of PSM and exact matching. We performed the comparisons in datasets with six continuous and six binary variables, with varying effect size on group assignment and outcome. In each case, there were 1,500 unexposed subjects and a varying number of exposed: N = 25, 50, 100, 150, 200, 250, or 300. The probability of outcome in unexposed subjects was set to 5% (rare), 20% (common), or 50% (frequent). We compared the methods based on the bias of estimate of risk difference, coverage of 95% confidence intervals for risk difference, and balance of covariates. We observed a difference in performance of matching methods in very small samples (N = 25-50) and in moderately small samples (N = 100-300). Our study showed that PSM performs better than other matching methods when number of exposed subjects is small, but the matching algorithm and the matching ratio should be considered carefully. We recommend using PSM with optimal algorithm and one-to-five matching ratio in very small samples, and PSM matching with any algorithm and one-to-one matching in moderately small samples.

Marginal Structural Models for Risk or Prevalence Ratios for a Point Exposure Using a Disease Risk Score

The disease risk score is a summary score that can be used to control for confounding with a potentially large set of covariates. While less widely used than the exposure propensity score, the disease risk score approach might be useful for novel or unusual exposures, when treatment indications or exposure patterns are rapidly changing, or when more is known about the nature of how covariates cause disease than is known about factors influencing propensity for the exposure of interest. Focusing on the simple case of a binary point exposure, we describe a marginal structural model for estimation of risk (or prevalence) ratios. The proposed model incorporates the disease risk score as an offset in a regression model, and it yields an estimate of a standardized risk ratio where the target population is the exposed group. Simulations are used to illustrate the approach, and an empirical example is provided. Confounder control based on the proposed method might be a useful alternative to approaches based on the exposure propensity score, or as a complement to them.

The Impact of Nondifferential Exposure Misclassification on the Performance of Propensity Scores for Continuous and Binary Outcomes: A Simulation Study

PURPOSE

To investigate the ability of the propensity score (PS) to reduce confounding bias in the presence of nondifferential misclassification of treatment, using simulations.

METHODS

Using an example from the pregnancy medication safety literature, we carried out simulations to quantify the effect of nondifferential misclassification of treatment under varying scenarios of sensitivity and specificity, exposure prevalence (10%, 50%), outcome type (continuous and binary), true outcome (null and increased risk), confounding direction, and different PS applications (matching, stratification, weighting, regression), and obtained measures of bias and 95% confidence interval coverage.

RESULTS

All methods were subject to substantial bias toward the null due to nondifferential exposure misclassification (range: 0%-47% for 50% exposure prevalence and 0%-80% for 10% exposure prevalence), particularly if specificity was low (<97%). PS stratification produced the least biased effect estimates. We observed that the impact of sensitivity and specificity on the bias and coverage for each adjustment method is strongly related to prevalence of exposure: as exposure prevalence decreases and/or outcomes are continuous rather than categorical, the effect of misclassification is magnified, producing larger biases and loss of coverage of 95% confidence intervals. PS matching resulted in unpredictably biased effect estimates.

CONCLUSIONS

The results of this study underline the importance of assessing exposure misclassification in observational studies in the context of PS methods. Although PS methods reduce confounding bias, bias owing to nondifferential misclassification is of potentially greater concern.

Evaluation of Complementary and Integrative Health Approaches Among US Veterans with Musculoskeletal Pain Using Propensity Score Methods

Objectives

To examine the treatment effectiveness of complementary and integrative health approaches (CIH) on chronic pain using Propensity Score (PS) methods.

Design, Settings, and Participants

A retrospective cohort of 309,277 veterans with chronic musculoskeletal pain assessed over three years after initial diagnosis.

Methods

CIH exposure was defined as one or more clinical visits for massage, acupuncture, or chiropractic care. The treatment effect of CIH on self-rated pain intensity was examined using a longitudinal model. PS-matching and inverse probability of treatment weighting (IPTW) were used to account for potential selection and confounding biases.

Results

At baseline, veterans with (7,621) and without (301,656) CIH exposure differed significantly in 21 out of 35 covariates. During the follow-up period, on average CIH recipients had 0.83 (95% confidence interval [CI] = 0.77 to 0.89) points higher pain intensity ratings (range = 0-10) than nonrecipients. This apparent unfavorable effect size was reduced to 0.37 (95% CI = 0.28 to 0.45) after PS matching, 0.36 (95% CI = 0.29 to 0.44) with IPTW on the treated (IPTW-T) weighting, and diminished to null when integrating IPTW-T with PS matching (0.004, 95% CI = -0.09 to 0.10). An alternative IPTW model and conventional covariate adjustment appeared least powerful in terms of potential bias reduction. Sensitivity analyses restricting the follow-up period to one year after CIH initiation derived consistent results.

Conclusions

PS-based causal methods successfully eliminated baseline difference between exposure groups in all measured covariates, yet they did not detect a significant difference in the self-rated pain intensity outcome between veterans who received CIHs and those who did not during the follow-up period.

Gender as an independent prognostic factor in small-cell lung cancer: Inha Lung Cancer Cohort study using propensity score matching

Introduction

The prognostic relevance of gender is undetermined in patients with small-cell lung cancer (SCLC). Therefore, we investigated whether gender is a prognostic factor in a SCLC cohort after controlling for confounding factors.

Materials and methods

Fifteen prognostic factors were classified into four groups (patient, stage migration, tumor, and treatment). The prognostic relevance of gender was evaluated using propensity score matching, Cox proportional hazards regression, and stepwise fashion adjustments.

Results

Of 591 patients with SCLC, 88 were women (14.9%). Women were more likely than men to have no history of smoking (48.9% vs. 2.0%, P < 0.001) and limited disease (48.9% vs. 37.8%, P = 0.050). Women had less progressive disease in M stage than men (52.3% vs. 62.8%, P = 0.031). Women had better survival than men in the entire cohort (median survival times [MSTs] and 95% confidence intervals [CIs]: 9.7 months and 7.8–11.6 for women, 8.0 months and 7.0–8.9 for men, log-rank P = 0.034) and in the matched cohort (MSTs and 95% CIs: 8.8 months and 5.8–11.8 for women, 5.9 months and 4.5–7.4 for men, log-rank P = 0.013). Female gender was a prognostic factor predicting better survival, even after stepwise and full adjustment with all prognostic variables (adjusted hazard ratios and 95% CIs: 0.51 and 0.34–0.77, P = 0.001 for entire cohort, 0.42 and 0.24–0.75, P = 0.003 for matched cohort).

Conclusions

Our results confirmed that gender is an independent prognostic factor in patients with SCLC.

Comparing external and internal validation methods in correcting outcome misclassification bias in logistic regression: A simulation study and application to the case of postsurgical venous thromboembolism following total hip and knee arthroplasty

PURPOSE

We assessed the validity of postsurgery venous thromboembolism (VTE) diagnoses identified from administrative databases and compared Bayesian and multiple imputation (MI) approaches in correcting for outcome misclassification in logistic regression models.

METHODS

Sensitivity and specificity of postsurgery VTE among patients undergoing total hip or knee replacement (THR/TKR) were assessed against chart review in six Montreal hospitals in 2009 to 2010. Administrative data on all THR/TKR Quebec patients in 2009 to 2010 were obtained. The performance of Bayesian external, Bayesian internal, and MI approaches to correct the odds ratio (OR) of postsurgery VTE in tertiary versus community hospitals was assessed using simulations. Bayesian external approach used prior information from external sources, while Bayesian internal and MI approaches used chart review.

RESULTS

In total, 17 319 patients were included, 2136 in participating hospitals, among whom 75 had VTE in administrative data versus 81 in chart review. VTE sensitivity was 0.59 (95% confidence interval, 0.48-0.69) and specificity was 0.99 (0.98-0.99), overall. The adjusted OR of VTE in tertiary versus community hospitals was 1.35 (1.12-1.64) using administrative data, 1.45 (0.97-2.19) when MI was used for misclassification correction, and 1.53 (0.83-2.87) and 1.57 (0.39-5.24) when Bayesian internal and external approaches were used, respectively. In simulations, all three approaches reduced the OR bias and had appropriate coverage for both nondifferential and differential misclassification.

CONCLUSION

VTE identified from administrative data had low sensitivity and high specificity. The Bayesian external approach was useful to reduce outcome misclassification bias in logistic regression; however, it required accurate specification of the misclassification properties and should be used with caution.

The Number of Events per Confounder for Valid Estimation of Risk Difference Using Modified Least-Squares Regression

Risk difference is a relevant effect measure in epidemiologic research. Although it is well known that when there are few events per confounder, logistic regression is not suitable for confounding control, it is not clear how many events per confounder are required for valid estimation of risk difference using linear binomial models. Because the maximum likelihood method has a convergence problem, we investigated the number of events per confounder necessary to validly estimate risk difference using modified least-squares regression in a simulation. We simulated 864 scenarios, according to the number of confounders (2-20), the number of events per confounder (2-12), marginal risk (0.5%-40%), exposure proportion (20% and 40%), and 3 sizes of risk difference. Our simulation showed that modified least-squares regression provided unbiased risk difference-regardless of the number of events per confounder-and reliable confidence intervals when more than 5 events were expected in the exposed and in the unexposed, irrespective of the number of events per confounder. We illustrated the modified least-squares regression analysis using perinatal epidemiologic data. Modified least-squares regression is considered to be a useful analytical tool for rare binary outcomes relative to the number of confounders.

Closed-form variance estimator for weighted propensity score estimators with survival outcome

Propensity score (PS) methods are widely used in observational studies for evaluating marginal treatment effects. PS-weighting is a popular PS-based method that allows for estimating both the average treatment effect on the overall population (ATE) and the average treatment effect on the treated population (ATT). Previous research has shown that the variance of the treatment effect is accurately estimated only if the variance estimator takes into account the fact that the propensity score is itself estimated from the available data in a first step of the analysis. In 2016, Austin showed that the bootstrap-based variance estimator was the only existing estimator resulting in approximately correct estimates of standard errors when evaluating a survival outcome and a Cox model was used to estimate a marginal hazard ratio (HR). This author stressed the need to develop a closed-form variance estimator of the marginal HR accounting for the estimation of the PS. In the present research, we developed such variance estimators both for the ATE and ATT. We evaluated their performance with an extensive simulation study and compared them to bootstrap-based variance estimators and to naive variance estimators that do not account for the estimation step. We found that the performance of the proposed variance estimators was similar to that of the bootstrap-based estimators. The proposed variance estimators provide an alternative to the bootstrap estimator, particularly interesting in situations in which time-consumption and/or reproducibility are an important issue. An implementation has been developed for the R software and is freely available (package hrIPW).

Comparison between treatment effects in a randomised controlled trial and an observational study using propensity scores in primary care

BACKGROUND

Although randomised controlled trials (RCTs) are considered 'gold standard' evidence, they are not always feasible or appropriate, and may represent a select population. Observational studies provide a useful alternative to enhance applicability, but results can be biased due to confounding.

AIM

To explore the utility of propensity scores for causal inference in an observational study.

DESIGN AND SETTING

Comparison of the effect of amoxicillin on key outcomes in an international RCT and observational study of lower respiratory tract infections.

METHOD

Propensity scores were calculated and applied as probability weights in the analyses. The adjusted results were compared with the effects reported in the RCT.

RESULTS

Groups were well balanced in the RCT but significantly imbalanced in the observational study, with evidence of confounding by indication: patients receiving antibiotics tended to be older and more unwell at baseline consultation. In the trial duration of symptoms (hazard ratio 1.06, 95% CI = 0.96 to 1.18) and symptom severity (-0.07, 95% CI = -0.15 to 0.007) did not differ between groups. Weighting by propensity score in the observational study resulted in very similar estimates of effect: duration of symptoms (hazard ratio 1.06, 95% CI = 0.80 to 1.40) and difference for symptom severity (-0.07, 95% CI = -0.34 to 0.20).

CONCLUSION

The observational study, after conditioning on propensity score, echoed the trial results. Provided that detailed information is available on potential sources of confounding, effects of interventions can probably be assessed reasonably well in observational datasets, allowing them to be more directly compared with the results of RCTs.

Aspirin and the risk of cardiovascular events in atherosclerosis patients with and without prior ischemic events

BACKGROUND

The benefit of aspirin among patients with stable atherosclerosis without a prior ischemic event is not well defined.

HYPOTHESIS

Aspirin would be of benefit in outpatients with atherosclerosis with prior ischemic events, but not in those without ischemic events.

METHODS

Subjects from the Reduction of Atherothrombosis for Continued Health registry were divided according to prior ischemic event (n =21 724) vs stable atherosclerosis, but no prior ischemic event (n = 11 872). Analyses were propensity score matched. Aspirin use was updated at each clinic visit and considered as a time-varying covariate. The primary outcome was the first occurrence of cardiovascular death, myocardial infarction, or stroke.

RESULTS

In the group with a prior ischemic event, aspirin use was associated with a marginally lower risk of the primary outcome at a median of 41 months (hazard ratio [HR]: 0.81, 95% confidence interval [CI]: 0.65-1.01, P = 0.06). In the group without a prior ischemic event, aspirin use was not associated with a lower risk of the primary outcome at a median of 36 months (HR: 1.03, 95% CI: 0.73-1.45, P = 0.86).

CONCLUSIONS

In this observational analysis of outpatients with stable atherosclerosis, aspirin was marginally beneficial among patients with a prior ischemic event; however, there was no apparent benefit among those with no prior ischemic event.

Estimation of conditional and marginal odds ratios using the prognostic score

Introduced by Hansen in 2008, the prognostic score (PGS) has been presented as 'the prognostic analogue of the propensity score' (PPS). PPS-based methods are intended to estimate marginal effects. Most previous studies evaluated the performance of existing PGS-based methods (adjustment, stratification and matching using the PGS) in situations in which the theoretical conditional and marginal effects are equal (i.e., collapsible situations). To support the use of PGS framework as an alternative to the PPS framework, applied researchers must have reliable information about the type of treatment effect estimated by each method. We propose four new PGS-based methods, each developed to estimate a specific type of treatment effect. We evaluated the ability of existing and new PGS-based methods to estimate the conditional treatment effect (CTE), the (marginal) average treatment effect on the whole population (ATE), and the (marginal) average treatment effect on the treated population (ATT), when the odds ratio (a non-collapsible estimator) is the measure of interest. The performance of PGS-based methods was assessed by Monte Carlo simulations and compared with PPS-based methods and multivariate regression analysis. Existing PGS-based methods did not allow for estimating the ATE and showed unacceptable performance when the proportion of exposed subjects was large. When estimating marginal effects, PPS-based methods were too conservative, whereas the new PGS-based methods performed better with low prevalence of exposure, and had coverages closer to the nominal value. When estimating CTE, the new PGS-based methods performed as well as traditional multivariate regression. Copyright © 2016 John Wiley & Sons, Ltd.

Value of Propensity Score Matching for Equalizing Comparator Groups in Observational Database Studies: A Case Study in Anti-infectives

PURPOSE

Propensity score methodologies can reduce bias and confounding in nonrandomized studies, including pharmaceutical comparative effectiveness studies. An observational case study was developed to demonstrate the impact of propensity score adjustments on outcomes (ie, discharge status) of patients hospitalized for complicated intra-abdominal infections.

METHODS

Two cohorts were examined: intensive care unit (ICU) (vs non-ICU) patients and tigecycline-treated patients (vs patients receiving other antibiotics). Discharge status was captured before propensity scoring.

FINDINGS

The impact of propensity scoring on discharge outcome was greater when comparing ICU patients versus non-ICU patients than when comparing tigecycline recipients versus nonrecipients.

IMPLICATIONS

Propensity scoring should be examined carefully to optimize its effects. Moreover, propensity scoring only addresses bias and confounding in nonrandomized studies that are attributable to variables contained within the dataset (ie, so called "observables") and not to other variables that may influence the relationship between outcomes and other independent variables.