Conditioning on the propensity score can result in biased estimation of common measures of treatment effect: a Monte Carlo study

Propensity score matching for estimating a marginal hazard ratio

Propensity score matching is commonly used to draw causal inference from observational survival data. However, its asymptotic properties have yet to be established, and variance estimation is still open to debate. We derive the statistical properties of the propensity score matching estimator of the marginal causal hazard ratio based on matching with replacement and a fixed number of matches. We also propose a double-resampling technique for variance estimation that takes into account the uncertainty due to propensity score estimation prior to matching.

A Cautionary Note on Using Propensity Score Calibration to Control for Unmeasured Confounding Bias When the Surrogacy Assumption Is Absent

Conventional propensity score methods encounter challenges when unmeasured confounding is present, as it becomes impossible to accurately estimate the gold-standard propensity score when data on certain confounders are unavailable. Propensity score calibration (PSC) addresses this issue by constructing a surrogate for the gold-standard propensity score under the surrogacy assumption. This assumption posits that the error-prone propensity score, based on observed confounders, is independent of the outcome when conditioned on the gold-standard propensity score and the exposure. However, this assumption implies that confounders cannot directly impact the outcome and that their effects on the outcome are solely mediated through the propensity score. This raises concerns regarding the applicability of PSC in practical settings where confounders can directly affect the outcome. While PSC aims to target a conditional treatment effect by conditioning on a subject's unobservable propensity score, the causal interest in the latter case lies in a conditional treatment effect conditioned on a subject's baseline characteristics. Our analysis reveals that PSC is generally biased unless the effects of confounders on the outcome and treatment are proportional to each other. Furthermore, we identify 2 sources of bias: 1) the noncollapsibility of effect measures, such as the odds ratio or hazard ratio and 2) residual confounding, as the calibrated propensity score may not possess the properties of a valid propensity score.

RKHS-based covariate balancing for survival causal effect estimation

Survival causal effect estimation based on right-censored data is of key interest in both survival analysis and causal inference. Propensity score weighting is one of the most popular methods in the literature. However, since it involves the inverse of propensity score estimates, its practical performance may be very unstable, especially when the covariate overlap is limited between treatment and control groups. To address this problem, a covariate balancing method is developed in this paper to estimate the counterfactual survival function. The proposed method is nonparametric and balances covariates in a reproducing kernel Hilbert space (RKHS) via weights that are counterparts of inverse propensity scores. The uniform rate of convergence for the proposed estimator is shown to be the same as that for the classical Kaplan-Meier estimator. The appealing practical performance of the proposed method is demonstrated by a simulation study as well as two real data applications to study the causal effect of smoking on survival time of stroke patients and that of endotoxin on survival time for female patients with lung cancer respectively.

Target trial emulation to assess real-world efficacy in the Epidemiological Strategy and Medical Economics metastatic breast cancer cohort

BACKGROUND

Real-world data studies usually consider biases related to measured confounders. We emulate a target trial implementing study design principles of randomized trials to observational studies; controlling biases related to selection, especially immortal time; and measured confounders.

METHODS

This comprehensive analysis emulating a randomized clinical trial compared overall survival in patients with HER2-negative metastatic breast cancer (MBC), receiving as first-line treatment, either paclitaxel alone or combined to bevacizumab. We used data from 5538 patients extracted from the Epidemiological Strategy and Medical Economics-MBC cohort to emulate a target trial using advanced statistical adjustment techniques including stabilized inverse-probability weighting and G-computation, dealing with missing data with multiple imputation, and performing a quantitative bias analysis for residual bias due to unmeasured confounders.

RESULTS

Emulation led to 3211 eligible patients, and overall survival estimates achieved with advanced statistical methods favored the combination therapy. Real-world effect sizes were close to that assessed in the existing E2100 randomized clinical trial (hazard ratio = 0.88, P = .16), but the increased sample size allowed to achieve a higher level of precision in real-world estimates (ie, reduced confidence intervals). Quantitative bias analysis confirmed the robustness of the results with respect to potential unmeasured confounding.

CONCLUSION

Target trial emulation with advanced statistical adjustment techniques is a promising approach to investigate long-term impact of innovative therapies in the French Epidemiological Strategy and Medical Economics-MBC cohort while minimizing biases and provides opportunities for comparative efficacy through the synthetic control arms provided.

DATABASE REGISTRATION

clinicaltrials.gov Identifier NCT03275311.

Allogeneic transplantation in advanced cutaneous T-cell lymphomas (CUTALLO): a propensity score matched controlled prospective study

BACKGROUND

Advanced-stage cutaneous T-cell lymphomas (CTCLs) are rare, usually refractory, and fatal diseases. Case series have suggested that allogeneic haematopoietic stem cell transplantation (HSCT) might improve the prognosis of advanced-stage CTCLs. The objective of this study was to investigate the effect of allogeneic HSCT compared with non-HSCT therapy on the outcome of individuals with advanced-stage CTCLs.

METHODS

In this prospective, multicentre, matched controlled trial, conducted at 30 hospitals, participants with advanced CTCLs were allocated treatment: if they had an available compatible related donor they were assigned to allogeneic HSCT, or if not they were allocated to non-allogeneic HSCT therapy. Key inclusion criteria were participants aged 18-70 years, with advanced stage mycosis fungoides or Sézary syndrome, and at least one poor prognostic criteria. Participants were excluded if they were not in complete or partial remission of the disease. Propensity score 1:1 matching with replacement (ie, that each participant treated with HSCT was matched to the participant with the closest propensity score treated with non-HSCT therapy, even if they had already been matched) was used to handle confounding factors, with the balance of covariate distribution between HSCT and non-HSCT groups assessed using standardised mean differences. The primary endpoint was progression-free survival in the matched intention-to-treat population. This trial is registered with ClinicalTrials.gov (NCT02520908), and is currently active but not recruiting.

FINDINGS

From June 1, 2016, to March 3, 2022, total of 99 participants were enrolled at 17 centres in France. Participants with a sibling or matched unrelated donor were assigned to allogeneic HSCT (HSCT group, n=55 [56%]) and participants without a donor were assigned to non-allogeneic HSCT treatment (non-HSCT group, n=44 [44%]). The median follow-up among survivors was 12·6 months (IQR 11·0-35·2). In the HSCT group, 51 participants (93%) were 1:1 matched to participants from the non-HSCT group. In the intention-to-treat analysis, median progression-free survival was significantly longer in the HSCT group (9·0 months [95% CI 6·6-30·5]) than in the non-HSCT group (3·0 months [2·0-6·3]), with a hazard ratio of 0·38 (95% CI 0·21-0·69; p<0·0001). In the per-protocol population, 40 participants (78%) in the HSCT group had 101 serious events and 29 participants (67%) in the non-HSCT group had 70 serious adverse events. The most common serious adverse event other than graft-versus-host disease in both groups was infections, occurring in 30 participants (59%) in the HSCT group and in 19 participants (44%) in the non-HSCT group.

INTERPRETATION

Allogeneic HSCT was associated with significantly longer progression-free survival in participants with advanced-stage CTCLs. These results indicate that allogeneic HSCT treatment should be made available to individuals with high-risk, advanced-stage mycosis fungoides or Sézary syndrome who achieve pre-transplant disease remission.

FUNDING

French Ministry of Health, National Cancer Institute, Programme Hospitalier de Recherche Clinique en Cancérologie.

Dealing with confounding in observational studies: A scoping review of methods evaluated in simulation studies with single-point exposure

The aim of this article was to perform a scoping review of methods available for dealing with confounding when analyzing the effect of health care treatments with single-point exposure in observational data. We aim to provide an overview of methods and their performance assessed by simulation studies indexed in PubMed. We searched PubMed for simulation studies published until January 2021. Our search was restricted to studies evaluating binary treatments and binary and/or continuous outcomes. Information was extracted on the methods' assumptions, performance, and technical properties. Of 28,548 identified references, 127 studies were eligible for inclusion. Of them, 84 assessed 14 different methods (ie, groups of estimators that share assumptions and implementation) for dealing with measured confounding, and 43 assessed 10 different methods for dealing with unmeasured confounding. Results suggest that there are large differences in performance between methods and that the performance of a specific method is highly dependent on the estimator. Furthermore, the methods' assumptions regarding the specific data features also substantially influence the methods' performance. Finally, the methods result in different estimands (ie, target of inference), which can even vary within methods. In conclusion, when choosing a method to adjust for measured or unmeasured confounding it is important to choose the most appropriate estimand, while considering the population of interest, data structure, and whether the plausibility of the methods' required assumptions hold.

Comparison of various heat waves definitions and the burden of heat-related mortality in France: Implications for existing early warning systems

INTRODUCTION

In France, a heat warning system (HWS) has been implemented almost two decades ago and rely on some official heat wave (HW) definitions. However, no study has compared the burden associated with a large set of alternative HW definitions to the official definitions. Such comparison could be particularly helpful to identify HW conditions for which effective HWS would minimize the health burden across various geographical contexts and possibly update thresholds to trigger HWS. The aim of this study is to identify (and rank) definitions that drive the highest health burden in terms of mortality to inform future HWS across multiple cities in France.

METHODS

Based on weather data for 16 French cities, we compared the two official definitions used in France to: i) the Excess Heat Factor (EHF) used in Australia, and ii) 18 alternative hypothetical HW definitions based on various combinations of temperature metrics, intensity, and duration. Propensity score matching and Poisson regressions were used to estimate the effect of each HW exposure on non-accidental mortality for the May-September period from 2000 to 2015.

RESULTS

The associations between HW and mortality differed greatly depending on the definition. The greatest burden of heat was 1,055 (95% confidence interval "CI": [856; 1,302]) deaths per summer and was obtained with the EHF. The EHF identified HW with 2.46 (95% CI: [1.92; 3.58]) or 8.18 (95% CI: [6.63; 10.61]) times the global burden at the national level obtained with the climatological indicator of the French national weather service and the HW indicator of the French national HWS, respectively and was the most impactful definition pattern for both temperate oceanic and Mediterranean climate types.

CONCLUSION

Identifying the set of extreme heat conditions that drive the highest health burden in a given geographical context is particularly helpful when designing or updating heat early warning systems.

Assessment of Glucose Lowering Medications' Effectiveness for Cardiovascular Clinical Risk Management of Real-World Patients with Type 2 Diabetes: Targeted Maximum Likelihood Estimation under Model Misspecification and Missing Outcomes

The results from many cardiovascular (CV) outcome trials suggest that glucose lowering medications (GLMs) are effective for the CV clinical risk management of type 2 diabetes (T2D) patients. The aim of this study is to compare the effectiveness of two GLMs (SGLT2i and GLP-1RA) for the CV clinical risk management of T2D patients in a real-world setting, by simultaneously reducing glycated hemoglobin, body weight, and systolic blood pressure. Data from the real-world Italian multicenter retrospective study Dapagliflozin Real World evideNce in Type 2 Diabetes (DARWINT 2D) are analyzed. Different statistical approaches are compared to deal with the real-world-associated issues, which can arise from model misspecification, nonrandomized treatment assignment, and a high percentage of missingness in the outcome, and can potentially bias the marginal treatment effect (MTE) estimate and thus have an influence on the clinical risk management of patients. We compare the logistic regression (LR), propensity score (PS)-based methods, and the targeted maximum likelihood estimator (TMLE), which allows for the use of machine learning (ML) models. Furthermore, a simulation study is performed, resembling the structure of the conditional dependencies among the main variables in DARWIN-T2D. LR and PS methods do not underline any difference in the effectiveness regarding the attainment of combined CV risk factor goals between the two treatments. TMLE suggests instead that dapagliflozin is significantly more effective than GLP-1RA for the CV risk management of T2D patients. The results from the simulation study suggest that TMLE has the lowest bias and SE for the estimate of the MTE.

Effect of postoperative administration of nafamostat mesilate on posthepatectomy liver failure

BACKGROUND

To investigate whether the administration of nafamostat mesilate (NM) reduces the risk of posthepatectomy liver failure (PHLF) in patients undergoing hepatectomy for hepatocellular carcinoma (HCC).

METHODS

We retrospectively reviewed the 1114 patients who underwent hepatectomy for HCC between 2004 and 2020. NM was selectively administered to patients undergoing major hepatectomy with an estimated blood loss of >500 mL. NM group was administered via intravenous of 20 mg of NM from immediately after surgery until postoperative day 4. We performed 1:1 propensity score matching and included 56 patients in each group. PHLF was defined according to the International Study Group of Liver Surgery (ISGLS).

RESULTS

The incidence of PHLF was lower in the NM group than control group (P = 0.018). The mean peak total bilirubin (P = 0.006), aspartate transaminase (P = 0.018), and alanine aminotransferase (P = 0.018) levels postoperatively were significantly lower in the NM group. The mean hospital stays (P = 0.012) and major complication rate (P = 0.023) were also significantly lower in the NM group.

CONCLUSION

Prophylactic administration of NM reduced the risks of complication and decreased the frequency of PHLF after hepatectomy. A further prospective study is needed to verify our findings.

A Review of Causal Inference for External Comparator Arm Studies

Randomized controlled trials (RCTs) are the gold standard design to establish the efficacy of new drugs and to support regulatory decision making. However, a marked increase in the submission of single-arm trials (SATs) has been observed in recent years, especially in the field of oncology due to the trend towards precision medicine contributing to the rise of new therapeutic interventions for rare diseases. SATs lack results for control patients, and information from external sources can be compiled to provide context for better interpretability of study results. External comparator arm (ECA) studies are defined as a clinical trial (most commonly a SAT) and an ECA of a comparable cohort of patients-commonly derived from real-world settings including registries, natural history studies, or medical records of routine care. This publication aims to provide a methodological overview, to sketch emergent best practice recommendations and to identify future methodological research topics. Specifically, existing scientific and regulatory guidance for ECA studies is reviewed and appropriate causal inference methods are discussed. Further topics include sample size considerations, use of estimands, handling of different data sources regarding differential baseline covariate definitions, differential endpoint measurements and timings. In addition, unique features of ECA studies are highlighted, specifically the opportunity to address bias caused by unmeasured ECA covariates, which are available in the SAT.

Assessing the effectiveness of empirical calibration under different bias scenarios

Background

Estimations of causal effects from observational data are subject to various sources of bias. One method for adjusting for the residual biases in the estimation of treatment effects is through the use of negative control outcomes, which are outcomes not believed to be affected by the treatment of interest. The empirical calibration procedure is a technique that uses negative control outcomes to calibrate p-values. An extension of this technique calibrates the coverage of the 95% confidence interval of a treatment effect estimate by using negative control outcomes as well as positive control outcomes, which are outcomes for which the treatment of interest has known effects. Although empirical calibration has been used in several large observational studies, there is no systematic examination of its effect under different bias scenarios.

Methods

The effect of empirical calibration of confidence intervals was analyzed using simulated datasets with known treatment effects. The simulations consisted of binary treatment and binary outcome, with biases resulting from unmeasured confounder, model misspecification, measurement error, and lack of positivity. The performance of the empirical calibration was evaluated by determining the change in the coverage of the confidence interval and the bias in the treatment effect estimate.

Results

Empirical calibration increased coverage of the 95% confidence interval of the treatment effect estimate under most bias scenarios but was inconsistent in adjusting the bias in the treatment effect estimate. Empirical calibration of confidence intervals was most effective when adjusting for the unmeasured confounding bias. Suitable negative controls had a large impact on the adjustment made by empirical calibration, but small improvements in the coverage of the outcome of interest were also observable when using unsuitable negative controls.

Conclusions

This work adds evidence to the efficacy of empirical calibration of the confidence intervals in observational studies. Calibration of confidence intervals is most effective where there are biases due to unmeasured confounding. Further research is needed on the selection of suitable negative controls.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01687-6.

Low-Volume Bowel Preparation Is Associated With Reduced Time to Colonoscopy in Hospitalized Patients: A Propensity-Matched Analysis

INTRODUCTION

Delays in inpatient colonoscopy are commonly caused by inadequate bowel preparation and result in increased hospital length of stay (LOS) and healthcare costs. Low-volume bowel preparation (LV-BP; sodium sulfate, potassium sulfate, and magnesium sulfate ) has been shown to improve outpatient bowel preparation quality compared with standard high-volume bowel preparations (HV-BP; polyethylene glycol ). However, its efficacy in hospitalized patients has not been well-studied. We assessed the impact of LV-BP on time to colonoscopy, hospital LOS, and bowel preparation quality among inpatients.

METHODS

We performed a propensity score-matched analysis of adult inpatients undergoing colonoscopy who received either LV-BP or HV-BP before colonoscopy at a quaternary academic medical center. Multivariate regression models with feature selection were developed to assess the association between LV-BP and study outcomes.

RESULTS

Among 1,807 inpatients included in this study, 293 and 1,514 patients received LV-BP and HV-BP, respectively. Among the propensity score-matched population, LV-BP was associated with a shorter time to colonoscopy (β: -0.43 [95% confidence interval: -0.56 to -0.30]) while having similar odds of adequate preparation (odds ratio: 1.02 [95% confidence interval: 0.71-1.46]; P = 0.92). LV-BP was also significantly associated with decreased hospital LOS among older patients (age ≥ 75 years), patients with chronic kidney disease, and patients who were hospitalized with gastrointestinal bleeding.

DISCUSSION

LV-BP is associated with decreased time to colonoscopy in hospitalized patients. Older inpatients, inpatients with chronic kidney disease, and inpatients with gastrointestinal bleeding may particularly benefit from LV-BP. Prospective studies are needed to further establish the role of LV-BP for inpatient colonoscopies.

Nationwide Analysis of Persistent Type II Endoleak and Late Outcomes of Endovascular Abdominal Aortic Aneurysm Repair in Japan: a Propensity-matched Analysis

We reviewed the results of endovascular aneurysm repair in patients from the Japanese Committee for Stentgraft Management registry to determine the significance of persistent type II endoleak (p-T2EL) and the risk of late adverse events, including aneurysm sac enlargement.

Medical Cannabis and Utilization of Nonhospice Palliative Care Services: Complements and Alternatives at End of Life

Background and Objectives

There is a need to know more about cannabis use among terminally diagnosed older adults, specifically whether it operates as a complement or alternative to palliative care. The objective is to explore differences among the terminal illness population within the Illinois Medical Cannabis Program (IMCP) by their use of palliative care.

Research Design and Methods

The study uses primary, cross-sectional survey data from 708 terminally diagnosed patients, residing in Illinois, and enrolled in the IMCP. We compared the sample on palliative care utilization through logistic regression models, examined associations between palliative care and self-reported outcome improvements using ordinary least squares regressions, and explored differences in average pain levels using independent t-tests.

Results

115 of 708 terminally diagnosed IMCP participants were receiving palliative care. We find increased odds of palliative care utilization for cancer (odds ratio [OR] [SE] = 2.15 [0.53], p < .01), low psychological well-being (OR [SE] = 1.97 [0.58], p < .05), medical complexity (OR [SE] = 2.05 [0.70], p < .05), and prior military service (OR [SE] = 2.01 [0.68], p < .05). Palliative care utilization is positively associated with improvement ratings for pain (7.52 [3.41], p < .05) and ability to manage health outcomes (8.29 [3.61], p < .01). Concurrent use of cannabis and opioids is associated with higher pain levels at initiation of cannabis dosing (p < .05).

Discussion and Implications

Our results suggest that cannabis is largely an alternative to palliative care for terminal patients. For those in palliative care, it is a therapeutic complement used at higher levels of pain.

Applying Propensity Score Methods in Clinical Research in Neurology

Propensity score-based analysis is increasingly being used in observational studies to estimate the effects of treatments, interventions, and exposures. We introduce the concept of the propensity score and how it can be used in observational research. We describe 4 different ways of using the propensity score: matching on the propensity score, inverse probability of treatment weighting using the propensity score, stratification on the propensity score, and covariate adjustment on the propensity score (with a focus on the first 2). We provide recommendations for the use and reporting of propensity score methods for the conduct of observational studies in neurologic research.

Use of Propensity Scoring and Its Application to Real-World Data: Advantages, Disadvantages, and Methodological Objectives Explained to Researchers Without Using Mathematical Equations

Real-time data collection of patient health status and medications is sped up with modern electronic devices and technologies. As real-world data provide enormous research opportunities, propensity score (PS) methods have been getting attention due to their theoretical grounds in a nonrandomized study setting. In contrast to randomized clinical trials, observational clinical data obtained from a real-world database may not have balanced distributions of patient characteristics between treatment and control groups at the beginning of the respective study. These imbalanced distributions may cause a bias in an estimated treatment effect, which needs to be eliminated. Propensity scoring is one class of statistical methods to address the imbalance issue of real-world data sets. This article provides basic concepts and assesses advantages, disadvantages, and methodological objectives of propensity scoring. Targeting clinical pharmacology researchers with limited statistical background, 5 representative methods are reviewed and visualized: matching, stratification, covariate modeling, inverse probability of treatment weighting, and doubly robust methods. Examples of applications of PS methods were selected from the literature of outcomes research and drug development, nephrology, and pediatrics. Opportunities of applications related to these examples are described. Furthermore, potential future applications of PS methods in clinical pharmacology are discussed. The 21st Century Cures Act signed in 2016 encourages scientists to find opportunities to apply propensity scoring to real-world data. This article underscores that scientists need to justify their choice of statistical methods, whether a PS method or an alternative method, based on their clinical study design, statistical assumptions, and research objectives.

Multiple imputation analysis for propensity score matching with missing causes of failure: An application to hepatocellular carcinoma data

Propensity score matching is widely used to determine the effects of treatments in observational studies. Competing risk survival data are common to medical research. However, there is a paucity of propensity score matching studies related to competing risk survival data with missing causes of failure. In this study, we provide guidelines for estimating the treatment effect on the cumulative incidence function when using propensity score matching on competing risk survival data with missing causes of failure. We examined the performances of different methods for imputing the data with missing causes. We then evaluated the gain from the missing cause imputation in an extensive simulation study and applied the proposed data imputation method to the data from a study on the risk of hepatocellular carcinoma in patients with chronic hepatitis B and chronic hepatitis C.

Effects of Compliance with the Early Management Bundle (SEP-1) on Mortality Changes among Medicare Beneficiaries with Sepsis: A Propensity Score Matched Cohort Study

BACKGROUND

U.S. hospitals have reported compliance with the SEP-1 quality measure to Medicare since 2015. Finding an association between compliance and outcomes is essential to gauge measure effectiveness.

RESEARCH QUESTION

What is the association between compliance with SEP-1 and 30-day mortality among Medicare beneficiaries?

STUDY DESIGN AND METHODS

Studying patient-level data reported to Medicare by 3,241 hospitals from October 1, 2015 to March 31, 2017, we used propensity score matching and a hierarchical general linear model (HGLM) to estimate the treatment effects associated with compliance with SEP-1. Compliance was defined as completion of all qualifying SEP-1 elements including lactate measurements, blood culture collection, broad-spectrum antibiotic administration, 30 ml/kg crystalloid fluid administration, application of vasopressors, and patient reassessment. The primary outcome was a change in 30-day mortality. Secondary outcomes included changes in length-of-stay.

RESULTS

We completed two matches to evaluate population-level treatment effects. In "Standard-match" 122,870 patients whose care was compliant were matched with the same number whose care was non-compliant. Compliance was associated with a reduction in 30-day mortality: 21.81% versus 27.48% yielding an ARR of 5.67% (95% confidence interval [CI]: 5.33-6.00; P < 0.001). In "Stringent-match" 107,016 patients whose care was compliant were matched with the same number whose care was non-compliant. Compliance was associated with a reduction in 30-day mortality: 22.22% versus 26.28% yielding an ARR of 4.06% (95% CI: 3.70-4.41; P < 0.001). At the subject-level, our HGLM model found compliance associated with lower 30-day risk-adjusted mortality (adjusted conditional odds ratio = 0.829; 95% CI: 0.812-0.846; P < 0001). Multiple elements correlated with lower mortality. Median length-of-stay was shorter among cases whose care was compliant (5 vs. 6 days; IQR: 3-9 vs. 4-10; P < 0.001).

INTERPRETATION

Compliance with SEP-1 was associated with lower 30-day mortality. Rendering SEP-1 compliant care may reduce the incidence of avoidable deaths.

Variance estimators for weighted and stratified linear dose-response function estimators using generalized propensity score

Propensity score methods are widely used in observational studies for evaluating marginal treatment effects. The generalized propensity score (GPS) is an extension of the propensity score framework, historically developed in the case of binary exposures, for use with quantitative or continuous exposures. In this paper, we proposed variance estimators for treatment effect estimators on continuous outcomes. Dose-response functions (DRFs) were estimated through weighting on the inverse of the GPS, or using stratification. Variance estimators were evaluated using Monte Carlo simulations. Despite the use of stabilized weights, the variability of the weighted estimator of the DRF was particularly high, and none of the variance estimators (a bootstrap-based estimator, a closed-form estimator especially developed to take into account the estimation step of the GPS, and a sandwich estimator) were able to adequately capture this variability, resulting in coverages below the nominal value, particularly when the proportion of the variation in the quantitative exposure explained by the covariates was large. The stratified estimator was more stable, and variance estimators (a bootstrap-based estimator, a pooled linearized estimator, and a pooled model-based estimator) more efficient at capturing the empirical variability of the parameters of the DRF. The pooled variance estimators tended to overestimate the variance, whereas the bootstrap estimator, which intrinsically takes into account the estimation step of the GPS, resulted in correct variance estimations and coverage rates. These methods were applied to a real data set with the aim of assessing the effect of maternal body mass index on newborn birth weight.

Performance of propensity score matching to estimate causal effects in small samples

Propensity score (PS) matching is a very popular causal estimator usually used to estimate the average treatment effect on the treated (ATT) from observational data. However, opting for this estimator may raise some efficiency issues when the sample size is limited. Therefore, we aimed to evaluate the performance of propensity score matching in this context. We started with a motivating example based on a cohort of 66 children with sickle cell anemia who received either allogeneic bone-marrow transplant or chronic transfusion. We found substantial differences in the ATT estimate according to the model selected for propensity score estimation and subsequent matching. Then, we assessed the performance of the different propensity score matching methods and post-matching analyses to estimate the ATT using a simulation study. Although all selected propensity score matching methods were based of previous recommendations, we found important discrepancies in the estimation of treatment effect between them, underlining the importance of thorough sensitivity analyses when using propensity score matching in the context of small sample sizes.

Adjustment for Baseline Covariates to Increase Efficiency in RCTs with Binary Endpoint: A Comparison of Bayesian and Frequentist Approaches

BACKGROUND

In a randomized controlled trial (RCT) with binary outcome the estimate of the marginal treatment effect can be biased by prognostic baseline covariates adjustment. Methods that target the marginal odds ratio, allowing for improved precision and power, have been developed.

METHODS

The performance of different estimators for the treatment effect in the frequentist (targeted maximum likelihood estimator, inverse-probability-of-treatment weighting, parametric G-computation, and the semiparametric locally efficient estimator) and Bayesian (model averaging), adjustment for confounding, and generalized Bayesian causal effect estimation frameworks are assessed and compared in a simulation study under different scenarios. The use of these estimators is illustrated on an RCT in type II diabetes.

RESULTS

Model mis-specification does not increase the bias. The approaches that are not doubly robust have increased standard error (SE) under the scenario of mis-specification of the treatment model. The Bayesian estimators showed a higher type II error than frequentist estimators if noisy covariates are included in the treatment model.

CONCLUSIONS

Adjusting for prognostic baseline covariates in the analysis of RCTs can have more power than intention-to-treat based tests. However, for some classes of model, when the regression model is mis-specified, inflated type I error and potential bias on treatment effect estimate may arise.

Modelling hospital outcome: problems with endogeneity

Background

Mortality modelling in the critical care paradigm traditionally uses logistic regression, despite the availability of estimators commonly used in alternate disciplines. Little attention has been paid to covariate endogeneity and the status of non-randomized treatment assignment. Using a large registry database, various binary outcome modelling strategies and methods to account for covariate endogeneity were explored.

Methods

Patient mortality data was sourced from the Australian & New Zealand Intensive Society Adult Patient Database for 2016. Hospital mortality was modelled using logistic, probit and linear probability (LPM) models with intensive care (ICU) providers as fixed (FE) and random (RE) effects. Model comparison entailed indices of discrimination and calibration, information criteria (AIC and BIC) and binned residual analysis. Suspect covariate and ventilation treatment assignment endogeneity was identified by correlation between predictor variable and hospital mortality error terms, using the Stata™ “eprobit” estimator. Marginal effects were used to demonstrate effect estimate differences between probit and “eprobit” models.

Results

The cohort comprised 92,693 patients from 124 intensive care units (ICU) in calendar year 2016. Patients mean age was 61.8 (SD 17.5) years, 41.6% were female and APACHE III severity of illness score 54.5(25.6); 43.7% were ventilated. Of the models considered in predicting hospital mortality, logistic regression (with or without ICU FE) and RE logistic regression dominated, more so the latter using information criteria indices. The LPM suffered from many predictions outside the unit [0,1] interval and both poor discrimination and calibration. Error terms of hospital length of stay, an independent risk of death score and ventilation status were correlated with the mortality error term. Marked differences in the ventilation mortality marginal effect was demonstrated between the probit and the "eprobit" models which were scenario dependent. Endogeneity was not demonstrated for the APACHE III score.

Conclusions

Logistic regression accounting for provider effects was the preferred estimator for hospital mortality modelling. Endogeneity of covariates and treatment variables may be identified using appropriate modelling, but failure to do so yields problematic effect estimates.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01251-8.

How to Apply Multiple Imputation in Propensity Score Matching with Partially Observed Confounders: A Simulation Study and Practical Recommendations

Propensity score matching (PSM) has been widely used to mitigate confounding in observational studies, although complications arise when the covariates used to estimate the PS are only partially observed. Multiple imputation (MI) is a potential solution for handling missing covariates in the estimation of the PS. However, it is not clear how to best apply MI strategies in the context of PSM. We conducted a simulation study to compare the performances of popular non-MI missing data methods and various MI-based strategies under different missing data mechanisms. We found that commonly applied missing data methods resulted in biased and inefficient estimates, and we observed large variation in performance across MI-based strategies. Based on our findings, we recommend 1) estimating the PS after applying MI to impute missing confounders; 2) conducting PSM within each imputed dataset followed by averaging the treatment effects to arrive at one summarized finding; 3) a bootstrapped-based variance to account for uncertainty of PS estimation, matching, and imputation; and 4) inclusion of key auxiliary variables in the imputation model.

Real-world discontinuation rate of teriflunomide and dimethyl fumarate in multiple sclerosis

BACKGROUND

For patients with MS, medication switches increase the risk of disease reactivation.

OBJECTIVE

Compare discontinuation rates due to treatment failure or side effects between teriflunomide and dimethyl fumarate, and investigate clinical variables affecting discontinuation rates.

METHODS

All patients who received teriflunomide or dimethyl fumarate at Haukeland University Hospital from 2013 until 2018 were identified. Clinical and demographic variables were extracted from the Norwegian MS Registry. Cause-specific Cox regression models estimated the rate of discontinuation due to treatment failure or side effects.

RESULTS

We included 354 patients treated with either dimethyl fumarate (n = 185) or teriflunomide (n = 169). We found 38% lower risk of discontinuation because of treatment failure for patients using dimethyl fumarate compared to teriflunomide (p < 0.05). In a treatment-naive subgroup (n = 183), we found a 38% reduced risk of discontinuation for any reason among patients using dimethyl fumarate (p < 0.05). There was no significant difference between treatment groups in discontinuation rate due to side effects, although more patients reported side effects when treated with dimethyl fumarate.

CONCLUSION

Our findings suggests that dimethyl fumarate has a lower risk of discontinuation because of treatment failure among both treatment-experienced and treatment-naive patients.

Propensity score matching in otolaryngologic literature: A systematic review and critical appraisal

BACKGROUND

Propensity score techniques can reduce confounding and bias in observational studies. Such analyses are able to measure and balance pre-determined covariates between treated and untreated groups, leading to results that can approximate those generated by randomized prospective studies when such trials are not feasible. The most commonly used propensity score -based analytic technique is propensity score matching (PSM). Although PSM popularity has continued to increase in medical literature, improper methodology or methodological reporting may lead to biased interpretation of treatment effects or limited scientific reproducibility and generalizability. In this study, we aim to characterize and assess the quality of PSM methodology reporting in high-impact otolaryngologic literature.

METHODS

PubMed and Embase based systematic review of the top 20 journals in otolaryngology, as measured by impact factor from the Journal Citations Reports from 2012 to 2018, for articles using PSM analysis throughout their publication history. Eligible articles were reviewed and assessed for quality and reporting of PSM methodology.

RESULTS

Our search yielded 101 studies, of which 92 were eligible for final analysis and review. The proportion of studies utilizing PSM increased significantly over time (p < 0.001). Nearly all studies (96.7%, n = 89) specified the covariates used to calculate propensity scores. Covariate balance was illustrated in 67.4% (n = 62) of studies, most frequently through p-values. A minority (17.4%, n = 16) of studies were found to be fully reproducible according to previously established criteria.

CONCLUSIONS

While PSM analysis is becoming increasingly prevalent in otolaryngologic literature, the quality of PSM methodology reporting can be improved. We provide potential recommendations for authors regarding optimal reporting for analyses using PSM.

Causal inference methods for small non-randomized studies: Methods and an application in COVID-19

The usual development cycles are too slow for the development of vaccines, diagnostics and treatments in pandemics such as the ongoing SARS-CoV-2 pandemic. Given the pressure in such a situation, there is a risk that findings of early clinical trials are overinterpreted despite their limitations in terms of size and design. Motivated by a non-randomized open-label study investigating the efficacy of hydroxychloroquine in patients with COVID-19, we describe in a unified fashion various alternative approaches to the analysis of non-randomized studies. A widely used tool to reduce the impact of treatment-selection bias are so-called propensity score (PS) methods. Conditioning on the propensity score allows one to replicate the design of a randomized controlled trial, conditional on observed covariates. Extensions include the g-computation approach, which is less frequently applied, in particular in clinical studies. Moreover, doubly robust estimators provide additional advantages. Here, we investigate the properties of propensity score based methods including three variations of doubly robust estimators in small sample settings, typical for early trials, in a simulation study. R code for the simulations is provided.

Comparison of Prognosis Between Juvenile and Adult Nasopharyngeal Carcinoma: A Propensity Score-Matched Analysis

Purpose

To investigate whether juvenile patients with nasopharyngeal carcinoma (NPC) in China have better prognosis than their adult counterparts in the intensity-modulated radiation therapy (IMRT) era, after controlling for potential confounding variables.

Methods

Data pertaining to 1139 patients with newly diagnosed NPC without metastasis, who were treated with IMRT at our hospital, were retrospectively analyzed. Of these, 60 patients were juvenile (age ≤18 years) diagnosed between January 2003 and December 2018, while 1079 patients were adults (≤65 years) diagnosed between January 2013 and December 2014. To minimize the influence of selection and confounding bias, 1:2 propensity score matching (PSM) was used. Overall survival (OS), disease-free survival (DFS), locoregional relapse-free survival (LRFS), and distant metastasis-free survival (DMFS) were estimated using the Kaplan–Meier method and between-group differences assessed using the Log rank test. The long-term toxicity of the juvenile patients was evaluated according to the criteria of the Radiation Therapy Oncology Group (RTOG) and the Common Terminology Criteria for Adverse Events (CTCAE) version 5.0.

Results

Five-year OS of juvenile and adult patients were 88.07% and 85.08%, respectively. Before PSM, OS, PFS, DMFS, or LRFS were comparable in the two groups (all P > 0.05). After PSM, OS, DFS, and LRFS in the juvenile group were markedly longer than that in adults (P = 0.005, P = 0.027, and P = 0.024, respectively). With respect to long-term toxicity, the most common adverse effects in juvenile patients were cervix fibrosis, ototoxicity, and xerostomia. However, except for two patients who developed grade 3 ototoxicity, all adverse effects were within grade 2.

Conclusion

In the IMRT era, juvenile Chinese patients with NPC had better 5-year OS, DFS, and LRFS than their adult counterparts. The adverse events in the juvenile cohort were relatively mild; however, the risk of severe ototoxicity should not be neglected.

Flexible regression approach to propensity score analysis and its relationship with matching and weighting

In propensity score analysis, the frequently used regression adjustment involves regressing the outcome on the estimated propensity score and treatment indicator. This approach can be highly efficient when model assumptions are valid, but can lead to biased results when the assumptions are violated. We extend the simple regression adjustment to a varying coefficient regression model that allows for nonlinear association between outcome and propensity score. We discuss its connection with some propensity score matching and weighting methods, and show that the proposed analytical framework can shed light on the intrinsic connection among some mainstream propensity score approaches (stratification, regression, kernel matching, and inverse probability weighting) and handle commonly used causal estimands. We derive analytic point and variance estimators that properly take into account the sampling variability in the estimated propensity score. Extensive simulations show that the proposed approach possesses desired finite sample properties and demonstrates competitive performance in comparison with other methods estimating the same causal estimand. The proposed methodology is illustrated with a study on right heart catheterization.

G-computation, propensity score-based methods, and targeted maximum likelihood estimator for causal inference with different covariates sets: a comparative simulation study

Controlling for confounding bias is crucial in causal inference. Distinct methods are currently employed to mitigate the effects of confounding bias. Each requires the introduction of a set of covariates, which remains difficult to choose, especially regarding the different methods. We conduct a simulation study to compare the relative performance results obtained by using four different sets of covariates (those causing the outcome, those causing the treatment allocation, those causing both the outcome and the treatment allocation, and all the covariates) and four methods: g-computation, inverse probability of treatment weighting, full matching and targeted maximum likelihood estimator. Our simulations are in the context of a binary treatment, a binary outcome and baseline confounders. The simulations suggest that considering all the covariates causing the outcome led to the lowest bias and variance, particularly for g-computation. The consideration of all the covariates did not decrease the bias but significantly reduced the power. We apply these methods to two real-world examples that have clinical relevance, thereby illustrating the real-world importance of using these methods. We propose an R package RISCA to encourage the use of g-computation in causal inference.

Importance of operating room case scheduling on analyses of observed reductions in surgical site infections from the purchase and installation of capital equipment in operating rooms

BACKGROUND

We review the impact of the consequences of operating room (OR) management decision making on power analyses for observational studies of surgical site infections (SSIs) among patients receiving care in ORs with interventions versus without interventions involving physical changes to ORs. Examples include ventilation systems, bactericidal lighting, and physical alterations to ORs.

METHODS

We performed a narrative review of operating room management and surgical site infection articles. We used 10-years of operating room data to estimate parameters for use in statistical power analyses.

RESULTS

Creating pivot tables or monthly control charts of SSI per case by OR and comparing among ORs with or without intervention is not recommended. This approach has low power to detect a difference in SSI rates among the ORs with or without the intervention. The reason is that appropriate OR case scheduling decision making causes risk factors for SSI to differ among ORs, even when stratifying by surgical specialty. Such risk factors include case duration, urgency, and American Society of Anesthesiologists' Physical Status. Instead, analyze SSI controlling for the OR, where the patient had surgery, and matching patients using these variables is preferable. With α = 0.05, 600 cases per OR, 5 intervention ORs, and 5 or 1 control patients for each intervention patient, reasonable power (≅94% or 78%, respectively) can be achieved to detect reductions (3.6% to 2.4%) in the incidence of SSI between ORs with or without the intervention.

CONCLUSIONS

By using this matched cohort design, the effect of the purchase and installation of capital equipment in ORs on SSI can be evaluated meaningfully.

Variance estimation when using propensity-score matching with replacement with survival or time-to-event outcomes

Propensity‐score matching is a popular analytic method to estimate the effects of treatments when using observational data. Matching on the propensity score typically requires a pool of potential controls that is larger than the number of treated or exposed subjects. The most common approach to matching on the propensity score is matching without replacement, in which each control subject is matched to at most one treated subject. Failure to find a matched control for each treated subject can lead to “bias due to incomplete matching.” To avoid this bias, it is important to identify a matched control subject for each treated subject. An alternative to matching without replacement is matching with replacement, in which control subjects are allowed to be matched to multiple treated subjects. A limitation to the use of matching with replacement is that variance estimation must account for both the matched nature of the sample and for some control subjects being included in multiple matched sets. While a variance estimator has been proposed for when outcomes are continuous, no such estimator has been proposed for use with time‐to‐event outcomes, which are common in medical and epidemiological research. We propose a variance estimator for the hazard ratio when matching with replacement. We conducted a series of Monte Carlo simulations to examine the performance of this estimator. We illustrate the utility of matching with replacement to estimate the effect of smoking cessation counseling on survival in smokers discharged from hospital with a heart attack.

Safety surveillance and the estimation of risk in select populations: Flexible methods to control for confounding while targeting marginal comparisons via standardization

We consider the critical problem of pharmacosurveillance for adverse events once a drug or medical product is incorporated into routine clinical care. When making inference on comparative safety using large-scale electronic health records, we often encounter an extremely rare binary adverse outcome with a large number of potential confounders. In this context, it is challenging to offer flexible methods to adjust for high-dimensional confounders, whereas use of the propensity score (PS) can help address this challenge by providing both confounding control and dimension reduction. Among PS methods, regression adjustment using the PS as a covariate in an outcome model has been incompletely studied and potentially misused. Previous studies have suggested that simple linear adjustment may not provide sufficient control of confounding. Moreover, no formal representation of the statistical procedure and associated inference has been detailed. In this paper, we characterize a three-step procedure, which performs flexible regression adjustment of the estimated PS followed by standardization to estimate the causal effect in a select population. We also propose a simple variance estimation method for performing inference. Through a realistic simulation mimicking data from the Food and Drugs Administration's Sentinel Initiative comparing the effect of angiotensin-converting enzyme inhibitors and beta blockers on incidence of angioedema, we show that flexible regression on the PS resulted in less bias without loss of efficiency, and can outperform other methods when the PS model is correctly specified. In addition, the direct variance estimation method is a computationally fast and reliable approach for inference.

Cataract surgery-related complications in patients with end-stage renal disease- a nationwide population-based study in Taiwan

This nationwide retrospective case-control study was aimed at elucidating the risk from cataract surgery in end-stage renal disease (ESRD) patients. Cataract surgery patients were identified using the diagnostic and procedural codes for International Classification of Diseases, 9^th Revision, Clinical Modification from Taiwan’s National Health Insurance Research Database. ESRD patients were selected as cases, while propensity scores for age, sex, comorbidities and year-of-surgery-matched patients without chronic kidney disease constituted the controls. Patients who had undergone eye surgery within 3 years before cataract surgery were excluded. The main outcome measures were target cataract surgery-related complications within 3 months after surgery. A total of 352 cases and 1,760 controls were analysed. Patients with ESRD had a 5.06-fold (95% CI: 2.36–10.87; p < 0.001) risk of vitreous haemorrhage and a 2.74-fold (95% CI: 1.20–6.27; p = 0.017) risk of re-operation for dropped nucleus or vitreous complications. Non-diabetic ESRD patients had a 3.49-fold (95% CI: 1.36–8.91; p = 0.009) risk of corneal oedema. In conclusion, ESRD patients have a higher risk of vitreous haemorrhage, re-operation for dropped nucleus or vitreous complications and corneal oedema (non-diabetic patients) after cataract surgery. Pre-surgery corneal examination, surgery procedure and medication adjustment, closer and longer post-surgery follow-up may lower the risk and improve the visual outcome.

How to obtain valid tests and confidence intervals after propensity score variable selection?

The problem of how to best select variables for confounding adjustment forms one of the key challenges in the evaluation of exposure or treatment effects in observational studies. Routine practice is often based on stepwise selection procedures that use hypothesis testing, change-in-estimate assessments or the lasso, which have all been criticised for – amongst other things – not giving sufficient priority to the selection of confounders. This has prompted vigorous recent activity in developing procedures that prioritise the selection of confounders, while preventing the selection of so-called instrumental variables that are associated with exposure, but not outcome (after adjustment for the exposure). A major drawback of all these procedures is that there is no finite sample size at which they are guaranteed to deliver treatment effect estimators and associated confidence intervals with adequate performance. This is the result of the estimator jumping back and forth between different selected models, and standard confidence intervals ignoring the resulting model selection uncertainty. In this paper, we will develop insight into this by evaluating the finite-sample distribution of the exposure effect estimator in linear regression, under a number of the aforementioned confounder selection procedures. We will show that by making clever use of propensity scores, a simple and generic solution is obtained in the context of generalized linear models, which overcomes this concern (under weaker conditions than competing proposals). Specifically, we propose to use separate regularized regressions for the outcome and propensity score models in order to construct a doubly robust ‘g-estimator’; when these models are sufficiently sparse and correctly specified, standard confidence intervals for the g-estimator implicitly incorporate the uncertainty induced by the variable selection procedure.

Controlling for confounding via propensity score methods can result in biased estimation of the conditional AUC: A simulation study

In the medical literature, there has been an increased interest in evaluating association between exposure and outcomes using nonrandomized observational studies. However, because assignments to exposure are not random in observational studies, comparisons of outcomes between exposed and nonexposed subjects must account for the effect of confounders. Propensity score methods have been widely used to control for confounding, when estimating exposure effect. Previous studies have shown that conditioning on the propensity score results in biased estimation of conditional odds ratio and hazard ratio. However, research is lacking on the performance of propensity score methods for covariate adjustment when estimating the area under the ROC curve (AUC). In this paper, AUC is proposed as measure of effect when outcomes are continuous. The AUC is interpreted as the probability that a randomly selected nonexposed subject has a better response than a randomly selected exposed subject. A series of simulations has been conducted to examine the performance of propensity score methods when association between exposure and outcomes is quantified by AUC; this includes determining the optimal choice of variables for the propensity score models. Additionally, the propensity score approach is compared with that of the conventional regression approach to adjust for covariates with the AUC. The choice of the best estimator depends on bias, relative bias, and root mean squared error. Finally, an example looking at the relationship of depression/anxiety and pain intensity in people with sickle cell disease is used to illustrate the estimation of the adjusted AUC using the proposed approaches.

Clinical significance of preoperative chemoradiotherapy for advanced esophageal cancer, evaluated by propensity score matching and weighting of inverse probability of treatment

The present study used inverse probability of treatment weighting (IPTW) and propensity score matching (PSM) to compare survival benefits among 112 patients with resectable, stage II–IV esophageal squamous cell carcinoma (SCC) treated between 1996 and 2016 with neoadjuvant chemoradiotherapy (NAC) plus surgery (Group A, n=55) or with surgery alone (Group B, n=57). Their propensity scores (PS) were calculated using a multivariable logistic regression model in which age, sex, cancer site, primary tumor length, cTNM stage, lymph node metastasis and depth of tumor invasion were the independent variables, and used to match Groups A and B according to the IPTW and matching method. After IPTW and PSM, univariate analysis was used to assess overall survival (OS) and disease-free survival (DFS), followed by Cox proportional hazard models for OS using IPTW between the two groups and the subgroups. After PSM, 5-year OS and DFS were significantly higher in Group A (OS: 65.2%, DFS: 65.2%) compared with Group B (OS: 31.2%, DFS: 20.87%). Similarly, after IPTW, OS and DFS were significantly higher in Group A compared with Group B patients. Five-year OS was 73.18% for Group A and 37.69% for Group B (hazard ratio: 0.2899, 95% confidence interval: 0.1167–0.7205). To conclude, treatment was more effective in Group A patients with clinical stage II, N0 and T3 disease involving the mid-esophagus. It was concluded that for patients with esophageal SCC, NAC plus esophagectomy exhibited improved survival compared with surgery alone, as demonstrated by use of IPTW and PSM methods.

Tracks as frames of reference for academic self-concept

This study compared the development of academic self-concepts between different educational programs. A longitudinal cohort study in Flanders (3205 students in 46 schools) was used to compare students' academic self-concepts during the first three years of secondary education. General academic self-concept, self-concept in mathematics and self-concept in Dutch were measured. The investigated educational programs, called tracks, differ in the extent they are academically or vocationally focused and differ in average student academic ability. To control for selection effects, students who are comparable across the four tracks were matched using propensity score matching, Mahalanobis distance matching and coarsened exact matching. By means of multiple indicator quadratic latent growth curves, pairs of tracks that are hierarchically consecutive were compared regarding the development in academic self-concepts. For the two highest tracks, it was beneficial to be allocated to the highest track, whereas the pairwise comparisons between the three lower tracks indicated a detrimental effect of being in a higher track. The findings from this study do not support the big-fish-little-pond hypothesis or the basking in reflected glory hypothesis. Differences between tracks for the development of self-concepts only became apparent after two years.

Propensity-score matching with competing risks in survival analysis

Propensity‐score matching is a popular analytic method to remove the effects of confounding due to measured baseline covariates when using observational data to estimate the effects of treatment. Time‐to‐event outcomes are common in medical research. Competing risks are outcomes whose occurrence precludes the occurrence of the primary time‐to‐event outcome of interest. All non‐fatal outcomes and all cause‐specific mortality outcomes are potentially subject to competing risks. There is a paucity of guidance on the conduct of propensity‐score matching in the presence of competing risks. We describe how both relative and absolute measures of treatment effect can be obtained when using propensity‐score matching with competing risks data. Estimates of the relative effect of treatment can be obtained by using cause‐specific hazard models in the matched sample. Estimates of absolute treatment effects can be obtained by comparing cumulative incidence functions (CIFs) between matched treated and matched control subjects. We conducted a series of Monte Carlo simulations to compare the empirical type I error rate of different statistical methods for testing the equality of CIFs estimated in the matched sample. We also examined the performance of different methods to estimate the marginal subdistribution hazard ratio. We recommend that a marginal subdistribution hazard model that accounts for the within‐pair clustering of outcomes be used to test the equality of CIFs and to estimate subdistribution hazard ratios. We illustrate the described methods by using data on patients discharged from hospital with acute myocardial infarction to estimate the effect of discharge prescribing of statins on cardiovascular death.

Matched or unmatched analyses with propensity-score-matched data?

Propensity-score matching has been used widely in observational studies to balance confounders across treatment groups. However, whether matched-pairs analyses should be used as a primary approach is still in debate. We compared the statistical power and type 1 error rate for four commonly used methods of analyzing propensity-score-matched samples with continuous outcomes: (1) an unadjusted mixed-effects model, (2) an unadjusted generalized estimating method, (3) simple linear regression, and (4) multiple linear regression. Multiple linear regression had the highest statistical power among the four competing methods. We also found that the degree of intraclass correlation within matched pairs depends on the dissimilarity between the coefficient vectors of confounders in the outcome and treatment models. Multiple linear regression is superior to the unadjusted matched-pairs analyses for propensity-score-matched data.

Closed-form variance estimator for weighted propensity score estimators with survival outcome

Propensity score (PS) methods are widely used in observational studies for evaluating marginal treatment effects. PS-weighting is a popular PS-based method that allows for estimating both the average treatment effect on the overall population (ATE) and the average treatment effect on the treated population (ATT). Previous research has shown that the variance of the treatment effect is accurately estimated only if the variance estimator takes into account the fact that the propensity score is itself estimated from the available data in a first step of the analysis. In 2016, Austin showed that the bootstrap-based variance estimator was the only existing estimator resulting in approximately correct estimates of standard errors when evaluating a survival outcome and a Cox model was used to estimate a marginal hazard ratio (HR). This author stressed the need to develop a closed-form variance estimator of the marginal HR accounting for the estimation of the PS. In the present research, we developed such variance estimators both for the ATE and ATT. We evaluated their performance with an extensive simulation study and compared them to bootstrap-based variance estimators and to naive variance estimators that do not account for the estimation step. We found that the performance of the proposed variance estimators was similar to that of the bootstrap-based estimators. The proposed variance estimators provide an alternative to the bootstrap estimator, particularly interesting in situations in which time-consumption and/or reproducibility are an important issue. An implementation has been developed for the R software and is freely available (package hrIPW).

A review and empirical comparison of causal inference methods for clustered observational data with application to the evaluation of the effectiveness of medical devices

Observational studies are commonplace in medicine. A frequent concern is confounding bias due to differences in patient characteristics across treatment groups, but other important issues include dependency among observations nested within clusters (e.g. patients clustered within physicians or surgeons) and confounding due to cluster characteristics (e.g. physician or surgeon experience or training). Given the frequency with which these issues arise in medical research, as well as their relative complexity, methods for the analysis of clustered observational data are reviewed. We argue for estimating causal treatment effects using marginal models that either match or weight observations using a suitable distance metric (e.g. the propensity score). Simulation results demonstrated that methods incorporating clustering into calculation of the variance were generally more accurate than those that did not. Moreover, methods that account for cluster confounding when estimating the treatment effect were least biased and most accurate. Throughout the paper we illustrate the proposed methods in a medical device setting that compares the effectiveness of femoral heads used in total hip replacements. Whenever possible the clustered aspect of the data should be considered in the design of the study when constructing the distance measure or in the matching process, as well as in the analysis when estimating the variance of the treatment effect.

Tuberculosis infection via the emergency department among inpatients in South Korea: a propensity score matched analysis of the National Inpatient Sample

BACKGROUND

Emergency departments (EDs) carry a high risk of infectious disease transmission and have also been implicated in tuberculosis (TB) outbreaks.

AIM

To determine if patients who visit EDs have an increased risk of TB infection. Using South Korean inpatient sample data (2012), the risk of TB occurrence during 90 days after hospitalization for patients admitted via EDs was compared with that for patients admitted via outpatient clinics.

METHODS

The data of the 2012 Health Insurance Review and Assessment Service - National Inpatient Sample were used. TB diagnosis was based on International Classification of Diseases Version 10 [all TB (A15-A19), pulmonary TB (A15-A16) and extrapulmonary TB (A17-A18)].

FINDINGS

After propensity score matching using the demographic and clinical characteristics of the patients, 191,997 patients (64,017 patients admitted via EDs and 127,908 patients admitted via outpatient clinics) were included in this study. There was no significant difference in baseline patient characteristics between the two groups. The percentage of patients with TB admitted via EDs was higher than that of patients admitted via outpatient clinics. The likelihood of active TB occurrence was 30% higher for all TB [hazard ratio (HR) 1.30; 95% confidence interval (CI) 1.12-1.52] and pulmonary TB (HR 1.30; 95% CI 1.10-1.53) in patients admitted via EDs compared with patients admitted via outpatient clinics; this difference was significant. However, no difference in the occurrence of extrapulmonary TB was observed between the two groups.

CONCLUSIONS

The likelihood of TB infection was greater in patients admitted via EDs than in patients admitted via outpatient clinics.

Association of Clinical and Economic Outcomes With Permanent Pacemaker Implantation After Transcatheter Aortic Valve Replacement

IMPORTANCE

The literature is inconsistent regarding the impact of permanent pacemaker implantation after transcatheter aortic valve replacement.

OBJECTIVE

To evaluate clinical and economic outcomes in patients who required permanent pacemaker implantation during the index hospitalization after transcatheter aortic valve replacement.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective, population-based cohort study using data from a multicenter registry included patients who underwent a transcatheter aortic valve replacement procedure from April 1, 2010, to March 31, 2015, in Ontario, Canada, with follow-up to March 31, 2017. Patients who had a previously implanted permanent pacemaker or who died during the index hospitalization were excluded. Inverse probability of treatment weighting using the propensity score was used to adjust for baseline differences between the pacemaker and nonpacemaker groups.

EXPOSURES

Patients received a permanent pacemaker during the index hospitalization after transcatheter aortic valve replacement.

MAIN OUTCOMES AND MEASURES

All-cause mortality, readmission, readmission for heart failure, emergency department visits, and cumulative 1-year health care costs.

RESULTS

The study cohort consisted of 1263 patients (mean [SD] age, 82.3 [7.2] years; 595 [47.1%] female; 137 [10.8%] rural), of whom 186 (14.7%) required permanent pacemaker insertion during the index hospitalization after transcatheter aortic valve replacement. Mean follow-up was 990 days. After propensity score weighting, over the entire follow-up period, pacemaker implantation was associated with significantly higher all-cause mortality (43.9% vs 31.7%; hazard ratio [HR], 1.40; 95% CI, 1.01-1.94; P = .04), all-cause readmission (80.9% vs 70.6%; HR, 1.28; 95% CI, 1.15-1.43; P < .001), and emergency department visits (95.5% vs 87.3%; HR, 1.28; 95% CI, 1.08-1.52; P = .004). Pacemaker implantation was also associated with significantly greater readmission for heart failure (33.9% vs 19.1%; HR, 1.90; 95% CI, 1.53-2.36; P < .001). There were no statistically significant differences between groups in adjusted cumulative health care costs 1 year after discharge.

CONCLUSIONS AND RELEVANCE

New permanent pacemaker implantation after transcatheter aortic valve replacement was associated with significantly greater morbidity and mortality at long-term follow-up. However, this did not translate to a difference in cumulative health care costs after hospital discharge.

Assessing the performance of the generalized propensity score for estimating the effect of quantitative or continuous exposures on binary outcomes

Propensity score methods are increasingly being used to estimate the effects of treatments and exposures when using observational data. The propensity score was initially developed for use with binary exposures. The generalized propensity score (GPS) is an extension of the propensity score for use with quantitative or continuous exposures (eg, dose or quantity of medication, income, or years of education). We used Monte Carlo simulations to examine the performance of different methods of using the GPS to estimate the effect of continuous exposures on binary outcomes. We examined covariate adjustment using the GPS and weighting using weights based on the inverse of the GPS. We examined both the use of ordinary least squares to estimate the propensity function and the use of the covariate balancing propensity score algorithm. The use of methods based on the GPS was compared with the use of G‐computation. All methods resulted in essentially unbiased estimation of the population dose‐response function. However, GPS‐based weighting tended to result in estimates that displayed greater variability and had higher mean squared error when the magnitude of confounding was strong. Of the methods based on the GPS, covariate adjustment using the GPS tended to result in estimates with lower variability and mean squared error when the magnitude of confounding was strong. We illustrate the application of these methods by estimating the effect of average neighborhood income on the probability of death within 1 year of hospitalization for an acute myocardial infarction.

Risk of acute coronary syndrome in patients with cervical spondylosis

BACKGROUND AND AIMS

Cervical spondylosis (CS) is reported to be associated with increased sympathetic activity and hypertension. However, the cardiovascular (CV) outcomes of patients with CS are largely unknown.

METHODS

A national insurance claims dataset of 22 million enrollees in Taiwan during 1999-2010 was used as the research database. We identified 27,948 patients with CS and age-, sex-, and comorbidity-matched controls. By using multivariate logistic regression analysis after adjustment for potential cardiovascular (CV) confounders, we calculated odds ratios (ORs) with 95% confidence intervals (CIs) to quantify the association between CS and acute coronary syndrome (ACS).

RESULTS

A total of 744 ACS events were identified among the 27,948 patients with CS. The overall incidence of ACS was 4.27 per 1000 person-years in the CS cohort and 3.90 per 1000 person-years in the non-CS cohort, with an adjusted hazard ratio (aHR) of 1.13 (95% CI = 1.08-1.18). The aHRs of ACS were 1.08 (95% CI = 1.03-1.15) in the CS cohort without myelopathy and 1.20 (95% CI = 1.13-1.28) in the CS cohort with myelopathy, compared with the non-CS cohort. Compared with patients with CS without neurological signs, patients with CS receiving rehabilitation exhibited a 0.67 aHRs of ACS (95% CI = 0.59-0.76), whereas those with neurological signs receiving spinal decompression exhibited 0.73 aHRs of ACS (95% CI = 0.63-0.84).

CONCLUSIONS

CS is associated with an increased risk of ACS. Receiving treatment for CS, either rehabilitation or spinal decompression, is associated with less risk of ACS.