Falsification Tests for Instrumental Variable Designs With an Application to Tendency to Operate

Instrumental variables in real-world clinical studies of dementia and neurodegenerative disease: Systematic review of the subject-matter argumentation, falsification test, and study design strategies to justify a valid instrument

OBJECTIVES

We systematically reviewed how investigators argued for and justified the validity of their instrumental variables (IV) in clinical studies of dementia and neurodegenerative disease.

METHODS

We included studies using IV analysis with observational data to investigate causal effects in clinical research studies of dementia and neurodegenerative disease. We reported the subject-matter argumentation, falsification test, and study design strategies used to satisfy the three assumptions of a valid IV: relevance, exclusion restriction, and exchangeability.

RESULTS

Justification for the relevance assumption was performed in all 12 included studies, exclusion restriction in seven studies, and exchangeability in nine studies. Two subject-matter argumentation strategies emerged from seven studies on the relevance of their IV. All studies except one provided quantitative evidence for the strength of the association between the IV and exposure variable. Four argumentation strategies emerged for exclusion restriction from six studies. Four falsification tests were performed across three studies. Three argumentation strategies emerged for exchangeability across four studies. Nine falsification tests were performed across nine studies. Two notable study design strategies were reported.

CONCLUSION

Our results reinforce IV analysis as a feasible option for clinical researchers in dementia and neurodegenerative disease by clarifying known strategies used to validate an IV.

A guide to regression discontinuity designs in medical applications

We present a practical guide for the analysis of regression discontinuity (RD) designs in biomedical contexts. We begin by introducing key concepts, assumptions, and estimands within both the continuity-based framework and the local randomization framework. We then discuss modern estimation and inference methods within both frameworks, including approaches for bandwidth or local neighborhood selection, optimal treatment effect point estimation, and robust bias-corrected inference methods for uncertainty quantification. We also overview empirical falsification tests that can be used to support key assumptions. Our discussion focuses on two particular features that are relevant in biomedical research: (i) fuzzy RD designs, which often arise when therapeutic treatments are based on clinical guidelines, but patients with scores near the cutoff are treated contrary to the assignment rule; and (ii) RD designs with discrete scores, which are ubiquitous in biomedical applications. We illustrate our discussion with three empirical applications: the effect CD4 guidelines for anti-retroviral therapy on retention of HIV patients in South Africa, the effect of genetic guidelines for chemotherapy on breast cancer recurrence in the United States, and the effects of age-based patient cost-sharing on healthcare utilization in Taiwan. Complete replication materials employing publicly available data and statistical software in Python, R and Stata are provided, offering researchers all necessary tools to conduct an RD analysis.

Conditional Effects of Race on Operative and Nonoperative Outcomes of Emergency General Surgery Conditions

INTRODUCTION

Many emergency general surgery (EGS) conditions can be managed both operatively or nonoperatively; however, it is unknown whether the decision to operate affects Black and White patients differentially.

METHODS

We identified a nationwide cohort of Black and White Medicare beneficiaries, hospitalized for common EGS conditions from July 2015 to June 2018. Using near-far matching to adjust for measurable confounding and an instrumental variable analysis to control for selection bias associated with treatment assignment, we compare outcomes of operative and nonoperative management in a stratified population of Black and White patients. Outcomes included in-hospital mortality, 30-day mortality, nonroutine discharge, and 30-day readmissions. An interaction test based on a t test was used to determine the conditional effects of operative versus nonoperative management between Black and White patients.

RESULTS

A total of 556,087 patients met inclusion criteria, of which 59,519 (10.7%) were Black and 496,568 (89.3%) were White. Overall, 165,932 (29.8%) patients had an operation and 390,155 (70.2%) were managed nonoperatively. Significant outcome differences were seen between operative and nonoperative management for some conditions; however, no significant differences were seen for the conditional effect of race on outcomes.

CONCLUSIONS

The decision to manage an EGS patient operatively versus nonoperatively has varying effects on surgical outcomes. These effects vary by EGS condition. There were no significant conditional effects of race on the outcomes of operative versus nonoperative management among universally insured older adults hospitalized with EGS conditions.

The COVID-19 pandemic and temporal change in metabolic risk factors for cardiovascular disease: A natural experiment within the HELIUS study

The coronavirus disease 2019 (COVID-19) pandemic, including the restrictive measures taken to reduce the spread of the virus, negatively affected people's health behavior. We explored whether the pandemic also had an effect on metabolic risk factors for cardiovascular disease (CVD) in women and men. We conducted a natural experiment, using data from 6962 participants without CVD at baseline (2011-2015) of six ethnic groups of the HELIUS study in Amsterdam, the Netherlands. We studied whether participants whose follow-up measurements were taken within the 11 months before the pandemic (control group) differed from those whose measurements were taken taken within 6 months after the first lockdown (exposed group). Using sex-stratified linear regressions with inverse probability weighting, we compared changes in baseline- and follow-up data between the control and exposed group in six metabolic risk factors: systolic and diastolic blood pressure (SBP, DBP), total cholesterol (TC), fasting plasma glucose (FPG), hemoglobin A1c (HbA1c), and estimated glomerular filtration rate (eGFR). Next, we explored the mediating effect of changes in body-mass index (BMI), alcohol, smoking, depressive symptoms and negative life events at follow-up. We observed less favorable changes in SBP (+1.12mmHg for women, +1.38mmHg for men), DBP (+0.85mmHg, +0.80mmHg) and FPG (only in women, +0.12 mmol/L) over time in the exposed group relative to the control group. Conversely, changes in HbA1c (-0.65 mmol/mol, -0.84 mmol/mol) and eGFR (+1.06 mL/min, +1.04 mL/min) were more favorable in the exposed compared to the control group, respectively. Changes in SBP, DBP, and FPG were partially mediated by changes in behavioral factors, in particular BMI and alcohol consumption. Concluding, the COVID-19 pandemic, in particular behavioral changes associated with restrictive lockdown measures, may have negatively affected several CVD risk factors, in both women and men.

Mendelian randomization

Mendelian randomization (MR) is a term that applies to the use of genetic variation to address causal questions about how modifiable exposures influence different outcomes. The principles of MR are based on Mendel's laws of inheritance and instrumental variable estimation methods, which enable the inference of causal effects in the presence of unobserved confounding. In this Primer, we outline the principles of MR, the instrumental variable conditions underlying MR estimation and some of the methods used for estimation. We go on to discuss how the assumptions underlying an MR study can be assessed and give methods of estimation that are robust to certain violations of these assumptions. We give examples of a range of studies in which MR has been applied, the limitations of current methods of analysis and the outlook for MR in the future. The difference between the assumptions required for MR analysis and other forms of non-interventional epidemiological studies means that MR can be used as part of a triangulation across multiple sources of evidence for causal inference.

The Influence of Increasing Levels of Provider-Patient Discussion on Quit Behavior: An Instrumental Variable Analysis of a National Survey

Objective: We aimed to examine the influence of increasing levels of discussion (both asked and advised, either asked or advised but not both, and neither asked nor advised) on quit behavior. Methods: We included 4133 adult current smokers from the 2015 National Health Interview Survey. The primary outcomes were quit intent and quit attempt, and the secondary outcomes were methods used for quitting. We used an instrumental variable analysis, as well as propensity score weighted and multivariable logistic regressions. Results: Compared to no discussion, having both or only one discussion, respectively, increased quit intent (OR = 1.65, 95% CI = 1.63–1.66 and OR = 1.02, 95% CI = 0.99–1.05), quit attempt (OR = 1.76, 95% CI = 1.75–1.77 and OR = 1.60, 95% CI = 1.57–1.63). Among those who attempted to quit (n = 1536), having both or only one discussion increased the use of pharmacologic (OR = 1.99, 95% CI = 1.97–2.02 and OR = 1.56, 95% CI = 1.49–1.63) or behavioral (OR = 2.01, 95% CI = 1.94–2.08 and OR = 2.91, 95% CI = 2.74–3.08) quit methods. Conclusions: Increasing levels of provider–patient discussion encourages quit behavior, and should be an integral part of reducing the health and economic burden of smoking. Strategies that promote the adherence and compliance of providers to communicate with patients may help increase the success of smoking cessation.

Evaluating a Key Instrumental Variable Assumption Using Randomization Tests

Instrumental variable (IV) analyses are becoming common in health services research and epidemiology. Most IV analyses use naturally occurring instruments, such as distance to a hospital. In these analyses, investigators must assume that the instrument is as-if randomly assigned. This assumption cannot be tested directly, but it can be falsified. Most IV falsification tests compare relative prevalence or bias in observed covariates between the instrument and exposure. These tests require investigators to make covariate-by-covariate judgments about the validity of the IV design. Often, only some covariates are well-balanced, making it unclear whether as-if randomization can be assumed for the instrument. We propose an alternative falsification test that compares IV balance or bias with the balance or bias that would have been produced under randomization. A key advantage of our test is that it allows for global balance measures as well as easily interpretable graphical comparisons. Furthermore, our test does not rely on parametric assumptions and can be used to validly assess whether the instrument is significantly closer to being as-if randomized than the exposure. We demonstrate our approach using data from (SPOT)light, a prospective cohort study carried out in 48 National Health Service hospitals in the United Kingdom between November 1, 2010, and December 31, 2011. This study used bed availability in the intensive care unit as an instrument for admission to the intensive care unit.

Does Transfer to Intensive Care Units Reduce Mortality? A Comparison of an Instrumental Variables Design to Risk Adjustment

BACKGROUND

Instrumental variable (IV) analysis can estimate treatment effects in the presence of residual or unmeasured confounding. In settings wherein measures of baseline risk severity are unavailable, IV designs are, therefore, particularly appealing, but, where established measures of risk severity are available, it is unclear whether IV methods are preferable.

OBJECTIVE

We compared regression with an IV design to estimate the effect of intensive care unit (ICU) transfer on mortality in a study with well-established measures of risk severity.

RESEARCH DESIGN

We use ICU bed availability at the time of assessment for ICU transfer as an instrument. Bed availability increases the chance of ICU admission, contains little information about patient characteristics, and it is unlikely that bed availability has any direct effect on in-hospital mortality.

SUBJECTS

We used a cohort study of deteriorating ward patients assessed for critical care unit admission, in 49 UK National Health Service hospitals between November 1, 2010, and December 31, 2011.

MEASURES

Detailed demographic, physiological, and comorbidity data were collected for all patients.

RESULTS

The risk adjustment methods reported that, after controlling for all measured covariates including measures of risk severity, ICU transfer was associated with higher 28-day mortality, with a risk difference of 7.2% (95% confidence interval=5.3%-9.1%). The IV estimate of ICU transfer was -5.4% (95% confidence interval=-47.1% to 36.3%) and applies to the subsample of patients whose transfer was "encouraged" by bed availability.

CONCLUSIONS

IV estimates indicate that ICU care is beneficial but are imprecisely estimated. Risk-adjusted estimates are more precise but, even with a rich set of covariates, report that ICU care is harmful.