Sample size importantly limits the usefulness of instrumental variable methods, depending on instrument strength and level of confounding

Assessing the performance of physician's prescribing preference as an instrumental variable in comparative effectiveness research with moderate and small sample sizes: a simulation study

Aim: This simulation study is to assess the utility of physician's prescribing preference (PPP) as an instrumental variable for moderate and smaller sample sizes. Materials & methods: We designed a simulation study to imitate a comparative effectiveness research under different sample sizes. We compare the performance of instrumental variable (IV) and non-IV approaches using two-stage least squares (2SLS) and ordinary least squares (OLS) methods, respectively. Further, we test the performance of different forms of proxies for PPP as an IV. Results: The percent bias of 2SLS is around approximately 20%, while the percent bias of OLS is close to 60%. The sample size is not associated with the level of bias for the PPP IV approach. Conclusion: Irrespective of sample size, the PPP IV approach leads to less biased estimates of treatment effectiveness than OLS adjusting for known confounding only. Particularly for smaller sample sizes, we recommend constructing PPP from long prescribing histories to improve statistical power.

Application of causal inference methods in individual-participant data meta-analyses in medicine: addressing data handling and reporting gaps with new proposed reporting guidelines

Observational data provide invaluable real-world information in medicine, but certain methodological considerations are required to derive causal estimates. In this systematic review, we evaluated the methodology and reporting quality of individual-level patient data meta-analyses (IPD-MAs) conducted with non-randomized exposures, published in 2009, 2014, and 2019 that sought to estimate a causal relationship in medicine. We screened over 16,000 titles and abstracts, reviewed 45 full-text articles out of the 167 deemed potentially eligible, and included 29 into the analysis. Unfortunately, we found that causal methodologies were rarely implemented, and reporting was generally poor across studies. Specifically, only three of the 29 articles used quasi-experimental methods, and no study used G-methods to adjust for time-varying confounding. To address these issues, we propose stronger collaborations between physicians and methodologists to ensure that causal methodologies are properly implemented in IPD-MAs. In addition, we put forward a suggested checklist of reporting guidelines for IPD-MAs that utilize causal methods. This checklist could improve reporting thereby potentially enhancing the quality and trustworthiness of IPD-MAs, which can be considered one of the most valuable sources of evidence for health policy.

Assessing the clinical and cost-effectiveness of inpatient mental health rehabilitation services provided by the NHS and independent sector (ACER): protocol

BACKGROUND

Mental health rehabilitation services provide specialist treatment to people with particularly severe and complex problems. In 2018, the Care Quality Commission reported that over half the 4,400 mental health inpatient rehabilitation beds in England were provided by the independent sector. They raised concerns that the length of stay and cost of independent sector care was double that of the NHS and that their services tended to be provided much further from people's homes. However, there has been no research comparing the two sectors and we therefore do not know if these concerns are justified. The ACER Study (Assessing the Clinical and cost-Effectiveness of inpatient mental health Rehabilitation services provided by the NHS and independent sector) is a national programme of research in England, funded from 2021 to 2026, that aims to investigate differences in inpatient mental health rehabilitation provided by the NHS and independent sector in terms of: patient characteristics; service quality; patient, carer and staff experiences; clinical and cost effectiveness.

METHODS

ACER comprises a:1) detailed survey of NHS and independent sector inpatient mental health rehabilitation services across England; 2) qualitative investigation of patient, family, staff and commissioners' experiences of the two sectors; 3) cohort study comparing clinical outcomes in the two sectors over 18 months; 4) comprehensive national comparison of inpatient service use in the two sectors, using instrumental variable analysis of routinely collected healthcare data over 18 months; 5) health economic evaluation of the relative cost-effectiveness of the two sectors. In Components 3 and 4, our primary outcome is 'successful rehabilitation' defined as a) being discharged from the inpatient rehabilitation unit without readmission and b) inpatient service use over the 18 months.

DISCUSSION

The ACER study will deliver the first empirical comparison of the clinical and cost-effectiveness of NHS and independent sector inpatient mental health rehabilitation services.

TRIAL REGISTRATION

ISRCTN17381762 retrospectively registered.

Effects of iloprost in patients with critical limb ischemia: Results of a cohort study from the COPART registry

BACKGROUND

Iloprost has been proposed as an alternative to amputation in Critical Limb Ischemia (CLI) patients when revascularization was unsuccessful or not possible. Nonetheless, there is limited evidence of its benefit. The main objective was to evaluate the effectiveness of iloprost and the secondary objective was to evaluate its safety.

METHODS

In this cohort study including CLI patients from the COPART registry from 2006/10 to 2021/01, patients exposed to iloprost were matched with up to three unexposed patients according to age, sex, and Propensity Score (PS) for exposure to iloprost. The main outcome combined the occurrence of all-cause death and major amputations; survival was assessed over one-year using Kaplan-Meier estimates and Cox model analyses. Major Adverse Cardiovascular Events (MACE) were chosen as the safety outcome; the association with iloprost was estimated using a logistic regression model.

RESULTS

Among 1850 CLI patients, 201 were exposed to iloprost (71.6% men; median age: 72 years vs. 72.1%; 75 years for unexposed). In 134 exposed patients matched to 375 unexposed patients, 14 major amputations and 24 deaths occurred in exposed patients (28.4%) vs. 33 and 46 respectively in the unexposed patients (20.9%). The hazard ratio (HR) was of 1.49 (95% Confidence Interval: 1.01-2.20). The association remained in the subgroup of "no option" patients (HR: 1.74; [1.01-2.20]). Regarding safety, 21/201 (10.7%) exposed patients experienced MACE vs. 146/1649 (9.41%) unexposed patients (unadjusted Odds Ratio [OR]: 1.17 [0.72-1.90]; adjusted OR: 1.23 [0.72-2.11]).

CONCLUSION

The study did not find any benefit of iloprost in CLI patients and even suggested a deleterious effect.

Application of Causal Inference Methods to Pooled Longitudinal Non- Randomized Studies: A Methodological Systematic Review

Observational data provide invaluable real-world information in medicine, but certain methodological considerations are required to derive causal estimates. In this systematic review, we evaluated the methodology and reporting quality of individual-level patient data meta-analyses (IPD-MAs) published in 2009, 2014, and 2019 that sought to estimate a causal relationship in medicine. We screened over 16,000 titles and abstracts, reviewed 45 full-text articles out of the 167 deemed potentially eligible, and included 29 into the analysis. Unfortunately, we found that causal methodologies were rarely implemented, and reporting was generally poor across studies. Specifically, only three of the 29 articles used quasi-experimental methods, and no study used G-methods to adjust for time-varying confounding. To address these issues, we propose stronger collaborations between physicians and methodologists to ensure that causal methodologies are properly implemented in IPD-MAs. In addition, we put forward a suggested checklist of reporting guidelines for IPD-MAs that utilize causal methods. This checklist could improve reporting thereby potentially enhancing the quality and trustworthiness of IPD-MAs, which can be considered one of the most valuable sources of evidence for health policy.

The challenge of dementia prevention trials and the role of quasi-experimental studies

Observational studies have shown consistently that modifiable risk factors during life are associated with increased dementia risk in old age but randomized controlled trials (RCTs) on dementia prevention evaluating the treatment of these risk factors did not find consistent effects on cognitive outcomes. The discrepancy in findings is potentially attributable to inherent differences between the two study designs. Although RCTs are the gold standard for establishing causality, designing and conducting an RCT for dementia prevention is complex. Quasi-experimental studies (QESs) may contribute to investigating causality without randomization. QESs use variation in exposure to a risk factor or intervention in an observational setting to deduct causal effects. Design-specific approaches are used to control for confounding, the main caveat of QESs. In this article we address the challenges, opportunities, and limitations of QESs for research into dementia prevention. HIGHLIGHTS: Despite consistent associations between modifiable risk factors and dementia, the mostly neutral effects of randomized controlled trials (RCTs) challenge the causality of these associations. RCTs in the field of dementia prevention are often problematic due to ethical, practical, or financial constraints, and their results may have limited generalizability. Four quasi-experimental study (QES) designs may be suitable to study causality between risk factors and dementia; we critically appraise these study designs for dementia-prevention studies. We describe how specific QES designs can be used to study the effects of risk-factor modification for 12 known risk factors for dementia.

Dealing with confounding in observational studies: A scoping review of methods evaluated in simulation studies with single-point exposure

The aim of this article was to perform a scoping review of methods available for dealing with confounding when analyzing the effect of health care treatments with single-point exposure in observational data. We aim to provide an overview of methods and their performance assessed by simulation studies indexed in PubMed. We searched PubMed for simulation studies published until January 2021. Our search was restricted to studies evaluating binary treatments and binary and/or continuous outcomes. Information was extracted on the methods' assumptions, performance, and technical properties. Of 28,548 identified references, 127 studies were eligible for inclusion. Of them, 84 assessed 14 different methods (ie, groups of estimators that share assumptions and implementation) for dealing with measured confounding, and 43 assessed 10 different methods for dealing with unmeasured confounding. Results suggest that there are large differences in performance between methods and that the performance of a specific method is highly dependent on the estimator. Furthermore, the methods' assumptions regarding the specific data features also substantially influence the methods' performance. Finally, the methods result in different estimands (ie, target of inference), which can even vary within methods. In conclusion, when choosing a method to adjust for measured or unmeasured confounding it is important to choose the most appropriate estimand, while considering the population of interest, data structure, and whether the plausibility of the methods' required assumptions hold.

Effect of Treatment in a Specialized Pediatric Hemato-Oncology Setting on 5-Year Survival in Acute Lymphoblastic Leukemia: A Quasi-Experimental Study

Simple Summary

Adolescents and young adults (AYAs) with acute lymphoblastic leukemia (ALL) have a worse prognosis than children. In addition to differences in biology—such as higher incidence of unfavorable genetic alterations in the AYA population—this might be related to the fact that ALL patients under a certain age (often 18 years) are generally treated in special pediatric hemato-oncology settings, which is associated with improved survival, while patients above that age are treated in adult hemato-oncology care settings. Based on previous research, adult treatment settings have increasingly adopted pediatric-inspired protocols, which appear to have led to increased survival of adolescent ALL patients. This study aims to assess whether there remains an effect of treatment of ALL patients in a specialized pediatric hemato-oncology setting on 5-year survival. This study provides insight into the effects of such treatment for ALL patients, and may stimulate further research into causal relationships in other oncological conditions.

Abstract

Survival rates of adolescents and young adults (AYAs) with acute lymphoblastic leukemia (ALL) are inferior to those of pediatric ALL patients. In part, this may be caused by differences in treatment setting. Generally, children are treated in specialized pediatric hemato-oncology settings, whereas AYAs are treated in adult hemato-oncology settings. Since 2005, adult treatment protocols have included pediatric-inspired chemotherapy, which has been the standard of care for AYAs from 2008 onwards. This study aims to assess whether, despite protocols in both settings having become more similar, there remains an effect of treatment in specialized pediatric hemato-oncology settings on 5-year survival for ALL patients in the Netherlands. We used nationwide registry data (2004–2013) on 472 ALL patients aged between 10 and 30 years old. A fuzzy regression discontinuity design was applied to estimate the treatment effect using two-stage least squares regression with the treatment threshold at 17 years and 7 months of age, adjusting for sex, age at diagnosis, and immunophenotype. We found a risk difference of 0.419 (p = 0.092; 95% CI = −0.0686; 0.907), meaning a 41.9 percentage point greater probability of surviving five years after diagnosis for ALL patients treated in specialized pediatric hemato-oncology settings. Our results suggest that ALL patients around the threshold could benefit from increased collaboration between pediatric and adult hemato-oncology in terms of survival.

Instrumental variable analysis to estimate treatment effects: a simulation study showing potential benefits of conditioning on hospital

Background

Instrumental variable (IV) analysis holds the potential to estimate treatment effects from observational data. IV analysis potentially circumvents unmeasured confounding but makes a number of assumptions, such as that the IV shares no common cause with the outcome. When using treatment preference as an instrument, a common cause, such as a preference regarding related treatments, may exist. We aimed to explore the validity and precision of a variant of IV analysis where we additionally adjust for the provider: adjusted IV analysis.

Methods

A treatment effect on an ordinal outcome was simulated (beta − 0.5 in logistic regression) for 15.000 patients, based on a large data set (the IMPACT data, n = 8799) using different scenarios including measured and unmeasured confounders, and a common cause of IV and outcome. We compared estimated treatment effects with patient-level adjustment for confounders, IV with treatment preference as the instrument, and adjusted IV, with hospital added as a fixed effect in the regression models.

Results

The use of patient-level adjustment resulted in biased estimates for all the analyses that included unmeasured confounders, IV analysis was less confounded, but also less reliable. With correlation between treatment preference and hospital characteristics (a common cause) estimates were skewed for regular IV analysis, but not for adjusted IV analysis.

Conclusion

When using IV analysis for comparing hospitals, some limitations of regular IV analysis can be overcome by adjusting for a common cause.

Trial registration

We do not report the results of a health care intervention.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01598-6.

Estimation of treatment effects in observational stroke care data: comparison of statistical approaches

Introduction

Various statistical approaches can be used to deal with unmeasured confounding when estimating treatment effects in observational studies, each with its own pros and cons. This study aimed to compare treatment effects as estimated by different statistical approaches for two interventions in observational stroke care data.

Patients and methods

We used prospectively collected data from the MR CLEAN registry including all patients (n = 3279) with ischemic stroke who underwent endovascular treatment (EVT) from 2014 to 2017 in 17 Dutch hospitals. Treatment effects of two interventions – i.e., receiving an intravenous thrombolytic (IVT) and undergoing general anesthesia (GA) before EVT – on good functional outcome (modified Rankin Scale ≤2) were estimated. We used three statistical regression-based approaches that vary in assumptions regarding the source of unmeasured confounding: individual-level (two subtypes), ecological, and instrumental variable analyses. In the latter, the preference for using the interventions in each hospital was used as an instrument.

Results

Use of IVT (range 66–87%) and GA (range 0–93%) varied substantially between hospitals. For IVT, the individual-level (OR ~ 1.33) resulted in significant positive effect estimates whereas in instrumental variable analysis no significant treatment effect was found (OR 1.11; 95% CI 0.58–1.56). The ecological analysis indicated no statistically significant different likelihood (β = − 0.002%; P = 0.99) of good functional outcome at hospitals using IVT 1% more frequently. For GA, we found non-significant opposite directions of points estimates the treatment effect in the individual-level (ORs ~ 0.60) versus the instrumental variable approach (OR = 1.04). The ecological analysis also resulted in a non-significant negative association (0.03% lower probability).

Discussion and conclusion

Both magnitude and direction of the estimated treatment effects for both interventions depend strongly on the statistical approach and thus on the source of (unmeasured) confounding. These issues should be understood concerning the specific characteristics of data, before applying an approach and interpreting the results. Instrumental variable analysis might be considered when unobserved confounding and practice variation is expected in observational multicenter studies.

Revisiting multi-stage models for upstream technology adoption: Evidence from rapid generation advance in rice breeding

Adoption of new plant varieties has played a significant role in eradicating global hunger. Previous research has mainly focused on farmer adoption and impact of new crop varieties, although upstream adoption of technologies in plant breeding can generate substantial multiplier effects on downstream impacts. This study moves upstream in the innovation system to generate policy advice on adoption and transfer of accelerated rice breeding technologies. More specifically, we assess the determinants of global adoption of rapid generation advance (RGA) through a sample of 158 rice breeders operating in various research institutes worldwide. Moving upstream in the innovation system has important theoretical and empirical implications due to the smaller number of decision-making units in the adoption process and the increasing role of institutional and managerial factors that may overrule individual adoption motivations. We revisit multi-stage models and devise the most robust estimation method that can be used in this situation. To generate insights on the impact of individual versus institutional adopter characteristics on upstream technology adoption, we juxtapose the response curves of the determinants of RGA adoption in rice breeding among alternative adoption stages, levels of conditionality and model specifications. Our findings confirm the importance of institutional and managerial factors and suggest that adoption and transfer of breeding technologies require breeding institutes to provide an enabling environment in which breeders are encouraged to take risks and are given sufficient freedom to experiment with and implement new technologies.

Commentary: Using potential outcomes causal methods to assess whether reductions in PM2.5 result in decreased mortality

Causal inference regarding exposures to ambient fine particulate matter (PM2.5) and mortality estimated from observational studies is limited by confounding, among other factors. In light of a variety of causal inference frameworks and methods that have been developed over the past century to specifically quantify causal effects, three research teams were selected in 2016 to evaluate the causality of PM2.5-mortality association among Medicare beneficiaries, using their own selections of causal inference methods and study designs but the same data sources. With a particular focus on controlling for unmeasured confounding, two research teams adopted an instrumental variables approach under a quasi-experiment or natural experiment study design, whereas one team adopted a structural nested mean model under the traditional cohort study design. All three research teams reported results supporting an estimated counterfactual causal relationship between ambient PM2.5 and all-cause mortality, and their estimated causal relationships are largely of similar magnitudes to recent epidemiological studies based on regression analyses with omitted potential confounders. The causal methods used by all three research teams were built upon the potential outcomes framework. This framework has marked conceptual advantages over regression-based methods in addressing confounding and yielding unbiased estimates of average treatment effect in observational epidemiological studies. However, potential violations of the unverifiable assumptions underlying each causal method leave the results from all three studies subject to biases. We also note that the studies are not immune to some other common sources of bias, including exposure measurement errors, ecological study design, model uncertainty and specification errors, and irrelevant exposure windows, that can undermine the validity of causal inferences in observational studies. As a result, despite some apparent consistency of study results from the three research teams with the wider epidemiological literature on PM2.5-mortality statistical associations, caution seems warranted in drawing causal conclusions from the results. A possible way forward is to improve study design and reduce dependence of conclusions on untested assumptions by complementing potential outcomes methods with structural causal modeling and information-theoretic methods that emphasize empirically tested and validated relationships.

Preference-based instrumental variables in health research rely on important and underreported assumptions: a systematic review

OBJECTIVE

Preference-based instrumental variables (PP IV) designs can identify causal effects when patients receive treatment due to variation in providers' treatment preference. We offer a systematic review and methodological assessment of PP IV applications in health research.

STUDY DESIGN AND SETTING

We included studies that applied PP IV for evaluation of any treatment in any population in health research (PROSPERO: CRD42020165014). We searched within four databases (Medline, Web of Science, ScienceDirect, SpringerLink) and four journals (including full-text and title and abstract sources) between January 1, 1998, and March 5, 2020. We extracted data on areas of applications and methodology, including assumptions using Swanson and Hernan's (2013) guideline.

RESULTS

We included 185 of 1087 identified studies. The use of PP IV has increased, being predominantly used for treatment effects in cancer, cardiovascular disease, and mental health. The most common PP IV was treatment variation at the facility-level, followed by physician- and regional-level. Only 12 percent of applications report the four main assumptions for PP IV. Selection on treatment may be a potential issue in 46 percent of studies.

CONCLUSION

The assumptions of PP IV are not sufficiently reported in existing work. PP IV-studies should use reporting guidelines.

Mendelian randomization: estimation of inpatient hospital costs attributable to obesity

BACKGROUND

Mendelian Randomization is a type of instrumental variable (IV) analysis that uses inherited genetic variants as instruments to estimate causal effects attributable to genetic factors. This study aims to estimate the impact of obesity on annual inpatient healthcare costs in the UK using linked data from the UK Biobank and Hospital Episode Statistics (HES).

METHODS

UK Biobank data for 482,127 subjects was linked with HES inpatient admission records, and costs were assigned to episodes of care. A two-stage least squares (TSLS) IV model and a TSLS two-part cost model were compared to a naïve regression of inpatient healthcare costs on body mass index (BMI).

RESULTS

The naïve analysis of annual cost on continuous BMI predicted an annual cost of £21.61 [95% CI £20.33 - £22.89] greater cost per unit increase in BMI. The TSLS IV model predicted an annual cost of £14.36 [95% CI £0.31 - £28.42] greater cost per unit increase in BMI. Modelled with a binary obesity variable, the naïve analysis predicted that obese subjects incurred £205.53 [95% CI £191.45 - £219.60] greater costs than non-obese subjects. The TSLS model predicted a cost £201.58 [95% CI £4.32 - £398.84] greater for obese subjects compared to non-obese subjects.

CONCLUSIONS

The IV models provide evidence for a causal relationship between obesity and higher inpatient healthcare costs. Compared to the naïve models, the binary IV model found a slightly smaller marginal effect of obesity, and the continuous IV model found a slightly smaller marginal effect of a single unit increase in BMI.

The causal effect of resilience on risk for drug abuse: a Swedish national instrumental variable, co-relative and propensity-score analysis

BACKGROUND

We sought to quantify and investigate the causal nature of the association between resilience at age 18 and future drug abuse (DA).

METHOD

In a national sample of Swedish men (n = 1 392 800), followed for a mean of 30.3 years, resilience was assessed during military conscription and DA defined from medical, criminal and pharmacy registers. For causal inference, we utilized three methods: (i) instrumental variable analyses with the month of birth as the instrument; (ii) co-relative analyses using the general population, cousins, siblings and monozygotic twins; and (iii) propensity scoring on a subsample (n = 48 548) with strong resilience predictors. Cox proportional hazards models were utilized to examine survival time till DA diagnosis.

RESULTS

Low resilience was most robustly predicted from internalizing symptoms. Lower levels of standardized resilience strongly predicted the risk for DA (HR = 2.31, 95% CIs 2.28-2.33). In instrumental, co-relative, and propensity score analyses, the association between resilience and DA was estimated at HR = 3.06 (2.44-3.85), 1.34 (1.28-1.39), and 1.40 (1.28-1.53), respectively. Sensitivity analyses suggested that our instrument was weak and, despite our large sample, likely under-estimated confounding.

CONCLUSIONS

Low resilience strongly predicts DA risk. Three different causal analysis methods, with divergent assumptions, concurred in estimating that an appreciable proportion of this association was causal, probably around 40%, with the remainder arising from confounding variables many of which are likely familial. Consistent with prior interventions focused on substance use prevention, our results suggest that prevention programs that increase resilience in adolescence should meaningfully reduce the long-term risk for DA.

Introduction to clinical research based on modern epidemiology

Over the past 20 years, recent advances in science technologies have dramatically changed the styles of clinical research. Currently, it has become more popular to use recent modern epidemiological techniques, such as propensity score, instrumental variable, competing risks, marginal structural modeling, mixed effects modeling, bootstrapping, and missing data analyses, than before. These advanced techniques, also known as modern epidemiology, may be strong tools for performing good clinical research, especially in large-scale observational studies, along with relevant research questions, good databases, and the passion of researchers. However, to use these methods effectively, we need to understand the basic assumptions behind them. Here, I will briefly introduce the concepts of these techniques and their implementation. In addition, I would like to emphasize that various types of clinical studies, not only large database studies but also small studies on rare and intractable diseases, are equally important because clinicians always do their best to take care of many kinds of patients who suffer from various kidney diseases and this is our most important mission.

Deciphering the Biological Mechanisms Underlying the Genome-Wide Associations between Computerized Device Use and Psychiatric Disorders

Computerized device use (CDU) is societally ubiquitous but its effects on mental health are unknown. We performed genetic correlation, Mendelian randomization, and latent causal variable analyses to identify shared genetic mechanisms between psychiatric disorders (Psychiatric Genomics Consortium; 14,477 < N < 150,064) and CDU (UK Biobank; N = 361,194 individuals). Using linkage disequilibrium score regression, we detected strong genetic correlations between "weekly usage of mobile phone in last 3 months" (PhoneUse) vs. attention deficit hyperactivity disorder (ADHD; rg = 0.425, p = 4.59 × 10-29) and "plays computer games" (CompGaming) vs. schizophrenia (SCZ; rg = -0.271, p = 7.16 × 10-26). Focusing on these correlations, we used two sample MRs to detect the causal relationships between trait pairs by treating single nucleotide polymorphisms as non-modifiable risk factors underlying both phenotypes. Significant bidirectional associations were detected (PhoneUse→ADHD β = 0.132, p = 1.89 × 10-4 and ADHD→PhoneUse β = 0.084, p = 2.86 × 10-10; CompGaming→SCZ β = -0.02, p = 6.46 × 10-25 and CompGaming→SCZ β = -0.194, p = 0.005) and the latent causal variable analyses did not support a causal relationship independent of the genetic correlations between these traits. This suggests that molecular pathways contribute to the genetic overlap between these traits. Dopamine transport enrichment (Gene Ontology:0015872, pSCZvsCompGaming = 2.74 × 10-10) and DRD2 association (pSCZ = 7.94 × 10-8; pCompGaming = 3.98 × 10-25) were detected in SCZ and CompGaming and support their negative correlative relationship. FOXP2 was significantly associated with ADHD (p = 9.32 × 10-7) and PhoneUse (p = 9.00 × 10-11) with effect directions concordant with their positive genetic correlation. Our study demonstrates that epidemiological associations between psychiatric disorders and CDUs are due, in part, to the molecular mechanisms shared between them rather than a causal relationship. Our findings imply that biological mechanisms underlying CDU contribute to the psychiatric phenotype manifestation.

Assessing causal treatment effect estimation when using large observational datasets

Background

Recently, there has been a heightened interest in developing and evaluating different methods for analysing observational data. This has been driven by the increased availability of large data resources such as Electronic Health Record (EHR) data alongside known limitations and changing characteristics of randomised controlled trials (RCTs). A wide range of methods are available for analysing observational data. However, various, sometimes strict, and often unverifiable assumptions must be made in order for the resulting effect estimates to have a causal interpretation. In this paper we will compare some common approaches to estimating treatment effects from observational data in order to highlight the importance of considering, and justifying, the relevant assumptions prior to conducting an observational analysis.

Methods

A simulation study was conducted based upon a small cohort of patients with chronic obstructive pulmonary disease. Two-stage least squares instrumental variables, propensity score, and linear regression models were compared under a range of different scenarios including different strengths of instrumental variable and unmeasured confounding. The effects of violating the assumptions of the instrumental variables analysis were also assessed. Sample sizes of up to 200,000 patients were considered.

Results

Two-stage least squares instrumental variable methods can yield unbiased treatment effect estimates in the presence of unmeasured confounding provided the sample size is sufficiently large. Adjusting for measured covariates in the analysis reduces the variability in the two-stage least squares estimates. In the simulation study, propensity score methods produced very similar results to linear regression for all scenarios. A weak instrument or strong unmeasured confounding led to an increase in uncertainty in the two-stage least squares instrumental variable effect estimates. A violation of the instrumental variable assumptions led to bias in the two-stage least squares effect estimates. Indeed, these were sometimes even more biased than those from a naïve linear regression model.

Conclusions

Instrumental variable methods can perform better than naïve regression and propensity scores. However, the assumptions need to be carefully considered and justified prior to conducting an analysis or performance may be worse than if the problem of unmeasured confounding had been ignored altogether.

Survival after radiotherapy versus radical cystectomy for primary muscle-invasive bladder cancer: A Swedish nationwide population-based cohort study

Background

Studies of survival comparing radical cystectomy (RC) and radiotherapy for muscle‐invasive bladder cancer have provided inconsistent results and have methodological limitations. The aim of the study was to investigate risk of death after radiotherapy as compared to RC.

Methods

We selected patients with muscle‐invasive urothelial carcinoma without distant metastases, treated with radiotherapy or RC from 1997 to 2014 in the Bladder Cancer Data Base Sweden (BladderBaSe) and estimated absolute and relative risk of bladder cancer death and all‐cause death. In a group of patients, theoretically eligible for a trial comparing radiotherapy and RC, we calculated risk difference in an instrumental variable analysis. We have not investigated chemoradiotherapy as this treatment was not used in the study time period.

Results

The study included 3 309 patients, of those 17% were treated with radiotherapy and 83% with RC. Patients treated with radiotherapy were older, had more advanced comorbidity, and had a higher risk of death as compared to patients treated with RC (relative risks of 1.5‐1.6). In the “trial population,” all‐cause death risk difference was 6 per 100 patients lower after radiotherapy at 5 years of follow‐up, 95% confidence interval −41 to 29.

Conclusion(s)

Patient selection between the treatments make it difficult to evaluate results from conventionally adjusted and propensity‐score matched survival analysis. When taking into account unmeasured confounding by instrumental variable analysis, no differences in survival was found between the treatments for a selected group of patients. Further clinical studies are needed to characterize this group of patients, which can serve as a basis for future comparison studies for treatment recommendations.

Mortality and Morbidity After Hartmann's Procedure Versus Primary Anastomosis Without a Diverting Stoma for Colorectal Perforation: A Nationwide Observational Study

BACKGROUND

The benefit of primary anastomosis (PA) without a diverting stoma over Hartmann's procedure (HP) for colorectal perforation remains controversial. We compared postoperative mortality and morbidity between HP and PA without a diverting stoma for colorectal perforation of various etiologies.

METHODS

Using the Japanese Diagnosis Procedure Combination database, we extracted data on patients who underwent emergency open laparotomy for colorectal perforation of various etiologies from July 1, 2010 to March 31, 2014. We compared 30-day mortality, postoperative complication rates, and postoperative critical care interventions between HP and PA groups using propensity score matching, inverse probability of treatment weighting, and instrumental variable analyses to adjust for measured and unmeasured confounding factors.

RESULTS

We identified 8500 eligible patients (5455 HP and 3045 PA). In the propensity score-matched model, a significant difference between the HP and PA groups was detected in 30-day mortality (7.7% vs. 9.6%; risk difference, 1.9%; 95% confidence interval [CI], 0.5-3.4). The inverse probability of treatment weighting showed similar results (8.8% vs. 10.7%; risk difference, 1.9%; 95% CI, 1.0-2.8). In the instrumental variable analysis, the point estimate suggested similar direction to that of the propensity score analyses (risk difference, 4.4%; 95% CI, -3.3 to 12.1). The PA group had significantly higher rates of secondary surgery for complications (4.6% vs. 8.4%; risk difference, 3.8%; 95% CI, 2.5-4.1) and slightly longer duration of postoperative critical care interventions.

CONCLUSIONS

This study revealed a significant difference in 30-day mortality between HP and PA without a diverting stoma.

Instrumental variable methods for a binary outcome were used to informatively address noncompliance in a randomized trial in surgery

Objectives

Randomization can be used as an instrumental variable (IV) to account for unmeasured confounding when seeking to assess the impact of noncompliance with treatment allocation in a randomized trial. We present and compare different methods to calculate the treatment effect on a binary outcome as a rate ratio in a randomized surgical trial.

Study Design and Setting

The effectiveness of peeling versus not peeling the internal limiting membrane of the retina as part of the surgery for a full thickness macular hole. We compared the IV-based estimates (nonparametric causal bound and two-stage residual inclusion approach [2SRI]) with standard treatment effect measures (intention to treat, per protocol and treatment received [TR]). Compliance was defined in two ways (initial and up to the time point of interest). Poisson regression was used for the model-based approaches with robust standard errors to calculate the risk ratio (RR) with 95% confidence intervals.

Results

Results were similar for 1-month macular hole status across methods. For 3- and 6-month macular hole status, nonparametric causal bounds provided a narrower range of uncertainty than other methods, though still had substantial imprecision. For 3-month macular hole status, the TR estimate was substantially different from the other point estimates.

Conclusion

Nonparametric causal bound approaches are a useful addition to an IV estimation approach, which tend to have large levels of uncertainty. Methods which allow RRs to be calculated when addressing noncompliance in randomized trials exist and may be superior to standard estimates. Further research is needed to explore the properties of different IV methods in a broad range of randomized controlled trial scenarios.

The effect of health on socioeconomic status: Using instrumental variables to revisit a successful randomized controlled trial

Poor health is widely recognized as a consequence of social disadvantage, but health problems may also help transmit social disadvantage over time and generations. Experimentally assigned health exposures, namely those tested in randomized controlled trials, may provide opportunities to estimate the causal effects of health on socioeconomic status (SES). We revisit data from the Diabetes Control and Complications Trial, a randomized controlled trial of glucose control therapy in Type 1 diabetic patients, and use treatment assignment as an instrument for health status to test the causal effect of treatment-related health improvement on subsequent SES measured during the trial's follow-up study, the Epidemiology of Diabetes Interventions and Complications study. We used the Two-Factor Hollingshead Index of Social Position, which comprises education and occupation, to measure SES. Glycated hemoglobin (HbA1c) served as a proxy for health status. Ordinary least squares (OLS) regression models showed that lower HbA1c at the trial's end was associated with higher SES at both the start of the follow-up and 17 years later. However, instrumental variable analyses showed no causal effect of HbA1c on SES, suggesting that OLS estimates are biased by endogeneity. Sensitivity analyses showed robustness to several alternate specifications. While the HbA1c advantage conferred by random assignment to treatment within the trial did not produce higher Hollingshead Index scores, we note that occupation and education categories may be harder to affect than are outcomes such as income. This analysis offers evidence that clinical trial data may be a rich and unrecognized resource for estimating health effects on SES.

Obstetric Facility Quality and Newborn Mortality in Malawi: A Cross-Sectional Study

BACKGROUND

Ending preventable newborn deaths is a global health priority, but efforts to improve coverage of maternal and newborn care have not yielded expected gains in infant survival in many settings. One possible explanation is poor quality of clinical care. We assess facility quality and estimate the association of facility quality with neonatal mortality in Malawi.

METHODS AND FINDINGS

Data on facility infrastructure as well as processes of routine and basic emergency obstetric care for all facilities in the country were obtained from 2013 Malawi Service Provision Assessment. Birth location and mortality for children born in the preceding two years were obtained from the 2013-2014 Millennium Development Goals Endline Survey. Facilities were classified as higher quality if they ranked in the top 25% of delivery facilities based on an index of 25 predefined quality indicators. To address risk selection (sicker mothers choosing or being referred to higher-quality facilities), we employed instrumental variable (IV) analysis to estimate the association of facility quality of care with neonatal mortality. We used the difference between distance to the nearest facility and distance to a higher-quality delivery facility as the instrument. Four hundred sixty-seven of the 540 delivery facilities in Malawi, including 134 rated as higher quality, were linked to births in the population survey. The difference between higher- and lower-quality facilities was most pronounced in indicators of basic emergency obstetric care procedures. Higher-quality facilities were located a median distance of 3.3 km further from women than the nearest delivery facility and were more likely to be in urban areas. Among the 6,686 neonates analyzed, the overall neonatal mortality rate was 17 per 1,000 live births. Delivery in a higher-quality facility (top 25%) was associated with a 2.3 percentage point lower newborn mortality (95% confidence interval [CI] -0.046, 0.000, p-value 0.047). These results imply a newborn mortality rate of 28 per 1,000 births at low-quality facilities and of 5 per 1,000 births at the top 25% of facilities, accounting for maternal and newborn characteristics. This estimate applies to newborns whose mothers would switch from a lower-quality to a higher-quality facility if one were more accessible. Although we did not find an indication of unmeasured associations between the instrument and outcome, this remains a potential limitation of IV analysis.

CONCLUSIONS

Poor quality of delivery facilities is associated with higher risk of newborn mortality in Malawi. A shift in focus from increasing utilization of delivery facilities to improving their quality is needed if global targets for further reductions in newborn mortality are to be achieved.

Effectiveness of a multidisciplinary heart failure disease management programme on 1-year mortality: Prospective cohort study

We performed a multicenter prospective observational cohort study (Epidémiologie et Pronostic de l'Insuffisance Cardiaque Aiguë en Lorraine, Epidemiology and Prognosis of Acute Heart Failure in Lorraine [EPICAL2]) to evaluate the effectiveness on mortality of a community-based multidisciplinary disease management programme (DMP) for heart failure (HF) patients.Between October 2011 and October 2012, 1816 patients, who were hospitalized for acute HF or who developed acute HF during a hospitalization, were included from 21 hospitals in a northeast region of France. At hospital admission, their mean age was 77.3 (standard deviation [SD] 11.6) years and mean left ventricular ejection fraction was 45.0 (SD 16.0)%. A subset of patients were enrolled in a multidimensional DMP for HF (n = 312, 17.2%), based on structured patient education, home monitoring visits by HF-trained nurses, and automatic alerts triggered by significant clinical and biological changes to the patient. The DMP involved general practitioners, nurses, and cardiologists collaborating via an individual web-based medical electronic record. The outcome was all-cause mortality from the 3rd to the 12th month after discharge. During the follow-up, a total of 377 (20.8%) patients died: 321 (21.3%) in the control group and 56 (17.9%) in the DMP group. In a propensity score analysis, DMP was associated with lower 1-year all-cause mortality (hazard ratio 0.65, 95% CI 0.46-0.92). Instrumental variable analysis gave similar results (hazard ratio 0.56, 0.27-1.16).In a real world setting, a multidimensional DMP for HF with structured patient education, home nurse monitoring, and appropriate physician alerts may improve survival when implemented after discharge from hospitalization due to worsening HF.

The proportion of all previous patients was a potential instrument for patients' actual prescriptions of nonsteroidal anti-inflammatory drugs

OBJECTIVES

To investigate whether physician's prescribing preference is a valid instrumental variable (IV) for patients' actual prescription of selective cyclooxygenase-2 (COX-2) inhibitors in the German Pharmacoepidemiological Research Database (GePaRD).

STUDY DESIGN AND SETTING

We compared the effect of COX-2 inhibitors vs. traditional nonsteroidal anti-inflammatory drugs (tNSAIDs) on the risk of gastrointestinal complications using physician's preference as IV. We used different definitions of physician's preference for COX-2 inhibitors. A retrospective cohort of new users was built which was further restricted to subcohorts. We compared IV-based risk difference estimates, using a two-stage approach, to estimates from conventional multivariate models.

RESULTS

We observed only a small proportion of COX-inhibitor users (3.2%) in our study. All instruments, in the full cohort and in the subcohorts, reduced the imbalance in most of the covariates. However, the IV treatment effect estimates had a highly inflated variance. Compared to the most recent prescription, the proportion of previous patients was a stronger instrument and reduced the variance of the estimates.

CONCLUSION

The proportion of all previous patients is a potential IV for comparing COX-2 inhibitors vs. tNSAIDs in GePaRD. Our study demonstrates that valid instruments in one health care system may not be directly applicable to others.

Performing both propensity score and instrumental variable analyses in observational studies often leads to discrepant results: a systematic review

OBJECTIVES

Propensity score (PS) and instrumental variable (IV) are analytical techniques used to adjust for confounding in observational research. More and more, they seem to be used simultaneously in studies evaluating health interventions. The present review aimed to analyze the agreement between PS and IV results in medical research published to date.

STUDY DESIGN AND SETTING

Review of all published observational studies that evaluated a clinical intervention using simultaneously PS and IV analyses, as identified in MEDLINE and Web of Science.

RESULTS

Thirty-seven studies, most of them published during the previous 5 years, reported 55 comparisons between results from PS and IV analyses. There was a slight/fair agreement between the methods [Cohen's kappa coefficient = 0.21 (95% confidence interval: 0.00, 0.41)]. In 23 cases (42%), results were nonsignificant for one method and significant for the other, and IV analysis results were nonsignificant in most situations (87%).

CONCLUSION

Discrepancies are frequent between PS and IV analyses and can be interpreted in various ways. This suggests that researchers should carefully consider their analytical choices, and readers should be cautious when interpreting results, until further studies clarify the respective roles of the two methods in observational comparative effectiveness research.