Bounding the per-protocol effect in randomized trials: an application to colorectal cancer screening

Instrumental variables methods reconcile intention-to-screen effects across pragmatic cancer screening trials

Pragmatic cancer screening trials mimic real-world scenarios in which patients and doctors are the ultimate arbiters of treatment. Intention-to-screen (ITS) analyses of such trials maintain randomization-based apples-to-apples comparisons, but differential adherence (the failure of subjects assigned to screening to get screened) makes ITS effects hard to compare across trials and sites. We show how instrumental variables (IV) methods address the nonadherence challenge in a comparison of estimates from 17 sites in five randomized trials measuring screening effects on colorectal cancer incidence. While adherence rates and ITS estimates vary widely across and within trials, IV estimates of per-protocol screening effects are remarkably consistent. An application of simple IV tools, including graphical analysis and formal statistical tests, shows how differential adherence explains variation in ITS impact. Screening compliers are also shown to have demographic characteristics similar to those of the full trial study sample. These findings argue for the clinical relevance of IV estimates of cancer screening effects.

Bounding the average causal effect in Mendelian randomisation studies with multiple proposed instruments: An application to prenatal alcohol exposure and attention deficit hyperactivity disorder

BACKGROUND

As large-scale observational data become more available, caution regarding causal assumptions remains critically important. This may be especially true for Mendelian randomisation (MR), an increasingly popular approach. Point estimation in MR usually requires strong, often implausible homogeneity assumptions beyond the core instrumental conditions. Bounding, which does not require homogeneity assumptions, is infrequently applied in MR.

OBJECTIVES

We aimed to demonstrate computing nonparametric bounds for the causal risk difference derived from multiple proposed instruments in an MR study where effect heterogeneity is expected.

METHODS

Using data from the Norwegian Mother, Father and Child Cohort Study (n = 2056) and Avon Longitudinal Study of Parents and Children (n = 6216) to study the average causal effect of maternal pregnancy alcohol use on offspring attention deficit hyperactivity disorder symptoms, we proposed 11 maternal SNPs as instruments. We computed bounds assuming subsets of SNPs were jointly valid instruments, for all combinations of SNPs where the MR model was not falsified.

RESULTS

The MR assumptions were violated for all sets with more than 4 SNPs in one cohort and for all sets with more than 2 SNPs in the other. Bounds assuming one SNP was an individually valid instrument barely improved on assumption-free bounds. Bounds tightened as more SNPs were assumed to be jointly valid instruments, and occasionally identified directions of effect, though bounds from different sets varied.

CONCLUSIONS

Our results suggest that, when proposing multiple instruments, bounds can contextualise plausible magnitudes and directions of effects. Computing bounds over multiple assumption sets, particularly in large, high-dimensional data, offers a means of triangulating results across different potential sources of bias within a study and may help researchers to better evaluate and emphasise which estimates are compatible with the most plausible assumptions for their specific setting.

Partial Identification of the Average Causal Effect in Multiple Study Populations: The Challenge of Combining Mendelian Randomization Studies

BACKGROUND

Researchers often use random-effects or fixed-effects meta-analysis to combine findings from multiple study populations. However, the causal interpretation of these models is not always clear, and they do not easily translate to settings where bounds, rather than point estimates, are computed.

METHODS

If bounds on an average causal effect of interest in a well-defined population are computed in multiple study populations under specified identifiability assumptions, then under those assumptions the average causal effect would lie within all study-specific bounds and thus the intersection of the study-specific bounds. We demonstrate this by pooling bounds on the average causal effect of prenatal alcohol exposure on attention deficit-hyperactivity disorder symptoms, computed in two European cohorts and under multiple sets of assumptions in Mendelian randomization (MR) analyses.

RESULTS

For all assumption sets considered, pooled bounds were wide and did not identify the direction of effect. The narrowest pooled bound computed implied the risk difference was between -4 and 34 percentage points.

CONCLUSIONS

All pooled bounds computed in our application covered the null, illustrating how strongly point estimates from prior MR studies of this effect rely on within-study homogeneity assumptions. We discuss how the interpretation of both pooled bounds and point estimation in MR is complicated by possible heterogeneity of effects across populations.

Invited Commentary: Conducting and Emulating Trials to Study Effects of Social Interventions

All else being equal, if we had 1 causal effect we wished to estimate, we would conduct a randomized trial with a protocol that mapped onto that causal question, or we would attempt to emulate that target trial with observational data. However, studying the social determinants of health often means there are not just 1 but several causal contrasts of simultaneous interest and importance, and each of these related but distinct causal questions may have varying degrees of feasibility in conducting trials. With this in mind, we discuss challenges and opportunities that arise when conducting and emulating such trials. We describe designing trials with the simultaneous goals of estimating the intention-to-treat effect, the per-protocol effect, effects of alternative protocols or joint interventions, effects within subgroups, and effects under interference, and we describe ways to make the most of all feasible randomized trials and emulated trials using observational data. Our comments are grounded in the study results of Courtin et al. (Am J Epidemiol. 2022;191(8):1444-1452).

Iron, folic acid, and multiple micronutrient supplementation strategies during pregnancy and adverse birth outcomes in Botswana

Background

Methods

Findings

Interpretation

Funding

Design of a pragmatic cluster-randomized trial comparing telehealth care and best practice clinic-based care for uncontrolled high blood pressure

BACKGROUND

Uncontrolled hypertension is the largest single contributor to all-cause and cardiovascular mortality in the U.S.

POPULATION

Nurse- and pharmacist-led team-based care and telehealth care interventions have been shown to result in large and lasting improvements in blood pressure (BP); however, it is unclear how successfully these can be implemented at scale in real-world settings. It is also uncertain how telehealth interventions impact patient experience compared to traditional clinic-based care.

AIMS/OBJECTIVES

To compare the effects of two evidence-based blood pressure care strategies in the primary care setting: (1) best-practice clinic-based care and (2) telehealth care with home BP telemonitoring and management by a clinical pharmacist. To evaluate implementation using mixed-methods supported by the RE-AIM framework and Consolidated Framework for Implementation Research.

METHODS

The design is a cluster-randomized comparative effectiveness pragmatic trial in 21 primary care clinics (9 clinic-based care, 12 telehealth care). Adult patients (age 18-85) with hypertension are enrolled via automated electronic health record (EHR) tools during primary care encounters if BP is elevated to ≥150/95 mmHg at two consecutive visits. The primary outcome is change in systolic BP over 12 months as extracted from the EHR. Secondary outcomes are change in key patient-reported outcomes over 6 months as measured by surveys. Qualitative data are collected at various time points to investigate implementation barriers and help explain intervention effects.

CONCLUSION

This pragmatic trial aims to inform health systems about the benefits, strengths, and limitations of implementing home BP telemonitoring with pharmacist management for uncontrolled hypertension in real-world primary care settings.

Using Bounds to Compare the Strength of Exchangeability Assumptions for Internal and External Validity

In the absence of strong assumptions (e.g., exchangeability), only bounds for causal effects can be identified. Here we describe bounds for the risk difference for an effect of a binary exposure on a binary outcome in 4 common study settings: observational studies and randomized studies, each with and without simple random selection from the target population. Through these scenarios, we introduce randomizations for selection and treatment, and the widths of the bounds are narrowed from 2 (the width of the range of the risk difference) to 0 (point identification). We then assess the strength of the assumptions of exchangeability for internal and external validity by comparing their contributions to the widths of the bounds in the setting of an observational study without random selection from the target population. We find that when less than two-thirds of the target population is selected into the study, the assumption of exchangeability for external validity of the risk difference is stronger than that for internal validity. The relative strength of these assumptions should be considered when designing, analyzing, and interpreting observational studies and will aid in determining the best methods for estimating the causal effects of interest.

The challenging interpretation of instrumental variable estimates under monotonicity

Background

Instrumental variable (IV) methods are often used to identify 'local' causal effects in a subgroup of the population of interest. Such 'local' effects may not be ideal for informing clinical or policy decision making. When the instrument is non-causal, additional difficulties arise for interpreting 'local' effects. Little attention has been paid to these difficulties, even though commonly proposed instruments in epidemiology are non-causal (e.g. proxies for physician's preference; genetic variants in some Mendelian randomization studies).

Methods

For IV estimates obtained from both causal and non-causal instruments under monotonicity, we present results to help investigators pose four questions about the local effect estimates obtained in their studies. (1) To what subgroup of the population does the effect pertain? Can we (2) estimate the size of or (3) describe the characteristics of this subgroup relative to the study population? (4) Can the sensitivity of the effect estimate to deviations from monotonicity be quantified?

Results

We show that the common interpretations and approaches for answering these four questions are generally only appropriate in the case of causal instruments.

Conclusions

Appropriate interpretation of an IV estimate under monotonicity as a 'local' effect critically depends on whether the proposed instrument is causal or non-causal. The results and formal proofs presented here can help in the transparent reporting of IV results and in enhancing the use of IV estimates in informing decision-making efforts.

Nonparametric Bounds for the Risk Function

Nonparametric bounds for the risk difference are straightforward to calculate and make no untestable assumptions about unmeasured confounding or selection bias due to missing data (e.g., dropout). These bounds are often wide and communicate uncertainty due to possible systemic errors. An illustrative example is provided.

Partial Identification of the Average Treatment Effect Using Instrumental Variables: Review of Methods for Binary Instruments, Treatments, and Outcomes

Several methods have been proposed for partially or point identifying the average treatment effect (ATE) using instrumental variable (IV) type assumptions. The descriptions of these methods are widespread across the statistical, economic, epidemiologic, and computer science literature, and the connections between the methods have not been readily apparent. In the setting of a binary instrument, treatment, and outcome, we review proposed methods for partial and point identification of the ATE under IV assumptions, express the identification results in a common notation and terminology, and propose a taxonomy that is based on sets of identifying assumptions. We further demonstrate and provide software for the application of these methods to estimate bounds. Supplementary materials for this article are available online.

Understanding the Assumptions Underlying Instrumental Variable Analyses: a Brief Review of Falsification Strategies and Related Tools

PURPOSE OF REVIEW

Instrumental variable (IV) methods continue to be applied to questions ranging from genetic to social epidemiology. In the epidemiologic literature, discussion of whether the assumptions underlying IV analyses hold is often limited to only certain assumptions and even then, arguments are mostly made using subject matter knowledge. To complement subject matter knowledge, there exist a variety of falsification strategies and other tools for weighing the plausibility of the assumptions underlying IV analyses.

RECENT FINDINGS

There are many tools that can refute the IV assumptions or help estimate the magnitude or direction of possible bias if the conditions do not hold perfectly. Many of these tools, including both recently developed strategies and strategies described decades ago, are underused or only used in specific applications of IV methods in epidemiology.

SUMMARY

Although estimating causal effects with IV analyses relies on unverifiable assumptions, the assumptions can sometimes be refuted. We suggest that the epidemiologists using IV analyses employ all the falsification strategies that apply to their research question in order to avoid settings that demonstrably violate a core condition for valid inference.

Nature as a Trialist?: Deconstructing the Analogy Between Mendelian Randomization and Randomized Trials

Mendelian randomization (MR) studies are often described as naturally occurring randomized trials in which genetic factors are randomly assigned by nature. Conceptualizing MR studies as randomized trials has profound implications for their design, conduct, reporting, and interpretation. For example, analytic practices that are discouraged in randomized trials should also be discouraged in MR studies. Here, we deconstruct the oft-made analogy between MR and randomized trials. We describe four key threats to the analogy between MR studies and randomized trials: (1) exchangeability is not guaranteed; (2) time zero (and therefore the time for setting eligibility criteria) is unclear; (3) the treatment assignment is often measured with error; and (4) adherence is poorly defined. By precisely defining the causal effects being estimated, we underscore that MR estimates are often vaguely analogous to per-protocol effects in randomized trials, and that current MR methods for estimating analogues of per-protocol effects could be biased in practice. We conclude that the analogy between randomized trials and MR studies provides further perspective on both the strengths and the limitations of MR studies as currently implemented, as well as future directions for MR methodology development and application. In particular, the analogy highlights potential future directions for some MR studies to produce more interpretable and informative numerical estimates.

Causal null hypotheses of sustained treatment strategies: What can be tested with an instrumental variable?

Sometimes instrumental variable methods are used to test whether a causal effect is null rather than to estimate the magnitude of a causal effect. However, when instrumental variable methods are applied to time-varying exposures, as in many Mendelian randomization studies, it is unclear what causal null hypothesis is tested. Here, we consider different versions of causal null hypotheses for time-varying exposures, show that the instrumental variable conditions alone are insufficient to test some of them, and describe additional assumptions that can be made to test a wider range of causal null hypotheses, including both sharp and average causal null hypotheses. Implications for interpretation and reporting of instrumental variable results are discussed.

An introduction to instrumental variable assumptions, validation and estimation

The instrumental variable method has been employed within economics to infer causality in the presence of unmeasured confounding. Emphasising the parallels to randomisation may increase understanding of the underlying assumptions within epidemiology. An instrument is a variable that predicts exposure, but conditional on exposure shows no independent association with the outcome. The random assignment in trials is an example of what would be expected to be an ideal instrument, but instruments can also be found in observational settings with a naturally varying phenomenon e.g. geographical variation, physical distance to facility or physician’s preference. The fourth identifying assumption has received less attention, but is essential for the generalisability of estimated effects. The instrument identifies the group of compliers in which exposure is pseudo-randomly assigned leading to exchangeability with regard to unmeasured confounders. Underlying assumptions can only partially be tested empirically and require subject-matter knowledge. Future studies employing instruments should carefully seek to validate all four assumptions, possibly drawing on parallels to randomisation.

Instrumental variable methods for a binary outcome were used to informatively address noncompliance in a randomized trial in surgery

Objectives

Randomization can be used as an instrumental variable (IV) to account for unmeasured confounding when seeking to assess the impact of noncompliance with treatment allocation in a randomized trial. We present and compare different methods to calculate the treatment effect on a binary outcome as a rate ratio in a randomized surgical trial.

Study Design and Setting

The effectiveness of peeling versus not peeling the internal limiting membrane of the retina as part of the surgery for a full thickness macular hole. We compared the IV-based estimates (nonparametric causal bound and two-stage residual inclusion approach [2SRI]) with standard treatment effect measures (intention to treat, per protocol and treatment received [TR]). Compliance was defined in two ways (initial and up to the time point of interest). Poisson regression was used for the model-based approaches with robust standard errors to calculate the risk ratio (RR) with 95% confidence intervals.

Results

Results were similar for 1-month macular hole status across methods. For 3- and 6-month macular hole status, nonparametric causal bounds provided a narrower range of uncertainty than other methods, though still had substantial imprecision. For 3-month macular hole status, the TR estimate was substantially different from the other point estimates.

Conclusion

Nonparametric causal bound approaches are a useful addition to an IV estimation approach, which tend to have large levels of uncertainty. Methods which allow RRs to be calculated when addressing noncompliance in randomized trials exist and may be superior to standard estimates. Further research is needed to explore the properties of different IV methods in a broad range of randomized controlled trial scenarios.

Causal inference as an emerging statistical approach in neurology: an example for epilepsy in the elderly

The elderly population faces an increasing number of cases of chronic neurological conditions, such as epilepsy and Alzheimer's disease. Because the elderly with epilepsy are commonly excluded from randomized controlled clinical trials, there are few rigorous studies to guide clinical practice. When the elderly are eligible for trials, they either rarely participate or frequently have poor adherence to therapy, thus limiting both generalizability and validity. In contrast, large observational data sets are increasingly available, but are susceptible to bias when using common analytic approaches. Recent developments in causal inference-analytic approaches also introduce the possibility of emulating randomized controlled trials to yield valid estimates. We provide a practical example of the application of the principles of causal inference to a large observational data set of patients with epilepsy. This review also provides a framework for comparative-effectiveness research in chronic neurological conditions.

Using instrumental variables to disentangle treatment and placebo effects in blinded and unblinded randomized clinical trials influenced by unmeasured confounders

Clinical trials traditionally employ blinding as a design mechanism to reduce the influence of placebo effects. In practice, however, it can be difficult or impossible to blind study participants and unblinded trials are common in medical research. Here we show how instrumental variables can be used to quantify and disentangle treatment and placebo effects in randomized clinical trials comparing control and active treatments in the presence of confounders. The key idea is to use randomization to separately manipulate treatment assignment and psychological encouragement conversations/interactions that increase the participants' desire for improved symptoms. The proposed approach is able to improve the estimation of treatment effects in blinded studies and, most importantly, opens the doors to account for placebo effects in unblinded trials.