Commentary

Target Validity and the Hierarchy of Study Designs

In recent years, increasing attention has been paid to problems of external validity, specifically to methodological approaches for both quantitative generalizability and transportability of study results. However, most approaches to these issues have considered external validity separately from internal validity. Here we argue that considering either internal or external validity in isolation may be problematic. Further, we argue that a joint measure of the validity of an effect estimate with respect to a specific population of interest may be more useful: We call this proposed measure target validity. In this work, we introduce and formally define target bias as the total difference between the true causal effect in the target population and the estimated causal effect in the study sample, and target validity as target bias = 0. We illustrate this measure with a series of examples and show how this measure may help us to think more clearly about comparisons between experimental and nonexperimental research results. Specifically, we show that even perfect internal validity does not ensure that a causal effect will be unbiased in a specific target population.

Generalizing Study Results: A Potential Outcomes Perspective

Great care is taken in epidemiologic studies to ensure the internal validity of causal effect estimates; however, external validity has received considerably less attention. When the study sample is not a random sample of the target population, the sample average treatment effect, even if internally valid, cannot usually be expected to equal the average treatment effect in the target population. The utility of an effect estimate for planning purposes and decision making will depend on the degree of departure from the true causal effect in the target population due to problems with both internal and external validity. Herein, we review concepts from recent literature on generalizability, one facet of external validity, using the potential outcomes framework. Identification conditions sufficient for external validity closely parallel identification conditions for internal validity, namely conditional exchangeability; positivity; the same distributions of the versions of treatment; no interference; and no measurement error. We also require correct model specification. Under these conditions, we discuss how a version of direct standardization (the g-formula, adjustment formula, or transport formula) or inverse probability weighting can be used to generalize a causal effect from a study sample to a well-defined target population, and demonstrate their application in an illustrative example.

Transportability of Trial Results Using Inverse Odds of Sampling Weights

Increasingly, the statistical and epidemiologic literature is focusing beyond issues of internal validity and turning its attention to questions of external validity. Here, we discuss some of the challenges of transporting a causal effect from a randomized trial to a specific target population. We present an inverse odds weighting approach that can easily operationalize transportability. We derive these weights in closed form and illustrate their use with a simple numerical example. We discuss how the conditions required for the identification of internally valid causal effects are translated to apply to the identification of externally valid causal effects. Estimating effects in target populations is an important goal, especially for policy or clinical decisions. Researchers and policy-makers should therefore consider use of statistical techniques such as inverse odds of sampling weights, which under careful assumptions can transport effect estimates from study samples to target populations.

Invited Commentary: Causal Inference Across Space and Time-Quixotic Quest, Worthy Goal, or Both?

The g-formula and agent-based models (ABMs) are 2 approaches used to estimate causal effects. In the current issue of the Journal, Murray et al. (Am J Epidemiol. 2017;186(2):131-142) compare the performance of the g-formula and ABMs to estimate causal effects in 3 target populations. In their thoughtful paper, the authors outline several reasons that a causal effect estimated using an ABM may be biased when parameterized from at least 1 source external to the target population. The authors have addressed an important issue in epidemiology: Often causal effect estimates are needed to inform public health decisions in settings without complete data. Because public health decisions are urgent, epidemiologists are frequently called upon to estimate a causal effect from existing data in a separate population rather than perform new data collection activities. The assumptions needed to transport causal effects to a specific target population must be carefully stated and assessed, just as one would explicitly state and analyze the assumptions required to draw internally valid causal inference in a specific study sample. Considering external validity in important target populations increases the impact of epidemiologic studies.

A typology of four notions of confounding in epidemiology

Confounding is a major concern in epidemiology. Despite its significance, the different notions of confounding have not been fully appreciated in the literature, leading to confusion of causal concepts in epidemiology. In this article, we aim to highlight the importance of differentiating between the subtly different notions of confounding from the perspective of counterfactual reasoning. By using a simple example, we illustrate the significance of considering the distribution of response types to distinguish causation from association, highlighting that confounding depends not only on the population chosen as the target of inference, but also on the notions of confounding in distribution and confounding in measure. This point has been relatively underappreciated, partly because some literature on the concept of confounding has only used the exposed and unexposed groups as the target populations, while it would be helpful to use the total population as the target population. Moreover, to clarify a further distinction between confounding “in expectation” and “realized” confounding, we illustrate the usefulness of examining the distribution of exposure status in the target population. To grasp the explicit distinction between confounding in expectation and realized confounding, we need to understand the mechanism that generates exposure events, not the product of that mechanism. Finally, we graphically illustrate this point, highlighting the usefulness of directed acyclic graphs in examining the presence of confounding in distribution, in the notion of confounding in expectation.

Invited commentary: every good randomization deserves observation

Preexposure prophylaxis (PrEP) is a promising approach to prevention of human immunodeficiency virus (HIV) transmission, in which an HIV-negative individual takes a single daily dose of an antiretroviral drug so that, if exposed to HIV, an active antiretroviral drug is already present in his or her system. In this issue of the Journal, Murnane et al. (Am J Epidemiol. 2015;182(10):848-856) use data from the Partners PrEP Study (Kenya and Uganda, 2008-2011) to estimate the efficacy of PrEP under perfect adherence. We discuss Murnane et al.'s work and then examine the larger issues of generalizability or transportability of findings from a randomized trial to a new setting when adherence to an intervention determines its effectiveness. We discuss sufficient conditions for generalizability and use causal directed acyclic graphs to show how these assumptions might play out when trials are used to make inferences about the effect of PrEP in current and future real-world target populations.

Risk

The epidemiologist primarily studies transitions between states of health and disease. The purpose of the present article is to define a foundational parameter for such studies, namely risk. We begin simply and build to the setting in which there is more than 1 event type (i.e., competing risks or competing events), as well as more than 1 treatment or exposure level of interest. In the presence of competing events, the risks are a set of counterfactual cumulative incidence functions for each treatment. These risks can be depicted visually and summarized numerically. We use an example from the study of human immunodeficiency virus to illustrate concepts.