The Importance of Making Assumptions in Bias Analysis

Quantitative bias analyses allow researchers to adjust for uncontrolled confounding, given specification of certain bias parameters. When researchers are concerned about unknown confounders, plausible values for these bias parameters will be difficult to specify. Ding and VanderWeele developed bounding factor and E-value approaches that require the user to specify only some of the bias parameters. We describe the mathematical meaning of bounding factors and E-values and the plausibility of these methods in an applied context. We encourage researchers to pay particular attention to the assumption made, when using E-values, that the prevalence of the uncontrolled confounder among the exposed is 100% (or, equivalently, the prevalence of the exposure among those without the confounder is 0%). We contrast methods that attempt to bound biases or effects and alternative approaches such as quantitative bias analysis. We provide an example where failure to make this distinction led to erroneous statements. If the primary concern in an analysis is with known but unmeasured potential confounders, then E-values are not needed and may be misleading. In cases where the concern is with unknown confounders, the E-value assumption of an extreme possible prevalence of the confounder limits its practical utility.

Bias Analysis Gone Bad

Quantitative bias analysis comprises the tools used to estimate the direction, magnitude, and uncertainty from systematic errors affecting epidemiologic research. Despite the availability of methods and tools, and guidance for good practices, few reports of epidemiologic research incorporate quantitative estimates of bias impacts. The lack of familiarity with bias analysis allows for the possibility of misuse, which is likely most often unintentional but could occasionally include intentional efforts to mislead. We identified 3 examples of suboptimal bias analysis, one for each common bias. For each, we describe the original research and its bias analysis, compare the bias analysis with good practices, and describe how the bias analysis and research findings might have been improved. We assert no motive to the suboptimal bias analysis by the original authors. Common shortcomings in the examples were lack of a clear bias model, computed example, and computing code; poor selection of the values assigned to the bias model's parameters; and little effort to understand the range of uncertainty associated with the bias. Until bias analysis becomes more common, community expectations for the presentation, explanation, and interpretation of bias analyses will remain unstable. Attention to good practices should improve quality, avoid errors, and discourage manipulation.