Bias Analysis Gone Bad

Quantitative bias analysis comprises the tools used to estimate the direction, magnitude, and uncertainty from systematic errors affecting epidemiologic research. Despite the availability of methods and tools, and guidance for good practices, few reports of epidemiologic research incorporate quantitative estimates of bias impacts. The lack of familiarity with bias analysis allows for the possibility of misuse, which is likely most often unintentional but could occasionally include intentional efforts to mislead. We identified 3 examples of suboptimal bias analysis, one for each common bias. For each, we describe the original research and its bias analysis, compare the bias analysis with good practices, and describe how the bias analysis and research findings might have been improved. We assert no motive to the suboptimal bias analysis by the original authors. Common shortcomings in the examples were lack of a clear bias model, computed example, and computing code; poor selection of the values assigned to the bias model's parameters; and little effort to understand the range of uncertainty associated with the bias. Until bias analysis becomes more common, community expectations for the presentation, explanation, and interpretation of bias analyses will remain unstable. Attention to good practices should improve quality, avoid errors, and discourage manipulation.

Good practices for the design, analysis, and interpretation of observational studies on birth spacing and perinatal health outcomes

BACKGROUND

Meta-analyses of observational studies have shown that women with a shorter interpregnancy interval (the time from delivery to start of a subsequent pregnancy) are more likely to experience adverse pregnancy outcomes, such as preterm delivery or small for gestational age birth, than women who space their births further apart. However, the studies used to inform these estimates have methodological shortcomings.

METHODS

In this commentary, we summarise the discussions of an expert workgroup describing good practices for the design, analysis, and interpretation of observational studies of interpregnancy interval and adverse perinatal health outcomes.

RESULTS

We argue that inferences drawn from research in this field will be improved by careful attention to elements such as: (a) refining the research question to clarify whether the goal is to estimate a causal effect vs describe patterns of association; (b) using directed acyclic graphs to represent potential causal networks and guide the analytic plan of studies seeking to estimate causal effects; (c) assessing how miscarriages and pregnancy terminations may have influenced interpregnancy interval classifications; (d) specifying how key factors such as previous pregnancy loss, pregnancy intention, and maternal socio-economic position will be considered; and (e) examining if the association between interpregnancy interval and perinatal outcome differs by factors such as maternal age.

CONCLUSION

This commentary outlines the discussions of this recent expert workgroup, and describes several suggested principles for study design and analysis that could mitigate many potential sources of bias.