1
|
Zivich PN, Edwards JK, Lofgren ET, Cole SR, Shook-Sa BE, Lessler J. Transportability Without Positivity: A Synthesis of Statistical and Simulation Modeling. Epidemiology 2024; 35:23-31. [PMID: 37757864 PMCID: PMC10841168 DOI: 10.1097/ede.0000000000001677] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/29/2023]
Abstract
Studies designed to estimate the effect of an action in a randomized or observational setting often do not represent a random sample of the desired target population. Instead, estimates from that study can be transported to the target population. However, transportability methods generally rely on a positivity assumption, such that all relevant covariate patterns in the target population are also observed in the study sample. Strict eligibility criteria, particularly in the context of randomized trials, may lead to violations of this assumption. Two common approaches to address positivity violations are restricting the target population and restricting the relevant covariate set. As neither of these restrictions is ideal, we instead propose a synthesis of statistical and simulation models to address positivity violations. We propose corresponding g-computation and inverse probability weighting estimators. The restriction and synthesis approaches to addressing positivity violations are contrasted with a simulation experiment and an illustrative example in the context of sexually transmitted infection testing uptake. In both cases, the proposed synthesis approach accurately addressed the original research question when paired with a thoughtfully selected simulation model. Neither of the restriction approaches was able to accurately address the motivating question. As public health decisions must often be made with imperfect target population information, model synthesis is a viable approach given a combination of empirical data and external information based on the best available knowledge.
Collapse
Affiliation(s)
- Paul N Zivich
- From the Institute of Global Health and Infectious Diseases, University of North Carolina at Chapel Hill, Chapel Hill, NC
- Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, NC
| | - Jessie K Edwards
- Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, NC
| | - Eric T Lofgren
- Paul G. Allen School for Global Health, Washington State University, Pullman, WA
| | - Stephen R Cole
- Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, NC
| | - Bonnie E Shook-Sa
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC
| | - Justin Lessler
- From the Institute of Global Health and Infectious Diseases, University of North Carolina at Chapel Hill, Chapel Hill, NC
- Carolina Population Center, University of North Carolina at Chapel Hill, Chapel Hill, NC
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD
| |
Collapse
|
2
|
Kühne F, Schomaker M, Stojkov I, Jahn B, Conrads-Frank A, Siebert S, Sroczynski G, Puntscher S, Schmid D, Schnell-Inderst P, Siebert U. Causal evidence in health decision making: methodological approaches of causal inference and health decision science. GERMAN MEDICAL SCIENCE : GMS E-JOURNAL 2022; 20:Doc12. [PMID: 36742460 PMCID: PMC9869404 DOI: 10.3205/000314] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Figures] [Subscribe] [Scholar Register] [Received: 12/10/2021] [Indexed: 02/07/2023]
Abstract
Objectives Public health decision making is a complex process based on thorough and comprehensive health technology assessments involving the comparison of different strategies, values and tradeoffs under uncertainty. This process must be based on best available evidence and plausible assumptions. Causal inference and health decision science are two methodological approaches providing information to help guide decision making in health care. Both approaches are quantitative methods that use statistical and modeling techniques and simplifying assumptions to mimic the complexity of the real world. We intend to review and lay out both disciplines with their aims, strengths and limitations based on a combination of textbook knowledge and expert experience. Methods To help understanding and differentiating the methodological approaches of causal inference and health decision science, we reviewed both methods with the focus on aims, research questions, methods, assumptions, limitations and challenges, and software. For each methodological approach, we established a group of four experts from our own working group to carefully review and summarize each method, followed by structured discussion rounds and written reviews, in which the experts from all disciplines including HTA and medicine were involved. The entire expert group discussed objectives, strengths and limitations of both methodological areas, and potential synergies. Finally, we derived recommendations for further research and provide a brief outlook on future trends. Results Causal inference methods aim for drawing causal conclusions from empirical data on the relationship of pre-specified interventions on a specific target outcome and apply a counterfactual framework and statistical techniques to derive causal effects of exposures or interventions from these data. Causal inference is based on a causal diagram, more specifically, a directed acyclic graph (DAG), which encodes the assumptions regarding the causal relations between variables. Depending on the type of confounding and selection bias, traditional statistical methods or more complex g-methods are needed to derive valid causal effects. Besides the correct specification of the DAG and the statistical model, assumptions such as consistency, positivity, and exchangeability must be checked when aiming at causal inference. Health decision science aims for guiding policy decision making regarding health interventions considering and balancing multiple competing objectives of a decision based on data from multiple sources and studies, for example prevalence studies, clinical trials and long-term observational routine effectiveness studies, and studies on preferences and costs. It involves decision analysis, a systematic, explicit and quantitative framework to guide decisions under uncertainty. Decision analyses are based on decision-analytic models to mimic the course of disease as well as aspects and consequences of the intervention in order to quantitatively optimize the decision. Depending on the type of decision problem, decision trees, state-transition models, discrete event simulation models, dynamic transmission models, or other model types are applied. Models must be validated against observed data, and comprehensive sensitivity analyses must be performed to assess uncertainty. Besides the appropriate choice of the model type and the valid specification of the model structure, it must be checked if input parameters of effects can be interpreted as causal parameters in the model. Otherwise results will be biased. Conclusions Both causal inference and health decision science aim for providing best causal evidence for informed health decision making. The strengths and limitations of both methods differ and a good understanding of both methods is essential for correct application but also for correct interpretation of findings from the described methods. Importantly, decision-analytic modeling should be combined with causal inference when developing guidance and recommendations regarding decisions on health care interventions.
Collapse
Affiliation(s)
- Felicitas Kühne
- Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
| | - Michael Schomaker
- Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
- Centre for Infectious Disease Epidemiology & Research, University of Cape Town, South Africa
| | - Igor Stojkov
- Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
| | - Beate Jahn
- Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
- Division of Health Technology Assessment, ONCOTYROL – Center for Personalized Cancer Medicine, Innsbruck, Austria
| | - Annette Conrads-Frank
- Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
| | - Silke Siebert
- Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
| | - Gaby Sroczynski
- Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
| | - Sibylle Puntscher
- Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
| | - Daniela Schmid
- Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
| | - Petra Schnell-Inderst
- Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
| | - Uwe Siebert
- Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
- Division of Health Technology Assessment, ONCOTYROL – Center for Personalized Cancer Medicine, Innsbruck, Austria
- Center for Health Decision Science, Departments of Epidemiology and Health Policy & Management, Harvard T.H. Chan School of Public Health, Boston, MA, USA
- Program on Cardiovascular Research, Institute for Technology Assessment and Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
3
|
Okoyo C, Onyango N, Orowe I, Mwandawiro C, Medley G. Sensitivity Analysis of a Transmission Interruption Model for the Soil-Transmitted Helminth Infections in Kenya. Front Public Health 2022; 10:841883. [PMID: 35400031 PMCID: PMC8990131 DOI: 10.3389/fpubh.2022.841883] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Accepted: 02/28/2022] [Indexed: 12/03/2022] Open
Abstract
As the world rallies toward the endgame of soil-transmitted helminths (STH) elimination by the year 2030, there is a need for efficient and robust mathematical models that would enable STH programme managers to target the scarce resources and interventions, increase treatment coverage among specific sub-groups of the population, and develop reliable surveillance systems that meet sensitivity and specificity requirements for the endgame of STH elimination. However, the considerable complexities often associated with STH-transmission models underpin the need for specifying a large number of parameters and inputs, which are often available with considerable degree of uncertainty. Additionally, the model may behave counter-intuitive especially when there are non-linearities in multiple input-output relationships. In this study, we performed a global sensitivity analysis (GSA), based on a variance decomposition method: extended Fourier Amplitude Sensitivity Test (eFAST), to a recently developed STH-transmission model in Kenya (an STH endemic country) to; (1) robustly compute sensitivity index (SI) for each parameter, (2) rank the parameters in order of their importance (from most to least influential), and (3) quantify the influence of each parameter, singly and cumulatively, on the model output. The sensitivity analysis (SA) results demonstrated that the model outcome (STH worm burden elimination in the human host) was significantly sensitive to some key parameter groupings: combined effect of improved water source and sanitation (ϕ), rounds of treatment offered (τ), efficacy of the drug used during treatment (h), proportion of the adult population treated (ga: akin to community-wide treatment), mortality rate of the mature worms in the human host (μ), and the strength of the -dependence of worm egg production (γ). For STH control programmes to effectively reach the endgame (STH elimination in the entire community), these key parameter groupings need to be targeted since together they contribute to a strategic public health intervention.
Collapse
Affiliation(s)
- Collins Okoyo
- Eastern and Southern Africa Centre of International Parasite Control (ESACIPAC), Kenya Medical Research Institute (KEMRI), Nairobi, Kenya
- School of Mathematics, University of Nairobi, Nairobi, Kenya
- *Correspondence: Collins Okoyo
| | - Nelson Onyango
- School of Mathematics, University of Nairobi, Nairobi, Kenya
| | - Idah Orowe
- School of Mathematics, University of Nairobi, Nairobi, Kenya
| | - Charles Mwandawiro
- Eastern and Southern Africa Centre of International Parasite Control (ESACIPAC), Kenya Medical Research Institute (KEMRI), Nairobi, Kenya
| | - Graham Medley
- Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine (LSHTM), London, United Kingdom
| |
Collapse
|
4
|
Mooney SJ, Shev AB, Keyes KM, Tracy M, Cerdá M. G-Computation and Agent-Based Modeling for Social Epidemiology: Can Population Interventions Prevent Posttraumatic Stress Disorder? Am J Epidemiol 2022; 191:188-197. [PMID: 34409437 PMCID: PMC8897987 DOI: 10.1093/aje/kwab219] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2020] [Revised: 08/02/2021] [Accepted: 08/11/2021] [Indexed: 11/13/2022] Open
Abstract
Agent-based modeling and g-computation can both be used to estimate impacts of intervening on complex systems. We explored each modeling approach within an applied example: interventions to reduce posttraumatic stress disorder (PTSD). We used data from a cohort of 2,282 adults representative of the adult population of the New York City metropolitan area from 2002-2006, of whom 16.3% developed PTSD over their lifetimes. We built 4 models: g-computation, an agent-based model (ABM) with no between-agent interactions, an ABM with violent-interaction dynamics, and an ABM with neighborhood dynamics. Three interventions were tested: 1) reducing violent victimization by 37.2% (real-world reduction); 2) reducing violent victimization by100%; and 3) supplementing the income of 20% of lower-income participants. The g-computation model estimated population-level PTSD risk reductions of 0.12% (95% confidence interval (CI): -0.16, 0.29), 0.28% (95% CI: -0.30, 0.70), and 1.55% (95% CI: 0.40, 2.12), respectively. The ABM with no interactions replicated the findings from g-computation. Introduction of interaction dynamics modestly decreased estimated intervention effects (income-supplement risk reduction dropped to 1.47%), whereas introduction of neighborhood dynamics modestly increased effectiveness (income-supplement risk reduction increased to 1.58%). Compared with g-computation, agent-based modeling permitted deeper exploration of complex systems dynamics at the cost of further assumptions.
Collapse
Affiliation(s)
- Stephen J Mooney
- Correspondence to Dr. Stephen Mooney, 1959 NE Pacific Street, Health Sciences Building F-262, Box 357236, Seattle, WA 98195 (e-mail: )
| | | | | | | | | |
Collapse
|
5
|
Murray EJ, Marshall BDL, Buchanan AL. Emulating Target Trials to Improve Causal Inference From Agent-Based Models. Am J Epidemiol 2021; 190:1652-1658. [PMID: 33595053 PMCID: PMC8484776 DOI: 10.1093/aje/kwab040] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2020] [Revised: 02/10/2021] [Accepted: 02/10/2021] [Indexed: 12/14/2022] Open
Abstract
Agent-based models are a key tool for investigating the emergent properties of population health settings, such as infectious disease transmission, where the exposure often violates the key "no interference" assumption of traditional causal inference under the potential outcomes framework. Agent-based models and other simulation-based modeling approaches have generally been viewed as a separate knowledge-generating paradigm from the potential outcomes framework, but this can lead to confusion about how to interpret the results of these models in real-world settings. By explicitly incorporating the target trial framework into the development of an agent-based or other simulation model, we can clarify the causal parameters of interest, as well as make explicit the assumptions required for valid causal effect estimation within or between populations. In this paper, we describe the use of the target trial framework for designing agent-based models when the goal is estimation of causal effects in the presence of interference, or spillover.
Collapse
Affiliation(s)
- Eleanor J Murray
- Correspondence to Dr. Eleanor J Murray, Department of Epidemiology, Boston University School of Public Health, 715 Albany Street, Boston, MA 02118 (e-mail: )
| | | | | |
Collapse
|
6
|
Kouser HN, Barnard-Mayers R, Murray E. Complex systems models for causal inference in social epidemiology. J Epidemiol Community Health 2021; 75:702-708. [PMID: 33172839 PMCID: PMC8849440 DOI: 10.1136/jech-2019-213052] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2020] [Revised: 09/24/2020] [Accepted: 10/21/2020] [Indexed: 02/06/2023]
Abstract
Systems models, which by design aim to capture multi-level complexity, are a natural choice of tool for bridging the divide between social epidemiology and causal inference. In this commentary, we discuss the potential uses of complex systems models for improving our understanding of quantitative causal effects in social epidemiology. To put systems models in context, we will describe how this approach could be used to optimise the distribution of COVID-19 response resources to minimise social inequalities during and after the pandemic.
Collapse
Affiliation(s)
- Hiba N Kouser
- Epidemiology, Boston University, Boston, Massachusetts, USA
| | | | - Eleanor Murray
- Epidemiology, Boston University, Boston, Massachusetts, USA
| |
Collapse
|
7
|
Laubach ZM, Murray EJ, Hoke KL, Safran RJ, Perng W. A biologist's guide to model selection and causal inference. Proc Biol Sci 2021; 288:20202815. [PMID: 33499782 PMCID: PMC7893255 DOI: 10.1098/rspb.2020.2815] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2020] [Accepted: 12/17/2020] [Indexed: 01/16/2023] Open
Abstract
A goal of many research programmes in biology is to extract meaningful insights from large, complex datasets. Researchers in ecology, evolution and behavior (EEB) often grapple with long-term, observational datasets from which they construct models to test causal hypotheses about biological processes. Similarly, epidemiologists analyse large, complex observational datasets to understand the distribution and determinants of human health. A key difference in the analytical workflows for these two distinct areas of biology is the delineation of data analysis tasks and explicit use of causal directed acyclic graphs (DAGs), widely adopted by epidemiologists. Here, we review the most recent causal inference literature and describe an analytical workflow that has direct applications for EEB. We start this commentary by defining four distinct analytical tasks (description, prediction, association, causal inference). The remainder of the text is dedicated to causal inference, specifically focusing on the use of DAGs to inform the modelling strategy. Given the increasing interest in causal inference and misperceptions regarding this task, we seek to facilitate an exchange of ideas between disciplinary silos and provide an analytical framework that is particularly relevant for making causal inference from observational data.
Collapse
Affiliation(s)
- Zachary M. Laubach
- Department of Ecology and Evolutionary Biology, University of Colorado, Boulder, CO, USA
- Department of Integrative Biology, Michigan State University, Lansing, MI, USA
| | - Eleanor J. Murray
- Department of Epidemiology, Boston University School of Public Health, Boston, MA, USA
| | - Kim L. Hoke
- Department of Biology, Colorado State University, Fort Collins, CO, USA
| | - Rebecca J. Safran
- Department of Ecology and Evolutionary Biology, University of Colorado, Boulder, CO, USA
| | - Wei Perng
- Department of Epidemiology, Lifecourse Epidemiology of Adiposity and Diabetes (LEAD) Center, University of Colorado Denver Anschutz Medical Campus, Aurora, CO, USA
| |
Collapse
|