1
|
Cao Z, Cho Y, Li F. Transporting randomized trial results to estimate counterfactual survival functions in target populations. Pharm Stat 2024; 23:442-465. [PMID: 38233102 DOI: 10.1002/pst.2354] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2023] [Revised: 08/27/2023] [Accepted: 11/30/2023] [Indexed: 01/19/2024]
Abstract
When the distributions of treatment effect modifiers differ between a randomized trial and an external target population, the sample average treatment effect in the trial may be substantially different from the target population average treatment, and accurate estimation of the latter requires adjusting for the differential distribution of effect modifiers. Despite the increasingly rich literature on transportability, little attention has been devoted to methods for transporting trial results to estimate counterfactual survival functions in target populations, when the primary outcome is time to event and subject to right censoring. In this article, we study inverse probability weighting and doubly robust estimators to estimate counterfactual survival functions and the target average survival treatment effect in the target population, and provide their respective approximate variance estimators. We focus on a common scenario where the target population information is observed only through a complex survey, and elucidate how the survey weights can be incorporated into each estimator we considered. Simulation studies are conducted to examine the finite-sample performances of the proposed estimators in terms of bias, efficiency and coverage, under both correct and incorrect model specifications. Finally, we apply the proposed method to assess transportability of the results in the Action to Control Cardiovascular Risk in Diabetes-Blood Pressure (ACCORD-BP) trial to all adults with Diabetes in the United States.
Collapse
Affiliation(s)
- Zhiqiang Cao
- Department of Mathematics, College of Big Data and Internet, Shenzhen Technology University, Shenzhen, People's Republic of China
| | - Youngjoo Cho
- Department of Applied Statistics, Konkuk University, Seoul, Republic of Korea
| | - Fan Li
- Department of Biostatistics, Yale University School of Public Health, New Haven, Connecticut, USA
- Center for Methods in Implementation and Prevention Science, Yale University School of Public Health, New Haven, Connecticut, USA
| |
Collapse
|
2
|
Gibbons LE, Mobley T, Mayeda ER, Lee CS, Gatto NM, LaCroix AZ, McEvoy LK, Crane PK, Hayes-Larson E. How Generalizable Are Findings from a Community-Based Prospective Cohort Study? Extending Estimates from the Adult Changes in Thought Study to Its Source Population. J Alzheimers Dis 2024; 100:163-174. [PMID: 38848188 DOI: 10.3233/jad-240247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/09/2024]
Abstract
Background The Adult Changes in Thought (ACT) study is a cohort of Kaiser Permanente Washington members ages 65+ that began in 1994. Objective We wanted to know how well ACT participants represented all older adults in the region, and how well ACT findings on eye disease and its relationship with Alzheimer's disease generalized to all older adults in the Seattle Metropolitan Region. Methods We used participation weights derived from pooling ACT and Behavioral Risk Factor Surveillance System (BRFSS) data to estimate prevalences of common eye diseases and their associations with Alzheimer's disease incidence. Cox proportional hazards models accounted for age, education, smoking, sex, and APOE genotype. Confidence intervals for weighted analyses were bootstrapped to account for error in estimating the weights. Results ACT participants were fairly similar to older adults in the region. The largest differences were more self-reported current cholesterol medication use in BRFSS and higher proportions with low education in ACT. Incorporating the weights had little impact on prevalence estimates for age-related macular degeneration or glaucoma. Weighted estimates were slightly higher for diabetic retinopathy (weighted 5.7% (95% Confidence Interval 4.3, 7.1); unweighted 4.1% (3.6, 4.6)) and cataract history (weighted 51.8% (49.6, 54.3); unweighted 48.6% (47.3, 49.9)). The weighted hazard ratio for recent diabetic retinopathy diagnosis and Alzheimer's disease was 1.84 (0.34, 4.29), versus 1.32 (0.87, 2.00) in unweighted ACT. Conclusions Most, but not all, associations were similar after participation weighting. Even in community-based cohorts, extending inferences to broader populations may benefit from evaluation with participation weights.
Collapse
Affiliation(s)
- Laura E Gibbons
- Department of Medicine, University of Washington, Seattle, WA, USA
| | - Taylor Mobley
- Department of Epidemiology, UCLA Fielding School of Public Health, University of California, Los Angeles, CA, USA
| | - Elizabeth Rose Mayeda
- Department of Epidemiology, UCLA Fielding School of Public Health, University of California, Los Angeles, CA, USA
| | - Cecilia S Lee
- Department of Ophthalmology, University of Washington, Seattle, WA, USA
- Roger and Angie Karalis Johnson Retina Center, Seattle, WA, USA
| | - Nicole M Gatto
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA
| | - Andrea Z LaCroix
- Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego, La Jolla, CA, USA
| | - Linda K McEvoy
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA
| | - Paul K Crane
- Department of Medicine, University of Washington, Seattle, WA, USA
| | - Eleanor Hayes-Larson
- Department of Epidemiology, UCLA Fielding School of Public Health, University of California, Los Angeles, CA, USA
| |
Collapse
|
3
|
Elliott MR, Carroll O, Grieve R, Carpenter J. Improving transportability of randomized controlled trial inference using robust prediction methods. Stat Methods Med Res 2023; 32:2365-2385. [PMID: 37936293 DOI: 10.1177/09622802231210944] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2023]
Abstract
Randomized trials have been the gold standard for assessing causal effects since their introduction by Fisher in the 1920s, since they can eliminate both observed and unobserved confounding. Estimates of causal effects at the population level from randomized controlled trials can still be biased if there are both effect modification and systematic differences between the trial sample and the ultimate population of inference with respect to these modifiers. Recent advances in the survey statistics literature to improve inference in nonprobability samples by using information from probability samples can provide an avenue for improving population causal inference in randomized controlled trials when relevant probability samples of the patient population are available. We review some recent work in "transporting" causal effect estimates from trials to populations, focusing on the setting where there is a "benchmark" or population-representative sample along with the RCT sample. We then propose estimators using either inverse probability weighting (IPWT) or prediction that can accommodate unequal probability of selection in the "benchmark" or population, and use Bayesian additive regression trees for both inverse probability of treatment weighting and prediction estimation that do not require specification of functional form or interaction. We also consider how the assumption of ignorability may be assessed from observed data and propose a sensitivity analysis under the failure of this assumption. We compare our proposed approach with existing methods in simulation and apply these alternative approaches to a study of pulmonary artery catheterization in critically ill patients. We also suggest next steps for future work.
Collapse
Affiliation(s)
- Michael R Elliott
- Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, MI, USA
- Survey Research Center, Institute for Social Research, University of Michigan, Ann Arbor, MI, USA
| | - Orlagh Carroll
- Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK
| | - Richard Grieve
- Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK
| | - James Carpenter
- Department of Medical Statistics, London School of Hygiene and Tropical Medicine, London, UK
| |
Collapse
|
4
|
Kaizar E, Lin CY, Faries D, Johnston J. Reweighting estimators to extend the external validity of clinical trials: methodological considerations. J Biopharm Stat 2023; 33:515-543. [PMID: 36688658 DOI: 10.1080/10543406.2022.2162067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Accepted: 12/10/2022] [Indexed: 01/24/2023]
Abstract
Methods to extend the strong internal validity of randomized controlled trials to reliably estimate treatment effects in target populations are gaining attention. This paper enumerates steps recommended for undertaking such extended inference, discusses currently viable choices for each one, and provides recommendations. We demonstrate a complete extended inference from a clinical trial studying a pharmaceutical treatment for Alzheimer's disease (AD) to a realistic target population of European residents diagnosed with AD. This case study highlights approaches to overcoming practical difficulties and demonstrates limitations of reliably extending inference from a trial to a real-world population.
Collapse
Affiliation(s)
- Eloise Kaizar
- Department of Statistics, Ohio State University, Columbus, Ohio, USA
| | - Chen-Yen Lin
- FSP Biometrics, Syneos Health, Toronto, Ontario, Canada
| | - Douglas Faries
- Real World Analytics, Eli Lilly and Company, Indianapolis, Indiana, USA
| | - Joseph Johnston
- Value, Evidence, and Outcomes, Eli Lilly and Company, Indianapolis, Indiana, USA
| |
Collapse
|
5
|
St Jean DT, Edwards JK, Rogawski McQuade ET, Thompson P, Thomas JC, Becker-Dreps S. Transporting monovalent rotavirus vaccine efficacy estimates to an external target population: a secondary analysis of data from a randomised controlled trial in Malawi. Epidemiol Infect 2023; 151:e49. [PMID: 36843494 PMCID: PMC10052556 DOI: 10.1017/s0950268823000286] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 02/14/2023] [Accepted: 02/15/2023] [Indexed: 02/28/2023] Open
Abstract
Oral rotavirus vaccine efficacy estimates from randomised controlled trials are highly variable across settings. Although the randomised study design increases the likelihood of internal validity of findings, results from trials may not always apply outside the context of the study due to differences between trial participants and the target population. Here, we used a weight-based method to transport results from a monovalent rotavirus vaccine clinical trial conducted in Malawi between 2005 and 2008 to a target population of all trial-eligible children in Malawi, represented by data from the 2015-2016 Malawi Demographic and Health Survey (DHS). We reweighted trial participants to reflect the population characteristics described by the Malawi DHS. Vaccine efficacy was estimated for 1008 trial participants after applying these weights such that they represented trial-eligible children in Malawi. We also conducted subgroup analyses to examine the heterogeneous treatment effects by stunting and tuberculosis vaccination status at enrolment. In the original trial, the estimates of one-year vaccine efficacy against severe rotavirus gastroenteritis and any-severity rotavirus gastroenteritis in Malawi were 49.2% (95% CI 15.6%-70.3%) and 32.1% (95% CI 2.5%-53.1%), respectively. After weighting trial participants to represent all trial-eligible children in Malawi, vaccine efficacy increased to 62.2% (95% CI 35.5%-79.0%) against severe rotavirus gastroenteritis and 38.9% (95% CI 11.4%-58.5%) against any-severity rotavirus gastroenteritis. Rotavirus vaccine efficacy may differ between trial participants and target populations when these two populations differ. Differences in tuberculosis vaccination status between the trial sample and DHS population contributed to varying trial and target population vaccine efficacy estimates.
Collapse
Affiliation(s)
- Denise T. St Jean
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Jessie K. Edwards
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | | | - Peyton Thompson
- Division of Infectious Diseases, Department of Pediatrics, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - James C. Thomas
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Sylvia Becker-Dreps
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
- Department of Family Medicine, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| |
Collapse
|
6
|
Cook RR, Foot C, Arah OA, Humphreys K, Rudolph KE, Luo SX, Tsui JI, Levander XA, Korthuis PT. Estimating the impact of stimulant use on initiation of buprenorphine and extended-release naltrexone in two clinical trials and real-world populations. Addict Sci Clin Pract 2023; 18:11. [PMID: 36788634 PMCID: PMC9930351 DOI: 10.1186/s13722-023-00364-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Accepted: 02/01/2023] [Indexed: 02/16/2023] Open
Abstract
BACKGROUND Co-use of stimulants and opioids is rapidly increasing. Randomized clinical trials (RCTs) have established the efficacy of medications for opioid use disorder (MOUD), but stimulant use may decrease the likelihood of initiating MOUD treatment. Furthermore, trial participants may not represent "real-world" populations who would benefit from treatment. METHODS We conducted a two-stage analysis. First, associations between stimulant use (time-varying urine drug screens for cocaine, methamphetamine, or amphetamines) and initiation of buprenorphine or extended-release naltrexone (XR-NTX) were estimated across two RCTs (CTN-0051 X:BOT and CTN-0067 CHOICES) using adjusted Cox regression models. Second, results were generalized to three target populations who would benefit from MOUD: Housed adults identifying the need for OUD treatment, as characterized by the National Survey on Drug Use and Health (NSDUH); adults entering OUD treatment, as characterized by Treatment Episodes Dataset (TEDS); and adults living in rural regions of the U.S. with high rates of injection drug use, as characterized by the Rural Opioids Initiative (ROI). Generalizability analyses adjusted for differences in demographic characteristics, substance use, housing status, and depression between RCT and target populations using inverse probability of selection weighting. RESULTS Analyses included 673 clinical trial participants, 139 NSDUH respondents (weighted to represent 661,650 people), 71,751 TEDS treatment episodes, and 1,933 ROI participants. The majority were aged 30-49 years, male, and non-Hispanic White. In RCTs, stimulant use reduced the likelihood of MOUD initiation by 32% (adjusted HR [aHR] = 0.68, 95% CI 0.49-0.94, p = 0.019). Stimulant use associations were slightly attenuated and non-significant among housed adults needing treatment (25% reduction, aHR = 0.75, 0.48-1.18, p = 0.215) and adults entering OUD treatment (28% reduction, aHR = 0.72, 0.51-1.01, p = 0.061). The association was more pronounced, but still non-significant among rural people injecting drugs (39% reduction, aHR = 0.61, 0.35-1.06, p = 0.081). Stimulant use had a larger negative impact on XR-NTX initiation compared to buprenorphine, especially in the rural population (76% reduction, aHR = 0.24, 0.08-0.69, p = 0.008). CONCLUSIONS Stimulant use is a barrier to buprenorphine or XR-NTX initiation in clinical trials and real-world populations that would benefit from OUD treatment. Interventions to address stimulant use among patients with OUD are urgently needed, especially among rural people injecting drugs, who already suffer from limited access to MOUD.
Collapse
Affiliation(s)
- R R Cook
- Section of Addiction Medicine, Department of Medicine, Oregon Health & Science University, Sam Jackson Hall, Suite 3370, 3245 SW Pavilion Loop, Portland, OR, 97239, USA.
| | - C Foot
- Section of Addiction Medicine, Department of Medicine, Oregon Health & Science University, Sam Jackson Hall, Suite 3370, 3245 SW Pavilion Loop, Portland, OR, 97239, USA
| | - O A Arah
- Department of Epidemiology, Fielding School of Public Health, University of California, Los Angeles (UCLA), Los Angeles, CA, USA
- Division of Physical Sciences, Department of Statistics, UCLA College, Los Angeles, CA, USA
- Research Unit for Epidemiology, Department of Public Health, Aarhus University, Aarhus, Denmark
| | - K Humphreys
- Center for Innovation to Implementation, VA Palo Alto Health Care System, Palo Alto, CA, USA
- Department of Psychiatry and Behavioral Sciences, Stanford University, Palo Alto, CA, USA
| | - K E Rudolph
- Department of Epidemiology, School of Public Health, Columbia University, New York, NY, USA
| | - S X Luo
- Division on Substance Use Disorders, Department of Psychiatry, Columbia University, New York, USA
| | - J I Tsui
- Department of Medicine, University of Washington, Seattle, WA, USA
| | - X A Levander
- Section of Addiction Medicine, Department of Medicine, Oregon Health & Science University, Sam Jackson Hall, Suite 3370, 3245 SW Pavilion Loop, Portland, OR, 97239, USA
| | - P T Korthuis
- Section of Addiction Medicine, Department of Medicine, Oregon Health & Science University, Sam Jackson Hall, Suite 3370, 3245 SW Pavilion Loop, Portland, OR, 97239, USA
| |
Collapse
|
7
|
Bernstein OM, Vegetabile BG, Salazar CR, Grill JD, Gillen DL. Adjustment for biased sampling using NHANES derived propensity weights. HEALTH SERVICES AND OUTCOMES RESEARCH METHODOLOGY 2022. [DOI: 10.1007/s10742-022-00283-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
|
8
|
Zivich PN, Shook-Sa BE, Edwards JK, Westreich D, Cole SR. On the Use of Covariate Supersets for Identification Conditions. Epidemiology 2022; 33:559-562. [PMID: 35384912 PMCID: PMC9156549 DOI: 10.1097/ede.0000000000001493] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
The union of distinct covariate sets, or the superset, is often used in proofs for the identification or the statistical consistency of an estimator when multiple sources of bias are present. However, the use of a superset can obscure important nuances. Here, we provide two illustrative examples: one in the context of missing data on outcomes, and one in which the average causal effect is transported to another target population. As these examples demonstrate, the use of supersets may indicate a parameter is not identifiable when the parameter is indeed identified. Furthermore, a series of exchangeability conditions may lead to successively weaker conditions. Future work on approaches to address multiple biases can avoid these pitfalls by considering the more general case of nonoverlapping covariate sets.
Collapse
Affiliation(s)
- Paul N Zivich
- From the Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, NC
| | - Bonnie E Shook-Sa
- Department of Biostatistics, UNC Gillings School of Global Public Health, Chapel Hill, NC
| | - Jessie K Edwards
- From the Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, NC
| | - Daniel Westreich
- From the Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, NC
| | - Stephen R Cole
- From the Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, NC
| |
Collapse
|
9
|
Li F, Buchanan AL, Cole SR. Generalizing trial evidence to target populations in non-nested designs: Applications to AIDS clinical trials. J R Stat Soc Ser C Appl Stat 2022; 71:669-697. [PMID: 35968541 PMCID: PMC9367209 DOI: 10.1111/rssc.12550] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023]
Abstract
Comparative effectiveness evidence from randomized trials may not be directly generalizable to a target population of substantive interest when, as in most cases, trial participants are not randomly sampled from the target population. Motivated by the need to generalize evidence from two trials conducted in the AIDS Clinical Trials Group (ACTG), we consider weighting, regression and doubly robust estimators to estimate the causal effects of HIV interventions in a specified population of people living with HIV in the USA. We focus on a non-nested trial design and discuss strategies for both point and variance estimation of the target population average treatment effect. Specifically in the generalizability context, we demonstrate both analytically and empirically that estimating the known propensity score in trials does not increase the variance for each of the weighting, regression and doubly robust estimators. We apply these methods to generalize the average treatment effects from two ACTG trials to specified target populations and operationalize key practical considerations. Finally, we report on a simulation study that investigates the finite-sample operating characteristics of the generalizability estimators and their sandwich variance estimators.
Collapse
Affiliation(s)
- Fan Li
- Department of Biostatistics, Yale University School of Public Health, New Haven, Connecticut, USA
- Center for Methods in Implementation and Prevention Science, Yale University, New Haven, Connecticut, USA
| | - Ashley L. Buchanan
- Department of Pharmacy Practice, College of Pharmacy, University of Rhode Island, Kingston, Rhode Island, USA
| | - Stephen R. Cole
- Department of Epidemiology, Gillings School of Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| |
Collapse
|
10
|
Kim MP, Kern C, Goldwasser S, Kreuter F, Reingold O. Universal adaptability: Target-independent inference that competes with propensity scoring. Proc Natl Acad Sci U S A 2022; 119:e2108097119. [PMID: 35046023 PMCID: PMC8794832 DOI: 10.1073/pnas.2108097119] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Accepted: 12/02/2021] [Indexed: 11/20/2022] Open
Abstract
The gold-standard approaches for gleaning statistically valid conclusions from data involve random sampling from the population. Collecting properly randomized data, however, can be challenging, so modern statistical methods, including propensity score reweighting, aim to enable valid inferences when random sampling is not feasible. We put forth an approach for making inferences based on available data from a source population that may differ in composition in unknown ways from an eventual target population. Whereas propensity scoring requires a separate estimation procedure for each different target population, we show how to build a single estimator, based on source data alone, that allows for efficient and accurate estimates on any downstream target data. We demonstrate, theoretically and empirically, that our target-independent approach to inference, which we dub "universal adaptability," is competitive with target-specific approaches that rely on propensity scoring. Our approach builds on a surprising connection between the problem of inferences in unspecified target populations and the multicalibration problem, studied in the burgeoning field of algorithmic fairness. We show how the multicalibration framework can be employed to yield valid inferences from a single source population across a diverse set of target populations.
Collapse
Affiliation(s)
- Michael P Kim
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA 94720
- Miller Institute for Basic Research in Science, Berkeley, CA 94720
| | - Christoph Kern
- School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany
| | - Shafi Goldwasser
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA 94720;
- Simons Institute for the Theory of Computation, Berkeley, CA 94720
| | - Frauke Kreuter
- Joint Program in Survey Methodology, University of Maryland, College Park, MD 20742
- Department of Statistics, Ludwig-Maximilians-Universität München, 80539 München, Germany
| | - Omer Reingold
- Department of Computer Science, Stanford University, Stanford, CA 94305
| |
Collapse
|
11
|
Sperrin M, Diaz-Ordaz K, Pajouheshnia R. Invited Commentary: Treatment Drop-in-Making the Case for Causal Prediction. Am J Epidemiol 2021; 190:2015-2018. [PMID: 33595073 PMCID: PMC8485150 DOI: 10.1093/aje/kwab030] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Revised: 01/25/2021] [Accepted: 02/02/2021] [Indexed: 11/12/2022] Open
Abstract
Clinical prediction models (CPMs) are often used to guide treatment initiation, with individuals at high risk offered treatment. This implicitly assumes that the probability quoted from a CPM represents the risk to an individual of an adverse outcome in absence of treatment. However, for a CPM to correctly target this estimand requires careful causal thinking. One problem that needs to be overcome is treatment drop-in: where individuals in the development data commence treatment after the time of prediction but before the outcome occurs. In this issue of the Journal, Xu et al. (Am J Epidemiol. 2021;190(10):2000-2014) use causal estimates from external data sources, such as clinical trials, to adjust CPMs for treatment drop-in. This represents a pragmatic and promising approach to address this issue, and it illustrates the value of utilizing causal inference in prediction. Building causality into the prediction pipeline can also bring other benefits. These include the ability to make and compare hypothetical predictions under different interventions, to make CPMs more explainable and transparent, and to improve model generalizability. Enriching CPMs with causal inference therefore has the potential to add considerable value to the role of prediction in healthcare.
Collapse
Affiliation(s)
- Matthew Sperrin
- Correspondence to Matthew Sperrin, Vaughan House, Portsmouth Street, University of Manchester, Manchester M13 9GB, UK (e-mail: )
| | | | | |
Collapse
|
12
|
Lesko CR, Ackerman B, Webster-Clark M, Edwards JK. Target validity: Bringing treatment of external validity in line with internal validity. CURR EPIDEMIOL REP 2021; 7:117-124. [PMID: 33585162 DOI: 10.1007/s40471-020-00239-0] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
Abstract
Purpose of Review "Target bias" is the difference between an estimate of association from a study sample and the causal effect in the target population of interest. It is the sum of internal and external bias. Given the extensive literature on internal validity, here, we review threats and methods to improve external validity. Recent findings External bias may arise when the distribution of modifiers of the effect of treatment differs between the study sample and the target population. Methods including those based on modeling the outcome, modeling sample membership, and doubly robust methods are available, assuming data on the target population is available. Summary The relevance of information for making policy decisions is dependent on both the actions that were studied and the sample in which they were evaluated. Combining methods for addressing internal and external validity can improve the policy relevance of study results.
Collapse
Affiliation(s)
- Catherine R Lesko
- Department of Epidemiology, Johns Hopkins School of Public Health, Baltimore, MD
| | - Benjamin Ackerman
- Department of Biostatistics, Johns Hopkins School of Public Health, Baltimore, MD
| | | | - Jessie K Edwards
- Department of Epidemiology, University of North Carolina, Chapel Hill, NC
| |
Collapse
|