1
|
Quan H, Li T, Chen X, Li G. Generalizing Treatment Effect to a Target Population Without Individual Patient Data in a Real-World Setting. Pharm Stat 2025; 24:e2435. [PMID: 39227179 DOI: 10.1002/pst.2435] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Revised: 07/16/2024] [Accepted: 08/09/2024] [Indexed: 09/05/2024]
Abstract
The innovative use of real-world data (RWD) can answer questions that cannot be addressed using data from randomized clinical trials (RCTs). While the sponsors of RCTs have a central database containing all individual patient data (IPD) collected from trials, analysts of RWD face a challenge: regulations on patient privacy make access to IPD from all regions logistically prohibitive. In this research, we propose a double inverse probability weighting (DIPW) approach for the analysis sponsor to estimate the population average treatment effect (PATE) for a target population without the need to access IPD. One probability weighting is for achieving comparable distributions in confounders across treatment groups; another probability weighting is for generalizing the result from a subpopulation of patients who have data on the endpoint to the whole target population. The likelihood expressions for propensity scores and the DIPW estimator of the PATE can be written to only rely on regional summary statistics that do not require IPD. Our approach hinges upon the positivity and conditional independency assumptions, prerequisites to most RWD analysis approaches. Simulations are conducted to compare the performances of the proposed method against a modified meta-analysis and a regular meta-analysis.
Collapse
Affiliation(s)
- Hui Quan
- Biostatistics and Programming, Sanofi, Bridgewater, New Jersey, USA
| | - Tong Li
- Biostatistics and Programming, Sanofi, Bridgewater, New Jersey, USA
| | - Xun Chen
- Biostatistics and Programming, Sanofi, Bridgewater, New Jersey, USA
| | - Gang Li
- Eisai Inc, Nutley, New Jersey, USA
| |
Collapse
|
2
|
Paget MA, Tockhorn-Heidenreich A, Belger M, Chartier F, Lantéri-Minet M. Generalizability of clinical trial efficacy results to a real-world population: An example in migraine prevention. J Manag Care Spec Pharm 2023; 29:1321-1330. [PMID: 38058137 PMCID: PMC10776265 DOI: 10.18553/jmcp.2023.29.12.1321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/08/2023]
Abstract
BACKGROUND Health care decision makers are often concerned about the external validity of randomized controlled trials (RCTs), as their results may not apply to certain patients in the real world who intend to receive treatment. OBJECTIVE To demonstrate a methodology for assessing the generalizability of clinical trial results to a real-world population, before sufficient and appropriate real-world effectiveness data are available, using individual patient-level data from an RCT and aggregated baseline data from a real-world French registry in migraine. METHODS The analyses were conducted in 2 steps. First, individual patient-level baseline data from the multinational CONQUER RCT were weighted to match aggregated real-world InovPain registry patient characteristic data. Matched patient characteristics were sex, age, migraine type and duration, number of monthly migraine headache days, and number of monthly headache days at baseline. Second, the weighted CONQUER patient data were used to reanalyze the primary endpoint of CONQUER (least squares mean change from baseline in the number of monthly migraine headache days during the 3-month double-blind treatment phase) using predefined methodology. Sensitivity analyses were conducted to assess the robustness of findings. RESULTS A total of 462 patients with migraine were randomized and treated with galcanezumab or placebo in CONQUER; aggregated InovPain data were available from 130 patients with migraine. We identified no important differences in baseline patient characteristics between the 2 prespecified populations, suggesting good external validity for CONQUER. Although this limited the extent of observed differences between the original and matched CONQUER populations, weighting of CONQUER data did help harmonize the 2 datasets and allow the results obtained in CONQUER to be generalized to patients more representative of the real-world French population with migraine. Results of weighted analyses suggested that galcanezumab would be superior to placebo for reducing monthly migraine headache days in a clinical trial in patients with episodic or chronic migraine who reflected the characteristics of patients eligible to receive the drug in France. CONCLUSIONS Findings suggest that our methods may be helpful for assessing the generalizability of clinical trial results to a real-world population before the availability of substantial real-world clinical data.
Collapse
Affiliation(s)
| | | | - Mark Belger
- Eli Lilly and Company, Indianapolis, IN, USA
| | | | - Michel Lantéri-Minet
- Pain Départment, CHU Nice and FHU InovPain Université Côte Azur, Nice, France
- INSERM U1107, Neuro-Dol, Trigeminal Pain and Migraine, Université Clermont Auvergne, France
| |
Collapse
|
3
|
Barker DH, Bie R, Steingrimsson JA. Addressing Systematic Missing Data in the Context of Causally Interpretable Meta-analysis. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2023; 24:1648-1658. [PMID: 37726579 DOI: 10.1007/s11121-023-01586-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/07/2023] [Indexed: 09/21/2023]
Abstract
Evidence synthesis involves drawing conclusions from trial samples that may differ from the target population of interest, and there is often heterogeneity among trials in sample characteristics, treatment implementation, study design, and assessment of covariates. Stitching together this patchwork of evidence requires subject-matter knowledge, a clearly defined target population, and guidance on how to weigh evidence from different trials. Transportability analysis has provided formal identifiability conditions required to make unbiased causal inference in the target population. In this manuscript, we review these conditions along with an additional assumption required to address systematic missing data. The identifiability conditions highlight the importance of accounting for differences in treatment effect modifiers between the populations underlying the trials and the target population. We perform simulations to evaluate the bias of conventional random effect models and multiply imputed estimates using the pooled trials sample and describe causal estimators that explicitly address trial-to-target differences in key covariates in the context of systematic missing data. Results indicate that the causal transportability estimators are unbiased when treatment effect modifiers are accounted for in the analyses. Results also highlight the importance of carefully evaluating identifiability conditions for each trial to reduce bias due to differences in participant characteristics between trials and the target population. Bias can be limited by adjusting for covariates that are strongly correlated with missing treatment effect modifiers, including data from trials that do not differ from the target on treatment modifiers, and removing trials that do differ from the target and did not assess a modifier.
Collapse
Affiliation(s)
- David H Barker
- Department of Psychiatry and Human Behavior, The Warren Alpert Medical School of Brown University, Providence, RI, USA.
- Bradley Hasbro Children's Research Center, Providence, RI, USA.
| | - Ruofan Bie
- Department of Biostatistics, Brown University, Providence, RI, USA
| | | |
Collapse
|
4
|
Tompsett D, Zylbersztejn A, Hardelid P, De Stavola B. Target Trial Emulation and Bias Through Missing Eligibility Data: An Application to a Study of Palivizumab for the Prevention of Hospitalization Due to Infant Respiratory Illness. Am J Epidemiol 2023; 192:600-611. [PMID: 36509514 PMCID: PMC10089079 DOI: 10.1093/aje/kwac202] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2021] [Revised: 08/11/2022] [Accepted: 11/09/2022] [Indexed: 12/15/2022] Open
Abstract
Target trial emulation (TTE) applies the principles of randomized controlled trials to the causal analysis of observational data sets. One challenge that is rarely considered in TTE is the sources of bias that may arise if the variables involved in the definition of eligibility for the trial are missing. We highlight patterns of bias that might arise when estimating the causal effect of a point exposure when restricting the target trial to individuals with complete eligibility data. Simulations consider realistic scenarios where the variables affecting eligibility modify the causal effect of the exposure and are missing at random or missing not at random. We discuss means to address these patterns of bias, namely: 1) controlling for the collider bias induced by the missing data on eligibility, and 2) imputing the missing values of the eligibility variables prior to selection into the target trial. Results are compared with the results when TTE is performed ignoring the impact of missing eligibility. A study of palivizumab, a monoclonal antibody recommended for the prevention of respiratory hospital admissions due to respiratory syncytial virus in high-risk infants, is used for illustration.
Collapse
Affiliation(s)
- Daniel Tompsett
- Correspondence to Dr. Daniel Tompsett, Population Policy and Practice Department, UCL GOS Institute of Child Health, 30 Guilford Street, London, WC1N 1EH, United Kingdom (e-mail: )
| | | | | | | |
Collapse
|
5
|
Barker DH, Dahabreh IJ, Steingrimsson JA, Houck C, Donenberg G, DiClemente R, Brown LK. Causally Interpretable Meta-analysis: Application in Adolescent HIV Prevention. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2022; 23:403-414. [PMID: 34241752 PMCID: PMC8742835 DOI: 10.1007/s11121-021-01270-3] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/15/2021] [Indexed: 12/30/2022]
Abstract
Endowing meta-analytic results with a causal interpretation is challenging when there are differences in the distribution of effect modifiers among the populations underlying the included trials and the target population where the results of the meta-analysis will be applied. Recent work on transportability methods has described identifiability conditions under which the collection of randomized trials in a meta-analysis can be used to draw causal inferences about the target population. When the conditions hold, the methods enable estimation of causal quantities such as the average treatment effect and conditional average treatment effect in target populations that differ from the populations underlying the trial samples. The methods also facilitate comparison of treatments not directly compared in a head-to-head trial and assessment of comparative effectiveness within subgroups of the target population. We briefly describe these methods and present a worked example using individual participant data from three HIV prevention trials among adolescents in mental health care. We describe practical challenges in defining the target population, obtaining individual participant data from included trials and a sample of the target population, and addressing systematic missing data across datasets. When fully realized, methods for causally interpretable meta-analysis can provide decision-makers valid estimates of how treatments will work in target populations of substantive interest as well as in subgroups of these populations.
Collapse
Affiliation(s)
- David H Barker
- Department of Psychiatry, Rhode Island Hospital, Providence, RI, USA.
- Department of Psychiatry and Human Behavior, The Warren Alpert Medical School of Brown University, Providence, RI, USA.
| | - Issa J Dahabreh
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | | | - Christopher Houck
- Department of Psychiatry, Rhode Island Hospital, Providence, RI, USA
- Department of Psychiatry and Human Behavior, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Geri Donenberg
- School of Public Health, University of Illinois At Chicago, Chicago, IL, USA
| | - Ralph DiClemente
- New York University College of Global Public Health, New York, NY, USA
| | - Larry K Brown
- Department of Psychiatry, Rhode Island Hospital, Providence, RI, USA
- Department of Psychiatry and Human Behavior, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| |
Collapse
|
6
|
Nagasaka M, Molife C, Cui ZL, Stefaniak V, Li X, Kim S, Lee HY, Beyrer J, Blumenschein G. Generalizability of ORIENT-11 trial results to a US standard of care cohort with advanced non-small-cell lung cancer. Future Oncol 2022; 18:1963-1977. [PMID: 35354280 DOI: 10.2217/fon-2022-0099] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022] Open
Abstract
Aim: This retrospective study estimated efficacy and safety of sintilimab + pemetrexed + platinum (SPP) versus placebo + pemetrexed + platinum (PPP) in untreated locally advanced/metastatic, nonsquamous non-small-cell lung cancer (NSCLC), after adjusting each ORIENT-11 trial patient's contribution to ORIENT-11 data based on characteristics of a target US population. Materials & methods: The target US population (n = 557) was selected from a real-world deidentified advanced NSCLC database based on key ORIENT-11 eligibility criteria. Inverse probability weights for ORIENT-11 patients (n = 397) relative to US patients were calculated. Efficacy and safety of SPP versus PPP were adjusted by inverse probability weights. Results: After adjustment, progression-free survival remained superior for SPP. Other efficacy and safety outcomes were consistent. Conclusion: These results provide evidence on how the effects observed with SPP in ORIENT-11 could translate to a US population with untreated locally advanced/metastatic nonsquamous NSCLC.
Collapse
Affiliation(s)
- Misako Nagasaka
- Division of Hematology & Oncology Department of Medicine, University of California Irvine, Orange County, CA 92868, USA
| | - Cliff Molife
- Value, Evidence, & Outcomes, Eli Lilly & Company, Indianapolis, IN 46225, USA
| | - Zhanglin Lin Cui
- Real World Analytics, Eli Lilly & Company, Indianapolis, IN 46225, USA
| | | | - Xiaohong Li
- Real World Analytics, Eli Lilly & Company, Indianapolis, IN 46225, USA
| | - Sangmi Kim
- Global Patient Safety, Eli Lilly & Company, Indianapolis, IN 46225, USA
| | - Hsui-Yung Lee
- Global Statistical Sciences, Eli Lilly & Company, Indianapolis, IN 46225, USA
| | - Julia Beyrer
- Value, Evidence, & Outcomes, Eli Lilly & Company, Indianapolis, IN 46225, USA
| | - George Blumenschein
- Department of Thoracic & Head & Neck Medical Oncology, The University of Texas M D Anderson Cancer Center, Houston, TX 77030, USA
| |
Collapse
|
7
|
Shahim B, Cohen DJ. Transporting Results of TAVR Trials to the Real World: A Long and Winding Road. JACC Cardiovasc Interv 2021; 14:2124-2126. [PMID: 34620390 DOI: 10.1016/j.jcin.2021.08.029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Accepted: 08/10/2021] [Indexed: 10/20/2022]
Affiliation(s)
- Bahira Shahim
- Cardiovascular Research Foundation, New York, New York, USA; Division of Cardiology, Department of Medicine, Karolinska University Hospital, Stockholm, Sweden
| | - David J Cohen
- Cardiovascular Research Foundation, New York, New York, USA; Department of Cardiology, St. Francis Hospital, Roslyn, New York, USA.
| |
Collapse
|
8
|
Butala NM, Secemsky E, Kazi DS, Song Y, Strom JB, Faridi KF, Brennan JM, Elmariah S, Shen C, Yeh RW. Applicability of Transcatheter Aortic Valve Replacement Trials to Real-World Clinical Practice: Findings From EXTEND-CoreValve. JACC Cardiovasc Interv 2021; 14:2112-2123. [PMID: 34620389 DOI: 10.1016/j.jcin.2021.08.006] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Revised: 07/28/2021] [Accepted: 08/03/2021] [Indexed: 01/15/2023]
Abstract
OBJECTIVES The aim of this study was to examine the applicability of pivotal transcatheter aortic valve replacement (TAVR) trials to the real-world population of Medicare patients undergoing TAVR. BACKGROUND It is unclear whether randomized controlled trial results of novel cardiovascular devices apply to patients encountered in clinical practice. METHODS Characteristics of patients enrolled in the U.S. CoreValve pivotal trials were compared with those of the population of Medicare beneficiaries who underwent TAVR in U.S. clinical practice between November 2, 2011, and December 31, 2017. Inverse probability weighting was used to reweight the trial cohort on the basis of Medicare patient characteristics, and a "real-world" treatment effect was estimated. RESULTS A total of 2,026 patients underwent TAVR in the U.S. CoreValve pivotal trials, and 135,112 patients underwent TAVR in the Medicare cohort. Trial patients were mostly similar to real-world patients at baseline, though trial patients were more likely to have hypertension (50% vs 39%) and coagulopathy (25% vs 17%), whereas real-world patients were more likely to have congestive heart failure (75% vs 68%) and frailty. The estimated real-world treatment effect of TAVR was an 11.4% absolute reduction in death or stroke (95% CI: 7.50%-14.92%) and an 8.7% absolute reduction in death (95% CI: 5.20%-12.32%) at 1 year with TAVR compared with conventional therapy (surgical aortic valve replacement for intermediate- and high-risk patients and medical therapy for extreme-risk patients). CONCLUSIONS The trial and real-world populations were mostly similar, with some notable differences. Nevertheless, the extrapolated real-world treatment effect was at least as high as the observed trial treatment effect, suggesting that the absolute benefit of TAVR in clinical trials is similar to the benefit of TAVR in the U.S. real-world setting.
Collapse
Affiliation(s)
- Neel M Butala
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA; Cardiology Division, Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Eric Secemsky
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Dhruv S Kazi
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Yang Song
- Baim Institute for Clinical Research, Boston, Massachusetts, USA
| | - Jordan B Strom
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Kamil F Faridi
- Section of Cardiology, Department of Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| | - J Matthew Brennan
- Duke Clinical Research Institute, Duke University School of Medicine, Durham, North Carolina, USA
| | - Sammy Elmariah
- Cardiology Division, Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Changyu Shen
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Robert W Yeh
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA.
| |
Collapse
|
9
|
Sampayo-Cordero M, Miguel-Huguet B, Malfettone A, Pérez-García JM, Llombart-Cussac A, Cortés J, Pardo A, Pérez-López J. The Impact of Excluding Nonrandomized Studies From Systematic Reviews in Rare Diseases: "The Example of Meta-Analyses Evaluating the Efficacy and Safety of Enzyme Replacement Therapy in Patients With Mucopolysaccharidosis". Front Mol Biosci 2021; 8:690615. [PMID: 34239895 PMCID: PMC8257960 DOI: 10.3389/fmolb.2021.690615] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2021] [Accepted: 05/24/2021] [Indexed: 12/01/2022] Open
Abstract
Nonrandomized studies are usually excluded from systematic reviews. This could lead to loss of a considerable amount of information on rare diseases. In this article, we explore the impact of excluding nonrandomized studies on the generalizability of meta-analyses results on mucopolysaccharidosis (MPS) disease. A comprehensive search of systematic reviews on MPS patients up to May 2020 was carried out (CRD42020191217). The primary endpoint was the rate of patients excluded from systematic reviews if only randomized studies were considered. Secondary outcomes included the differences in patient and study characteristics between randomized and nonrandomized studies, the methods used to combine data from studies with different designs, and the number of patients excluded from systematic reviews if case reports were not considered. More than 50% of the patients analyzed have been recruited in nonrandomized studies. Patient characteristics, duration of follow-up, and the clinical outcomes evaluated differ between the randomized and nonrandomized studies. There are feasible strategies to combine the data from different randomized and nonrandomized designs. The analyses suggest the relevance of including case reports in the systematic reviews, since the smaller the number of patients in the reference population, the larger the selection bias associated to excluding case reports. Our results recommend including nonrandomized studies in the systematic reviews of MPS to increase the representativeness of the results and to avoid a selection bias. The recommendations obtained from this study should be considered when conducting systematic reviews on rare diseases.
Collapse
Affiliation(s)
| | | | | | - José Manuel Pérez-García
- Medica Scientia Innovation Research (MedSIR), Barcelona, Spain
- IOB Institute of Oncology, Quiron Salud Group, Madrid, Spain
| | - Antonio Llombart-Cussac
- Medica Scientia Innovation Research (MedSIR), Barcelona, Spain
- Hospital Arnau de Vilanova, Universidad Católica de Valencia San Vicente Mártir, Valencia, Spain
| | - Javier Cortés
- Medica Scientia Innovation Research (MedSIR), Barcelona, Spain
- IOB Institute of Oncology, Quiron Salud Group, Madrid, Spain
- Vall d´Hebron Institute of Oncology (VHIO), Barcelona, Spain
| | - Almudena Pardo
- Albiotech Consultores y Redacción Científica S.L., Madrid, Spain
| | - Jordi Pérez-López
- Department of Internal Medicine, Hospital Vall d’Hebron, Barcelona, Spain
| |
Collapse
|
10
|
Lesko CR, Ackerman B, Webster-Clark M, Edwards JK. Target validity: Bringing treatment of external validity in line with internal validity. CURR EPIDEMIOL REP 2021; 7:117-124. [PMID: 33585162 DOI: 10.1007/s40471-020-00239-0] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
Abstract
Purpose of Review "Target bias" is the difference between an estimate of association from a study sample and the causal effect in the target population of interest. It is the sum of internal and external bias. Given the extensive literature on internal validity, here, we review threats and methods to improve external validity. Recent findings External bias may arise when the distribution of modifiers of the effect of treatment differs between the study sample and the target population. Methods including those based on modeling the outcome, modeling sample membership, and doubly robust methods are available, assuming data on the target population is available. Summary The relevance of information for making policy decisions is dependent on both the actions that were studied and the sample in which they were evaluated. Combining methods for addressing internal and external validity can improve the policy relevance of study results.
Collapse
Affiliation(s)
- Catherine R Lesko
- Department of Epidemiology, Johns Hopkins School of Public Health, Baltimore, MD
| | - Benjamin Ackerman
- Department of Biostatistics, Johns Hopkins School of Public Health, Baltimore, MD
| | | | - Jessie K Edwards
- Department of Epidemiology, University of North Carolina, Chapel Hill, NC
| |
Collapse
|
11
|
Lund JL, Webster-Clark MA, Hinton SP, Shmuel S, Stürmer T, Sanoff HK. Effectiveness of adjuvant FOLFOX vs 5FU/LV in adults over age 65 with stage II and III colon cancer using a novel hybrid approach. Pharmacoepidemiol Drug Saf 2020; 29:1579-1587. [PMID: 33015888 DOI: 10.1002/pds.5148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2020] [Revised: 09/04/2020] [Accepted: 09/30/2020] [Indexed: 11/08/2022]
Abstract
PURPOSE Estimates of cancer therapy effects can differ in clinical trials and clinical practice, partly due to underrepresentation of certain patient subgroups in trials. We utilize a hybrid approach, combining clinical trial and real-world data, to estimate the comparative effectiveness of two adjuvant chemotherapy regimens for colon cancer. METHODS We identified patients aged 66 and older enrolled in the Multicenter International Study of Oxaliplatin/5FU-LV in the Adjuvant Treatment of Colon Cancer. Similar patients were identified in the Surveillance, Epidemiology, and End Results (SEER)-Medicare database, initiating adjuvant chemotherapy with either 5-fluorouracil (5FU) alone or in combination with oxaliplatin (FOLFOX). We used logistic regression to estimate the likelihood of trial enrollment as a function of age, sex, and substage. Using inverse odds of sampling weights (IOSW), we compared 5-year mortality in patients randomized to FOLFOX vs 5FU using weighted Cox proportional hazards regression, the Nelson-Aalen estimator for cumulative hazards, and bootstrapping for 95% confidence intervals (CIs). RESULTS There were 690 trial participants and 3834 SEER-Medicare patients. The SEER-Medicare population was older and had a higher proportion of stage IIIB and IIIC patients than the trial. After controlling for differences between populations, the IOSW 5-year HR was 1.21 (0.89, 1.65), slightly farther from the null than the trial estimate (HR = 1.14, 95%CI: 0.87, 1.49). CONCLUSIONS This study supports mounting evidence of little to no incremental reduction in 5-year mortality for FOLFOX vs 5FU in older adults with stage II-III colon cancer, emphasizing the importance of combining clinical trial and real-world data to support such conclusions.
Collapse
Affiliation(s)
- Jennifer L Lund
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA.,Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Michael A Webster-Clark
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Sharon Peacock Hinton
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Shahar Shmuel
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Til Stürmer
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA.,Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Hanna K Sanoff
- Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA.,Division of Hematology/Oncology, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| |
Collapse
|
12
|
Happich M, Brnabic A, Faries D, Abrams K, Winfree KB, Girvan A, Jonsson P, Johnston J, Belger M. Reweighting Randomized Controlled Trial Evidence to Better Reflect Real Life - A Case Study of the Innovative Medicines Initiative. Clin Pharmacol Ther 2020; 108:817-825. [PMID: 32301116 PMCID: PMC7540324 DOI: 10.1002/cpt.1854] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2019] [Accepted: 03/31/2020] [Indexed: 01/25/2023]
Abstract
Evidence from randomized controlled trials available for timely health technology assessments of new pharmacological treatments and regulatory decision making may not be generalizable to local patient populations, often resulting in decisions being made under uncertainty. In recent years, several reweighting approaches have been explored to address this important question of generalizability to a target population. We present a case study of the Innovative Medicines Initiative to illustrate the inverse propensity score reweighting methodology, which may allow us to estimate the expected treatment benefit if a clinical trial had been run in a broader real‐world target population. We learned that identifying treatment effect modifiers, understanding and managing differences between patient characteristic data sets, and balancing the closeness of trial and target patient populations with effective sample size are key to successfully using this methodology and potentially mitigating some of this uncertainty around local decision making.
Collapse
Affiliation(s)
| | - Alan Brnabic
- Eli Lilly and Company, Sydney, New South Wales, Australia
| | - Douglas Faries
- Lilly Corporate Center, Eli Lilly and Company, Indianapolis, Indiana, USA
| | - Keith Abrams
- Department of Health Sciences, University of Leicester, Leicester, UK
| | | | - Allicia Girvan
- Lilly Corporate Center, Eli Lilly and Company, Indianapolis, Indiana, USA
| | - Pall Jonsson
- National Institute for Health and Care Excellence (NICE), Manchester, UK
| | - Joseph Johnston
- Lilly Corporate Center, Eli Lilly and Company, Indianapolis, Indiana, USA
| | - Mark Belger
- Lilly Research Centre, Eli Lilly and Company, Surrey, UK
| | | |
Collapse
|
13
|
Sciannameo V, Berchialla P, Orsi E, Lamacchia O, Morano S, Querci F, Consoli A, Avogaro A, Fadini GP. Enrolment criteria for diabetes cardiovascular outcome trials do not inform on generalizability to clinical practice: The case of glucagon-like peptide-1 receptor agonists. Diabetes Obes Metab 2020; 22:817-827. [PMID: 31943710 DOI: 10.1111/dom.13962] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/15/2019] [Revised: 12/26/2019] [Accepted: 01/08/2020] [Indexed: 02/06/2023]
Abstract
AIM To evaluate the generalizability of cardiovascular outcome trials (CVOTs) on glucagon-like peptide-1 receptor agonists (GLP-1RAs), we assessed what proportion of real-world patients with type 2 diabetes (T2D) constitute true CVOT-like populations. MATERIALS AND METHODS We applied inclusion/exclusion (I/E) criteria of each GLP-1RA CVOT to a cross-sectional database of 281 380 T2D patients from Italian diabetes outpatient clinics. We calculated the proportion of patients eligible for each CVOT and compared their clinical characteristics with those of trial patients. In addition, we used a Bayesian network-based method to sample the greatest subsets of real-world patients yielding true CVOT-like populations. RESULTS Between 98 725 and 124 164 T2D patients could be evaluated for CVOT eligibility. After excluding patients who were already on GLP-1RAs and applying I/E criteria, 35.8% of patients would be eligible for REWIND, 34.1% for PIONEER-6, 13.4% for EXSCEL, 10.1% for SUSTAIN-6, 9.5% for HARMONY and 9.4% for LEADER. Overall, 45.4% of patients could be eligible for at least one of the CVOTs. These patients, however, were extremely different to trial patients in most of the clinical characteristics, including demographics, concomitant medications and complications. The greatest CVOT-like subsets of real-world patients were 0.5% for SUSTAIN-6, 1.0% for EXSCEL, 1.2% for LEADER, 1.8% for PIONEER-6 and 7.9% for REWIND. CONCLUSIONS A very small proportion of real-world patients constitute true CVOT-like populations. These findings question whether any meaningful information can be drawn from applying trial enrolment criteria to real-world T2D patients.
Collapse
Affiliation(s)
- Veronica Sciannameo
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, Padova, Italy
| | - Paola Berchialla
- Department of Clinical and Biological Sciences, University of Turin, Turin, Italy
| | - Emanuela Orsi
- Unit of Endocrinology and Metabolic Diseases, Fondazione IRCCS Ca' Granda-Ospedale Maggiore Policlinico di Milano, Milan, Italy
| | - Olga Lamacchia
- Unit of Endocrinology, Department of Medical and Surgical Sciences, University Hospital of Foggia, Foggia, Italy
| | - Susanna Morano
- Unit of Diabetes Complications, V Clinica Medica, Department of Experimental Medicine, University of Rome "La Sapienza", Rome, Italy
| | - Fabrizio Querci
- Unit of Diabetology, ASST Bergamo Est, Alzano Lombardo, Italy
| | - Agostino Consoli
- Department of Medicine and Aging Science, "G. D'Annunzio" University of Chieti, Chieti, Italy
| | - Angelo Avogaro
- Department of Medicine, University of Padova, Padova, Italy
| | | |
Collapse
|
14
|
He Z, Tang X, Yang X, Guo Y, George TJ, Charness N, Quan Hem KB, Hogan W, Bian J. Clinical Trial Generalizability Assessment in the Big Data Era: A Review. Clin Transl Sci 2020; 13:675-684. [PMID: 32058639 PMCID: PMC7359942 DOI: 10.1111/cts.12764] [Citation(s) in RCA: 71] [Impact Index Per Article: 14.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2019] [Accepted: 01/25/2020] [Indexed: 01/04/2023] Open
Abstract
Clinical studies, especially randomized, controlled trials, are essential for generating evidence for clinical practice. However, generalizability is a long‐standing concern when applying trial results to real‐world patients. Generalizability assessment is thus important, nevertheless, not consistently practiced. We performed a systematic review to understand the practice of generalizability assessment. We identified 187 relevant articles and systematically organized these studies in a taxonomy with three dimensions: (i) data availability (i.e., before or after trial (a priori vs. a posteriori generalizability)); (ii) result outputs (i.e., score vs. nonscore); and (iii) populations of interest. We further reported disease areas, underrepresented subgroups, and types of data used to profile target populations. We observed an increasing trend of generalizability assessments, but < 30% of studies reported positive generalizability results. As a priori generalizability can be assessed using only study design information (primarily eligibility criteria), it gives investigators a golden opportunity to adjust the study design before the trial starts. Nevertheless, < 40% of the studies in our review assessed a priori generalizability. With the wide adoption of electronic health records systems, rich real‐world patient databases are increasingly available for generalizability assessment; however, informatics tools are lacking to support the adoption of generalizability assessment practice.
Collapse
Affiliation(s)
- Zhe He
- School of Information, Florida State University, Tallahassee, Florida, USA
| | - Xiang Tang
- Department of Statistics, Florida State University, Tallahassee, Florida, USA
| | - Xi Yang
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Yi Guo
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Thomas J George
- Hematology & Oncology, Department of Medicine, College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Neil Charness
- Department of Psychology, Florida State University, Tallahassee, Florida, USA
| | - Kelsa Bartley Quan Hem
- Calder Memorial Library, Miller School of Medicine, University of Miami, Miami, Florida, USA
| | - William Hogan
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Jiang Bian
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
| |
Collapse
|
15
|
Yiu Z, Mason K, Barker J, Hampton P, McElhone K, Smith C, Warren R, Griffiths C, Lunt M, Burden A. A standardization approach to compare treatment safety and effectiveness outcomes between clinical trials and real-world populations in psoriasis. Br J Dermatol 2019; 181:1265-1271. [PMID: 30822358 PMCID: PMC6916305 DOI: 10.1111/bjd.17849] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/28/2019] [Indexed: 12/23/2022]
Abstract
BACKGROUND Patients recruited in randomized controlled trials (RCTs) for biologic therapies in psoriasis are not fully representative of the real-world psoriasis population. OBJECTIVES Firstly, to investigate whether patient characteristics are associated with being included in a psoriasis RCT. Secondly, to estimate the differences in the incidence of severe adverse events (SAEs) and the response rate between RCT and real-world populations of patients on biologic therapies for psoriasis using a standardization method. METHODS Data from the British Association of Dermatologists Biologics and Immunomodulators Register (BADBIR) were appended to individual participant-level data from two RCTs assessing ustekinumab in patients with psoriasis. Baseline variables were assessed for association of being in an RCT using a multivariable logistic regression model. Propensity score weights were derived to reweigh the registry population so that variables had the distribution of the trial population. We measured the C-statistic of the model with trial status as the dependent variable, and the risk differences in the incidence rate of SAEs in the first year and Psoriasis Area and Severity Index (PASI) after 6 months in the BADBIR cohort before and after weighting. RESULTS In total 6790 registry and 2021 RCT participants were included. The multivariable logistic regression model had a C-statistic of 0.82 [95% confidence interval (CI) 0.81-0.83]. The risk differences for the incidence rate of SAEs and the proportion of patients with PASI < 1.5 were 9.27 (95% CI -3.91-22.5) per 1000 person-years and 0.95 (95% CI -1.98-4.15), respectively. CONCLUSIONS Our results suggest that RCTs of biologic therapies in patients with psoriasis are not fully representative of the real-world population, but this lack of external validity does not account for the efficacy-effectiveness gap. What's already known about this topic? Patients with psoriasis who would not be eligible for randomized controlled trials (RCTs) investigating biologic therapies have a greater risk of serious adverse events and lower treatment effectiveness than patients who would have been eligible. What does this study add? Baseline patient characteristics were shown to be predictive of whether a patient would have been eligible for enrolment in an RCT for psoriasis biologic therapy. We did not find any efficacy-effectiveness gap between the sample representative of the real-world population of patients with psoriasis and the sample representative of the RCT population. Factors outside of baseline patient characteristics, such as observer effect and higher adherence in RCTs, may be more influential in any efficacy-effectiveness gap between trial and real-world populations of patients with psoriasis.
Collapse
Affiliation(s)
- Z.Z.N. Yiu
- Dermatology CentreSalford Royal NHS Foundation TrustThe University of ManchesterManchester Academic Health Science CentreNIHR Manchester Biomedical Research CentreManchesterM13 9PTU.K
| | - K.J. Mason
- Dermatology CentreSalford Royal NHS Foundation TrustThe University of ManchesterManchester Academic Health Science CentreNIHR Manchester Biomedical Research CentreManchesterM13 9PTU.K
| | - J.N.W.N. Barker
- St John's Institute of DermatologyGuy's and St Thomas’ NHS Foundation TrustLondonSE1 9RTU.K
| | - P.J. Hampton
- Dermatological Sciences, Institute of Cellular Medicine, Medical SchoolNewcastle University, and Department of Dermatology, Royal Victoria Infirmary, Newcastle Hospitals NHS Foundation TrustNewcastle upon TyneNE2 4HHU.K
| | - K. McElhone
- Arthritis Research U.K. Epidemiology UnitThe University of ManchesterManchesterM13 9PTU.K
| | - C.H. Smith
- St John's Institute of DermatologyGuy's and St Thomas’ NHS Foundation TrustLondonSE1 9RTU.K
| | - R.B. Warren
- Dermatology CentreSalford Royal NHS Foundation TrustThe University of ManchesterManchester Academic Health Science CentreNIHR Manchester Biomedical Research CentreManchesterM13 9PTU.K
| | - C.E.M. Griffiths
- Dermatology CentreSalford Royal NHS Foundation TrustThe University of ManchesterManchester Academic Health Science CentreNIHR Manchester Biomedical Research CentreManchesterM13 9PTU.K
| | - M. Lunt
- Arthritis Research U.K. Epidemiology UnitThe University of ManchesterManchesterM13 9PTU.K
| | - A.D. Burden
- Institute of Infection, Immunity and InflammationUniversity of GlasgowGlasgowG12 8TAU.K
| | | |
Collapse
|
16
|
Lodi S, Phillips A, Lundgren J, Logan R, Sharma S, Cole SR, Babiker A, Law M, Chu H, Byrne D, Horban A, Sterne JAC, Porter K, Sabin C, Costagliola D, Abgrall S, Gill J, Touloumi G, Pacheco AG, van Sighem A, Reiss P, Bucher HC, Montoliu Giménez A, Jarrin I, Wittkop L, Meyer L, Perez-Hoyos S, Justice A, Neaton JD, Hernán MA. Effect Estimates in Randomized Trials and Observational Studies: Comparing Apples With Apples. Am J Epidemiol 2019; 188:1569-1577. [PMID: 31063192 DOI: 10.1093/aje/kwz100] [Citation(s) in RCA: 72] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2018] [Accepted: 04/17/2019] [Indexed: 12/25/2022] Open
Abstract
Effect estimates from randomized trials and observational studies might not be directly comparable because of differences in study design, other than randomization, and in data analysis. We propose a 3-step procedure to facilitate meaningful comparisons of effect estimates from randomized trials and observational studies: 1) harmonization of the study protocols (eligibility criteria, treatment strategies, outcome, start and end of follow-up, causal contrast) so that the studies target the same causal effect, 2) harmonization of the data analysis to estimate the causal effect, and 3) sensitivity analyses to investigate the impact of discrepancies that could not be accounted for in the harmonization process. To illustrate our approach, we compared estimates of the effect of immediate with deferred initiation of antiretroviral therapy in individuals positive for the human immunodeficiency virus from the Strategic Timing of Antiretroviral Therapy (START) randomized trial and the observational HIV-CAUSAL Collaboration.
Collapse
Affiliation(s)
- Sara Lodi
- Department of Biostatistics, Boston University School of Public Health, Boston, Massachusetts
| | - Andrew Phillips
- Institute for Global Health, University College London, United Kingdom
| | - Jens Lundgren
- Department of Infectious Diseases, Rigshospitalet, University of Copenhagen, Denmark
| | - Roger Logan
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
| | - Shweta Sharma
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota
| | | | - Abdel Babiker
- Medical Research Council, Clinical Trials Unit in University College London, London, United Kingdom
| | | | - Haitao Chu
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota
| | - Dana Byrne
- Division of Infectious Diseases, Department of Medicine, Cooper University Hospital, Cooper Medical School at Rowan University, New Jersey
| | - Andrzej Horban
- Medical University of Warsaw, Department for Adult's Infectious Diseases, Warsaw, Poland
| | - Jonathan A C Sterne
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, North Carolina
- Department of Population Health Sciences, University of Bristol, Bristol, United Kingdom
| | - Kholoud Porter
- Institute for Global Health, University College London, United Kingdom
| | - Caroline Sabin
- Institute for Global Health, University College London, United Kingdom
| | - Dominique Costagliola
- INSERM, Sorbonne Université, Institut Pierre Louis d’Épidémiologie et de Santé Publique (IPLESP), Paris, France
| | - Sophie Abgrall
- INSERM, Sorbonne Université, Institut Pierre Louis d’Épidémiologie et de Santé Publique (IPLESP), Paris, France
- AP-HP, Hôpital Antoine Béclère, Service de Médecine Interne, Clamart, France
| | - John Gill
- Southern Alberta Clinic, Calgary, Canada
- Department of Medicine, University of Calgary, Canada
| | - Giota Touloumi
- National and Kapodistrian University of Athens, Faculty of Medicine, Dept. of Hygiene, Epidemiology and Medical Statistics, Greece
| | - Antonio G Pacheco
- Programa de Computação Científica, Fundacao Oswaldo Cruz, Rio de Janeiro, Brasil
| | | | - Peter Reiss
- Stichting HIV Monitoring, Amsterdam, the Netherlands
- Amsterdam University Medical Centres, University of Amsterdam, Department of Global Health and Division of Infectious Diseases, Amsterdam, the Netherlands
- Amsterdam Institute for Global Health and Development, and Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
| | - Heiner C Bucher
- Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Switzerland
| | - Alexandra Montoliu Giménez
- Centre for Epidemiological Studies on HIV/STI in Catalonia (CEEISCAT), Agència de Salut Pública de Catalunya (ASPC), Badalona, Spain
| | - Inmaculada Jarrin
- Centro Nacional de Epidemiología, Instituto de Salud Carlos III, Madrid, Spain
| | - Linda Wittkop
- Univ. Bordeaux, ISPED, Inserm, Bordeaux Population Health Research Center, team MORPH3EUS, UMR 1219, CIC-EC 1401, Bordeaux, France
| | - Laurence Meyer
- CHU de Bordeaux, Pôle de santé publique, Service d'information médicale, Bordeaux, France
- Université Paris Sud, UMR 1018, le Kremlin Bicêtre, France
| | | | - Amy Justice
- Yale University School of Medicine, New Haven, Connecticut
| | - James D Neaton
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota
| | - Miguel A Hernán
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
- Department of Biostatistics, Harvard T.H. Chan School of Public Health
- Harvard-MIT Division of Health Sciences and Technology, Boston, Massachusetts
| | | |
Collapse
|
17
|
Goldstein BA, Phelan M, Pagidipati NJ, Holman RR, Pencina MJ, Stuart EA. An outcome model approach to transporting a randomized controlled trial results to a target population. J Am Med Inform Assoc 2019; 26:429-437. [PMID: 30869798 DOI: 10.1093/jamia/ocy188] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2018] [Revised: 11/12/2018] [Accepted: 12/19/2018] [Indexed: 02/01/2023] Open
Abstract
OBJECTIVE Participants enrolled into randomized controlled trials (RCTs) often do not reflect real-world populations. Previous research in how best to transport RCT results to target populations has focused on weighting RCT data to look like the target data. Simulation work, however, has suggested that an outcome model approach may be preferable. Here, we describe such an approach using source data from the 2 × 2 factorial NAVIGATOR (Nateglinide And Valsartan in Impaired Glucose Tolerance Outcomes Research) trial, which evaluated the impact of valsartan and nateglinide on cardiovascular outcomes and new-onset diabetes in a prediabetic population. MATERIALS AND METHODS Our target data consisted of people with prediabetes serviced at the Duke University Health System. We used random survival forests to develop separate outcome models for each of the 4 treatments, estimating the 5-year risk difference for progression to diabetes, and estimated the treatment effect in our local patient populations, as well as subpopulations, and compared the results with the traditional weighting approach. RESULTS Our models suggested that the treatment effect for valsartan in our patient population was the same as in the trial, whereas for nateglinide treatment effect was stronger than observed in the original trial. Our effect estimates were more efficient than the weighting approach and we effectively estimated subgroup differences. CONCLUSIONS The described method represents a straightforward approach to efficiently transporting an RCT result to any target population.
Collapse
Affiliation(s)
- Benjamin A Goldstein
- Department of Biostatistics & Bioinformatics, Duke University, Durham, North Carolina, USA.,Center for Predictive Medicine, Duke Clinical Research Institute, Durham, North Carolina, USA
| | - Matthew Phelan
- Center for Predictive Medicine, Duke Clinical Research Institute, Durham, North Carolina, USA
| | - Neha J Pagidipati
- Center for Predictive Medicine, Duke Clinical Research Institute, Durham, North Carolina, USA.,Department of Medicine, Duke Clinical Research Institute, Center for Predictive Medicine, Duke University, Durham, North Carolina, USA
| | - Rury R Holman
- Endocrinology and Metabolism, University of Oxford, Oxford, United Kingdom
| | - Michael J Pencina
- Department of Biostatistics & Bioinformatics, Duke University, Durham, North Carolina, USA.,Center for Predictive Medicine, Duke Clinical Research Institute, Durham, North Carolina, USA
| | - Elizabeth A Stuart
- Department of Biostatistics John Hopkins University, Baltimore, Maryland, USA.,Department of Mental Health, John Hopkins University, Baltimore, Maryland, USA
| |
Collapse
|
18
|
Hong JL, Webster-Clark M, Jonsson Funk M, Stürmer T, Dempster SE, Cole SR, Herr I, LoCasale R. Comparison of Methods to Generalize Randomized Clinical Trial Results Without Individual-Level Data for the Target Population. Am J Epidemiol 2019; 188:426-437. [PMID: 30312378 DOI: 10.1093/aje/kwy233] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2018] [Accepted: 10/05/2018] [Indexed: 01/24/2023] Open
Abstract
Our study explored the application of methods to generalize randomized controlled trial results to a target population without individual-level data. We compared 4 methods using aggregate data for the target population to generalize results from the international trial, Justification for the Use of Statins in Prevention: an Intervention Trial Evaluating Rosuvastatin (JUPITER), to a target population of trial-eligible patients in the UK Clinical Practice Research Datalink (CPRD). The gold-standard method used individual data from both the trial and CPRD to predict probabilities of being sampled in the trial and to reweight trial participants to reflect CPRD patient characteristics. Methods 1 and 2 used weighting methods based on simulated individual data or the method of moments, respectively. Method 3 weighted the trial's subgroup-specific treatment effects to match the distribution of an effect modifier in CPRD. Method 4 calculated the expected absolute benefits in CPRD assuming homogeneous relative treatment effect. Methods based on aggregate data for the target population generally yielded results between the trial and gold-standard estimates. Methods 1 and 2 yielded estimates closest to the gold-standard estimates when continuous effect modifiers were represented as categorical variables. Although individual data or data on joint distributions remains the best approach to generalize trial results, these methods using aggregate data might be useful tools for timely assessment of randomized trial generalizability.
Collapse
Affiliation(s)
- Jin-Liern Hong
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Michael Webster-Clark
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Michele Jonsson Funk
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Til Stürmer
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | | | - Stephen R Cole
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Iksha Herr
- Medical Evidence and Observational Research, AstraZeneca, Gaithersburg, Maryland
| | - Robert LoCasale
- Medical Evidence and Observational Research, AstraZeneca, Gaithersburg, Maryland
| |
Collapse
|
19
|
Webster-Clark MA, Sanoff HK, Stürmer T, Peacock Hinton S, Lund JL. Diagnostic Assessment of Assumptions for External Validity: An Example Using Data in Metastatic Colorectal Cancer. Epidemiology 2019; 30:103-111. [PMID: 30252687 PMCID: PMC6269648 DOI: 10.1097/ede.0000000000000926] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
BACKGROUND Methods developed to estimate intervention effects in external target populations assume that all important effect measure modifiers have been identified and appropriately modeled. Propensity score-based diagnostics can be used to assess the plausibility of these assumptions for weighting methods. METHODS We demonstrate the use of these diagnostics when assessing the transportability of treatment effects from the standard of care for metastatic colorectal cancer control arm in a phase III trial (HORIZON III) to a target population of 1,942 Medicare beneficiaries age 65+ years. RESULTS In an unadjusted comparison, control arm participants had lower mortality compared with target population patients treated with the standard of care therapy (trial vs. target hazard ratio [HR] = 0.72, 95% confidence interval [CI], 0.58, 0.89). Applying inverse odds of sampling weights attenuated the trial versus target HR (weighted HR = 0.96, 95% CI = 0.73, 1.26). However, whether unadjusted or weighted, hazards did not appear proportional. At 6 months of follow-up, mortality was lower in the weighted trial population than the target population (weighted trial vs. target risk difference [RD] = -0.07, 95% CI = -0.13, -0.01), but not at 12 months (weighted RD = 0.00, 95% CI = -0.09, 0.09). CONCLUSION These diagnostics suggest that direct transport of treatment effects from HORIZON III to the Medicare population is not valid. However, the proposed sampling model might allow valid transport of the treatment effects on longer-term mortality from HORIZON III to the Medicare population treated in clinical practice. See video abstract at, http://links.lww.com/EDE/B435.
Collapse
Affiliation(s)
| | - Hanna K Sanoff
- Department of Medicine, University of North Carolina, Chapel Hill, NC
| | - Til Stürmer
- From the Department of Epidemiology, University of North Carolina, Chapel Hill, NC
| | | | - Jennifer L Lund
- From the Department of Epidemiology, University of North Carolina, Chapel Hill, NC
| |
Collapse
|
20
|
Whitlock EL, Diaz-Ramirez LG, Smith AK, Boscardin WJ, Avidan MS, Glymour MM. Cognitive Change After Cardiac Surgery Versus Cardiac Catheterization: A Population-Based Study. Ann Thorac Surg 2018; 107:1119-1125. [PMID: 30578068 DOI: 10.1016/j.athoracsur.2018.10.021] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/11/2018] [Revised: 09/24/2018] [Accepted: 10/09/2018] [Indexed: 11/29/2022]
Abstract
BACKGROUND Despite concern that cardiac surgery may adversely affect cognition, little evidence is available from population-based studies using presurgery data. With the use of the Health and Retirement Study, we compared memory change after participant-reported cardiac catheterization or cardiac surgery. METHODS Participants were community-dwelling adults aged 65 years and older who self-reported cardiac catheterization or "heart surgery" at any biennial Health and Retirement Study interview between 2000 and 2014. Participants may have undergone the index procedure any time in the preceding 2 years. We modeled preprocedure to postprocedure change in composite memory score, derived from objective memory testing, using linear mixed effects models. We modeled postprocedure subjective memory decline with logistic regression. To quantify clinical relevance, we used the predicted memory change to estimate impact on ability to manage medications and finances independently. RESULTS Of 3,105 participants, 1,921 (62%) underwent catheterization and 1,184 (38%) underwent operation. In adjusted analyses, surgery participants had little difference in preprocedure to postprocedure memory change compared with participants undergoing cardiac catheterization (-0.021 memory units; 95% confidence interval: -0.046 to 0.005 memory units, p = 0.12). If the relationship were causal, the point estimate for memory decline would confer an absolute 0.26% or 0.19% decrease in ability to manage finances or medications, respectively, corresponding to 4.6 additional months of cognitive aging. Cardiac surgery was not associated with subjective memory decline (adjusted odds ratio 0.93, 95% confidence interval: 0.74 to 1.18). CONCLUSIONS In this large, population-based cohort, memory declines after heart surgery and cardiac catheterization were similar. These findings suggest intermediate-term population-level adverse cognitive effects of cardiac surgery, if any, are likely subtle.
Collapse
Affiliation(s)
- Elizabeth L Whitlock
- Department of Anesthesia and Perioperative Care, University of California San Francisco, San Francisco, California.
| | - L Grisell Diaz-Ramirez
- Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, California
| | - Alexander K Smith
- Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, California
| | - W John Boscardin
- Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, California; Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, California
| | - Michael S Avidan
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, Missouri
| | - M Maria Glymour
- Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, California
| |
Collapse
|
21
|
Barry EL, Lund JL, Westreich D, Mott LA, Ahnen DJ, Beck GJ, Bostick RM, Bresalier RS, Burke CA, Church TR, Rees JR, Robertson DJ, Baron JA. Body mass index, calcium supplementation and risk of colorectal adenomas. Int J Cancer 2018; 144:448-458. [PMID: 30117164 DOI: 10.1002/ijc.31803] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2018] [Revised: 07/25/2018] [Accepted: 07/27/2018] [Indexed: 12/20/2022]
Abstract
Calcium supplementation (1,200 mg/day) did not significantly reduce colorectal adenomas in our recent randomized, controlled trial (Vitamin D/Calcium Polyp Prevention Study, VCPPS, 2004-2013) in contrast to our previous trial (Calcium Polyp Prevention Study, CPPS, 1988-1996). To reconcile these findings, we identified participant characteristics that differed between the study populations and modified the effect of calcium supplementation on adenomas or high-risk findings (advanced or multiple adenomas). Compared to the CPPS, more participants in the VCPPS were obese (body mass index (BMI) ≥30 kg/m2 ; 37.5% vs. 24.4%) and fewer had normal BMI (BMI <25 kg/m2 ; 18.5% vs. 31%). BMI appeared to modify the effect of calcium supplementation on adenomas and especially on high risk-findings: in the VCPPS, there was a 44% reduction in high-risk findings among individuals whose BMI was normal (RR = 0.56, 95% CI = 0.26-1.23), but not among overweight (RR = 1.09, 95% CI = 0.62-1.91) or obese (RR = 1.54, 95% CI = 0.92-2.57) individuals (pinteraction = 0.03). Similarly, in the CPPS, there was a 56% reduction in high-risk findings among individuals whose BMI was normal (RR = 0.44, 95% CI = 0.26-0.74), but not among overweight (RR = 0.87, 95% CI = 0.55-1.39) or obese (RR = 1.02, 95% CI = 0.57-1.82) individuals (pinteraction = 0.02). Standardization of each trial's findings to the BMI distribution in the other attenuated calcium's protective effect on adenomas in the CPPS but enhanced it in the VCPPS. In conclusion, 1,200 mg/day calcium supplementation may reduce risk of colorectal adenomas among those with normal BMI but not in overweight or obese individuals; and differences in BMI distribution partially account for the apparent difference in calcium efficacy between the two trials.
Collapse
Affiliation(s)
- Elizabeth L Barry
- Department of Epidemiology, Geisel School of Medicine at Dartmouth, Lebanon, NH
| | - Jennifer L Lund
- Department of Epidemiology, University of North Carolina at Chapel Hill, Gillings School of Public Health, Chapel Hill, NC
| | - Daniel Westreich
- Department of Epidemiology, University of North Carolina at Chapel Hill, Gillings School of Public Health, Chapel Hill, NC
| | - Leila A Mott
- Department of Epidemiology, Geisel School of Medicine at Dartmouth, Lebanon, NH
| | - Dennis J Ahnen
- Division of Gastroenterology and Hepatology, University of Colorado School of Medicine, Denver, CO
| | - Gerald J Beck
- Department of Quantitative Health Sciences, Cleveland Clinic, Cleveland, OH
| | - Roberd M Bostick
- Department of Epidemiology, Rollins School of Public Health, Emory University and Winship Cancer Institute, Atlanta, GA
| | - Robert S Bresalier
- Department of Gastroenterology, Hepatology, and Nutrition, University of Texas M.D. Anderson Cancer Center, Houston, TX
| | - Carol A Burke
- Department of Gastroenterology and Hepatology, Cleveland Clinic, Cleveland, OH
| | - Timothy R Church
- Division of Environmental Health Sciences, University of Minnesota School of Public Health, Minneapolis, MN
| | - Judy R Rees
- Department of Epidemiology, Geisel School of Medicine at Dartmouth, Lebanon, NH
| | - Douglas J Robertson
- VA Medical Center, White River Junction, VT and Geisel School of Medicine at Dartmouth, Hanover, NH
| | - John A Baron
- Department of Epidemiology, Geisel School of Medicine at Dartmouth, Lebanon, NH.,Department of Epidemiology, University of North Carolina at Chapel Hill, Gillings School of Public Health, Chapel Hill, NC.,Department of Medicine, University of North Carolina at Chapel Hill, School of Medicine, Chapel Hill, NC
| |
Collapse
|