1
|
Steingrimsson JA, Barker DH, Bie R, Dahabreh IJ. Systematically missing data in causally interpretable meta-analysis. Biostatistics 2024; 25:289-305. [PMID: 36977366 PMCID: PMC11017122 DOI: 10.1093/biostatistics/kxad006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 02/15/2023] [Accepted: 03/13/2023] [Indexed: 03/30/2023] Open
Abstract
Causally interpretable meta-analysis combines information from a collection of randomized controlled trials to estimate treatment effects in a target population in which experimentation may not be possible but from which covariate information can be obtained. In such analyses, a key practical challenge is the presence of systematically missing data when some trials have collected data on one or more baseline covariates, but other trials have not, such that the covariate information is missing for all participants in the latter. In this article, we provide identification results for potential (counterfactual) outcome means and average treatment effects in the target population when covariate data are systematically missing from some of the trials in the meta-analysis. We propose three estimators for the average treatment effect in the target population, examine their asymptotic properties, and show that they have good finite-sample performance in simulation studies. We use the estimators to analyze data from two large lung cancer screening trials and target population data from the National Health and Nutrition Examination Survey (NHANES). To accommodate the complex survey design of the NHANES, we modify the methods to incorporate survey sampling weights and allow for clustering.
Collapse
Affiliation(s)
- Jon A Steingrimsson
- Department of Biostatistics, Brown University, 121 South Main Street, Providence, RI 02903, USA
| | - David H Barker
- Department of Psychiatry, Rhode Island Hospital, Providence, RI 02904, USA
| | - Ruofan Bie
- Department of Biostatistics, Brown University, 121 South Main Street, Providence, RI 02903, USA
| | - Issa J Dahabreh
- Departments of Epidemiology and Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA 02115, USA and CAUSALab, Harvard T.H. Chan School of Public Health, Boston, MA 02115, USA
| |
Collapse
|
2
|
Debray TPA, Simoneau G, Copetti M, Platt RW, Shen C, Pellegrini F, de Moor C. Methods for comparative effectiveness based on time to confirmed disability progression with irregular observations in multiple sclerosis. Stat Methods Med Res 2023; 32:1284-1299. [PMID: 37303120 PMCID: PMC10500950 DOI: 10.1177/09622802231172032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Real-world data sources offer opportunities to compare the effectiveness of treatments in practical clinical settings. However, relevant outcomes are often recorded selectively and collected at irregular measurement times. It is therefore common to convert the available visits to a standardized schedule with equally spaced visits. Although more advanced imputation methods exist, they are not designed to recover longitudinal outcome trajectories and typically assume that missingness is non-informative. We, therefore, propose an extension of multilevel multiple imputation methods to facilitate the analysis of real-world outcome data that is collected at irregular observation times. We illustrate multilevel multiple imputation in a case study evaluating two disease-modifying therapies for multiple sclerosis in terms of time to confirmed disability progression. This survival outcome is derived from repeated measurements of the Expanded Disability Status Scale, which is collected when patients come to the healthcare center for a clinical visit and for which longitudinal trajectories can be estimated. Subsequently, we perform a simulation study to compare the performance of multilevel multiple imputation to commonly used single imputation methods. Results indicate that multilevel multiple imputation leads to less biased treatment effect estimates and improves the coverage of confidence intervals, even when outcomes are missing not at random.
Collapse
Affiliation(s)
- Thomas PA Debray
- Julius Centrum voor Gezondheidswetenschappen en Eerstelijns Geneeskunde, Utrecht, Netherlands
- Smart Data Analysis and Statistics B.V., Utrecht, Netherlands
| | | | | | - Robert W Platt
- Department of Epidemiology, Bioastatistics and Occupational Health, McGill University, Quebec, Canada
| | | | | | | |
Collapse
|
3
|
Seo M, Furukawa TA, Karyotaki E, Efthimiou O. Developing prediction models when there are systematically missing predictors in individual patient data meta-analysis. Res Synth Methods 2023; 14:455-467. [PMID: 36755407 DOI: 10.1002/jrsm.1625] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Revised: 01/23/2023] [Accepted: 01/30/2023] [Indexed: 02/10/2023]
Abstract
Clinical prediction models are widely used in modern clinical practice. Such models are often developed using individual patient data (IPD) from a single study, but often there are IPD available from multiple studies. This allows using meta-analytical methods for developing prediction models, increasing power and precision. Different studies, however, often measure different sets of predictors, which may result to systematically missing predictors, that is, when not all studies collect all predictors of interest. This situation poses challenges in model development. We hereby describe various approaches that can be used to develop prediction models for continuous outcomes in such situations. We compare four approaches: a "restrict predictors" approach, where the model is developed using only predictors measured in all studies; a multiple imputation approach that ignores study-level clustering; a multiple imputation approach that accounts for study-level clustering; and a new approach that develops a prediction model in each study separately using all predictors reported, and then synthesizes all predictions in a multi-study ensemble. We explore in simulations the performance of all approaches under various scenarios. We find that imputation methods and our new method outperform the restrict predictors approach. In several scenarios, our method outperformed imputation methods, especially for few studies, when predictor effects were small, and in case of large heterogeneity. We use a real dataset of 12 trials in psychotherapies for depression to illustrate all methods in practice, and we provide code in R.
Collapse
Affiliation(s)
- Michael Seo
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland.,Graduate School for Health Sciences, University of Bern, Bern, Switzerland
| | - Toshi A Furukawa
- Departments of Health Promotion and Human Behavior and of Clinical Epidemiology, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan
| | - Eirini Karyotaki
- Department of Global Health and Social Medicine, Harvard Medical School, Boston, USA.,Department of Clinical Neuro- and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.,Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| | - Orestis Efthimiou
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland.,Department of Psychiatry, University of Oxford, Oxford, UK
| |
Collapse
|
4
|
Cao Y, Allore H, Vander Wyk B, Gutman R. Review and evaluation of imputation methods for multivariate longitudinal data with mixed-type incomplete variables. Stat Med 2022; 41:5844-5876. [PMID: 36220138 PMCID: PMC9771917 DOI: 10.1002/sim.9592] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Revised: 07/16/2022] [Accepted: 09/23/2022] [Indexed: 12/24/2022]
Abstract
Estimating relationships between multiple incomplete patient measurements requires methods to cope with missing values. Multiple imputation is one approach to address missing data by filling in plausible values for those that are missing. Multiple imputation procedures can be classified into two broad types: joint modeling (JM) and fully conditional specification (FCS). JM fits a multivariate distribution for the entire set of variables, but it may be complex to define and implement. FCS imputes missing data variable-by-variable from a set of conditional distributions. In many studies, FCS is easier to define and implement than JM, but it may be based on incompatible conditional models. Imputation methods based on multilevel modeling show improved operating characteristics when imputing longitudinal data, but they can be computationally intensive, especially when imputing multiple variables simultaneously. We review current MI methods for incomplete longitudinal data and their implementation on widely accessible software. Using simulated data from the National Health and Aging Trends Study, we compare their performance for monotone and intermittent missing data patterns. Our simulations demonstrate that in a longitudinal study with a limited number of repeated observations and time-varying variables, FCS-Standard is a computationally efficient imputation method that is accurate and precise for univariate single-level and multilevel regression models. When the analyses comprise multivariate multilevel models, FCS-LMM-latent is a statistically valid procedure with overall more accurate estimates, but it requires more intensive computations. Imputation methods based on generalized linear multilevel models can lead to biased subject-level variance estimates when the statistical analyses involve hierarchical models.
Collapse
Affiliation(s)
- Yi Cao
- Department of Biostatistics, Brown University, Providence, RI, USA
| | - Heather Allore
- Section of Geriatrics, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, USA
- Department of Biostatistics, Yale School of Medicine, New Haven, CT, USA
| | - Brent Vander Wyk
- Section of Geriatrics, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, USA
| | - Roee Gutman
- Department of Biostatistics, Brown University, Providence, RI, USA
| |
Collapse
|
5
|
Elfadaly FG, Adamson A, Patel J, Potts L, Potts J, Blangiardo M, Thompson J, Minelli C. BIMAM—a tool for imputing variables missing across datasets using a Bayesian imputation and analysis model. Int J Epidemiol 2021. [PMCID: PMC8580266 DOI: 10.1093/ije/dyab177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022] Open
Abstract
Motivation Combination of multiple datasets is routine in modern epidemiology. However, studies may have measured different sets of variables; this is often inefficiently dealt with by excluding studies or dropping variables. Multilevel multiple imputation methods to impute these ‘systematically’ missing data (as opposed to ‘sporadically’ missing data within a study) are available, but problems may arise when many random effects are needed to allow for heterogeneity across studies. We show that the Bayesian IMputation and Analysis Model (BIMAM) implemented in our tool works well in this situation. General features BIMAM performs imputation and analysis simultaneously. It imputes both binary and continuous systematically and sporadically missing data, and analyses binary and continuous outcomes. BIMAM is a user-friendly, freely available tool that does not require knowledge of Bayesian methods. BIMAM is an R Shiny application. It is downloadable to a local machine and it automatically installs the required freely available packages (R packages, including R2MultiBUGS and MultiBUGS). Availability BIMAM is available at [www.alecstudy.org/bimam].
Collapse
Affiliation(s)
- Fadlalla G Elfadaly
- School of Mathematics and Statistics, The Open University, Milton Keynes, UK
| | - Alex Adamson
- National Heart and Lung Institute, Imperial College London, London, UK
| | - Jaymini Patel
- National Heart and Lung Institute, Imperial College London, London, UK
| | - Laura Potts
- Department of Biostatistics & Health Informatics, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK
| | - James Potts
- National Heart and Lung Institute, Imperial College London, London, UK
| | - Marta Blangiardo
- MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK
| | - John Thompson
- Department of Health Sciences, University of Leicester, Leicester, UK
| | - Cosetta Minelli
- National Heart and Lung Institute, Imperial College London, London, UK
| |
Collapse
|
6
|
Hunt NB, Gardarsdottir H, Bazelier MT, Klungel OH, Pajouheshnia R. A systematic review of how missing data are handled and reported in multi-database pharmacoepidemiologic studies. Pharmacoepidemiol Drug Saf 2021; 30:819-826. [PMID: 33834576 PMCID: PMC8252545 DOI: 10.1002/pds.5245] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2020] [Revised: 03/24/2021] [Accepted: 04/05/2021] [Indexed: 01/24/2023]
Abstract
Purpose Pharmacoepidemiologic multi‐database studies (MDBS) provide opportunities to better evaluate the safety and effectiveness of medicines. However, the issue of missing data is often exacerbated in MDBS, potentially resulting in bias and precision loss. We sought to measure how missing data are being recorded and addressed in pharmacoepidemiologic MDBS. Methods We conducted a systematic literature search in PubMed for pharmacoepidemiologic MDBS published between 1st January 2018 and 31st December 2019. Included studies were those that used ≥2 distinct databases to assess the same safety/effectiveness outcome associated with a drug exposure. Outcome variables extracted from the studies included strategies to execute a MDBS, reporting of missing data (type, bias evaluation) and the methods used to account for missing data. Results Two thousand seven hundred and twenty‐six articles were identified, and 62 studies were included: using data from either North America (56%), Europe (31%), multiple regions (11%) or East‐Asia (2%). Thirty‐five (56%) articles reported missing data: 11 of these studies reported that this could have introduced bias and 19 studies reported a method to address missing data. Thirteen (68%) carried out a complete case analysis, 2 (11%) applied multiple imputation, 2 (11%) used both methods, 1 (5%) used mean imputation and 1 (5%) substituted information from a similar variable. Conclusions Just over half of the recent pharmacoepidemiologic MDBS reported missing data and two‐thirds of these studies reported how they accounted for it. We should increase our vigilance for database completeness in MDBS by reporting and addressing the missing data that could introduce bias.
Collapse
Affiliation(s)
- Nicholas B Hunt
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences (UIPS), Utrecht University, Utrecht, The Netherlands
| | - Helga Gardarsdottir
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences (UIPS), Utrecht University, Utrecht, The Netherlands.,Department of Clinical Pharmacy, University Medical Centre Utrecht, Utrecht, The Netherlands.,Department of Pharmaceutical Sciences, School of Health Sciences, University of Iceland, Reykjavik, Iceland
| | - Marloes T Bazelier
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences (UIPS), Utrecht University, Utrecht, The Netherlands
| | - Olaf H Klungel
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences (UIPS), Utrecht University, Utrecht, The Netherlands
| | - Romin Pajouheshnia
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences (UIPS), Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
7
|
Peres LC, Bethea TN, Camacho TF, Bandera EV, Beeghly-Fadiel A, Chyn DL, Harris HR, Joslin CE, Moorman PG, Myers E, Ochs-Balcom HM, Rosenow W, Setiawan VW, Wu AH, Rosenberg L, Schildkraut JM. Racial Differences in Population Attributable Risk for Epithelial Ovarian Cancer in the OCWAA Consortium. J Natl Cancer Inst 2020; 113:710-718. [PMID: 33252629 DOI: 10.1093/jnci/djaa188] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2020] [Revised: 08/31/2020] [Accepted: 10/14/2020] [Indexed: 12/14/2022] Open
Abstract
BACKGROUND The causes of racial disparities in epithelial ovarian cancer (EOC) incidence remain unclear. Differences in the prevalence of ovarian cancer risk factors may explain disparities in EOC incidence among African American (AA) and White women. METHODS We used data from 4 case-control studies and 3 case-control studies nested within prospective cohorts in the Ovarian Cancer in Women of African Ancestry Consortium to estimate race-specific associations of 10 known or suspected EOC risk factors using logistic regression. Using the Bruzzi method, race-specific population attributable risks (PAR) were estimated for each risk factor individually and collectively, including groupings of exposures (reproductive factors and modifiable factors). All statistical tests were 2-sided. RESULTS Among 3244 White EOC cases and 9638 controls and 1052 AA EOC cases and 2410 controls, AA women had a statistically significantly higher PAR (false discovery rate [FDR] P < .001) for first-degree family history of breast cancer (PAR = 10.1%, 95% confidence interval [CI] = 6.5% to 13.7%) compared with White women (PAR = 2.6%, 95% CI = 0.8% to 4.4%). After multiple test correction, AA women had a higher PAR than White women when evaluating all risk factors collectively (PAR = 61.6%, 95% CI = 48.6% to 71.3% vs PAR = 43.0%, 95% CI = 32.8% to 51.4%, respectively; FDR P = .06) and for modifiable exposures, including body mass index, oral contraceptives, aspirin, and body powder (PAR = 36.0%, 95% CI = 21.0% to 48.8% vs PAR = 13.8%, 95% CI = 4.5% to 21.8%, respectively; FDR P = .04). CONCLUSIONS Collectively, the selected risk factors accounted for slightly more of the risk among AA than White women, and interventions to reduce EOC incidence that are focused on multiple modifiable risk factors may be slightly more beneficial to AA women than White women at risk for EOC.
Collapse
Affiliation(s)
- Lauren C Peres
- Department of Cancer Epidemiology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA
| | - Traci N Bethea
- Office of Minority Health and Health Disparities Research, Georgetown Lombardi Comprehensive Cancer Center, Georgetown University Medical Campus, Washington, DC, USA
| | - Tareq F Camacho
- Department of Public Health Sciences, University of Virginia, Charlottesville, VA, USA
| | - Elisa V Bandera
- Cancer Epidemiology and Health Outcomes, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Alicia Beeghly-Fadiel
- Department of Medicine, Division of Epidemiology, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Deanna L Chyn
- Department of Public Health Sciences, University of Virginia, Charlottesville, VA, USA
| | - Holly R Harris
- Program in Epidemiology, Division of Public Health Sciences, Fred Hutchinson Cancer Research Center, Seattle, WA, USA.,Department of Epidemiology, School of Public Health, University of Washington, Seattle, WA, USA
| | - Charlotte E Joslin
- Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago School of Medicine and Division of Epidemiology and Biostatistics, School of Public Health, Chicago, IL, USA
| | - Patricia G Moorman
- Department of Obstetrics and Gynecology, Duke University Medical Center, Durham, NC, USA
| | - Evan Myers
- Department of Family Medicine and Community Health, Duke University Medical Center, Durham, NC, USA
| | - Heather M Ochs-Balcom
- Department of Epidemiology and Environmental Health, School of Public Health and Health Professions, University at Buffalo, The State University of New York, Buffalo, NY, USA
| | - Will Rosenow
- Department of Public Health Sciences, University of Virginia, Charlottesville, VA, USA
| | - V Wendy Setiawan
- Department of Preventive Medicine, Keck School of Medicine, University of Southern California Norris Comprehensive Cancer Center, Los Angeles, CA, USA
| | - Anna H Wu
- Department of Preventive Medicine, Keck School of Medicine, University of Southern California Norris Comprehensive Cancer Center, Los Angeles, CA, USA
| | - Lynn Rosenberg
- Slone Epidemiology Center at Boston University, Boston, MA, USA
| | - Joellen M Schildkraut
- Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, USA
| |
Collapse
|
8
|
Steyerberg EW, Nieboer D, Debray TPA, van Houwelingen HC. Assessment of heterogeneity in an individual participant data meta-analysis of prediction models: An overview and illustration. Stat Med 2019; 38:4290-4309. [PMID: 31373722 PMCID: PMC6772012 DOI: 10.1002/sim.8296] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2017] [Revised: 03/23/2019] [Accepted: 06/06/2019] [Indexed: 02/06/2023]
Abstract
Clinical prediction models aim to provide estimates of absolute risk for a diagnostic or prognostic endpoint. Such models may be derived from data from various studies in the context of a meta‐analysis. We describe and propose approaches for assessing heterogeneity in predictor effects and predictions arising from models based on data from different sources. These methods are illustrated in a case study with patients suffering from traumatic brain injury, where we aim to predict 6‐month mortality based on individual patient data using meta‐analytic techniques (15 studies, n = 11 022 patients). The insights into various aspects of heterogeneity are important to develop better models and understand problems with the transportability of absolute risk predictions.
Collapse
Affiliation(s)
- Ewout W Steyerberg
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands.,Department of Public Health, Erasmus MC, Rotterdam, The Netherlands
| | - Daan Nieboer
- Department of Public Health, Erasmus MC, Rotterdam, The Netherlands
| | - Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, Utrecht University Medical Center, Utrecht, The Netherlands.,Cochrane Netherlands, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Hans C van Houwelingen
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands
| |
Collapse
|
9
|
Audigier V, White IR, Jolani S, Debray TPA, Quartagno M, Carpenter J, van Buuren S, Resche-Rigon M. Multiple Imputation for Multilevel Data with Continuous and Binary Variables. Stat Sci 2018. [DOI: 10.1214/18-sts646] [Citation(s) in RCA: 53] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|