1
|
Chalkou K, Hamza T, Benkert P, Kuhle J, Zecca C, Simoneau G, Pellegrini F, Manca A, Egger M, Salanti G. Combining randomized and non-randomized data to predict heterogeneous effects of competing treatments. Res Synth Methods 2024; 15:641-656. [PMID: 38501273 DOI: 10.1002/jrsm.1717] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 01/26/2024] [Accepted: 02/16/2024] [Indexed: 03/20/2024]
Abstract
Some patients benefit from a treatment while others may do so less or do not benefit at all. We have previously developed a two-stage network meta-regression prediction model that synthesized randomized trials and evaluates how treatment effects vary across patient characteristics. In this article, we extended this model to combine different sources of types in different formats: aggregate data (AD) and individual participant data (IPD) from randomized and non-randomized evidence. In the first stage, a prognostic model is developed to predict the baseline risk of the outcome using a large cohort study. In the second stage, we recalibrated this prognostic model to improve our predictions for patients enrolled in randomized trials. In the third stage, we used the baseline risk as effect modifier in a network meta-regression model combining AD, IPD randomized clinical trial to estimate heterogeneous treatment effects. We illustrated the approach in the re-analysis of a network of studies comparing three drugs for relapsing-remitting multiple sclerosis. Several patient characteristics influence the baseline risk of relapse, which in turn modifies the effect of the drugs. The proposed model makes personalized predictions for health outcomes under several treatment options and encompasses all relevant randomized and non-randomized evidence.
Collapse
Affiliation(s)
- Konstantina Chalkou
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
- Department of Clinical Research, University of Bern, Bern, Switzerland
| | - Tasnim Hamza
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
| | - Pascal Benkert
- Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Jens Kuhle
- Multiple Sclerosis Centre, Neurologic Clinic and Policlinic, Department of Head, Spine and Neuromedicine, University Hospital Basel, University of Basel, Basel, Switzerland
- Multiple Sclerosis Centre, Neurologic Clinic and Policlinic, Department of Biomedicine, University Hospital Basel, University of Basel, Basel, Switzerland
- Multiple Sclerosis Centre, Neurologic Clinic and Policlinic, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
- Research Center for Clinical Neuroimmunology and Neuroscience (RC2NB), University Hospital, University of Basel, Basel, Switzerland
| | - Chiara Zecca
- Multiple Sclerosis Center, Neurocenter of Southern Switzerland, EOC, Lugano, Switzerland
- Faculty of Biomedical Sciences, Università della Svizzera Italiana, Lugano, Switzerland
| | | | | | - Andrea Manca
- Centre for Health Economics, University of York, York, UK
| | - Matthias Egger
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Georgia Salanti
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| |
Collapse
|
2
|
Woodyard KC, Hogan E, Dembinski DR, Madzia J, Guyton L, Janowak CF, Pan BS, Gobble RM. A Review of Meta-Analyses in Plastic Surgery: Need for Adequate Assessment of Publication Bias. J Surg Res 2024; 296:781-789. [PMID: 37543495 DOI: 10.1016/j.jss.2023.06.052] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Revised: 06/10/2023] [Accepted: 06/25/2023] [Indexed: 08/07/2023]
Abstract
INTRODUCTION Publication bias describes a phenomenon in which significant positive results have a higher likelihood of being published compared to negative or nonsignificant results. Publication bias can confound the estimated therapeutic effect in meta-analyses and needs to be adequately assessed in the surgical literature. METHODS A review of meta-analyses published in five plastic surgery journals from 2002 to 2022 was conducted. The inclusion criteria for meta-analyses were factors that demonstrated an obligation to assess publication bias, such as interventions with comparable treatment groups and enough power for statistical analysis. Acknowledgment of publication bias risk, quality of bias assessment, methods used in assessment, and individual article factors were analyzed. RESULTS 318 unique meta-analyses were identified in literature search, and after full-text reviews, 143 met the inclusion criteria for obligation to assess publication bias. 64% of eligible meta-analyses acknowledged the confounding potential of publication bias, and only 46% conducted a formal assessment. Of those who conducted an assessment, 49% used subjective inspection of funnel plots alone, while 47% used any statistical testing in analysis. Overall, only 9/143 (6.3%) assessed publication bias and attempted to correct for its effect. Journals with a higher average impact factor were associated with mention and assessment of publication bias, but more recent publication year and higher number of primary articles analyzed were not. CONCLUSIONS This review identified low rates of proper publication bias assessment in meta-analyses published in five major plastic surgery journals. Assessment of publication bias using objective statistical testing is necessary to ensure quality literature within surgical disciplines.
Collapse
Affiliation(s)
- Kiersten C Woodyard
- Division of Plastic and Reconstructive Surgery, University of Cincinnati, Cincinnati, Ohio; Division of Pediatric Plastic Surgery, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
| | - Elise Hogan
- Division of Plastic and Reconstructive Surgery, University of Cincinnati, Cincinnati, Ohio
| | - Douglas R Dembinski
- Division of Plastic and Reconstructive Surgery, University of Cincinnati, Cincinnati, Ohio
| | - Jules Madzia
- Division of Plastic and Reconstructive Surgery, University of Cincinnati, Cincinnati, Ohio
| | - Lane Guyton
- Division of Plastic and Reconstructive Surgery, University of Cincinnati, Cincinnati, Ohio
| | - Christopher F Janowak
- Division of Trauma and Critical Care Surgery, University of Cincinnati, Cincinnati, Ohio
| | - Brian S Pan
- Division of Pediatric Plastic Surgery, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
| | - Ryan M Gobble
- Division of Plastic and Reconstructive Surgery, University of Cincinnati, Cincinnati, Ohio.
| |
Collapse
|
3
|
Brantner CL, Nguyen TQ, Tang T, Zhao C, Hong H, Stuart EA. Comparison of methods that combine multiple randomized trials to estimate heterogeneous treatment effects. Stat Med 2024; 43:1291-1314. [PMID: 38273647 PMCID: PMC11086055 DOI: 10.1002/sim.9955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Revised: 10/06/2023] [Accepted: 10/23/2023] [Indexed: 01/27/2024]
Abstract
Individualized treatment decisions can improve health outcomes, but using data to make these decisions in a reliable, precise, and generalizable way is challenging with a single dataset. Leveraging multiple randomized controlled trials allows for the combination of datasets with unconfounded treatment assignment to better estimate heterogeneous treatment effects. This article discusses several nonparametric approaches for estimating heterogeneous treatment effects using data from multiple trials. We extend single-study methods to a scenario with multiple trials and explore their performance through a simulation study, with data generation scenarios that have differing levels of cross-trial heterogeneity. The simulations demonstrate that methods that directly allow for heterogeneity of the treatment effect across trials perform better than methods that do not, and that the choice of single-study method matters based on the functional form of the treatment effect. Finally, we discuss which methods perform well in each setting and then apply them to four randomized controlled trials to examine effect heterogeneity of treatments for major depressive disorder.
Collapse
Affiliation(s)
- Carly Lupton Brantner
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA
| | - Trang Quynh Nguyen
- Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA
| | - Tengjie Tang
- Department of Statistical Science, Duke University, Durham, North Carolina, USA
| | - Congwen Zhao
- Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, USA
| | - Hwanhee Hong
- Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, USA
| | - Elizabeth A. Stuart
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA
- Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA
- Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA
| |
Collapse
|
4
|
Bouvier F, Chaimani A, Peyrot E, Gueyffier F, Grenet G, Porcher R. Estimating individualized treatment effects using an individual participant data meta-analysis. BMC Med Res Methodol 2024; 24:74. [PMID: 38528447 DOI: 10.1186/s12874-024-02202-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Accepted: 03/15/2024] [Indexed: 03/27/2024] Open
Abstract
BACKGROUND One key aspect of personalized medicine is to identify individuals who benefit from an intervention. Some approaches have been developed to estimate individualized treatment effects (ITE) with a single randomized control trial (RCT) or observational data, but they are often underpowered for the ITE estimation. Using individual participant data meta-analyses (IPD-MA) might solve this problem. Few studies have investigated how to develop risk prediction models with IPD-MA, and it remains unclear how to combine those methods with approaches used for ITE estimation. In this article, we compared different approaches using both simulated and real data with binary and time-to-event outcomes to estimate the individualized treatment effects from an IPD-MA in a one-stage approach. METHODS We compared five one-stage models: naive model (NA), random intercept (RI), stratified intercept (SI), rank-1 (R1), and fully stratified (FS), built with two different strategies, the S-learner and the T-learner constructed with a Monte Carlo simulation study in which we explored different scenarios with a binary or a time-to-event outcome. To evaluate the performance of the models, we used the c-statistic for benefit, the calibration of predictions, and the mean squared error. The different models were also used on the INDANA IPD-MA, comparing an anti-hypertensive treatment to no treatment or placebo ( N = 40 237 , 836 events). RESULTS Simulation results showed that using the S-learner led to better ITE estimation performances for both binary and time-to-event outcomes. None of the risk models stand out and had significantly better results. For the INDANA dataset with a binary outcome, the naive and the random intercept models had the best performances. CONCLUSIONS For the choice of the strategy, using interactions with treatment (the S-learner) is preferable. For the choice of the method, no approach is better than the other.
Collapse
Affiliation(s)
- Florie Bouvier
- Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France.
| | - Anna Chaimani
- Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
- Cochrane France, Paris, France
| | - Etienne Peyrot
- Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
| | - François Gueyffier
- Laboratoire de Biométrie et Biologie Evolutive UMR 5558, CNRS, Université Lyon 1, Université de Lyon, Villeurbanne, France
| | - Guillaume Grenet
- Laboratoire de Biométrie et Biologie Evolutive UMR 5558, CNRS, Université Lyon 1, Université de Lyon, Villeurbanne, France
| | - Raphaël Porcher
- Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
- Centre d'Épidémiologie Clinique, AP-HP, Hôtel-Dieu, Paris, France
| |
Collapse
|
5
|
Godolphin PJ, Marlin N, Cornett C, Fisher DJ, Tierney JF, White IR, Rogozińska E. Use of multiple covariates in assessing treatment-effect modifiers: A methodological review of individual participant data meta-analyses. Res Synth Methods 2024; 15:107-116. [PMID: 37771175 DOI: 10.1002/jrsm.1674] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Revised: 08/22/2023] [Accepted: 09/18/2023] [Indexed: 09/30/2023]
Abstract
Individual participant data (IPD) meta-analyses of randomised trials are considered a reliable way to assess participant-level treatment effect modifiers but may not make the best use of the available data. Traditionally, effect modifiers are explored one covariate at a time, which gives rise to the possibility that evidence of treatment-covariate interaction may be due to confounding from a different, related covariate. We aimed to evaluate current practice when estimating treatment-covariate interactions in IPD meta-analysis, specifically focusing on involvement of additional covariates in the models. We reviewed 100 IPD meta-analyses of randomised trials, published between 2015 and 2020, that assessed at least one treatment-covariate interaction. We identified four approaches to handling additional covariates: (1) Single interaction model (unadjusted): No additional covariates included (57/100 IPD meta-analyses); (2) Single interaction model (adjusted): Adjustment for the main effect of at least one additional covariate (35/100); (3) Multiple interactions model: Adjustment for at least one two-way interaction between treatment and an additional covariate (3/100); and (4) Three-way interaction model: Three-way interaction formed between treatment, the additional covariate and the potential effect modifier (5/100). IPD is not being utilised to its fullest extent. In an exemplar dataset, we demonstrate how these approaches lead to different conclusions. Researchers should adjust for additional covariates when estimating interactions in IPD meta-analysis providing they adjust their main effects, which is already widely recommended. Further, they should consider whether more complex approaches could provide better information on who might benefit most from treatments, improving patient choice and treatment policy and practice.
Collapse
Affiliation(s)
- Peter J Godolphin
- MRC Clinical Trials Unit at University College London, Institute of Clinical Trials and Methodology, London, UK
| | - Nadine Marlin
- Pragmatic Clinical Trials Unit, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, UK
| | - Chantelle Cornett
- MRC Clinical Trials Unit at University College London, Institute of Clinical Trials and Methodology, London, UK
| | - David J Fisher
- MRC Clinical Trials Unit at University College London, Institute of Clinical Trials and Methodology, London, UK
| | - Jayne F Tierney
- MRC Clinical Trials Unit at University College London, Institute of Clinical Trials and Methodology, London, UK
| | - Ian R White
- MRC Clinical Trials Unit at University College London, Institute of Clinical Trials and Methodology, London, UK
| | - Ewelina Rogozińska
- MRC Clinical Trials Unit at University College London, Institute of Clinical Trials and Methodology, London, UK
| |
Collapse
|
6
|
Papola D, Karyotaki E, Purgato M, Sijbrandij M, Tedeschi F, Cuijpers P, Orestis E, Furukawa TA, Patel V, Barbui C. Dismantling and personalising task-sharing psychosocial interventions for common mental disorders: a study protocol for an individual participant data component network meta-analysis. BMJ Open 2023; 13:e077037. [PMID: 37918937 PMCID: PMC10626809 DOI: 10.1136/bmjopen-2023-077037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Accepted: 10/10/2023] [Indexed: 11/04/2023] Open
Abstract
INTRODUCTION Common mental disorders, including depression, anxiety and related somatic health symptoms, are leading causes of disability worldwide. Especially in low-resource settings, psychosocial interventions delivered by non-specialist providers through task-sharing modalities proved to be valid options to expand access to mental healthcare. However, such interventions are usually eclectic multicomponent interventions consisting of different combinations of evidence-based therapeutic strategies. Which of these various components (or combinations thereof) are more efficacious (and for whom) to reduce common mental disorder symptomatology is yet to be substantiated by evidence. METHODS AND ANALYSIS Comprehensive search was performed in electronic databases MEDLINE, Embase, PsycINFO and the Cochrane Register of Controlled Trials-CENTRAL from database inception to 15 March 2023 to systematically identify all randomised controlled trials that compared any single component or multicomponent psychosocial intervention delivered through the task-sharing modality against any active or inactive control condition in the treatment of adults suffering from common mental disorders. From these trials, individual participant data (IPD) of all measured outcomes and covariates will be collected. We will dismantle psychosocial interventions creating a taxonomy of components and then apply the IPD component network meta-analysis (IPD-cNMA) methodology to assess the efficacy of individual components (or combinations thereof) according to participant-level prognostic factors and effect modifiers. ETHICS AND DISSEMINATION Ethics approval is not applicable for this study since no original data will be collected. Results from this study will be published in peer-reviewed journals and presented at relevant conferences.
Collapse
Affiliation(s)
- Davide Papola
- Department of Global Health and Social Medicine, Harvard Medical School, Boston, Massachusetts, USA
- WHO Collaborating Centre for Research and Training in Mental Health and Service Evaluation, Department of Neuroscience, Biomedicine and Movement Science, Section of Psychiatry, University of Verona, Verona, Italy
| | - Eirini Karyotaki
- Department of Clinical, Neuro and Developmental Psychology, WHO Collaborating Centre for Research and Dissemination of Psychological Interventions, Amsterdam Public Health research institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Marianna Purgato
- WHO Collaborating Centre for Research and Training in Mental Health and Service Evaluation, Department of Neuroscience, Biomedicine and Movement Science, Section of Psychiatry, University of Verona, Verona, Italy
| | - Marit Sijbrandij
- Department of Clinical, Neuro and Developmental Psychology, WHO Collaborating Centre for Research and Dissemination of Psychological Interventions, Amsterdam Public Health research institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Federico Tedeschi
- WHO Collaborating Centre for Research and Training in Mental Health and Service Evaluation, Department of Neuroscience, Biomedicine and Movement Science, Section of Psychiatry, University of Verona, Verona, Italy
| | - Pim Cuijpers
- Department of Clinical, Neuro and Developmental Psychology, WHO Collaborating Centre for Research and Dissemination of Psychological Interventions, Amsterdam Public Health research institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Efthimiou Orestis
- Institute of Primary Health Care (BIHAM), University of Bern, Bern, Switzerland
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| | - Toshi A Furukawa
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto University, Kyoto, Japan
| | - Vikram Patel
- Department of Global Health and Social Medicine, Harvard Medical School, Boston, Massachusetts, USA
| | - Corrado Barbui
- WHO Collaborating Centre for Research and Training in Mental Health and Service Evaluation, Department of Neuroscience, Biomedicine and Movement Science, Section of Psychiatry, University of Verona, Verona, Italy
| |
Collapse
|
7
|
Brantner CL, Chang TH, Nguyen TQ, Hong H, Stefano LD, Stuart EA. Methods for Integrating Trials and Non-experimental Data to Examine Treatment Effect Heterogeneity. Stat Sci 2023; 38:640-654. [PMID: 38638306 PMCID: PMC11025720 DOI: 10.1214/23-sts890] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/20/2024]
Abstract
Estimating treatment effects conditional on observed covariates can improve the ability to tailor treatments to particular individuals. Doing so effectively requires dealing with potential confounding, and also enough data to adequately estimate effect moderation. A recent influx of work has looked into estimating treatment effect heterogeneity using data from multiple randomized controlled trials and/or observational datasets. With many new methods available for assessing treatment effect heterogeneity using multiple studies, it is important to understand which methods are best used in which setting, how the methods compare to one another, and what needs to be done to continue progress in this field. This paper reviews these methods broken down by data setting: aggregate-level data, federated learning, and individual participant-level data. We define the conditional average treatment effect and discuss differences between parametric and nonparametric estimators, and we list key assumptions, both those that are required within a single study and those that are necessary for data combination. After describing existing approaches, we compare and contrast them and reveal open areas for future research. This review demonstrates that there are many possible approaches for estimating treatment effect heterogeneity through the combination of datasets, but that there is substantial work to be done to compare these methods through case studies and simulations, extend them to different settings, and refine them to account for various challenges present in real data.
Collapse
Affiliation(s)
- Carly Lupton Brantner
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland 21205, USA
| | - Ting-Hsuan Chang
- Department of Biostatistics, Columbia Mailman School of Public Health, New York, New York 10032, USA
| | - Trang Quynh Nguyen
- Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland 21205, USA
| | - Hwanhee Hong
- Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina 27710, USA
| | - Leon Di Stefano
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland 21205, USA
| | - Elizabeth A Stuart
- Departments of Biostatistics, Mental Health, and Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland 21205, USA
| |
Collapse
|
8
|
Hoogland J, Takada T, van Smeden M, Rovers MM, de Sutter AI, Merenstein D, Kaiser L, Liira H, Little P, Bucher HC, Moons KGM, Reitsma JB, Venekamp RP. Prognosis and prediction of antibiotic benefit in adults with clinically diagnosed acute rhinosinusitis: an individual participant data meta-analysis. Diagn Progn Res 2023; 7:16. [PMID: 37667327 PMCID: PMC10478354 DOI: 10.1186/s41512-023-00154-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Accepted: 07/20/2023] [Indexed: 09/06/2023] Open
Abstract
BACKGROUND A previous individual participant data meta-analysis (IPD-MA) of antibiotics for adults with clinically diagnosed acute rhinosinusitis (ARS) showed a marginal overall effect of antibiotics, but was unable to identify patients that are most likely to benefit from antibiotics when applying conventional (i.e. univariable or one-variable-at-a-time) subgroup analysis. We updated the systematic review and investigated whether multivariable prediction of patient-level prognosis and antibiotic treatment effect may lead to more tailored treatment assignment in adults presenting to primary care with ARS. METHODS An IPD-MA of nine double-blind placebo-controlled trials of antibiotic treatment (n=2539) was conducted, with the probability of being cured at 8-15 days as the primary outcome. A logistic mixed effects model was developed to predict the probability of being cured based on demographic characteristics, signs and symptoms, and antibiotic treatment assignment. Predictive performance was quantified based on internal-external cross-validation in terms of calibration and discrimination performance, overall model fit, and the accuracy of individual predictions. RESULTS Results indicate that the prognosis with respect to risk of cure could not be reliably predicted (c-statistic 0.58 and Brier score 0.24). Similarly, patient-level treatment effect predictions did not reliably distinguish between those that did and did not benefit from antibiotics (c-for-benefit 0.50). CONCLUSIONS In conclusion, multivariable prediction based on patient demographics and common signs and symptoms did not reliably predict the patient-level probability of cure and antibiotic effect in this IPD-MA. Therefore, these characteristics cannot be expected to reliably distinguish those that do and do not benefit from antibiotics in adults presenting to primary care with ARS.
Collapse
Affiliation(s)
- Jeroen Hoogland
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
- Department of Epidemiology and Data Science, Amsterdam University Medical Centres, Amsterdam University, Amsterdam, The Netherlands.
| | - Toshihiko Takada
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Department of General Medicine, Shirakawa Satellite for Teaching And Research (STAR), Fukushima Medical University, Fukushima, Japan
| | - Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Maroeska M Rovers
- Radboud Institute for Health Sciences (RIHS), Radboud University Medical Center, Nijmegen, The Netherlands
| | - An I de Sutter
- Department of Public Health and Primary Care, Ghent University, Ghent, Belgium
| | - Daniel Merenstein
- Department of Family Medicine, Georgetown University Medical Center, Washington, DC, USA
| | - Laurent Kaiser
- Department of Medicine, Division of Infectious Diseases, University Hospital Geneva, Geneva, Switzerland
| | - Helena Liira
- Department of General Practice, School of Primary, Aboriginal and Rural Health Care, University of Western Australia, Perth, Australia
- Department of General Practice and Primary Care, University of Helsinki, Helsinki, Finland
| | - Paul Little
- Primary Care & Population Sciences Unit, Aldermoor Health Centre, University of Southampton, Southampton, UK
| | - Heiner C Bucher
- Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel and University of Basel, Basel, Switzerland
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Johannes B Reitsma
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Roderick P Venekamp
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
9
|
Chalkou K, Vickers AJ, Pellegrini F, Manca A, Salanti G. Decision Curve Analysis for Personalized Treatment Choice between Multiple Options. Med Decis Making 2023; 43:337-349. [PMID: 36511470 PMCID: PMC10021120 DOI: 10.1177/0272989x221143058] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Accepted: 11/03/2022] [Indexed: 12/15/2022]
Abstract
BACKGROUND Decision curve analysis can be used to determine whether a personalized model for treatment benefit would lead to better clinical decisions. Decision curve analysis methods have been described to estimate treatment benefit using data from a single randomized controlled trial. OBJECTIVES Our main objective is to extend the decision curve analysis methodology to the scenario in which several treatment options exist and evidence about their effects comes from a set of trials, synthesized using network meta-analysis (NMA). METHODS We describe the steps needed to estimate the net benefit of a prediction model using evidence from studies synthesized in an NMA. We show how to compare personalized versus one-size-fit-all treatment decision-making strategies, such as "treat none" or "treat all patients with a specific treatment" strategies. First, threshold values for each included treatment need to be defined (i.e., the minimum risk difference compared with control that renders a treatment worth taking). The net benefit per strategy can then be plotted for a plausible range of threshold values to reveal the most clinically useful strategy. We applied our methodology to an NMA prediction model for relapsing-remitting multiple sclerosis, which can be used to choose between natalizumab, dimethyl fumarate, glatiramer acetate, and placebo. RESULTS We illustrated the extended decision curve analysis methodology using several threshold value combinations for each available treatment. For the examined threshold values, the "treat patients according to the prediction model" strategy performs either better than or close to the one-size-fit-all treatment strategies. However, even small differences may be important in clinical decision making. As the advantage of the personalized model was not consistent across all thresholds, improving the existing model (by including, for example, predictors that will increase discrimination) is needed before advocating its clinical usefulness. CONCLUSIONS This novel extension of decision curve analysis can be applied to NMA-based prediction models to evaluate their use to aid treatment decision making. HIGHLIGHTS Decision curve analysis is extended into a (network) meta-analysis framework.Personalized models predicting treatment benefit are evaluated when several treatment options are available and evidence about their effects comes from a set of trials.Detailed steps to compare personalized versus one-size-fit-all treatment decision-making strategies are outlined.This extension of decision curve analysis can be applied to (network) meta-analysis-based prediction models to evaluate their use to aid treatment decision making.
Collapse
Affiliation(s)
- Konstantina Chalkou
- Institute of Social and Preventive Medicine,
University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University
of Bern, Switzerland
| | - Andrew J. Vickers
- Department of Epidemiology and Biostatistics,
Memorial Sloan-Kettering Cancer Center, New York, NY, USA
| | | | - Andrea Manca
- Centre for Health Economics, University of
York, York, UK
| | - Georgia Salanti
- Institute of Social and Preventive Medicine,
University of Bern, Bern, Switzerland
| |
Collapse
|
10
|
Godolphin PJ, White IR, Tierney JF, Fisher DJ. Estimating interactions and subgroup-specific treatment effects in meta-analysis without aggregation bias: A within-trial framework. Res Synth Methods 2023; 14:68-78. [PMID: 35833636 PMCID: PMC10087172 DOI: 10.1002/jrsm.1590] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2021] [Revised: 04/28/2022] [Accepted: 06/12/2022] [Indexed: 01/21/2023]
Abstract
Estimation of within-trial interactions in meta-analysis is crucial for reliable assessment of how treatment effects vary across participant subgroups. However, current methods have various limitations. Patients, clinicians and policy-makers need reliable estimates of treatment effects within specific covariate subgroups, on relative and absolute scales, in order to target treatments appropriately-which estimation of an interaction effect does not in itself provide. Also, the focus has been on covariates with only two subgroups, and may exclude relevant data if only a single subgroup is reported. Therefore, in this article we further develop the "within-trial" framework by providing practical methods to (1) estimate within-trial interactions across two or more subgroups; (2) estimate subgroup-specific ("floating") treatment effects that are compatible with the within-trial interactions and make maximum use of available data; and (3) clearly present this data using novel implementation of forest plots. We described the steps involved and apply the methods to two examples taken from previously published meta-analyses, and demonstrate a straightforward implementation in Stata based upon existing code for multivariate meta-analysis. We discuss how the within-trial framework and plots can be utilised with aggregate (or "published") source data, as well as with individual participant data, to effectively demonstrate how treatment effects differ across participant subgroups.
Collapse
Affiliation(s)
- Peter J Godolphin
- MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, University College London, London, UK
| | - Ian R White
- MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, University College London, London, UK
| | - Jayne F Tierney
- MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, University College London, London, UK
| | - David J Fisher
- MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, University College London, London, UK
| |
Collapse
|
11
|
Yoshida K, Seo M, Luo Y, Sahker E, Cipriani A, Leucht S, Iwatsubo T, Efthimiou O, Furukawa TA. Personalized Prediction of Alzheimer’s Disease and Its Treatment Effects by Donepezil: An Individual Participant Data Meta-Analysis of Eight Randomized Controlled Trials. J Alzheimers Dis 2022; 89:1143-1157. [DOI: 10.3233/jad-220263] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Background: Patient characteristics may predict the progression of Alzheimer’s disease (AD) and may moderate the effects of donepezil. Objective: To build a personalized prediction model for patients with AD and to estimate patient-specific treatment effects of donepezil, using individual patient characteristics. Methods: We systematically searched for all double-masked randomized controlled trials comparing oral donepezil and pill placebo in the treatment of AD and requested individual participant data through its developer, Eisai. The primary outcome was cognitive function at 24 weeks, measured with the Alzheimer’s Disease Assessment Scale-cognitive component (ADAS-cog). We built a Bayesian meta-analytical prediction model for patients receiving placebo and we performed an individual patient data meta-analysis to estimate patient-level treatment effects. Results: Eight studies with 3,156 participants were included. The Bayesian prediction model suggested that more severe cognitive and global function at baseline and younger age were associated with worse cognitive function at 24 weeks. The individual participant data meta-analysis showed that, on average, donepezil was superior to placebo in cognitive function (ADAS-cog scores, –3.2; 95% Credible Interval (CrI) –4.2 to –2.1). In addition, our results suggested that antipsychotic drug use at baseline might be associated with a lower effect of donepezil in ADAS-cog (2.0; 95% CrI, –0.02 to 4.3). Conclusion: Although our results suggested that donepezil is somewhat efficacious for cognitive function for most patients with AD, use of antipsychotic drugs may be associated with lower efficacy of the drug. Future research with larger sample sizes, more patient covariates, and longer treatment duration is needed.
Collapse
Affiliation(s)
- Kazufumi Yoshida
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine, School of Public Health, Kyoto, Japan
| | - Michael Seo
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Yan Luo
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine, School of Public Health, Kyoto, Japan
| | - Ethan Sahker
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine, School of Public Health, Kyoto, Japan
- Population Health and Policy Research Unit, Medical Education Center, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Andrea Cipriani
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom
- Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, United Kingdom
| | - Stefan Leucht
- Department of Psychiatry and Psychotherapy, Technical University of Munich, School of Medicine, Munich, Germany
| | - Takeshi Iwatsubo
- Department of Neuropathology, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan
| | - Orestis Efthimiou
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom
- Institute of Primary Health Care (BIHAM), University of Bern, Bern, Switzerland
| | - Toshiaki A. Furukawa
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine, School of Public Health, Kyoto, Japan
| |
Collapse
|
12
|
Riley RD, Dias S, Donegan S, Tierney JF, Stewart LA, Efthimiou O, Phillippo DM. Using individual participant data to improve network meta-analysis projects. BMJ Evid Based Med 2022; 28:197-203. [PMID: 35948411 DOI: 10.1136/bmjebm-2022-111931] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 07/01/2022] [Indexed: 11/04/2022]
Abstract
A network meta-analysis combines the evidence from existing randomised trials about the comparative efficacy of multiple treatments. It allows direct and indirect evidence about each comparison to be included in the same analysis, and provides a coherent framework to compare and rank treatments. A traditional network meta-analysis uses aggregate data (eg, treatment effect estimates and standard errors) obtained from publications or trial investigators. An alternative approach is to obtain, check, harmonise and meta-analyse the individual participant data (IPD) from each trial. In this article, we describe potential advantages of IPD for network meta-analysis projects, emphasising five key benefits: (1) improving the quality and scope of information available for inclusion in the meta-analysis, (2) examining and plotting distributions of covariates across trials (eg, for potential effect modifiers), (3) standardising and improving the analysis of each trial, (4) adjusting for prognostic factors to allow a network meta-analysis of conditional treatment effects and (5) including treatment-covariate interactions (effect modifiers) to allow relative treatment effects to vary by participant-level covariate values (eg, age, baseline depression score). A running theme of all these benefits is that they help examine and reduce heterogeneity (differences in the true treatment effect between trials) and inconsistency (differences in the true treatment effect between direct and indirect evidence) in the network. As a consequence, an IPD network meta-analysis has the potential for more precise, reliable and informative results for clinical practice and even allows treatment comparisons to be made for individual patients and targeted populations conditional on their particular characteristics.
Collapse
Affiliation(s)
| | - Sofia Dias
- Centre for Reviews and Dissemination, University of York, York, UK
| | - Sarah Donegan
- Department of Health Data Science, University of Liverpool, Liverpool, UK
| | | | - Lesley A Stewart
- Centre for Reviews and Dissemination, University of York, York, UK
| | - Orestis Efthimiou
- Institute of Social and Preventive Medicine (ISPMU), University of Bern, Bern, Switzerland
| | - David M Phillippo
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| |
Collapse
|
13
|
Seo M, Debray TP, Ruffieux Y, Gsteiger S, Bujkiewicz S, Finckh A, Egger M, Efthimiou O. Combining individual patient data from randomized and non-randomized studies to predict real-world effectiveness of interventions. Stat Methods Med Res 2022; 31:1355-1373. [PMID: 35469504 PMCID: PMC9251754 DOI: 10.1177/09622802221090759] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Meta-analysis of randomized controlled trials is generally considered the most reliable source of estimates of relative treatment effects. However, in the last few years, there has been interest in using non-randomized studies to complement evidence from randomized controlled trials. Several meta-analytical models have been proposed to this end. Such models mainly focussed on estimating the average relative effects of interventions. In real-life clinical practice, when deciding on how to treat a patient, it might be of great interest to have personalized predictions of absolute outcomes under several available treatment options. This paper describes a general framework for developing models that combine individual patient data from randomized controlled trials and non-randomized study when aiming to predict outcomes for a set of competing medical interventions applied in real-world clinical settings. We also discuss methods for measuring the models' performance to identify the optimal model to use in each setting. We focus on the case of continuous outcomes and illustrate our methods using a data set from rheumatoid arthritis, comprising patient-level data from three randomized controlled trials and two registries from Switzerland and Britain.
Collapse
Affiliation(s)
- Michael Seo
- Institute of Social and Preventive Medicine, 27210University of Bern, Bern, Switzerland.,Graduate School for Health Sciences, 27210University of Bern, Bern, Switzerland
| | - Thomas Pa Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, 8125Utrecht University, Utrecht, The Netherlands.,Smart Data Analysis and Statistics B.V., Utrecht, The Netherlands
| | - Yann Ruffieux
- Institute of Social and Preventive Medicine, 27210University of Bern, Bern, Switzerland
| | - Sandro Gsteiger
- Pharmaceuticals Division, Global Access, F. Hoffmann-La Roche, Basel, Switzerland
| | - Sylwia Bujkiewicz
- Biostatistics Research Group, Department of Health Sciences, 4488University of Leicester, Leicester, UK
| | - Axel Finckh
- Division of Rheumatology, 30576University Hospitals of Geneva, Geneva, Switzerland
| | - Matthias Egger
- Institute of Social and Preventive Medicine, 27210University of Bern, Bern, Switzerland.,Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Orestis Efthimiou
- Institute of Social and Preventive Medicine, 27210University of Bern, Bern, Switzerland.,Department of Psychiatry, 6396University of Oxford, Oxford, UK
| |
Collapse
|
14
|
Efthimiou O, Seo M, Karyotaki E, Cuijpers P, Furukawa TA, Schwarzer G, Rücker G, Mavridis D. Bayesian models for aggregate and individual patient data component network meta-analysis. Stat Med 2022; 41:2586-2601. [PMID: 35261053 PMCID: PMC9314605 DOI: 10.1002/sim.9372] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2021] [Revised: 12/17/2021] [Accepted: 02/21/2022] [Indexed: 12/28/2022]
Abstract
Network meta-analysis can synthesize evidence from studies comparing multiple treatments for the same disease. Sometimes the treatments of a network are complex interventions, comprising several independent components in different combinations. A component network meta-analysis (CNMA) can be used to analyze such data and can in principle disentangle the individual effect of each component. However, components may interact with each other, either synergistically or antagonistically. Deciding which interactions, if any, to include in a CNMA model may be difficult, especially for large networks with many components. In this article, we present two Bayesian CNMA models that can be used to identify prominent interactions between components. Our models utilize Bayesian variable selection methods, namely the stochastic search variable selection and the Bayesian LASSO, and can benefit from the inclusion of prior information about important interactions. Moreover, we extend these models to combine data from studies providing aggregate information and studies providing individual patient data (IPD). We illustrate our models in practice using three real datasets, from studies in panic disorder, depression, and multiple myeloma. Finally, we describe methods for developing web-applications that can utilize results from an IPD-CNMA, to allow for personalized estimates of relative treatment effects given a patient's characteristics.
Collapse
Affiliation(s)
- Orestis Efthimiou
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland.,Department of Psychiatry, University of Oxford, Oxford, UK.,Institute of Primary Health Care (BIHAM), Université of Bern, Bern, Switzerland
| | - Michael Seo
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland.,Graduate School for Health Sciences, University of Bern, Bern, Switzerland
| | - Eirini Karyotaki
- Department of Global Health and Social Medicine, Harvard Medical School, Boston, Massachusetts, USA.,Department of Clinical Neuro- and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.,Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| | - Pim Cuijpers
- Department of Clinical Neuro- and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.,Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| | - Toshi A Furukawa
- Departments of Health Promotion and Human Behavior and of Clinical Epidemiology, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan
| | - Guido Schwarzer
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Gerta Rücker
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Dimitris Mavridis
- Department of Primary Education, University of Ioannina, Ioannina, Greece.,Faculté de Médecine, Université Paris Descartes, Paris, France
| |
Collapse
|
15
|
Ollier E, Blanchard P, Le Teuff G, Michiels S. Penalized Poisson model for network meta-analysis of individual patient time-to-event data. Stat Med 2021; 41:340-355. [PMID: 34710951 DOI: 10.1002/sim.9240] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2020] [Revised: 10/14/2021] [Accepted: 10/15/2021] [Indexed: 12/15/2022]
Abstract
Network meta-analysis (NMA) allows the combination of direct and indirect evidence from a set of randomized clinical trials. Performing NMA using individual patient data (IPD) is considered as a "gold standard" approach as it provides several advantages over NMA based on aggregate data. For example, it allows to perform advanced modeling of covariates or covariate-treatment interactions. An important issue in IPD NMA is the selection of influential parameters among terms that account for inconsistency, covariates, covariate-by-treatment interactions or nonproportionality of treatments effect for time to event data. This issue has not been deeply studied in the literature yet and in particular not for time-to-event data. A major difficulty is to jointly account for between-trial heterogeneity which could have a major influence on the selection process. The use of penalized generalized mixed effect model is a solution, but existing implementations have several shortcomings and an important computational cost that precludes their use for complex IPD NMA. In this article, we propose a penalized Poisson regression model to perform IPD NMA of time-to-event data. It is based only on fixed effect parameters which improve its computational cost over the use of random effects. It could be easily implemented using existing penalized regression package. Computer code is shared for implementation. The methods were applied on simulated data to illustrate the importance to take into account between trial heterogeneity during the selection procedure. Finally, it was applied to an IPD NMA of overall survival of chemotherapy and radiotherapy in nasopharyngeal carcinoma.
Collapse
Affiliation(s)
- Edouard Ollier
- Service de Biostatistique et d'Épidémiologie, Gustave Roussy, Université Paris-Saclay, Villejuif, France.,Oncostat U1018, Inserm, Université Paris-Saclay, Labeled Ligue Contre le Cancer, Villejuif, France.,SAINBIOSE U1059, Equipe DVH, Université Jean Monnet, Saint-Etienne, France
| | - Pierre Blanchard
- Oncostat U1018, Inserm, Université Paris-Saclay, Labeled Ligue Contre le Cancer, Villejuif, France.,Département de Radiothérapie, Gustave Roussy, Université Paris-Saclay, Villejuif, France
| | - Gwénaël Le Teuff
- Service de Biostatistique et d'Épidémiologie, Gustave Roussy, Université Paris-Saclay, Villejuif, France.,Oncostat U1018, Inserm, Université Paris-Saclay, Labeled Ligue Contre le Cancer, Villejuif, France
| | - Stefan Michiels
- Service de Biostatistique et d'Épidémiologie, Gustave Roussy, Université Paris-Saclay, Villejuif, France.,Oncostat U1018, Inserm, Université Paris-Saclay, Labeled Ligue Contre le Cancer, Villejuif, France
| |
Collapse
|
16
|
Furukawa TA, Suganuma A, Ostinelli EG, Andersson G, Beevers CG, Shumake J, Berger T, Boele FW, Buntrock C, Carlbring P, Choi I, Christensen H, Mackinnon A, Dahne J, Huibers MJH, Ebert DD, Farrer L, Forand NR, Strunk DR, Ezawa ID, Forsell E, Kaldo V, Geraedts A, Gilbody S, Littlewood E, Brabyn S, Hadjistavropoulos HD, Schneider LH, Johansson R, Kenter R, Kivi M, Björkelund C, Kleiboer A, Riper H, Klein JP, Schröder J, Meyer B, Moritz S, Bücker L, Lintvedt O, Johansson P, Lundgren J, Milgrom J, Gemmill AW, Mohr DC, Montero-Marin J, Garcia-Campayo J, Nobis S, Zarski AC, O'Moore K, Williams AD, Newby JM, Perini S, Phillips R, Schneider J, Pots W, Pugh NE, Richards D, Rosso IM, Rauch SL, Sheeber LB, Smith J, Spek V, Pop VJ, Ünlü B, van Bastelaar KMP, van Luenen S, Garnefski N, Kraaij V, Vernmark K, Warmerdam L, van Straten A, Zagorscak P, Knaevelsrud C, Heinrich M, Miguel C, Cipriani A, Efthimiou O, Karyotaki E, Cuijpers P. Dismantling, optimising, and personalising internet cognitive behavioural therapy for depression: a systematic review and component network meta-analysis using individual participant data. Lancet Psychiatry 2021; 8:500-511. [PMID: 33957075 PMCID: PMC8838916 DOI: 10.1016/s2215-0366(21)00077-8] [Citation(s) in RCA: 79] [Impact Index Per Article: 26.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/24/2020] [Revised: 02/12/2021] [Accepted: 02/17/2021] [Indexed: 02/07/2023]
Abstract
BACKGROUND Internet cognitive behavioural therapy (iCBT) is a viable delivery format of CBT for depression. However, iCBT programmes include training in a wide array of cognitive and behavioural skills via different delivery methods, and it remains unclear which of these components are more efficacious and for whom. METHODS We did a systematic review and individual participant data component network meta-analysis (cNMA) of iCBT trials for depression. We searched PubMed, PsycINFO, Embase, and the Cochrane Library for randomised controlled trials (RCTs) published from database inception to Jan 1, 2019, that compared any form of iCBT against another or a control condition in the acute treatment of adults (aged ≥18 years) with depression. Studies with inpatients or patients with bipolar depression were excluded. We sought individual participant data from the original authors. When these data were unavailable, we used aggregate data. Two independent researchers identified the included components. The primary outcome was depression severity, expressed as incremental mean difference (iMD) in the Patient Health Questionnaire-9 (PHQ-9) scores when a component is added to a treatment. We developed a web app that estimates relative efficacies between any two combinations of components, given baseline patient characteristics. This study is registered in PROSPERO, CRD42018104683. FINDINGS We identified 76 RCTs, including 48 trials contributing individual participant data (11 704 participants) and 28 trials with aggregate data (6474 participants). The participants' weighted mean age was 42·0 years and 12 406 (71%) of 17 521 reported were women. There was suggestive evidence that behavioural activation might be beneficial (iMD -1·83 [95% credible interval (CrI) -2·90 to -0·80]) and that relaxation might be harmful (1·20 [95% CrI 0·17 to 2·27]). Baseline severity emerged as the strongest prognostic factor for endpoint depression. Combining human and automated encouragement reduced dropouts from treatment (incremental odds ratio, 0·32 [95% CrI 0·13 to 0·93]). The risk of bias was low for the randomisation process, missing outcome data, or selection of reported results in most of the included studies, uncertain for deviation from intended interventions, and high for measurement of outcomes. There was moderate to high heterogeneity among the studies and their components. INTERPRETATION The individual patient data cNMA revealed potentially helpful, less helpful, or harmful components and delivery formats for iCBT packages. iCBT packages aiming to be effective and efficient might choose to include beneficial components and exclude ones that are potentially detrimental. Our web app can facilitate shared decision making by therapist and patient in choosing their preferred iCBT package. FUNDING Japan Society for the Promotion of Science.
Collapse
Affiliation(s)
- Toshi A Furukawa
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan.
| | - Aya Suganuma
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan
| | - Edoardo G Ostinelli
- Department of Psychiatry, Warneford Hospital, University of Oxford, Oxford, UK
| | - Gerhard Andersson
- Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institutet and Stockholm Health Care Services, Stockholm County Council, Stockholm, Sweden; Department of Behavioral Sciences and Learning, Linköping University, Linköping, Sweden
| | - Christopher G Beevers
- Department of Psychology and Institute for Mental Health Research, University of Texas at Austin, Austin, TX, USA
| | - Jason Shumake
- Department of Psychology and Institute for Mental Health Research, University of Texas at Austin, Austin, TX, USA
| | - Thomas Berger
- Department of Clinical Psychology and Psychotherapy, University of Bern, Bern, Switzerland
| | - Florien Willemijn Boele
- Patient Centred Outcomes Research Group, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Claudia Buntrock
- Department of Clinical Psychology and Psychotherapy, Friedrich-Alexander-University Erlangen-Nürnberg, Erlangen, Germany
| | - Per Carlbring
- Department of Psychology, Stockholm University, Stockholm, Sweden
| | - Isabella Choi
- Central Clinical School, Brain and Mind Centre, Faculty of Medicine and Health, University of Sydney, Sydney, NSW, Australia
| | - Helen Christensen
- Black Dog Institute and University of New South Wales, Prince of Wales Hospital, Sydney, NSW, Australia
| | - Andrew Mackinnon
- Black Dog Institute and University of New South Wales, Prince of Wales Hospital, Sydney, NSW, Australia
| | - Jennifer Dahne
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
| | - Marcus J H Huibers
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - David D Ebert
- Department for Sport and Health Sciences, Chair for Psychology & Digital Mental Health Care, Technical University Munich, Germany
| | - Louise Farrer
- Centre for Mental Health Research, The Australian National University, Canberra, Australia
| | - Nicholas R Forand
- Department of Psychiatry, The Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA
| | - Daniel R Strunk
- Department of Psychology, The Ohio State University, Columbus, OH, USA
| | - Iony D Ezawa
- Department of Psychology, The Ohio State University, Columbus, OH, USA
| | - Erik Forsell
- Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institutet and Stockholm Health Care Services, Stockholm County Council, Stockholm, Sweden
| | - Viktor Kaldo
- Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institutet and Stockholm Health Care Services, Stockholm County Council, Stockholm, Sweden; Department of Psychology, Faculty of Health and Life Sciences, Linnaeus University, Växjö, Sweden
| | | | - Simon Gilbody
- Department of Health Sciences, University of York, York, UK
| | | | - Sally Brabyn
- Department of Health Sciences, University of York, York, UK
| | | | - Luke H Schneider
- Anxiety Treatment and Research Clinic, St Joseph's Healthcare Hamilton, Hamilton, ON, Canada
| | - Robert Johansson
- Department of Psychology, Stockholm University, Stockholm, Sweden
| | - Robin Kenter
- Department of Clinical Psychology, Faculty of Psychology, University of Bergen, Bergen, Norway
| | - Marie Kivi
- Department of Psychology, University of Gothenburg, Gothenburg, Sweden
| | - Cecilia Björkelund
- Primary Health Care, School of Public Health and Community Medicine, Institute of Medicine, University of Gothenburg, Gothenburg, Sweden
| | - Annet Kleiboer
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - Heleen Riper
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - Jan Philipp Klein
- Department of Psychiatry and Psychotherapy, Luebeck University, Luebeck, Germany
| | - Johanna Schröder
- Institute for Sex Research, Sexual Medicine and Forensic Psychiatry, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Björn Meyer
- Research Department, GAIA AG, Hamburg, Germany
| | - Steffen Moritz
- Department of Psychiatry and Psychotherapy, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Lara Bücker
- Department of Psychiatry and Psychotherapy, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Ove Lintvedt
- Norwegian Center for E-health research, Tromsø, Norway
| | - Peter Johansson
- Department of Health, Medicine and Caring Sciences, Linköping University, Norrköping, Sweden
| | - Johan Lundgren
- Department of Health, Medicine and Caring Sciences, Linköping University, Norrköping, Sweden
| | - Jeannette Milgrom
- Parent-Infant Research Institute and Austin Health, Melbourne School of Psychological Sciences, University of Melbourne, VIC, Australia
| | - Alan W Gemmill
- Parent-Infant Research Institute and Austin Health, Melbourne School of Psychological Sciences, University of Melbourne, VIC, Australia
| | - David C Mohr
- Center for Behavioral Intervention Technologies, Department of Preventive Medicine, Northwestern University, Chicago, IL, USA
| | - Jesus Montero-Marin
- Department of Psychiatry, Warneford Hospital, University of Oxford, Oxford, UK
| | - Javier Garcia-Campayo
- Aragon Institute for Health Research, Miguel Servet University Hospital, Zaragoza, Spain; Primary Care Prevention and Health Promotion Research Network, RedIAPP, Madrid, Spain
| | | | - Anna-Carlotta Zarski
- Department of Clinical Psychology and Psychotherapy, Friedrich-Alexander-University Erlangen-Nürnberg, Erlangen, Germany
| | - Kathleen O'Moore
- Black Dog Institute and University of New South Wales, Prince of Wales Hospital, Sydney, NSW, Australia
| | - Alishia D Williams
- Department of Psychology, Faculty of Science, The University of New South Wales, Sydney, NSW, Australia
| | - Jill M Newby
- School of Psychology, University of New South Wales at the Black Dog Institute, Sydney, NSW, Australia
| | - Sarah Perini
- Clinical Research Unit for Anxiety and Depression, St Vincent's Hospital, Sydney, NSW, Australia
| | - Rachel Phillips
- Faculty of Medicine, School of Public Health, Imperial College London, London, UK
| | - Justine Schneider
- School of Sociology & Social Policy and Institute of Mental Health, University of Nottingham, Nottingham, UK
| | - Wendy Pots
- Department of Psychology, Health & Technology, University of Twente, Enschede, Netherlands
| | | | - Derek Richards
- University of Dublin, Trinity College, School of Psychology, E-mental Health Research Group, Dublin, Ireland; SilverCloud Health, Clinical Research & Innovation, Dublin, Ireland
| | | | | | | | - Jessica Smith
- Imperial Clinical Trials Unit, Imperial College London, London, UK
| | - Viola Spek
- School of Applied Psychology, Fontys University of Applied Science, Eindhoven, Netherlands
| | - Victor J Pop
- Department of Medical & Clinical Psychology, Tilburg University, Tilburg, Netherlands
| | | | | | - Sanne van Luenen
- Department of Clinical Psychology, Leiden University, Leiden, Netherlands
| | - Nadia Garnefski
- Department of Clinical Psychology, Leiden University, Leiden, Netherlands
| | - Vivian Kraaij
- Department of Clinical Psychology, Leiden University, Leiden, Netherlands
| | - Kristofer Vernmark
- Department of Behavioral Sciences and Learning, Linköping University, Linköping, Sweden
| | | | - Annemieke van Straten
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - Pavle Zagorscak
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
| | | | - Manuel Heinrich
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
| | - Clara Miguel
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - Andrea Cipriani
- Department of Psychiatry, Warneford Hospital, University of Oxford, Oxford, UK; Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
| | - Orestis Efthimiou
- Department of Psychiatry, Warneford Hospital, University of Oxford, Oxford, UK; Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Eirini Karyotaki
- Department of Global Health and Social Medicine, Harvard Medical School, Boston, MA, USA
| | - Pim Cuijpers
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| |
Collapse
|
17
|
Chalkou K, Steyerberg E, Egger M, Manca A, Pellegrini F, Salanti G. A two-stage prediction model for heterogeneous effects of treatments. Stat Med 2021; 40:4362-4375. [PMID: 34048066 PMCID: PMC9291845 DOI: 10.1002/sim.9034] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2020] [Revised: 04/19/2021] [Accepted: 04/21/2021] [Indexed: 12/23/2022]
Abstract
Treatment effects vary across different patients, and estimation of this variability is essential for clinical decision‐making. We aimed to develop a model estimating the benefit of alternative treatment options for individual patients, extending a risk modeling approach in a network meta‐analysis framework. We propose a two‐stage prediction model for heterogeneous treatment effects by combining prognosis research and network meta‐analysis methods where individual patient data are available. In the first stage, a prognostic model to predict the baseline risk of the outcome. In the second stage, we use the baseline risk score from the first stage as a single prognostic factor and effect modifier in a network meta‐regression model. We apply the approach to a network meta‐analysis of three randomized clinical trials comparing the relapses in Natalizumab, Glatiramer Acetate, and Dimethyl Fumarate, including 3590 patients diagnosed with relapsing‐remitting multiple sclerosis. We find that the baseline risk score modifies the relative and absolute treatment effects. Several patient characteristics, such as age and disability status, impact the baseline risk of relapse, which in turn moderates the benefit expected for each of the treatments. For high‐risk patients, the treatment that minimizes the risk of relapse in 2 years is Natalizumab, whereas Dimethyl Fumarate might be a better option for low‐risk patients. Our approach can be easily extended to all outcomes of interest and has the potential to inform a personalized treatment approach.
Collapse
Affiliation(s)
- Konstantina Chalkou
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Ewout Steyerberg
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands
| | - Matthias Egger
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland.,Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Andrea Manca
- Centre for Health Economics, University of York, York, UK
| | | | - Georgia Salanti
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| |
Collapse
|
18
|
Seo M, White IR, Furukawa TA, Imai H, Valgimigli M, Egger M, Zwahlen M, Efthimiou O. Comparing methods for estimating patient-specific treatment effects in individual patient data meta-analysis. Stat Med 2020; 40:1553-1573. [PMID: 33368415 PMCID: PMC7898845 DOI: 10.1002/sim.8859] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2020] [Revised: 09/28/2020] [Accepted: 12/07/2020] [Indexed: 12/31/2022]
Abstract
Meta‐analysis of individual patient data (IPD) is increasingly used to synthesize data from multiple trials. IPD meta‐analysis offers several advantages over meta‐analyzing aggregate data, including the capacity to individualize treatment recommendations. Trials usually collect information on many patient characteristics. Some of these covariates may strongly interact with treatment (and thus be associated with treatment effect modification) while others may have little effect. It is currently unclear whether a systematic approach to the selection of treatment‐covariate interactions in an IPD meta‐analysis can lead to better estimates of patient‐specific treatment effects. We aimed to answer this question by comparing in simulations the standard approach to IPD meta‐analysis (no variable selection, all treatment‐covariate interactions included in the model) with six alternative methods: stepwise regression, and five regression methods that perform shrinkage on treatment‐covariate interactions, that is, least absolute shrinkage and selection operator (LASSO), ridge, adaptive LASSO, Bayesian LASSO, and stochastic search variable selection. Exploring a range of scenarios, we found that shrinkage methods performed well for both continuous and dichotomous outcomes, for a variety of settings. In most scenarios, these methods gave lower mean squared error of the patient‐specific treatment effect as compared with the standard approach and stepwise regression. We illustrate the application of these methods in two datasets from cardiology and psychiatry. We recommend that future IPD meta‐analysis that aim to estimate patient‐specific treatment effects using multiple effect modifiers should use shrinkage methods, whereas stepwise regression should be avoided.
Collapse
Affiliation(s)
- Michael Seo
- Institute of Social and Preventive MedicineUniversity of BernBernSwitzerland
- Graduate School for Health SciencesUniversity of BernBernSwitzerland
| | - Ian R. White
- MRC Clinical Trials Unit, Institute of Clinical Trials and MethodologyUniversity College LondonLondonUK
| | - Toshi A. Furukawa
- Departments of Health Promotion and Human Behavior and of Clinical EpidemiologyKyoto University Graduate School of Medicine/School of Public HealthKyotoJapan
| | - Hissei Imai
- Departments of Health Promotion and Human Behavior and of Clinical EpidemiologyKyoto University Graduate School of Medicine/School of Public HealthKyotoJapan
| | - Marco Valgimigli
- Department of Cardiology, Bern University HospitalUniversity of BernBernSwitzerland
| | - Matthias Egger
- Institute of Social and Preventive MedicineUniversity of BernBernSwitzerland
| | - Marcel Zwahlen
- Institute of Social and Preventive MedicineUniversity of BernBernSwitzerland
| | - Orestis Efthimiou
- Institute of Social and Preventive MedicineUniversity of BernBernSwitzerland
| |
Collapse
|