1
|
Hamza T, Schwarzer G, Salanti G. crossnma: An R package to synthesize cross-design evidence and cross-format data using network meta-analysis and network meta-regression. BMC Med Res Methodol 2024; 24:169. [PMID: 39103781 DOI: 10.1186/s12874-023-02130-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Accepted: 12/19/2023] [Indexed: 08/07/2024] Open
Abstract
BACKGROUND Although aggregate data (AD) from randomised clinical trials (RCTs) are used in the majority of network meta-analyses (NMAs), other study designs (e.g., cohort studies and other non-randomised studies, NRS) can be informative about relative treatment effects. The individual participant data (IPD) of the study, when available, are preferred to AD for adjusting for important participant characteristics and to better handle heterogeneity and inconsistency in the network. RESULTS We developed the R package crossnma to perform cross-format (IPD and AD) and cross-design (RCT and NRS) NMA and network meta-regression (NMR). The models are implemented as Bayesian three-level hierarchical models using Just Another Gibbs Sampler (JAGS) software within the R environment. The R package crossnma includes functions to automatically create the JAGS model, reformat the data (based on user input), assess convergence and summarize the results. We demonstrate the workflow within crossnma by using a network of six trials comparing four treatments. CONCLUSIONS The R package crossnma enables the user to perform NMA and NMR with different data types in a Bayesian framework and facilitates the inclusion of all types of evidence recognising differences in risk of bias.
Collapse
Affiliation(s)
- Tasnim Hamza
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland.
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland.
| | - Guido Schwarzer
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Georgia Salanti
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| |
Collapse
|
2
|
Siafis S, Wu H, Nomura N, Schneider-Thoma J, Bighelli I, Lorenz C, Dib JE, Tharyan P, Calver LA, Isbister GK, Chan EWY, Knott JC, Yap CYL, Mantovani C, Martel ML, Barbic D, Honer WG, Hansen WP, Huf G, Alexander J, Raveendran NS, Coutinho ESF, Priller J, Adams CE, Salanti G, Leucht S. Effectiveness of pharmacological treatments for severe agitation in real-world emergency settings: protocol of individual-participant-data network meta-analysis. Syst Rev 2024; 13:205. [PMID: 39095865 PMCID: PMC11295517 DOI: 10.1186/s13643-024-02623-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Accepted: 07/20/2024] [Indexed: 08/04/2024] Open
Abstract
BACKGROUND Severe psychomotor agitation and aggression often require immediate pharmacological intervention, but clear evidence-based recommendations for choosing among the multiple options are lacking. To address this gap, we plan a systematic review and individual-participant-data network meta-analysis to investigate their comparative effectiveness in real-world emergency settings with increased precision. METHODS We will include randomized controlled trials investigating intramuscular or intravenous pharmacological interventions, as monotherapy or in combination, in adults with severe psychomotor agitation irrespective of the underlying diagnosis and requiring rapid tranquilization in general or psychiatric emergency settings. We will exclude studies before 2002, those focusing on specific reasons for agitation and placebo-controlled trials to avoid concerns related to the transitivity assumption and potential selection biases. We will search for eligible studies in BIOSIS, CENTRAL, CINAHL Plus, Embase, LILACS, MEDLINE via Ovid, PubMed, ProQuest, PsycINFO, ClinicalTrials.gov, and WHO-ICTRP. Individual-participant data will be requested from the study authors and harmonized into a uniform format, and aggregated data will also be extracted from the studies. At least two independent reviewers will conduct the study selection, data extraction, risk-of-bias assessment using RoB 2, and applicability evaluation using the RITES tool. The primary outcome will be the number of patients achieving adequate sedation within 30 min after treatment, with secondary outcomes including the need for additional interventions and adverse events, using odds ratios as the effect size. If enough individual-participant data will be collected, we will synthesize them in a network meta-regression model within a Bayesian framework, incorporating study- and participant-level characteristics to explore potential sources of heterogeneity. In cases where individual-participant data are unavailable, potential data availability bias will be explored, and models allowing for the inclusion of studies reporting only aggregated data will be considered. We will assess the confidence in the evidence using the Confidence in Network Meta-Analysis (CINeMA) approach. DISCUSSION This individual-participant-data network meta-analysis aims to provide a fine-tuned synthesis of the evidence on the comparative effectiveness of pharmacological interventions for severe psychomotor agitation in real-world emergency settings. The findings from this study can greatly be provided clearer evidence-based guidance on the most effective treatments. SYSTEMATIC REVIEW REGISTRATION PROSPERO CRD42023402365.
Collapse
Affiliation(s)
- Spyridon Siafis
- Technical University of Munich, TUM School of Medicine and Health, Department of Psychiatry and Psychotherapy, Klinikum rechts der Isar, Munich, Germany.
- German Center for Mental Health (DZPG), Partner site München/Augsburg, Munich, Germany.
| | - Hui Wu
- Technical University of Munich, TUM School of Medicine and Health, Department of Psychiatry and Psychotherapy, Klinikum rechts der Isar, Munich, Germany
| | - Nobuyuki Nomura
- Technical University of Munich, TUM School of Medicine and Health, Department of Psychiatry and Psychotherapy, Klinikum rechts der Isar, Munich, Germany
- German Center for Mental Health (DZPG), Partner site München/Augsburg, Munich, Germany
- Department of Neuropsychiatry, Keio University School of Medicine, Tokyo, Japan
| | - Johannes Schneider-Thoma
- Technical University of Munich, TUM School of Medicine and Health, Department of Psychiatry and Psychotherapy, Klinikum rechts der Isar, Munich, Germany
- German Center for Mental Health (DZPG), Partner site München/Augsburg, Munich, Germany
| | - Irene Bighelli
- Technical University of Munich, TUM School of Medicine and Health, Department of Psychiatry and Psychotherapy, Klinikum rechts der Isar, Munich, Germany
- German Center for Mental Health (DZPG), Partner site München/Augsburg, Munich, Germany
| | - Carolin Lorenz
- Technical University of Munich, TUM School of Medicine and Health, Department of Psychiatry and Psychotherapy, Klinikum rechts der Isar, Munich, Germany
| | - Joseph E Dib
- Division of Psychiatry & Applied Psychology, Institute of Mental Health, School of Medicine, University of Nottingham, Nottingham, Nottinghamshire, UK
| | - Prathap Tharyan
- Clinical Epidemiology Unit, Christian Medical Centre, Vellore, India
| | - Leonie A Calver
- School of Nursing and Midwifery, University of Newcastle, Callaghan, NSW, Australia
| | - Geoffrey K Isbister
- Clinical Toxicology Research Group, University of Newcastle, Newcastle, NSW, Australia
| | - Esther W Y Chan
- Department of Pharmacology and Pharmacy, University of Hong Kong, Pok Fu Lam, Hong Kong SAR, China
- Laboratory of Data Discovery for Health (D24H), Science Park, Hong Kong SAR, China
- Department of Pharmacy, HKU-Shenzhen Hospital, Shenzhen, China
- HKU-SZ Institute of Research and Innovation (SIRI), Shenzhen, China
| | - Jonathan C Knott
- Department of Critical Care, The University of Melbourne, Melbourne, VIC, Australia
| | - Celene Y L Yap
- Department of Nursing, Faculty of Medicine, Dentistry and Health Sciences, Melbourne School of Health Sciences, The University of Melbourne, Melbourne, VIC, Australia
| | - Célia Mantovani
- Department of Neurosciences and Behavior, Ribeirao Preto School of Medicine, Universidade de Sao Paulo, São Paulo, Brazil
| | - Marc L Martel
- Department of Emergency Medicine, Hennepin County Medical Center, Minneapolis, MN, USA
| | - David Barbic
- Department of Emergency Medicine, University of British Columbia, Vancouver, BC, Canada
- Centre for Health Evaluation & Outcomes Sciences, St. Paul's Hospital, Vancouver, BC, Canada
| | - William G Honer
- Department of Psychiatry, Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada
- BC Mental Health and Substance Use Services Research Institute, Vancouver, BC, Canada
| | | | - Gisele Huf
- National Institute of Quality Control in Health, Oswaldo Cruz Foundation, Av. Brasil 4365, Manguinhos, Rio de Janeiro, Brazil
| | | | - Nirmal S Raveendran
- Department of Psychiatry, Christian Medical College, Vellore, Tamil Nadu, India
| | - Evandro S F Coutinho
- Department of Epidemiology, Institute of Social Medicine, State University of Rio de Janeiro, Rio de Janeiro, Brazil
| | - Josef Priller
- Technical University of Munich, TUM School of Medicine and Health, Department of Psychiatry and Psychotherapy, Klinikum rechts der Isar, Munich, Germany
- German Center for Mental Health (DZPG), Partner site München/Augsburg, Munich, Germany
- Neuropsychiatry and Laboratory of Molecular Psychiatry, Charité - Universitätsmedizin Berlin and DZNE, Berlin, Germany
- University of Edinburgh and UK DRI, Edinburgh, UK
| | - Clive E Adams
- Institute of Mental Health, University of Nottingham, Nottingham, Nottinghamshire, UK
| | - Georgia Salanti
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| | - Stefan Leucht
- Technical University of Munich, TUM School of Medicine and Health, Department of Psychiatry and Psychotherapy, Klinikum rechts der Isar, Munich, Germany
- German Center for Mental Health (DZPG), Partner site München/Augsburg, Munich, Germany
| |
Collapse
|
3
|
Jackson D, Zhang L, Hettle R, Cooper M. 'Twenty years of network meta-analysis: Continuing controversies and recent developments': A health technology assessment perspective. Res Synth Methods 2024. [PMID: 39076063 DOI: 10.1002/jrsm.1740] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2024] [Revised: 06/21/2024] [Accepted: 07/08/2024] [Indexed: 07/31/2024]
Abstract
We respond to some of the methodological issues raised in a recent review of network meta-analysis (NMA). We also provide a health technology developer's perspective and consider the future application of NMA to health technology assessment.
Collapse
Affiliation(s)
- Dan Jackson
- Statistical Innovation Group, AstraZeneca, Cambridge, UK
| | - Landan Zhang
- Statistical Innovation Group, AstraZeneca, Cambridge, UK
| | - Robert Hettle
- Health Technology Assessment and Modelling Science, AstraZeneca, Cambridge, UK
| | | |
Collapse
|
4
|
Ades AE, Welton NJ, Dias S, Caldwell DM, Phillippo DM. Response to discussant comments on "NMA, the first 20 years". Res Synth Methods 2024. [PMID: 39054934 DOI: 10.1002/jrsm.1745] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2024] [Accepted: 07/09/2024] [Indexed: 07/27/2024]
Abstract
We respond to discussant comments on our paper "Twenty years of network meta-analysis: Continuing controversies and recent developments" (https://doi.org/10.1002/jrsm.1700) and raise some additional points for consideration, including: the way in which methodological guidance is generated; integration of the estimand framework with evidence synthesis; and implications of the European Joint Clinical Assessment. We ask: what properties are required of population adjustment methods to enable transparent and consistent decision-making? We also ask why individual patient data is not routinely made available to re-imbursement authorities and clinical guideline developers.
Collapse
Affiliation(s)
- A E Ades
- Population Health Sciences, Bristol Medical School, Bristol, UK
| | - Nicky J Welton
- Population Health Sciences, Bristol Medical School, Bristol, UK
| | - Sofia Dias
- Centre for Reviews and Dissemination, University of York, York, UK
| | | | | |
Collapse
|
5
|
Chalkou K, Hamza T, Benkert P, Kuhle J, Zecca C, Simoneau G, Pellegrini F, Manca A, Egger M, Salanti G. Combining randomized and non-randomized data to predict heterogeneous effects of competing treatments. Res Synth Methods 2024; 15:641-656. [PMID: 38501273 DOI: 10.1002/jrsm.1717] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 01/26/2024] [Accepted: 02/16/2024] [Indexed: 03/20/2024]
Abstract
Some patients benefit from a treatment while others may do so less or do not benefit at all. We have previously developed a two-stage network meta-regression prediction model that synthesized randomized trials and evaluates how treatment effects vary across patient characteristics. In this article, we extended this model to combine different sources of types in different formats: aggregate data (AD) and individual participant data (IPD) from randomized and non-randomized evidence. In the first stage, a prognostic model is developed to predict the baseline risk of the outcome using a large cohort study. In the second stage, we recalibrated this prognostic model to improve our predictions for patients enrolled in randomized trials. In the third stage, we used the baseline risk as effect modifier in a network meta-regression model combining AD, IPD randomized clinical trial to estimate heterogeneous treatment effects. We illustrated the approach in the re-analysis of a network of studies comparing three drugs for relapsing-remitting multiple sclerosis. Several patient characteristics influence the baseline risk of relapse, which in turn modifies the effect of the drugs. The proposed model makes personalized predictions for health outcomes under several treatment options and encompasses all relevant randomized and non-randomized evidence.
Collapse
Affiliation(s)
- Konstantina Chalkou
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
- Department of Clinical Research, University of Bern, Bern, Switzerland
| | - Tasnim Hamza
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
| | - Pascal Benkert
- Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Jens Kuhle
- Multiple Sclerosis Centre, Neurologic Clinic and Policlinic, Department of Head, Spine and Neuromedicine, University Hospital Basel, University of Basel, Basel, Switzerland
- Multiple Sclerosis Centre, Neurologic Clinic and Policlinic, Department of Biomedicine, University Hospital Basel, University of Basel, Basel, Switzerland
- Multiple Sclerosis Centre, Neurologic Clinic and Policlinic, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
- Research Center for Clinical Neuroimmunology and Neuroscience (RC2NB), University Hospital, University of Basel, Basel, Switzerland
| | - Chiara Zecca
- Multiple Sclerosis Center, Neurocenter of Southern Switzerland, EOC, Lugano, Switzerland
- Faculty of Biomedical Sciences, Università della Svizzera Italiana, Lugano, Switzerland
| | | | | | - Andrea Manca
- Centre for Health Economics, University of York, York, UK
| | - Matthias Egger
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Georgia Salanti
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| |
Collapse
|
6
|
Ades AE, Welton NJ, Dias S, Phillippo DM, Caldwell DM. Twenty years of network meta-analysis: Continuing controversies and recent developments. Res Synth Methods 2024. [PMID: 38234221 DOI: 10.1002/jrsm.1700] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Revised: 12/15/2023] [Accepted: 12/18/2023] [Indexed: 01/19/2024]
Abstract
Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.
Collapse
Affiliation(s)
- A E Ades
- Population Health Sciences, Bristol Medical School, Bristol, UK
| | - Nicky J Welton
- Population Health Sciences, Bristol Medical School, Bristol, UK
| | - Sofia Dias
- Centre for Reviews and Dissemination, University of York, York, UK
| | | | | |
Collapse
|
7
|
Chiocchia V, Holloway A, Salanti G. Semi-automated assessment of the risk of bias due to missing evidence in network meta-analysis: a guidance paper for the ROB-MEN web-application. BMC Med Res Methodol 2023; 23:223. [PMID: 37805460 PMCID: PMC10559514 DOI: 10.1186/s12874-023-02038-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Accepted: 09/20/2023] [Indexed: 10/09/2023] Open
Abstract
Network meta-analysis compares multiple interventions and estimates the relative treatment effects between all interventions, combining both direct and indirect evidence. Recently, a framework was developed to assess the Risk Of Bias due to Missing Evidence in Network meta-analysis (ROB-MEN) which is part of the more comprehensive framework to evaluate the Confidence In the evidence for Network Meta-Analysis (CINeMA). To produce an overall risk of bias judgement for each network estimate, ROB-MEN: performs an assessment of the bias due to missing evidence in each possible pairwise comparison; combines the assessment with the contribution from the direct pairwise comparisons; considers the potential for small-study effects. To facilitate and semi-automate this process, ROB-MEN has been implemented in a user-friendly web-application ( https://cinema.ispm.unibe.ch/rob-men ). Here we provide a tutorial detailing the functionality and use of the application consisting of data upload, analysis configuration, output visualisation, and production of the tool's output tables for recording the risk of bias assessment. We also illustrate an example application using the demo dataset available for download on the application's homepage. The ROB-MEN web-application is open-source and freely available ( https://github.com/esm-ispm-unibe-ch/rob-men ).
Collapse
Affiliation(s)
- Virginia Chiocchia
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland.
| | - Alexander Holloway
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Georgia Salanti
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| |
Collapse
|
8
|
Hussein H, Abrams KR, Gray LJ, Anwer S, Dias S, Bujkiewicz S. Hierarchical network meta-analysis models for synthesis of evidence from randomised and non-randomised studies. BMC Med Res Methodol 2023; 23:97. [PMID: 37087450 PMCID: PMC10122363 DOI: 10.1186/s12874-023-01925-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Accepted: 04/14/2023] [Indexed: 04/24/2023] Open
Abstract
BACKGROUND With the increased interest in the inclusion of non-randomised data in network meta-analyses (NMAs) of randomised controlled trials (RCTs), analysts need to consider the implications of the differences in study designs as such data can be prone to increased bias due to the lack of randomisation and unmeasured confounding. This study aims to explore and extend a number of NMA models that account for the differences in the study designs, assessing their impact on the effect estimates and uncertainty. METHODS Bayesian random-effects meta-analytic models, including naïve pooling and hierarchical models differentiating between the study designs, were extended to allow for the treatment class effect and accounting for bias, with further extensions allowing for bias terms to vary depending on the treatment class. Models were applied to an illustrative example in type 2 diabetes; using data from a systematic review of RCTs and non-randomised studies of two classes of glucose-lowering medications: sodium-glucose co-transporter 2 inhibitors and glucagon-like peptide-1 receptor agonists. RESULTS Across all methods, the estimated mean differences in glycated haemoglobin after 24 and 52 weeks remained similar with the inclusion of observational data. The uncertainty around these estimates reduced when conducting naïve pooling, compared to NMA of RCT data alone, and remained similar when applying hierarchical model allowing for class effect. However, the uncertainty around these effect estimates increased when fitting hierarchical models allowing for the differences in study design. The impact on uncertainty varied between treatments when applying the bias adjustment models. Hierarchical models and bias adjustment models all provided a better fit in comparison to the naïve-pooling method. CONCLUSIONS Hierarchical and bias adjustment NMA models accounting for study design may be more appropriate when conducting a NMA of RCTs and observational studies. The degree of uncertainty around the effectiveness estimates varied depending on the method but use of hierarchical models accounting for the study design resulted in increased uncertainty. Inclusion of non-randomised data may, however, result in inferences that are more generalisable and the models accounting for the differences in the study design allow for more detailed and appropriate modelling of complex data, preventing overly optimistic conclusions.
Collapse
Affiliation(s)
- Humaira Hussein
- Biostatistics Research Group, Department of Population Health Sciences, University of Leicester, University Road, Leicester, LE1 7RH, UK
| | - Keith R Abrams
- Department of Statistics, University of Warwick, Coventry, CV4 7AL, UK
- Centre for Health Economics, University of York, York, YO10 5DD, UK
| | - Laura J Gray
- Biostatistics Research Group, Department of Population Health Sciences, University of Leicester, University Road, Leicester, LE1 7RH, UK
| | - Sumayya Anwer
- Centre for Reviews and Dissemination, University of York, York, YO10 5DD, UK
| | - Sofia Dias
- Centre for Reviews and Dissemination, University of York, York, YO10 5DD, UK
| | - Sylwia Bujkiewicz
- Biostatistics Research Group, Department of Population Health Sciences, University of Leicester, University Road, Leicester, LE1 7RH, UK.
| |
Collapse
|