Synthesizing cross-design evidence and cross-format data using network meta-regression

crossnma: An R package to synthesize cross-design evidence and cross-format data using network meta-analysis and network meta-regression

BACKGROUND

Although aggregate data (AD) from randomised clinical trials (RCTs) are used in the majority of network meta-analyses (NMAs), other study designs (e.g., cohort studies and other non-randomised studies, NRS) can be informative about relative treatment effects. The individual participant data (IPD) of the study, when available, are preferred to AD for adjusting for important participant characteristics and to better handle heterogeneity and inconsistency in the network.

RESULTS

We developed the R package crossnma to perform cross-format (IPD and AD) and cross-design (RCT and NRS) NMA and network meta-regression (NMR). The models are implemented as Bayesian three-level hierarchical models using Just Another Gibbs Sampler (JAGS) software within the R environment. The R package crossnma includes functions to automatically create the JAGS model, reformat the data (based on user input), assess convergence and summarize the results. We demonstrate the workflow within crossnma by using a network of six trials comparing four treatments.

CONCLUSIONS

The R package crossnma enables the user to perform NMA and NMR with different data types in a Bayesian framework and facilitates the inclusion of all types of evidence recognising differences in risk of bias.

Effectiveness of pharmacological treatments for severe agitation in real-world emergency settings: protocol of individual-participant-data network meta-analysis

BACKGROUND

Severe psychomotor agitation and aggression often require immediate pharmacological intervention, but clear evidence-based recommendations for choosing among the multiple options are lacking. To address this gap, we plan a systematic review and individual-participant-data network meta-analysis to investigate their comparative effectiveness in real-world emergency settings with increased precision.

METHODS

We will include randomized controlled trials investigating intramuscular or intravenous pharmacological interventions, as monotherapy or in combination, in adults with severe psychomotor agitation irrespective of the underlying diagnosis and requiring rapid tranquilization in general or psychiatric emergency settings. We will exclude studies before 2002, those focusing on specific reasons for agitation and placebo-controlled trials to avoid concerns related to the transitivity assumption and potential selection biases. We will search for eligible studies in BIOSIS, CENTRAL, CINAHL Plus, Embase, LILACS, MEDLINE via Ovid, PubMed, ProQuest, PsycINFO, ClinicalTrials.gov, and WHO-ICTRP. Individual-participant data will be requested from the study authors and harmonized into a uniform format, and aggregated data will also be extracted from the studies. At least two independent reviewers will conduct the study selection, data extraction, risk-of-bias assessment using RoB 2, and applicability evaluation using the RITES tool. The primary outcome will be the number of patients achieving adequate sedation within 30 min after treatment, with secondary outcomes including the need for additional interventions and adverse events, using odds ratios as the effect size. If enough individual-participant data will be collected, we will synthesize them in a network meta-regression model within a Bayesian framework, incorporating study- and participant-level characteristics to explore potential sources of heterogeneity. In cases where individual-participant data are unavailable, potential data availability bias will be explored, and models allowing for the inclusion of studies reporting only aggregated data will be considered. We will assess the confidence in the evidence using the Confidence in Network Meta-Analysis (CINeMA) approach.

DISCUSSION

This individual-participant-data network meta-analysis aims to provide a fine-tuned synthesis of the evidence on the comparative effectiveness of pharmacological interventions for severe psychomotor agitation in real-world emergency settings. The findings from this study can greatly be provided clearer evidence-based guidance on the most effective treatments.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42023402365.

'Twenty years of network meta-analysis: Continuing controversies and recent developments': A health technology assessment perspective

We respond to some of the methodological issues raised in a recent review of network meta-analysis (NMA). We also provide a health technology developer's perspective and consider the future application of NMA to health technology assessment.

Response to discussant comments on "NMA, the first 20 years"

We respond to discussant comments on our paper "Twenty years of network meta-analysis: Continuing controversies and recent developments" (https://doi.org/10.1002/jrsm.1700) and raise some additional points for consideration, including: the way in which methodological guidance is generated; integration of the estimand framework with evidence synthesis; and implications of the European Joint Clinical Assessment. We ask: what properties are required of population adjustment methods to enable transparent and consistent decision-making? We also ask why individual patient data is not routinely made available to re-imbursement authorities and clinical guideline developers.

Combining randomized and non-randomized data to predict heterogeneous effects of competing treatments

Some patients benefit from a treatment while others may do so less or do not benefit at all. We have previously developed a two-stage network meta-regression prediction model that synthesized randomized trials and evaluates how treatment effects vary across patient characteristics. In this article, we extended this model to combine different sources of types in different formats: aggregate data (AD) and individual participant data (IPD) from randomized and non-randomized evidence. In the first stage, a prognostic model is developed to predict the baseline risk of the outcome using a large cohort study. In the second stage, we recalibrated this prognostic model to improve our predictions for patients enrolled in randomized trials. In the third stage, we used the baseline risk as effect modifier in a network meta-regression model combining AD, IPD randomized clinical trial to estimate heterogeneous treatment effects. We illustrated the approach in the re-analysis of a network of studies comparing three drugs for relapsing-remitting multiple sclerosis. Several patient characteristics influence the baseline risk of relapse, which in turn modifies the effect of the drugs. The proposed model makes personalized predictions for health outcomes under several treatment options and encompasses all relevant randomized and non-randomized evidence.

Twenty years of network meta-analysis: Continuing controversies and recent developments

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.

Semi-automated assessment of the risk of bias due to missing evidence in network meta-analysis: a guidance paper for the ROB-MEN web-application

Network meta-analysis compares multiple interventions and estimates the relative treatment effects between all interventions, combining both direct and indirect evidence. Recently, a framework was developed to assess the Risk Of Bias due to Missing Evidence in Network meta-analysis (ROB-MEN) which is part of the more comprehensive framework to evaluate the Confidence In the evidence for Network Meta-Analysis (CINeMA). To produce an overall risk of bias judgement for each network estimate, ROB-MEN: performs an assessment of the bias due to missing evidence in each possible pairwise comparison; combines the assessment with the contribution from the direct pairwise comparisons; considers the potential for small-study effects. To facilitate and semi-automate this process, ROB-MEN has been implemented in a user-friendly web-application ( https://cinema.ispm.unibe.ch/rob-men ). Here we provide a tutorial detailing the functionality and use of the application consisting of data upload, analysis configuration, output visualisation, and production of the tool's output tables for recording the risk of bias assessment. We also illustrate an example application using the demo dataset available for download on the application's homepage. The ROB-MEN web-application is open-source and freely available ( https://github.com/esm-ispm-unibe-ch/rob-men ).

Hierarchical network meta-analysis models for synthesis of evidence from randomised and non-randomised studies

BACKGROUND

With the increased interest in the inclusion of non-randomised data in network meta-analyses (NMAs) of randomised controlled trials (RCTs), analysts need to consider the implications of the differences in study designs as such data can be prone to increased bias due to the lack of randomisation and unmeasured confounding. This study aims to explore and extend a number of NMA models that account for the differences in the study designs, assessing their impact on the effect estimates and uncertainty.

METHODS

Bayesian random-effects meta-analytic models, including naïve pooling and hierarchical models differentiating between the study designs, were extended to allow for the treatment class effect and accounting for bias, with further extensions allowing for bias terms to vary depending on the treatment class. Models were applied to an illustrative example in type 2 diabetes; using data from a systematic review of RCTs and non-randomised studies of two classes of glucose-lowering medications: sodium-glucose co-transporter 2 inhibitors and glucagon-like peptide-1 receptor agonists.

RESULTS

Across all methods, the estimated mean differences in glycated haemoglobin after 24 and 52 weeks remained similar with the inclusion of observational data. The uncertainty around these estimates reduced when conducting naïve pooling, compared to NMA of RCT data alone, and remained similar when applying hierarchical model allowing for class effect. However, the uncertainty around these effect estimates increased when fitting hierarchical models allowing for the differences in study design. The impact on uncertainty varied between treatments when applying the bias adjustment models. Hierarchical models and bias adjustment models all provided a better fit in comparison to the naïve-pooling method.

CONCLUSIONS

Hierarchical and bias adjustment NMA models accounting for study design may be more appropriate when conducting a NMA of RCTs and observational studies. The degree of uncertainty around the effectiveness estimates varied depending on the method but use of hierarchical models accounting for the study design resulted in increased uncertainty. Inclusion of non-randomised data may, however, result in inferences that are more generalisable and the models accounting for the differences in the study design allow for more detailed and appropriate modelling of complex data, preventing overly optimistic conclusions.