Addressing between-study heterogeneity and inconsistency in mixed treatment comparisons: Application to stroke prevention treatments in individuals with non-rheumatic atrial fibrillation

Effects of classical psychedelics on implicit and explicit emotional empathy and cognitive empathy: a meta-analysis of MET task

This meta-analysis investigates the effect of classic psychedelic drugs on empathy and focuses on cognitive and emotional empathy measured using the Multifaceted Empathy Test (MET). Empathy entails the ability to understand and share the feelings of another and is a significant component of social interaction. Several studies have examined the effects of psychedelic drugs such as LSD, psilocybin and ayahuasca on empathy, yet their overall effect has not been studied so far. In this meta-analysis, we reviewed data from studies up to November 2023 with the aim of examining the effects of various psychedelic drugs on empathic abilities broadly. Our findings suggest that classical psychedelics significantly enhance explicit and implicit emotional empathy without affecting measures of cognitive empathy. The results emphasize the need to continue testing the therapeutic potential of classic psychedelic drugs.

Combining GLP-1 receptor agonists and SGLT-2 inhibitors for cardiovascular disease prevention in type 2 diabetes: A systematic review with multiple network meta-regressions

BACKGROUND

Glucagon-like peptide-1 receptor agonists (GLP-1RA) and sodium-glucose co-transporter-2 inhibitors (SGLT-2I) are associated with significant cardiovascular benefit in type 2 diabetes (T2D). However, GLP-1RA or SGLT-2I alone may not improve some cardiovascular outcomes in patients with prior cardiovascular co-morbidities.

AIM

To explore whether combining GLP-1RA and SGLT-2I can achieve additional benefit in preventing cardiovascular diseases in T2D.

METHODS

The systematic review was conducted according to PRISMA recommendations. The protocol was registered on PROSPERO (ID: 42022385007). A total of 107049 participants from eligible cardiovascular outcomes trials of GLP-1RA and SGLT-2I were included in network meta-regressions to estimate cardiovascular benefit of the combination treatment. Effect modification of prior myocardial infarction (MI) and heart failure (HF) was also explored to provide clinical insight as to when the combination treatment should be considered.

RESULTS

The estimated hazard ratios (HR)_{GLP-1RA/SGLT-2IvsPlacebo} (0.75-0.98) and HR_{CombinationvsGLP-1RA/SGLT-2I} (0.26-0.86) for primary and secondary cardiovascular outcomes suggested that the combination treatment may achieve additional cardiovascular benefit compared with GLP-1RA or SGLT-2I alone. In patients with prior MI or HF, the mono-therapies may not improve the overall cardiovascular outcomes, as the estimated HR_MI+/HF+ (0.57-1.52) suggested that GLP-1RA or SGLT-2I alone may be associated with lower risks of hospitalization for HF but not cardiovascular death.

CONCLUSION

Considering its greater cardiovascular benefit in T2D, the combination treatment of GLP-1RA and SGLT-2I might be prioritized in patients with prior MI or HF, where the monotherapies may not provide sufficient cardiovascular protection.

Brain volume loss in relapsing multiple sclerosis: indirect treatment comparisons of available disease-modifying therapies

BACKGROUND

Brain volume loss (BVL) has been identified as a predictor of disability progression in relapsing multiple sclerosis (RMS). As many available disease-modifying treatments (DMTs) have shown an effect on slowing BVL, this is becoming an emerging clinical endpoint in RMS clinical trials.

METHODS

In this study, a systematic literature review was conducted to identify BVL results from randomized controlled trials of DMTs in RMS. Indirect treatment comparisons (ITCs) were conducted to estimate the relative efficacy of DMTs on BVL using two approaches: a model-based meta-analysis (MBMA) with adjustment for measurement timepoint and DMT dosage, and a network meta-analysis (NMA).

RESULTS

In the MBMA, DMTs associated with significantly reduced BVL versus placebo at two years included fingolimod (mean difference [MD] = 0.25; 95% confidence interval [CI] = 0.15 - 0.36), ozanimod (MD = 0.26; 95% CI = 0.12 - 0.41), teriflunomide (MD = 0.38; 95% CI = 0.20 - 0.55), alemtuzumab (MD = 0.38; 95% CI = 0.10 - 0.67) and ponesimod (MD = 0.71; 95% CI = 0.48 - 0.95), whereas interferons and natalizumab performed the most poorly. The results of NMA analysis were generally comparable with those of the MBMA.

CONCLUSIONS

Limitations of these analyses included the potential for confounding due to pseudoatrophy, and a lack of long-term clinical data for BVL. Our findings suggest that important differences in BVL may exist between DMTs. Continued investigation of BVL in studies of RMS is important to complement traditional disability endpoints, and to foster a better understanding of the mechanisms by which DMTs can slow BVL.

Implementing Multilevel Network Meta-Regression for Time-To-Event Outcomes: A Case Study in Relapsed Refractory Multiple Myeloma

OBJECTIVES

Multilevel network meta-regression (ML-NMR) leverages individual patient data (IPD) and aggregate data from a network of randomized controlled trials (RCTs) to assess the comparative efficacy of multiple treatments, while adjusting for between-study differences. We provide an overview of ML-NMR for time-to-event outcomes and apply it to an illustrative case study, including example R code.

METHODS

The case study evaluated the comparative efficacy of idecabtagene vicleucel (ide-cel), selinexor+dexamethasone (Sd), belantamab mafodotin (BM), and conventional care (CC) for patients with triple-class exposed relapsed/refractory multiple myeloma in terms of overall survival. Single-arm clinical trials and real-world data were naively combined to create an aggregate data artificial RCT (aRCT) (MAMMOTH-CC versus DREAMM-2-BM versus STORM-2-Sd) and an IPD aRCT (KarMMa-ide-cel versus KarMMa-RW-CC). With some assumptions, we incorporated continuous covariates with skewed distributions, reported as median and range. The ML-NMR models adjusted for number of prior lines, triple-class refractory status, and age and were compared using the leave-one-out information criterion. We summarized predicted hazard ratios and survival (95% credible intervals) in the IPD aRCT population.

RESULTS

The Weibull ML-NMR model had the lowest leave-one-out information criterion. Ide-cel was more efficacious than Sd, BM, and CC in terms of overall survival. Effect modifiers had minimal impact on the model, and only triple-class refractory was a prognostic factor.

CONCLUSIONS

We demonstrate an application of ML-NMR for time-to-event outcomes and introduce code that can be used to aid implementation. Given its benefits, we encourage practitioners to utilize ML-NMR when population adjustment is necessary for comparisons of multiple treatments.

Effects of non-invasive respiratory support in post-operative patients: a systematic review and network meta-analysis

BACKGROUND

Re-intubation secondary to post-extubation respiratory failure in post-operative patients is associated with increased patient morbidity and mortality. Non-invasive respiratory support (NRS) alternative to conventional oxygen therapy (COT), i.e., high-flow nasal oxygen, continuous positive airway pressure, and non-invasive ventilation (NIV), has been proposed to prevent or treat post-extubation respiratory failure. Aim of the present study is assessing the effects of NRS application, compared to COT, on the re-intubation rate (primary outcome), and time to re-intubation, incidence of nosocomial pneumonia, patient discomfort, intensive care unit (ICU) and hospital length of stay, and mortality (secondary outcomes) in adult patients extubated after surgery.

METHODS

A systematic review and network meta-analysis of randomized and non-randomized controlled trials. A search from Medline, Embase, Scopus, Cochrane Central Register of Controlled Trials, and Web of Science from inception until February 2, 2024 was performed.

RESULTS

Thirty-three studies (11,292 patients) were included. Among all NRS modalities, only NIV reduced the re-intubation rate, compared to COT (odds ratio 0.49, 95% confidence interval 0.28; 0.87, p = 0.015, I2 = 60.5%, low certainty of evidence). In particular, this effect was observed in patients receiving NIV for treatment, while not for prevention, of post-extubation respiratory failure, and in patients at high, while not low, risk of post-extubation respiratory failure. NIV reduced the rate of nosocomial pneumonia, ICU length of stay, and ICU, hospital, and long-term mortality, while not worsening patient discomfort.

CONCLUSIONS

In post-operative patients receiving NRS after extubation, NIV reduced the rate of re-intubation, compared to COT, when used for treatment of post-extubation respiratory failure and in patients at high risk of post-extubation respiratory failure.

Heterogeneity in meta-analysis: a comprehensive overview

In recent years, meta-analysis has evolved to a critically important field of Statistics, and has significant applications in Medicine and Health Sciences. In this work we briefly present existing methodologies to conduct meta-analysis along with any discussion and recent developments accompanying them. Undoubtedly, studies brought together in a systematic review will differ in one way or another. This yields a considerable amount of variability, any kind of which may be termed heterogeneity. To this end, reports of meta-analyses commonly present a statistical test of heterogeneity when attempting to establish whether the included studies are indeed similar in terms of the reported output or not. We intend to provide an overview of the topic, discuss the potential sources of heterogeneity commonly met in the literature and provide useful guidelines on how to address this issue and to detect heterogeneity. Moreover, we review the recent developments in the Bayesian approach along with the various graphical tools and statistical software that are currently available to the analyst. In addition, we discuss sensitivity analysis issues and other approaches of understanding the causes of heterogeneity. Finally, we explore heterogeneity in meta-analysis for time to event data in a nutshell, pointing out its unique characteristics.

Pembrolizumab plus chemotherapy for first-line treatment of advanced triple-negative breast cancer

Aim: A systematic review and network meta-analysis (NMA) was performed to evaluate the efficacy of first-line treatments for locally recurrent unresectable or metastatic triple-negative breast cancer (TNBC) patients.Materials & methods: Databases were searched for randomized controlled trials evaluating first-line treatments for locally recurrent unresectable or metastatic TNBC patients. NMA was performed to estimate relative treatment effects on overall and progression-free survival between pembrolizumab + chemotherapy and other interventions.Results: NMA including eight trials showed that the relative efficacy of pembrolizumab + chemotherapy was statistically superior to that of other immunotherapy- or chemotherapy-based treatment regimens.Conclusion: Pembrolizumab + chemotherapy confers benefits in survival outcomes versus alternative interventions for the first-line treatment of locally recurrent unresectable or metastatic TNBC patients.

Effectiveness of Rehabilitation for Osteoarthritis of the Knee: A Scoping Review of Network Meta-Analyses

BACKGROUND

 Although an increasing number of network meta-analyses have been conducted on the effectiveness of conservative therapy for knee osteoarthritis, these may have been poorly planned and executed.We aimed to review the qualities of a comprehensive set of network meta-analyses on rehabilitation therapies for knee osteoarthritis and provide an overview of the effectiveness of each therapy.

METHODS

The eligibility criteria were as follows: (i) conservative rehabilitation was the primary treatment in the intervention group, (ii) included patients were diagnosed with knee osteoarthritis, and (iii) patient groups were aged ≤75 years, and rehabilitation interventions comprised exercise, orthotic, or physical therapies or patient education. Two independent reviewers screened the titles and abstracts of the identified records and selected the eligible reviews; their full texts were further assessed for eligibility. Then, a checklist derived from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension statement for the reporting of systematic reviews incorporating network meta-analyses of healthcare Interventions was used to validate the completeness of the reporting of each network meta-analysis. Furthermore, the statistical and outcome data regarding the quality of life, knee joint function and pain, adverse events, and physical functions were extracted using a customized spreadsheet.

RESULTS

Overall, 2701 titles and abstracts were screened, and eight network meta-analyses were ultimately selected. Nearly all reviews adequately addressed the PRISMA extension checklist, and the completeness of reporting was adequate; therefore, all expected information could be extracted. However, the methodology used to confirm the transitivity assumption was insufficient in many reviews. The following interventions were effective in reducing pain and improving physical function: (i) strengthening, flexibility, aerobic, and mind-body exercises, (ii) pulsed ultrasound, (iii) focused and radial extracorporeal shockwave therapy, and (iv) continuous ultrasound. The following interventions were effective in improving the quality of life: (i) strengthening, (ii) mixed, and (iii) mind-body exercises.

CONCLUSIONS

Our results suggested that exercise therapies, including muscle-strengthening, aerobic, flexibility, and mind-body exercises, are likely to be effective for pain relief and functional improvement in knee osteoarthritis. This may be the first review to provide a comprehensive perspective for considering priorities for future rehabilitation interventions for knee osteoarthritis.

Low awareness of the transitivity assumption in complex networks of interventions: a systematic survey from 721 network meta-analyses

BACKGROUND

The transitivity assumption is the cornerstone of network meta-analysis (NMA). Violating transitivity compromises the credibility of the indirect estimates and, by extent, the estimated treatment effects of the comparisons in the network. The present study offers comprehensive empirical evidence on the completeness of reporting and evaluating transitivity in systematic reviews with multiple interventions.

METHODS

We screened the datasets of two previous empirical studies, resulting in 361 systematic reviews with NMA published between January 2011 and April 2015. We updated our evidence base with an additional 360 systematic reviews with NMA published between 2016 and 2021, employing a pragmatic approach. We devised assessment criteria for reporting and evaluating transitivity using relevant methodological literature and compared their reporting frequency before and after the PRISMA-NMA statement.

RESULTS

Systematic reviews published after PRISMA-NMA were more likely to provide a protocol (odds ratio (OR): 3.94, 95% CI: 2.79-5.64), pre-plan the transitivity evaluation (OR: 3.01, 95% CI: 1.54-6.23), and report the evaluation and results (OR: 2.10, 95% CI: 1.55-2.86) than those before PRISMA-NMA. However, systematic reviews after PRISMA-NMA were less likely to define transitivity (OR: 0.57, 95% CI: 0.42-0.79) and discuss the implications of transitivity (OR: 0.48, 95% CI: 0.27-0.85) than those published before PRISMA-NMA. Most systematic reviews evaluated transitivity statistically than conceptually (40% versus 12% before PRISMA-NMA, and 54% versus 11% after PRISMA-NMA), with consistency evaluation being the most preferred (34% before versus 47% after PRISMA-NMA). One in five reviews inferred the plausibility of the transitivity (22% before versus 18% after PRISMA-NMA), followed by 11% of reviews that found it difficult to judge transitivity due to insufficient data. In justifying their conclusions, reviews considered mostly the comparability of the trials (24% before versus 30% after PRISMA-NMA), followed by the consistency evaluation (23% before versus 16% after PRISMA-NMA).

CONCLUSIONS

Overall, there has been a slight improvement in reporting and evaluating transitivity since releasing PRISMA-NMA, particularly in items related to the systematic review report. Nevertheless, there has been limited attention to pre-planning the transitivity evaluation and low awareness of the conceptual evaluation methods that align with the nature of the assumption.

Efficacy of biomaterials for lateral bone augmentation performed with guided bone regeneration. A network meta-analysis

Bone regeneration is often required concomitant with implant placement to treat a bone fenestration, a dehiscence, and for contouring. This systematic review assessed the impact of different biomaterials employed for guided bone regeneration (GBR) simultaneous to implant placement on the stability of radiographic peri-implant bone levels at ≥12 months of follow-up (focused question 1), as well as on bone defect dimension (width/height) changes at re-assessment after ≥4 months (focused question 2). Only randomized controlled trials (RCTs) and controlled clinical trials (CCTs) that compared different biomaterials for GBR were considered. A Bayesian network meta-analysis (NMA) was performed using a random-effects model. A ranking probability between treatments was obtained, as well as an estimation of the surface under the cumulative ranking value (SUCRA). Overall, whenever the biological principle of GBR was followed, regeneration occurred in a predictable way, irrespective of the type of biomaterial used. A lower efficacy of GBR treatments was suggested for initially large defects, despite the trend did not reach statistical significance. Regardless of the biomaterial employed, a certain resorption of the augmented bone was observed overtime. While GBR was shown to be a safe and predictable treatment, several complications (including exposure, infection, and soft tissue dehiscence) were reported, which tend to be higher when using cross-linked collagen membranes.

Neoadjuvant immunotherapy and chemotherapy regimens for the treatment of high-risk, early-stage triple-negative breast cancer: a systematic review and network meta-analysis

BACKGROUND

Patients with triple-negative breast cancer (TNBC) are generally younger and more likely to experience disease recurrence and have the shortest survival among all breast cancer patients. Recently, neoadjuvant delivery of the programmed cell death protein-1 inhibitor pembrolizumab plus chemotherapy followed by adjuvant pembrolizumab was approved for patients with high-risk, early-stage TNBC, but this treatment regimen has not been evaluated in head-to-head trials with other neoadjuvant treatment regimens. Therefore, the objective of this study was to estimate the relative efficacy of neoadjuvant pembrolizumab + chemotherapy followed by adjuvant pembrolizumab versus other neoadjuvant treatments for early-stage TNBC through a systematic review and network meta-analysis (NMA).

METHODS

EMBASE, MEDLINE, Cochrane Central Register of Controlled Trials, conference abstracts, and clinical trial registries were searched for randomized controlled trials evaluating neoadjuvant treatments for early-stage TNBC. NMA was performed to estimate relative treatment effects among evaluated interventions.

RESULTS

Five trials met the inclusion criteria and were included in the NMA. The relative efficacy of neoadjuvant pembrolizumab + chemotherapy followed by adjuvant pembrolizumab was favorable to paclitaxel followed by anthracycline + cyclophosphamide in terms of pathologic complete response (pCR), event-free survival (EFS), and overall survival; paclitaxel + carboplatin followed by anthracycline + cyclophosphamide in terms of pCR and EFS; paclitaxel + bevacizumab followed by anthracycline + cyclophosphamide + bevacizumab in terms of pCR; and paclitaxel + carboplatin + veliparib followed by anthracycline + cyclophosphamide in terms of EFS.

CONCLUSIONS

Neoadjuvant pembrolizumab + chemotherapy followed by adjuvant pembrolizumab confers benefits in response and survival outcomes versus alternative neoadjuvant treatments for early-stage TNBC.

Network meta-interpolation: Effect modification adjustment in network meta-analysis using subgroup analyses

Effect modification (EM) may cause bias in network meta-analysis (NMA). Existing population adjustment NMA methods use individual patient data to adjust for EM but disregard available subgroup information from aggregated data in the evidence network. Additionally, these methods often rely on the shared effect modification (SEM) assumption. In this paper, we propose Network Meta-Interpolation (NMI): a method using subgroup analyses to adjust for EM that does not assume SEM. NMI balances effect modifiers across studies by turning treatment effect (TE) estimates at the subgroup- and study level into TE and standard errors at EM values common to all studies. In an extensive simulation study, we simulate two evidence networks consisting of four treatments, and assess the impact of departure from the SEM assumption, variable EM correlation across trials, trial sample size and network size. NMI was compared to standard NMA, network meta-regression (NMR) and Multilevel NMR (ML-NMR) in terms of estimation accuracy and credible interval (CrI) coverage. In the base case non-SEM dataset, NMI achieved the highest estimation accuracy with root mean squared error (RMSE) of 0.228, followed by standard NMA (0.241), ML-NMR (0.447) and NMR (0.541). In the SEM dataset, NMI was again the most accurate method with RMSE of 0.222, followed by ML-NMR (0.255). CrI coverage followed a similar pattern. NMI's dominance in terms of estimation accuracy and CrI coverage appeared to be consistent across all scenarios. NMI represents an effective option for NMA in the presence of study imbalance and available subgroup data.

Comparative efficacy and complications of long-acting and intermediate-acting insulin regimens for adults with type 1 diabetes: an individual patient data network meta-analysis

OBJECTIVE

To examine the comparative efficacy and complications of long-acting and intermediate-acting insulin for different patient characteristics for type 1 diabetes mellitus (T1DM).

DESIGN

Systematic review and individual patient data (IPD) network meta-analysis (NMA).

DATA SOURCES

MEDLINE, EMBASE and the Cochrane Central Register of Controlled Trials were searched through June 2015.

ELIGIBILITY CRITERIA

Randomised controlled trials (RCTs) on adults with T1DM assessing glycosylated haemoglobin (A1c) and severe hypoglycaemia in long-acting and intermediate-acting insulin regimens.

DATA EXTRACTION AND SYNTHESIS

We requested IPD from authors and funders. When IPD were not available, we used aggregate data. We conducted a random-effects model, and specifically a one-stage IPD-NMA for those studies providing IPD and a two-stage IPD-NMA to incorporate those studies not providing IPD.

RESULTS

We included 28 RCTs plus one companion report, after screening 6680 titles/abstracts and 205 full-text articles. Of the 28 RCTs, 27 studies provided data for the NMA with 7394 participants, of which 12 RCTs had IPD on 4943 participants. The IPD-NMA for A1c suggested that glargine once daily (mean difference [MD]=-0.31, 95% confidence interval [CI]: -0.48 to -0.14) and detemir once daily (MD=-0.25, 95% CI: -0.41 to -0.09) were superior to neutral protamine Hagedorn (NPH) once daily. NPH once/two times per day improved A1c compared with NPH once daily (MD=-0.30, 95% CI: -0.50 to -0.11). Results regarding complications in severe hypoglycaemia should be considered with great caution due to inconsistency in the evidence network. Accounting for missing data, there was no evidence of inconsistency and long-acting insulin regimens ranked higher regarding reducing severe hypoglycaemia compared with intermediate-acting insulin regimens (two-stage NMA: glargine two times per day SUCRA (Surface Under the Cumulative Ranking curve)=89%, detemir once daily SUCRA=77%; one-stage NMA: detemir once daily/two times per day SUCRA=85%). Using multiple imputations and IPD only, complications in severe hypoglycaemia increased with diabetes-related comorbidities (regression coefficient: 1.03, 95% CI: 1.02 to 1.03).

CONCLUSIONS

Long-acting insulin regimens reduced A1c compared with intermediate-acting insulin regimens and were associated with lower severe hypoglycaemia. Of the observed differences, only glargine once daily achieved a clinically significant reduction of 0.30%. Results should be interpreted with caution due to very low quality of evidence.

PROSPERO REGISTRATION NUMBER

CRD42015023511.

Herbal Granules of Heat-Clearing and Detoxifying for Children with Mild Hand, Foot, and Mouth Disease: A Bayesian Network Meta-Analysis

Background

Regarding ethical considerations of randomized controlled trials (RCTs) in children, limited evidence for mild hand, foot, and mouth disease (HFMD) is available. Recently, with the increasing but result-conflicting RCTs published around herbal granules of heat-clearing and detoxifying (HGs-HD), a head-to-head comparison is urgently needed to choose a suitable therapy for clinical practice.

Materials and Methods

This study was conducted according to the preferred reporting items for systematic review and meta-analysis (PRISMA) extension statement for network meta-analysis (NMA). Eight databases (Medline, Embase, and so on) and two trial registry platforms (https://www.clinicaltrials.gov and https://www.chictr.org.cn) were searched from inception to May 26, 2021. The NMA was performed using a random-effect model. The treatment hierarchy was summarized and reported as the surface under the cumulative ranking curve (SUCRA) probability values. The rankings of each HGs-HD at primary outcomes were estimated by the inverse probability weighting (IPW) approach and averaged, which presents the comprehensive improvement effect.

Results

Forty-five RCTs involving 18 interventions were included that studied 5,652 children with mild HFMD. The best performance probability for improving symptoms were respectively presented in terms of fever (Xiao'er Resuqing granules, XRGs, 94.9%), rash (Xiao'er Jinqiao granules, 83.9%), hospitalization (Xiao'er Chiqiao Qingre granules, XCQGs, 92.7%), vesicles (Jinlianhua granules, 91.0%), appetite (Xiao'er Chiqiao Qingre granules, XCQGs, 86.7%), and ulcers (Kouyanqing granules, KouGs, 88.8%). Furthermore, the top 5 rankings for comprehensive improvement effect were Yanning granules (YNGs, 2.256), XCQGs (2.858), XRGs (3.270), KouGs (7.223), and Houerhuan Xiaoyan granules (HXGs, 7.597).

Conclusions

This is the first NMA of HGs-HD head-to-head comparisons for children with mild HFMD. Of those, YNGs, XCQGs, XRGs, KouGs, and HXGs could be recommended as potential choices for clinical practice. Of course, the results should be interpreted with caution due to the limited high-quality RCTs.

A comparison of methods for enriching network meta-analyses in the absence of individual patient data

During drug development, a biomarker is sometimes identified as separating a patient population into those with more and those with less benefit from evaluated treatments. Consequently, later studies might be targeted, while earlier ones are performed in mixed patient populations. This poses a challenge in evidence synthesis, especially if only aggregated data are available. Starting from this scenario, we investigate three commonly used network meta-analytic estimation methods, the naive estimation approach, the stand-alone analysis, and the network meta-regression. Additionally, we adapt and modify two methods which are used in evidence synthesis to combine randomized controlled trials with observational studies, the enrichment-through-weighting approach and the informative prior estimation. We evaluate all five methods in a simulation study with 32 scenarios using bias, RMSE, coverage, precision, and power. Additionally, we re-visit a clinical data set to exemplify and discuss the application. In the simulation study, none of the methods was observed to be clearly favorable over all investigated scenarios. However, the stand-alone analysis and the naive estimation performed comparably or worse than the other methods in all evaluated performance measures and simulation scenarios and are therefore not recommended. While substantial between-trial heterogeneity is challenging for all estimation approaches, the performance of the network meta-regression, the enriching-through weighting approach and the informative prior approach was dependent on the simulation scenario and the performance measure of interest. Furthermore, as these estimation methods are drawing slightly different assumptions, some of which require the presence of additional information for estimation, we recommend sensitivity-analyses wherever possible. This article is protected by copyright. All rights reserved.

Bayesian models for aggregate and individual patient data component network meta-analysis

Network meta-analysis can synthesize evidence from studies comparing multiple treatments for the same disease. Sometimes the treatments of a network are complex interventions, comprising several independent components in different combinations. A component network meta-analysis (CNMA) can be used to analyze such data and can in principle disentangle the individual effect of each component. However, components may interact with each other, either synergistically or antagonistically. Deciding which interactions, if any, to include in a CNMA model may be difficult, especially for large networks with many components. In this article, we present two Bayesian CNMA models that can be used to identify prominent interactions between components. Our models utilize Bayesian variable selection methods, namely the stochastic search variable selection and the Bayesian LASSO, and can benefit from the inclusion of prior information about important interactions. Moreover, we extend these models to combine data from studies providing aggregate information and studies providing individual patient data (IPD). We illustrate our models in practice using three real datasets, from studies in panic disorder, depression, and multiple myeloma. Finally, we describe methods for developing web-applications that can utilize results from an IPD-CNMA, to allow for personalized estimates of relative treatment effects given a patient's characteristics.

Physical therapy and orthopaedic equipment-induced reduction in the biomechanical risk factors related to knee osteoarthritis: a systematic review and Bayesian network meta-analysis of randomised controlled trials

OBJECTIVE

Are physical therapy or orthopaedic equipment efficacious in reducing the biomechanical risk factors in people with tibiofemoral osteoarthritis (OA)? Is there a better therapeutic intervention than others to improve these outcomes?

DESIGN

Systematic review with network meta-analysis (NMA) of randomised trials.

DATA SOURCES

PubMed, Web of Science, Cochrane Library, Embase and MEDLINE were searched through January 2021.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

We included randomised controlled trials exploring the benefits of using physical therapy or orthopaedic equipment in reducing the biomechanical risk factors which included knee adduction moment (KAM) and knee adduction angular impulse (KAAI) in individuals with tibiofemoral OA.

DATA EXTRACTION AND SYNTHESIS

Two authors extracted data independently and assessed risk of bias. We conducted an NMA to compare multiple interventions, including both direct and indirect evidences. Heterogeneity was assessed (sensitivity analysis) and quantified (I2 statistic). Grading of Recommendations Assessment, Development and Evaluation assessed the certainty of the evidence.

RESULTS

Eighteen randomised controlled trials, including 944 participants, met the inclusion criteria, of which 14 trials could be included in the NMA. Based on the collective probability of being the overall best therapy for reducing the first peak KAM, lateral wedge insoles (LWI) plus knee brace was closely followed by gait retraining, and knee brace only. Although no significant difference was observed among the eight interventions, variable-stiffness shoes and neuromuscular exercise exhibited an increase in the first peak KAM compared with the control condition group. And based on the collective probability of being the overall best therapy for reducing KAAI, gait retraining was followed by LWI only, and lower limb exercise.

CONCLUSION

The results of our study support the use of LWI plus knee brace for reducing the first peak KAM. Gait retraining did not rank highest but it influenced both KAM and KAAI and therefore it was the most recommended therapy for reducing the biomechanical risk factors.

Digital interventions in mental health: evidence syntheses and economic modelling

BACKGROUND

Economic evaluations provide evidence on whether or not digital interventions offer value for money, based on their costs and outcomes relative to the costs and outcomes of alternatives.

OBJECTIVES

(1) Evaluate and summarise published economic studies about digital interventions across different technologies, therapies, comparators and mental health conditions; (2) synthesise clinical evidence about digital interventions for an exemplar mental health condition; (3) construct an economic model for the same exemplar mental health condition using the previously synthesised clinical evidence; and (4) consult with stakeholders about how they understand and assess the value of digital interventions.

METHODS

We completed four work packages: (1) a systematic review and quality assessment of economic studies about digital interventions; (2) a systematic review and network meta-analysis of randomised controlled trials on digital interventions for generalised anxiety disorder; (3) an economic model and value-of-information analysis on digital interventions for generalised anxiety disorder; and (4) a series of knowledge exchange face-to-face and digital seminars with stakeholders.

RESULTS

In work package 1, we reviewed 76 economic evaluations: 11 economic models and 65 within-trial analyses. Although the results of the studies are not directly comparable because they used different methods, the overall picture suggests that digital interventions are likely to be cost-effective, compared with no intervention and non-therapeutic controls, whereas the value of digital interventions compared with face-to-face therapy or printed manuals is unclear. In work package 2, we carried out two network meta-analyses of 20 randomised controlled trials of digital interventions for generalised anxiety disorder with a total of 2350 participants. The results were used to inform our economic model, but when considered on their own they were inconclusive because of the very wide confidence intervals. In work package 3, our decision-analytic model found that digital interventions for generalised anxiety disorder were associated with lower net monetary benefit than medication and face-to-face therapy, but greater net monetary benefit than non-therapeutic controls and no intervention. Value for money was driven by clinical outcomes rather than by intervention costs, and a value-of-information analysis suggested that uncertainty in the treatment effect had the greatest value (£12.9B). In work package 4, stakeholders identified several areas of benefits and costs of digital interventions that are important to them, including safety, sustainability and reducing waiting times. Four factors may influence their decisions to use digital interventions, other than costs and outcomes: increasing patient choice, reaching underserved populations, enabling continuous care and accepting the 'inevitability of going digital'.

LIMITATIONS

There was substantial uncertainty around effect estimates of digital interventions compared with alternatives. This uncertainty was driven by the small number of studies informing most comparisons, the small samples in some of these studies and the studies' high risk of bias.

CONCLUSIONS

Digital interventions may offer good value for money as an alternative to 'doing nothing' or 'doing something non-therapeutic' (e.g. monitoring or having a general discussion), but their added value compared with medication, face-to-face therapy and printed manuals is uncertain. Clinical outcomes rather than intervention costs drive 'value for money'.

FUTURE WORK

There is a need to develop digital interventions that are more effective, rather than just cheaper, than their alternatives.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42018105837.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 1. See the NIHR Journals Library website for further project information.

A frequentist one-step model for a simple network meta-analysis of time-to-event data in presence of an effect modifier

INTRODUCTION

Individual patient data (IPD) present particular advantages in network meta-analysis (NMA) because interactions may lead an aggregated data (AD)-based model to wrong a treatment effect (TE) estimation. However, fewer works have been conducted for IPD with time-to-event contrary to binary outcomes. We aimed to develop a general frequentist one-step model for evaluating TE in the presence of interaction in a three-node NMA for time-to-event data.

METHODS

One-step, frequentist, IPD-based Cox and Poisson generalized linear mixed models were proposed. We simulated a three-node network with or without a closed loop with (1) no interaction, (2) covariate-treatment interaction, and (3) covariate distribution heterogeneity and covariate-treatment interaction. These models were applied to the NMA (Meta-analyses of Chemotherapy in Head and Neck Cancer [MACH-NC] and Radiotherapy in Carcinomas of Head and Neck [MARCH]), which compared the addition of chemotherapy or modified radiotherapy (mRT) to loco-regional treatment with two direct comparisons. AD-based (contrast and meta-regression) models were used as reference.

RESULTS

In the simulated study, no IPD models failed to converge. IPD-based models performed well in all scenarios and configurations with small bias. There were few variations across different scenarios. In contrast, AD-based models performed well when there were no interactions, but demonstrated some bias when interaction existed and a larger one when the modifier was not distributed evenly. While meta-regression performed better than contrast-based only, it demonstrated a large variability in estimated TE. In the real data example, Cox and Poisson IPD-based models gave similar estimations of the model parameters. Interaction decomposition permitted by IPD explained the ecological bias observed in the meta-regression.

CONCLUSION

The proposed general one-step frequentist Cox and Poisson models had small bias in the evaluation of a three-node network with interactions. They performed as well or better than AD-based models and should also be undertaken whenever possible.

A method for assessing robustness of the results of a star-shaped network meta-analysis under the unidentifiable consistency assumption

Background

In a star-shaped network, pairwise comparisons link treatments with a reference treatment (often placebo or standard care), but not with each other. Thus, comparisons between non-reference treatments rely on indirect evidence, and are based on the unidentifiable consistency assumption, limiting the reliability of the results. We suggest a method of performing a sensitivity analysis through data imputation to assess the robustness of results with an unknown degree of inconsistency.

Methods

The method involves imputation of data for randomized controlled trials comparing non-reference treatments, to produce a complete network. The imputed data simulate a situation that would allow mixed treatment comparison, with a statistically acceptable extent of inconsistency. By comparing the agreement between the results obtained from the original star-shaped network meta-analysis and the results after incorporating the imputed data, the robustness of the results of the original star-shaped network meta-analysis can be quantified and assessed. To illustrate this method, we applied it to two real datasets and some simulated datasets.

Results

Applying the method to the star-shaped network formed by discarding all comparisons between non-reference treatments from a real complete network, 33% of the results from the analysis incorporating imputed data under acceptable inconsistency indicated that the treatment ranking would be different from the ranking obtained from the star-shaped network. Through a simulation study, we demonstrated the sensitivity of the results after data imputation for a star-shaped network with different levels of within- and between-study variability. An extended usability of the method was also demonstrated by another example where some head-to-head comparisons were incorporated.

Conclusions

Our method will serve as a practical technique to assess the reliability of results from a star-shaped network meta-analysis under the unverifiable consistency assumption.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01290-1.

Classifying information-sharing methods

BACKGROUND

Sparse relative effectiveness evidence is a frequent problem in Health Technology Assessment (HTA). Where evidence directly pertaining to the decision problem is sparse, it may be feasible to expand the evidence-base to include studies that relate to the decision problem only indirectly: for instance, when there is no evidence on a comparator, evidence on other treatments of the same molecular class could be used; similarly, a decision on children may borrow-strength from evidence on adults. Usually, in HTA, such indirect evidence is either included by ignoring any differences ('lumping') or not included at all ('splitting'). However, a range of more sophisticated methods exists, primarily in the biostatistics literature. The objective of this study is to identify and classify the breadth of the available information-sharing methods.

METHODS

Forwards and backwards citation-mining techniques were used on a set of seminal papers on the topic of information-sharing. Papers were included if they specified (network) meta-analytic methods for combining information from distinct populations, interventions, outcomes or study-designs.

RESULTS

Overall, 89 papers were included. A plethora of evidence synthesis methods have been used for information-sharing. Most papers (n=79) described methods that shared information on relative treatment effects. Amongst these, there was a strong emphasis on methods for information-sharing across multiple outcomes (n=42) and treatments (n=25), with fewer papers focusing on study-designs (n=23) or populations (n=8). We categorise and discuss the methods under four 'core' relationships of information-sharing: functional, exchangeability-based, prior-based and multivariate relationships, and explain the assumptions made within each of these core approaches.

CONCLUSIONS

This study highlights the range of information-sharing methods available. These methods often impose more moderate assumptions than lumping or splitting. Hence, the degree of information-sharing that they impose could potentially be considered more appropriate. Our identification of four 'core' methods of information-sharing allows for an improved understanding of the assumptions underpinning the different methods. Further research is required to understand how the methods differ in terms of the strength of sharing they impose and the implications of this for health care decisions.

EstablishINg the best STEp-up treatments for children with uncontrolled asthma despite INhaled corticosteroids (EINSTEIN): protocol for a systematic review, network meta-analysis and cost-effectiveness analysis using individual participant data (IPD)

INTRODUCTION

Asthma affects millions of children worldwide-1.1 million children in the UK. Asthma symptoms cannot be cured but can be controlled with low-dose inhaled corticosteroids (ICSs) in the majority of individuals. Treatment with a low-dose ICS, however, fails to control asthma symptoms in around 10%-15% of children and this places the individual at increased risk for an asthma attack. At present, there is no clear preferred treatment option for a child whose asthma is not controlled by low-dose ICS and international guidelines currently recommend at least three treatment options. Herein, we propose a systematic review and individual participant data network meta-analysis (IPD-NMA) aiming to synthesise all available published and unpublished evidence from randomised controlled trials (RCTs) to establish the clinical effectiveness of pharmacological treatments in children and adolescents with uncontrolled asthma on ICS and help to make evidence-informed treatment choices. This will be used to parameterise a Markov-based economic model to assess the cost-effectiveness of alternative treatment options in order to inform decisions in the context of drug formularies and clinical guidelines.

METHODS AND ANALYSIS

We will search in MEDLINE, the Cochrane Library, the Cochrane Central Register of Controlled Trials (CENTRAL), Embase, NICE Technology Appraisals and the National Institute for Health Research (NIHR) Health Technology Assessment series for RCTs of interventions in patients with uncontrolled asthma on ICS. All studies where children and adolescents were eligible for inclusion will be considered, and authors or sponsors will be contacted to request IPD on patients aged <18. The reference lists of existing clinical guidelines, along with included studies and relevant reviews, will be checked to identify further relevant studies. Unpublished studies will be located by searching across a range of clinical trial registries, including internal trial registers for pharmaceutical companies. All studies will be appraised for inclusion against predefined inclusion and exclusion criteria by two independent reviewers with disagreements resolved through discussion with a third reviewer. We will perform an IPD-NMA-eventually supplemented with aggregate data for the RCTs without IPD-to establish both the probability that a treatment is best and the probability that a particular treatment is most likely to be effective for a specific profile of the patient. The IPD-NMA will be performed for each outcome variable within a Bayesian framework, using the WinBUGS software. Also, potential patient-level characteristics that may modify treatment effects will be explored, which represents one of the strengths of this study.

ETHICS AND DISSEMINATION

The Committee on Research Ethics, University of Liverpool, has confirmed that ethics review is not required. The dissemination plan consists of publishing the results in an open-access medical journal, a plain-language summary available for parents and children, dissemination via local, national and international meetings and conferences and the press offices of our Higher Education Institutions (HEIs). A synopsis of results will be disseminated to NICE and British Thoracic Society/Scottish Intercollegiate Guidelines Network (SIGN) as highly relevant to future clinical guideline updates.

PROSPERO REGISTRATION NUMBER

CRD42019127599.

Bayesian splines versus fractional polynomials in network meta-analysis

Background

Network meta-analysis (NMA) provides a powerful tool for the simultaneous evaluation of multiple treatments by combining evidence from different studies, allowing for direct and indirect comparisons between treatments. In recent years, NMA is becoming increasingly popular in the medical literature and underlying statistical methodologies are evolving both in the frequentist and Bayesian framework. Traditional NMA models are often based on the comparison of two treatment arms per study. These individual studies may measure outcomes at multiple time points that are not necessarily homogeneous across studies.

Methods

In this article we present a Bayesian model based on B-splines for the simultaneous analysis of outcomes across time points, that allows for indirect comparison of treatments across different longitudinal studies.

Results

We illustrate the proposed approach in simulations as well as on real data examples available in the literature and compare it with a model based on P-splines and one based on fractional polynomials, showing that our approach is flexible and overcomes the limitations of the latter.

Conclusions

The proposed approach is computationally efficient and able to accommodate a large class of temporal treatment effect patterns, allowing for direct and indirect comparisons of widely varying shapes of longitudinal profiles.

Effectiveness and adverse events of topical and allergen immunotherapy for atopic dermatitis: a systematic review and network meta-analysis protocol

BACKGROUND

Atopic dermatitis (AD) is an inflammatory chronic condition that affects the skin of children and adults and has an important impact on the quality of life. Treatments for AD are based on environmental controls, topical and systemic therapies, and allergen-specific immunotherapy (AIT). However, it remains unclear the effectiveness and adverse events of AIT and all conventional topical treatments compared with placebo and each other for AD.

METHODS

We will search five electronic databases [Central Cochrane register of controlled trials (CENTRAL), MEDLINE, EMBASE, CINAHL, and LILACS] from inception until November 2019 with no language restriction, and we will include experimental studies [randomized controlled trials (RCTs), and quasi-RCTs]. The primary outcome is global and specific skin symptoms assessment. Secondary outcomes are hospital length of stay, quality of life, and adverse events. Reviewers independently will extract data from the studies that meet our inclusion criteria and will assess the risk of bias of individual primary studies. We will conduct random effects pairwise meta-analyses for the observed pairwise comparisons with at least two trials. Then, we will perform random-effects Bayesian network meta-analysis (NMA) to obtain treatment effects for all possible comparisons and to provide a hierarchy of all interventions for each outcome. Possible incoherence between direct and indirect sources of evidence will be investigated locally (if possible) and globally. To investigate sources of statistical heterogeneity, we will perform a series of meta-regression analyses based on pre-specified important effect modifiers. Two authors will appraise the certainty of the evidence for each outcome applying the GRADE's framework for NMA.

DISCUSSION

The findings of this systematic review will shed the light on the effectiveness and adverse events of all possible comparisons for treating AD and on the quality of the collated evidence for recommendations. It will also provide critical information to health care professionals to comprehend and manage this disease at different age stages, treatment type, duration, and severity of atopic dermatitis.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO Protocol ID CRD42019147106.

The impact of clinical heterogeneity on conducting network meta-analyses in transthyretin amyloidosis with polyneuropathy

Objective: The comparative safety and efficacy of tafamidis, patisiran and inotersen treatments for transthyretin amyloidosis with polyneuropathy (ATTR-PN) has not been evaluated in clinical trials. In the absence of head-to-head evidence, indirect treatment comparisons such as network meta-analyses (NMAs) can be performed to evaluate relative effects of treatments. This study aims to assess the feasibility of conducting an NMA of available therapies for ATTR-PN patients.Methods: Pivotal trials for three approved ATTR-PN treatments, tafamidis (Fx-005), patisiran (APOLLO) and inotersen (NEURO-TTR), were compared in terms of study design, baseline population characteristics, outcome definitions and baseline risk. These assessments of heterogeneity informed the decision to perform Bayesian NMAs.Results: Despite similar study designs, clear differences in eligibility criteria between trials were accompanied by imbalances in baseline population characteristics considered to be plausible effect modifiers, such as disease stage and previous treatment. Of the outcomes assessed, only quality of life and adverse events were similarly reported in all trials. Neuropathy outcomes were not evaluated consistently between trials.Conclusions: An NMA of ATTR-PN treatments was not feasible, given the observed cross-trial heterogeneity. This decision highlights the importance of careful consideration for clinical heterogeneity that may threaten the validity of indirect comparisons.

A comparison of Bayesian and frequentist methods in random-effects network meta-analysis of binary data

The performance of statistical methods is often evaluated by means of simulation studies. In case of network meta-analysis of binary data, however, simulations are not currently available for many practically relevant settings. We perform a simulation study for sparse networks of trials under between-trial heterogeneity and including multi-arm trials. Results of the evaluation of two popular frequentist methods and a Bayesian approach using two different prior specifications are presented. Methods are evaluated using coverage, width of intervals, bias, and root mean squared error (RMSE). In addition, deviations from the theoretical surface under the cumulative rankings (SUCRAs) or P-scores of the treatments are evaluated. Under low heterogeneity and when a large number of trials informs the contrasts, all methods perform well with respect to the evaluated performance measures. Coverage is observed to be generally higher for the Bayesian than the frequentist methods. The width of credible intervals is larger than those of confidence intervals and is increasing when using a flatter prior for between-trial heterogeneity. Bias was generally small, but increased with heterogeneity, especially in netmeta. In some scenarios, the direction of bias differed between frequentist and Bayesian methods. The RMSE was comparable between methods but larger in indirectly than in directly estimated treatment effects. The deviation of the SUCRAs or P-scores from their theoretical values was mostly comparable over the methods but differed depending on the heterogeneity and the geometry of the investigated network. Multivariate meta-regression or Bayesian estimation using a half-normal prior scaled to 0.5 seems to be promising with respect to the evaluated performance measures in network meta-analysis of sparse networks.

Charting the landscape of graphical displays for meta-analysis and systematic reviews: a comprehensive review, taxonomy, and feature analysis

BACKGROUND

Data-visualization methods are essential to explore and communicate meta-analytic data and results. With a large number of novel graphs proposed quite recently, a comprehensive, up-to-date overview of available graphing options for meta-analysis is unavailable.

METHODS

We applied a multi-tiered search strategy to find the meta-analytic graphs proposed and introduced so far. We checked more than 150 retrievable textbooks on research synthesis methodology cover to cover, six different software programs regularly used for meta-analysis, and the entire content of two leading journals on research synthesis. In addition, we conducted Google Scholar and Google image searches and cited-reference searches of prior reviews of the topic. Retrieved graphs were categorized into a taxonomy encompassing 11 main classes, evaluated according to 24 graph-functionality features, and individually presented and described with explanatory vignettes.

RESULTS

We ascertained more than 200 different graphs and graph variants used to visualize meta-analytic data. One half of these have accrued within the past 10 years alone. The most prevalent classes were graphs for network meta-analysis (45 displays), graphs showing combined effect(s) only (26), funnel plot-like displays (24), displays showing more than one outcome per study (19), robustness, outlier and influence diagnostics (15), study selection and p-value based displays (15), and forest plot-like displays (14). The majority of graphs (130, 62.5%) possessed a unique combination of graph features.

CONCLUSIONS

The rich and diverse set of available meta-analytic graphs offers a variety of options to display many different aspects of meta-analyses. This comprehensive overview of available graphs allows researchers to make better-informed decisions on which graphs suit their needs and therefore facilitates using the meta-analytic tool kit of graphs to its full potential. It also constitutes a roadmap for a goal-driven development of further graphical displays for research synthesis.

Effectiveness of interventions for the reversal of a metabolic syndrome diagnosis: An update of a meta-analysis of mixed treatment comparison studies

Introducción.

El conocer las intervenciones más efectivas para revertir el síndrome metabólico es clave para el diseño de estrategias clínicas de prevención de enfermedades como la diabetes mellitus de tipo 2 y la enfermedad cardiovascular.

Objetivo.

Sintetizar el tamaño del efecto de las intervenciones disponibles para revertir un diagnóstico de síndrome metabólico.

Materiales y métodos.

Se hizo la búsqueda en Embase y Medline, incluyendo los ensayos clínicos en los que la variable “respuesta” se definía como la reversión del diagnóstico del síndrome metabólico. Se categorizaron las intervenciones en cuatro dimensiones: 1) estilo de vida (dieta y ejercicio); 2) farmacia; 3) combinación de estilo de vida y farmacia, y 4) grupos de control; finalmente, se hizo una comparación mixta de tratamientos.

Resultados.

Se detectaron dos estudios adicionales a los incluidos en el metaanálisis publicado por Dunkley, et al., en el 2012. Se estimó que las intervenciones relacionadas con el estilo de vida tuvieron 2,61 veces (intervalo de credibilidad entre 1,00 y 5,47) más probabilidades de revertir el síndrome metabólico que las de los grupos de control y las relacionadas con los tratamientos farmacéuticos, una probabilidad de 3,39 veces más que las del grupo de control, pero con un intervalo de credibilidad entre 0,81 y 9,99. Las intervenciones sobre el estilo de vida tuvieron 1,59 veces más probabilidades de revertir el síndrome metabólico que las del tratamiento farmacéutico.

Conclusión.

Las estrategias basadas en la dieta y la actividad física de las personas, tuvieron una mayor probabilidad de ser más efectivas para revertir el diagnóstico de síndrome metabólico.

Identifying inconsistency in network meta-analysis: Is the net heat plot a reliable method?

One of the biggest challenges for network meta‐analysis is inconsistency, which occurs when the direct and indirect evidence conflict. Inconsistency causes problems for the estimation and interpretation of treatment effects and treatment contrasts. Krahn and colleagues proposed the net heat approach as a graphical tool for identifying and locating inconsistency within a network of randomized controlled trials. For networks with a treatment loop, the net heat plot displays statistics calculated by temporarily removing each design one at a time, in turn, and assessing the contribution of each remaining design to the inconsistency. The net heat plot takes the form of a matrix which is displayed graphically with coloring indicating the degree of inconsistency in the network. Applied to a network of individual participant data assessing overall survival in 7531 patients with lung cancer, we were surprised to find no evidence of important inconsistency from the net heat approach; this contradicted other approaches for assessing inconsistency such as the Bucher approach, Cochran's Q statistic, node‐splitting, and the inconsistency parameter approach, which all suggested evidence of inconsistency within the network at the 5% level. Further theoretical work shows that the calculations underlying the net heat plot constitute an arbitrary weighting of the direct and indirect evidence which may be misleading. We illustrate this further using a simulation study and a network meta‐analysis of 10 treatments for diabetes. We conclude that the net heat plot does not reliably signal inconsistency or identify designs that cause inconsistency.

MetaInsight: An interactive web-based tool for analyzing, interrogating, and visualizing network meta-analyses using R-shiny and netmeta

Background

Network meta‐analysis (NMA) is a powerful analysis method used to identify the best treatments for a condition and is used extensively by health care decision makers. Although software routines exist for conducting NMA, they require considerable statistical programming expertise to use, which limits the number of researchers able to conduct such analyses.

Objectives

To develop a web‐based tool allowing users with only standard internet browser software to be able to conduct NMAs using an intuitive “point and click” interface and present the results using visual plots.

Methods

Using the existing netmeta and Shiny packages for R to conduct the analyses, and to develop the user interface, we created the MetaInsight tool which is freely available to use via the web.

Results

A package was created for conducting NMA which satisfied our objectives, and this is described, and its application demonstrated, using an illustrative example.

Conclusions

We believe that many researchers will find our package helpful for facilitating NMA as well as allowing decision makers to scrutinize presented results visually and in real time. This will impact on the relevance of statistical analyses for health care decision making and sustainably increase capacity by empowering informed nonspecialists to be able to conduct more clinically relevant reviews. It is also hoped that others will be inspired to create similar tools for other advanced specialist analyses methods using the freely available technologies we have adopted.

Interventions for maintenance of surgically induced remission in Crohn's disease: a network meta-analysis

BACKGROUND

Crohn's disease (CD) is a chronic disease of the gut. About 75% of people with CD undergo surgery at least once in their lifetime to induce remission. However, as there is no known cure for the disease, patients usually experience a recurrence even after surgery. Different interventions are routinely used in maintaining postsurgical remission. There is currently no consensus on which treatment is the most effective.

OBJECTIVES

To assess the effects and harms of interventions for the maintenance of surgically induced remission in Crohn's disease and rank the treatments in order of effectiveness.

SEARCH METHODS

We searched the Cochrane IBD Group Specialized Register, CENTRAL, MEDLINE, and Embase from inception to 15 January 2019. We also searched reference lists of relevant articles, abstracts from major gastroenterology meetings, ClinicalTrials.gov, and the WHO ICTRP. There was no restriction on language, date, or publication status.

SELECTION CRITERIA

We considered for inclusion randomised controlled trials (RCTs) that compared different interventions used for maintaining surgically induced remission in people with CD who were in postsurgical remission. Participants had to have received maintenance treatment for at least three months. We excluded studies assessing enteral diet, diet manipulation, herbal medicine, and nutritional supplementation.

DATA COLLECTION AND ANALYSIS

Two review authors independently selected relevant studies, extracted data, and assessed the risk of bias. Any disagreements were resolved by discussion or by arbitration of a third review author when necessary. We conducted a network meta-analysis (NMA) using a Bayesian approach through Markov Chain Monte Carlo (MCMC) simulation. For the pairwise comparisons carried out in Review Manager 5, we calculated risk ratios (RR) with their corresponding 95% confidence intervals (95% CI). For the NMA, we presented hazard ratios (HR) with corresponding 95% credible intervals (95% CrI) and reported ranking probabilities for each intervention. For the NMA, we focused on three main outcomes: clinical relapse, endoscopic relapse, and withdrawals due to adverse events. Data were insufficient to assess time to relapse and histologic relapse. Adverse events and serious adverse events were not sufficiently or objectively reported to permit an NMA. We used CINeMA (Confidence in Network Meta-Analysis) methods to evaluate our confidence in the findings within networks, and GRADE for entire networks.

MAIN RESULTS

We included 35 RCTs (3249 participants) in the review. The average age of study participants ranged between 33.6 and 38.8 years. Risk of bias was high in 18 studies, low in four studies, and unclear in 13 studies. Of the 35 included RCTs, 26 studies (2581 participants; 9 interventions) were considered eligible for inclusion in the NMA. The interventions studied included 5-aminosalicylic acid (5-ASA), adalimumab, antibiotics, budesonide, infliximab, probiotics, purine analogues, sulfasalazine, and a combination of sulfasalazine and prednisolone. This resulted in 30 direct contrasts, which informed 102 mixed-treatment contrasts.The evidence for the clinical relapse network (21 studies; 2245 participants) and endoscopic relapse (12 studies; 1128 participants) were of low certainty while the evidence for withdrawal due to adverse events (15 studies; 1498 participants) was of very low certainty. This assessment was due to high risk of bias in most of the studies, inconsistency, and imprecision across networks. We mainly judged individual contrasts as of low or very low certainty, except 5-ASA versus placebo, the evidence for which was judged as of moderate certainty.We ranked the treatments based on effectiveness and the certainty of the evidence. For clinical relapse, the five most highly ranked treatments were adalimumab, infliximab, budesonide, 5-ASA, and purine analogues. We found some evidence that adalimumab (HR 0.11, 95% Crl 0.02 to 0.33; low-certainty evidence) and 5-ASA may reduce the probability of clinical relapse compared to placebo (HR 0.69, 95% Crl 0.53 to 0.87; moderate-certainty evidence). However, budesonide may not be effective in preventing clinical relapse (HR 0.66, 95% CrI 0.27 to 1.34; low-certainty evidence). We are less confident about the effectiveness of infliximab (HR 0.36, 95% CrI 0.02 to 1.74; very low-certainty evidence) and purine analogues (HR 0.75, 95% CrI 0.55 to 1.00; low-certainty evidence). It was unclear whether the other interventions reduced the probability of a clinical relapse, as the certainty of the evidence was very low.Due to high risk of bias and limited data across the network, we are uncertain about the effectiveness of interventions for preventing endoscopic relapse. Whilst there might be some evidence of prevention of endoscopic relapse with adalimumab (HR 0.10, 95% CrI 0.01 to 0.32; low-certainty evidence), no other intervention studied appeared to be effective.Due to high risk of bias and limited data across the network, we are uncertain about the effectiveness of interventions for preventing withdrawal due to adverse events. Withdrawal due to adverse events appeared to be least likely with sulfasalazine (HR 1.96, 95% Crl 0.00 to 8.90; very low-certainty evidence) and most likely with antibiotics (HR 53.92, 95% Crl 0.43 to 259.80; very low-certainty evidence). When considering the network as a whole, two adverse events leading to study withdrawal (i.e. pancreatitis and leukopenia) occurred in more than 1% of participants treated with an intervention. Pancreatitis occurred in 2.8% (11/399) of purine analogue participants compared to 0.17% (2/1210) of all other groups studied. Leukopenia occurred in 2.5% (10/399) of purine analogue participants compared to 0.08% (1/1210) of all other groups studied.

AUTHORS' CONCLUSIONS

Due to low-certainty evidence in the networks, we are unable to draw conclusions on which treatment is most effective for preventing clinical relapse and endoscopic relapse. Evidence on the safety of the interventions was inconclusive, however cases of pancreatitis and leukopenia from purine analogues were evident in the studies. Larger trials are needed to further understand the effect of the interventions on endoscopic relapse.

Botulinum Toxins Type A (Bont-A) in the Management of Lower Limb Spasticity in Children: A Systematic Literature Review and Bayesian Network Meta-analysis

BACKGROUND

Botulinum neurotoxins type A (BoNT-As) are used in pediatric lower limb spasticity, which affects more than 2.5 million children worldwide. Botulinum neurotoxins type-A improve active function and delay musculoskeletal complications. The objective of this analysis was to evaluate the efficacy and safety of abobotulinumtoxinA versus other botulinum neurotoxins type A in pediatric spasticity, in the absence of head-to-head evidence.

METHODS

A systematic literature review was conducted to identify relevant randomized controlled trials. The evidence base was synthesized with Bayesian network meta-analyses. Outcomes analyzed included Modified Ashworth Scale, Tardieu Scale (TS) spasticity grade, and Goal Attainment Scale (standard mean difference only) at 12 weeks postinjection, and adverse events occurring during study periods.

RESULTS

Thirty-eight studies were identified, 10 of which met inclusion criteria for quantitative synthesis. On Modified Ashworth Scale, abobotulinumtoxinA 15 U/kg/leg was significantly more efficacious than onabotulinumtoxinA 4 U/kg/leg (-0.99 [-1.34, -0.64]) and onabotulinumtoxinA 4 U/kg/leg+casting (-0.81 [-1.16, -0.46]) and numerically better than onabotulinumtoxinA 8 U/kg (-0.40 [-0.67, -0.14]). AbobotulinumtoxinA 10 U/kg/leg was significantly more efficacious than onabotulinumtoxinA 4 U/kg/leg (±casting). On Goal Attainment Scale, abobotulinumtoxinA 15 U/kg/leg and 10 U/kg/leg were significantly more efficacious than onabotulinumtoxinA 12 U/kg/leg. On Tardieu Scale spasticity grade, abobotulinumtoxinA was comparable to other treatments. AbobotulinumtoxinA demonstrated adverse event rates comparable to all doses of onabotulinumtoxinA.

CONCLUSIONS

In pediatric lower limb spasticity, abobotulinumtoxinA offered significant or numerical efficacy advantages on tone (Modified Ashworth Scale) and functional outcomes (Goal Attainment Scale), and comparable efficacy on Tardieu Scale spasticity grade. AbobotulinumtoxinA was comparable to onabotulinumtoxinA on tolerability. Results should be interpreted in the context of heterogeneity and sparsity of the evidence base.

Trends in comparative efficacy and safety of malaria control interventions for maternal and child health outcomes in Africa: a study protocol for a Bayesian network meta-regression exploring the effect of HIV and malaria endemicity spectrum

INTRODUCTION

Unprecedented global efforts to prevent malaria morbidity and mortality in sub-Saharan Africa have saved hundreds of thousands of lives across the continent in the last two decades. This study aims to determine how the comparative efficacy and safety of available malaria control interventions intended to improve maternal and child health outcomes have changed over time considering the varied epidemiological contexts on the continent.

METHODS

We will review all randomised controlled trials that investigated malaria control interventions in pregnant women in sub-Saharan Africa and were published between January 1980 and December 2018. We will subsequently use network meta-regression to estimate temporal trends in the relative and absolute efficacy and safety of Intermittent Preventive Treatments, Intermittent Screening and Treatments, Insecticide-treated bed nets, and their combinations, and predict their ranking according to their relative and absolute efficacy and safety over time. Our outcomes will include 12 maternal and 7 child mortality and morbidity outcomes, known to be associated with either malaria infection or control. We will use intention-to-treat analysis to derive our estimates and meta-regression to estimate temporal trends and the effect modification by HIV infection, malaria endemicity and Plasmodium falciparum resistance to sulfadoxine-pyrimethamine, while adjusting for multiple potential confounders via propensity score calibration.

PROSPERO REGISTRATION NUMBER

CRD42018095138.

Interventions for maintenance of surgically‐induced remission in Crohn’s disease: a network meta‐analysis

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

To assess the effects and harms of interventions for the maintenance of surgically‐induced remission in Crohn's disease and to rank treatments in order of effectiveness.

Assessing the consistency assumptions underlying network meta-regression using aggregate data

When numerous treatments exist for a disease (Treatments 1, 2, 3, etc), network meta‐regression (NMR) examines whether each relative treatment effect (eg, mean difference for 2 vs 1, 3 vs 1, and 3 vs 2) differs according to a covariate (eg, disease severity). Two consistency assumptions underlie NMR: consistency of the treatment effects at the covariate value 0 and consistency of the regression coefficients for the treatment by covariate interaction. The NMR results may be unreliable when the assumptions do not hold. Furthermore, interactions may exist but are not found because inconsistency of the coefficients is masking them, for example, when the treatment effect increases as the covariate increases using direct evidence but the effect decreases with the increasing covariate using indirect evidence.

We outline existing NMR models that incorporate different types of treatment by covariate interaction. We then introduce models that can be used to assess the consistency assumptions underlying NMR for aggregate data. We extend existing node‐splitting models, the unrelated mean effects inconsistency model, and the design by treatment inconsistency model to incorporate covariate interactions. We propose models for assessing both consistency assumptions simultaneously and models for assessing each of the assumptions in turn to gain a more thorough understanding of consistency.

We apply the methods in a Bayesian framework to trial‐level data comparing antimalarial treatments using the covariate average age and to four fabricated data sets to demonstrate key scenarios.

We discuss the pros and cons of the methods and important considerations when applying models to aggregated data.

Adalimumab, etanercept and ustekinumab for treating plaque psoriasis in children and young people: systematic review and economic evaluation

BACKGROUND

Psoriasis is a chronic inflammatory disease that predominantly affects the skin. Adalimumab (HUMIRA^®, AbbVie, Maidenhead, UK), etanercept (Enbrel^®, Pfizer, New York, NY, USA) and ustekinumab (STELARA^®, Janssen Biotech, Inc., Titusville, NJ, USA) are the three biological treatments currently licensed for psoriasis in children.

OBJECTIVE

To determine the clinical effectiveness and cost-effectiveness of adalimumab, etanercept and ustekinumab within their respective licensed indications for the treatment of plaque psoriasis in children and young people.

DATA SOURCES

Searches of the literature and regulatory sources, contact with European psoriasis registries, company submissions and clinical study reports from manufacturers, and previous National Institute for Health and Care Excellence (NICE) technology appraisal documentation.

REVIEW METHODS

Included studies were summarised and subjected to detailed critical appraisal. A network meta-analysis incorporating adult data was developed to connect the effectiveness data in children and young people and populate a de novo decision-analytic model. The model estimated the cost-effectiveness of adalimumab, etanercept and ustekinumab compared with each other and with either methotrexate or best supportive care (BSC), depending on the position of the intervention in the management pathway.

RESULTS

Of the 2386 non-duplicate records identified, nine studies (one randomised controlled trial for each drug plus six observational studies) were included in the review of clinical effectiveness and safety. Etanercept and ustekinumab resulted in significantly greater improvements in psoriasis symptoms than placebo at 12 weeks' follow-up. The magnitude and persistence of the effects beyond 12 weeks is less certain. Adalimumab resulted in significantly greater improvements in psoriasis symptoms than methotrexate for some but not all measures at 16 weeks. Quality-of-life benefits were inconsistent across different measures. There was limited evidence of excess short-term adverse events; however, the possibility of rare events cannot be excluded. The majority of the incremental cost-effectiveness ratios for the use of biologics in children and young people exceeded NICE's usual threshold for cost-effectiveness and were reduced significantly only when combined assumptions that align with those made in the management of psoriasis in adults were adopted.

LIMITATIONS

The clinical evidence base for short- and long-term outcomes was limited in terms of total participant numbers, length of follow-up and the absence of young children.

CONCLUSIONS

The paucity of clinical and economic evidence to inform the cost-effectiveness of biological treatments in children and young people imposed a number of strong assumptions and uncertainties. Health-related quality-of-life (HRQoL) gains associated with treatment and the number of hospitalisations in children and young people are areas of considerable uncertainty. The findings suggest that biological treatments may not be cost-effective for the management of psoriasis in children and young people at a willingness-to-pay threshold of £30,000 per quality-adjusted life-year, unless a number of strong assumptions about HRQoL and the costs of BSC are combined. Registry data on biological treatments would help determine safety, patterns of treatment switching, impact on comorbidities and long-term withdrawal rates. Further research is also needed into the resource use and costs associated with BSC. Adequately powered randomised controlled trials (including comparisons against placebo) could substantially reduce the uncertainty surrounding the effectiveness of biological treatments in biologic-experienced populations of children and young people, particularly in younger children. Such trials should establish the impact of biological therapies on HRQoL in this population, ideally by collecting direct estimates of EuroQol-5 Dimensions for Youth (EQ-5D-Y) utilities.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42016039494.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

A framework for identifying treatment-covariate interactions in individual participant data network meta-analysis

Background: Stratified medicine seeks to identify patients most likely to respond to treatment. Individual participant data (IPD) network meta‐analysis (NMA) models have greater power than individual trials to identify treatment‐covariate interactions (TCIs). Treatment‐covariate interactions contain “within” and “across” trial interactions, where the across‐trial interaction is more susceptible to confounding and ecological bias.

Methods: We considered a network of IPD from 37 trials (5922 patients) for cervical cancer (2394 events), where previous research identified disease stage as a potential interaction covariate. We compare 2 models for NMA with TCIs: (1) 2 effects separating within‐ and across‐trial interactions and (2) a single effect combining within‐ and across‐trial interactions. We argue for a visual assessment of consistency of within‐ and across‐trial interactions and consider more detailed aspects of interaction modelling, eg, common vs trial‐specific effects of the covariate. This leads us to propose a practical framework for IPD NMA with TCIs.

Results: Following our framework, we found no evidence in the cervical cancer network for a treatment‐stage interaction on the basis of the within‐trial interaction. The NMA provided additional power for an across‐trial interaction over and above the pairwise evidence. Following our proposed framework, we found that the within‐ and across‐trial interactions should not be combined.

Conclusion: Across‐trial interactions are susceptible to confounding and ecological bias. It is important to separate the sources of evidence to check their consistency and identify which sources of evidence are driving the conclusion. Our framework provides practical guidance for researchers, reducing the risk of unduly optimistic interpretation of TCIs.

Graphs of study contributions and covariate distributions for network meta-regression

Background

Meta‐regression results must be interpreted taking into account the range of covariate values of the contributing studies. Results based on interpolation or extrapolation may be unreliable. In network meta‐regression (NMR) models, which include covariates in network meta‐analyses, results are estimated using direct and indirect evidence; therefore, it may be unclear which studies and covariate values contribute to which result. We propose graphs to help understand which trials and covariate values contribute to each NMR result and to highlight extrapolation or interpolation.

Methods

We introduce methods to calculate the contribution that each trial and covariate value makes to each result and compare them with existing methods. We show how to construct graphs including a network covariate distribution diagram, covariate‐contribution plot, heat plot, contribution‐NMR plot, and heat‐NMR plot. We demonstrate the methods using a dataset with treatments for malaria using the covariate average age and a dataset of topical fluoride interventions for preventing dental caries using the covariate randomisation year.

Results

For the malaria dataset, no contributing trials had an average age between 7–25 years and therefore results were interpolated within this range. For the fluoride dataset, there are no contributing trials randomised between 1954–1959 for most comparisons therefore, within this range, results would be extrapolated.

Conclusions

Even in a fully connected network, an NMR result may be estimated from trials with a narrower covariate range than the range of the whole dataset. Calculating contributions and graphically displaying them aids interpretation of NMR result by highlighting extrapolated or interpolated results.

A cost-effectiveness modelling study of strategies to reduce risk of infection following primary hip replacement based on a systematic review

BACKGROUND

A deep infection of the surgical site is reported in 0.7% of all cases of total hip arthroplasty (THA). This often leads to revision surgery that is invasive, painful and costly. A range of strategies is employed in NHS hospitals to reduce risk, yet no economic analysis has been undertaken to compare the value for money of competing prevention strategies.

OBJECTIVES

To compare the costs and health benefits of strategies that reduce the risk of deep infection following THA in NHS hospitals. To make recommendations to decision-makers about the cost-effectiveness of the alternatives.

DESIGN

The study comprised a systematic review and cost-effectiveness decision analysis.

SETTING

77,321 patients who had a primary hip arthroplasty in NHS hospitals in 2012.

INTERVENTIONS

Nine different treatment strategies including antibiotic prophylaxis, antibiotic-impregnated cement and ventilation systems used in the operating theatre.

MAIN OUTCOME MEASURES

Change in the number of deep infections, change in the total costs and change in the total health benefits in quality-adjusted life-years (QALYs).

DATA SOURCES

Literature searches using MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature and the Cochrane Central Register of Controlled Trials were undertaken to cover the period 1966-2012 to identify infection prevention strategies. Relevant journals, conference proceedings and bibliographies of retrieved papers were hand-searched. Orthopaedic surgeons and infection prevention experts were also consulted.

REVIEW METHODS

English-language papers only. The selection of evidence was by two independent reviewers. Studies were included if they were interventions that reported THA-related deep surgical site infection (SSI) as an outcome. Mixed-treatment comparisons were made to produce estimates of the relative effects of competing infection control strategies.

RESULTS

Twelve studies, six randomised controlled trials and six observational studies, involving 123,788 total hip replacements (THRs) and nine infection control strategies, were identified. The quality of the evidence was judged against four categories developed by the National Institute for Health and Care Excellence Methods for Development of NICE Public Health Guidance ( http://publications.nice.org.uk/methods-for-the-development-of-nice-public-health-guidance-third-edition-pmg4 ), accessed March 2012. All evidence was found to fit the two highest categories of 1 and 2. Nine competing infection control interventions [treatments (Ts) 1-9] were used in a cohort simulation model of 77,321 patients who had a primary THR in 2012. Predictions were made for cases of deep infection and total costs, and QALY outcomes. Compared with a baseline of T1 (no systemic antibiotics, plain cement and conventional ventilation) all other treatment strategies reduced risk. T6 was the most effective (systemic antibiotics, antibiotic-impregnated cement and conventional ventilation) and prevented a further 1481 cases of deep infection, and led to the largest annual cost savings and the greatest gains to QALYs. The additional uses of laminar airflow and body exhaust suits indicate higher costs and worse health outcomes.

CONCLUSIONS

T6 is an optimal strategy for reducing the risk of SSI following THA. The other strategies that are commonly used among NHS hospitals lead to higher cost and worse QALY outcomes. Policy-makers, therefore, have an opportunity to save resources and improve health outcomes. The effects of laminar air flow and body exhaust suits might be further studied if policy-makers are to consider disinvesting in these technologies.

LIMITATIONS

A wide range of evidence sources was synthesised and there is large uncertainty in the conclusions.

FUNDING

The National Institute for Health Research Health Technology Assessment programme and the Queensland Health Quality Improvement and Enhancement Programme (grant number 2008001769).

A design-by-treatment interaction model for network meta-analysis and meta-regression with integrated nested Laplace approximations

Network meta-analysis (NMA) is gaining popularity for comparing multiple treatments in a single analysis. Generalized linear mixed models provide a unifying framework for NMA, allow us to analyze datasets with dichotomous, continuous or count endpoints, and take into account multiarm trials, potential heterogeneity between trials and network inconsistency. To perform inference within such NMA models, the use of Bayesian methods is often advocated. The standard inference tool is Markov chain Monte Carlo (MCMC), which is computationally expensive and requires convergence diagnostics. A deterministic approach to do fully Bayesian inference for latent Gaussian models can be achieved by integrated nested Laplace approximations (INLA), which is a fast and accurate alternative to MCMC. We show how NMA models fit in the class of latent Gaussian models and how NMA models are implemented using INLA and demonstrate that the estimates obtained by INLA are in close agreement with the ones obtained by MCMC. Specifically, we emphasize the design-by-treatment interaction model with random inconsistency parameters (also known as the Jackson model). Also, we have proposed a network meta-regression model, which is constructed by incorporating trial-level covariates to the Jackson model to explain possible sources of heterogeneity and/or inconsistency in the network. A publicly available R package, nmaINLA, is developed to automate the INLA implementation of NMA models, which are considered in this paper. Three applications illustrate the use of INLA for a NMA.

Klinische Studien zu neuen direkten oralen Antikoagulanzien

Um die Wirksamkeit und Verträglichkeit neuer direkter oraler Antikoagulanzien (DOAK) zu vergleichen, müssten idealerweise eine direkte Gegenüberstellung der Substanzen in klinischen Studien durchgeführt werden. Auf Grund des logistischen und finanziellen Aufwandes wird dies aber in absehbarer Zeit nicht geschehen. Indirekte Vergleiche, auch Netzwerkanalysen (NMA) genannt, lassen sich mit den vorliegenden Studien zwischen den DOAKs vornehmen. In der postoperativen Phase nach Knie- und Hüftgelenkersatz und bei Patienten mit Vorhofflimmern sind derzeit diese Vergleich möglich. Indirekte Vergleiche sind jedoch auch mit Einschränkungen behaftet.

Diese Arbeit gibt eine Übersicht der aktuell vorliegenden NMAs und versucht, deren Ergebnisse mit der Pharmakologie der DOAKs und den methodischen Problemen von NMAs zu beleuchten.

Multiple Score Comparison: a network meta-analysis approach to comparison and external validation of prognostic scores

BACKGROUND

Prediction models and prognostic scores have been increasingly popular in both clinical practice and clinical research settings, for example to aid in risk-based decision making or control for confounding. In many medical fields, a large number of prognostic scores are available, but practitioners may find it difficult to choose between them due to lack of external validation as well as lack of comparisons between them.

METHODS

Borrowing methodology from network meta-analysis, we describe an approach to Multiple Score Comparison meta-analysis (MSC) which permits concurrent external validation and comparisons of prognostic scores using individual patient data (IPD) arising from a large-scale international collaboration. We describe the challenges in adapting network meta-analysis to the MSC setting, for instance the need to explicitly include correlations between the scores on a cohort level, and how to deal with many multi-score studies. We propose first using IPD to make cohort-level aggregate discrimination or calibration scores, comparing all to a common comparator. Then, standard network meta-analysis techniques can be applied, taking care to consider correlation structures in cohorts with multiple scores. Transitivity, consistency and heterogeneity are also examined.

RESULTS

We provide a clinical application, comparing prognostic scores for 3-year mortality in patients with chronic obstructive pulmonary disease using data from a large-scale collaborative initiative. We focus on the discriminative properties of the prognostic scores. Our results show clear differences in performance, with ADO and eBODE showing higher discrimination with respect to mortality than other considered scores. The assumptions of transitivity and local and global consistency were not violated. Heterogeneity was small.

CONCLUSIONS

We applied a network meta-analytic methodology to externally validate and concurrently compare the prognostic properties of clinical scores. Our large-scale external validation indicates that the scores with the best discriminative properties to predict 3 year mortality in patients with COPD are ADO and eBODE.

Network meta-analysis including treatment by covariate interactions: Consistency can vary across covariate values

BACKGROUND

Many reviews aim to compare numerous treatments and report results stratified by subgroups (eg, by disease severity). In such cases, a network meta-analysis model including treatment by covariate interactions can estimate the relative effects of all treatment pairings for each subgroup of patients. Two key assumptions underlie such models: consistency of treatment effects and consistency of the regression coefficients for the interactions. Consistency may differ depending on the covariate value at which consistency is assessed. For valid inference, we need to be confident of consistency for the relevant range of covariate values. In this paper, we demonstrate how to assess consistency of treatment effects from direct and indirect evidence at various covariate values.

METHODS

Consistency is assessed using visual inspection, inconsistency estimates, and probabilities. The method is applied to an individual patient dataset comparing artemisinin combination therapies for treating uncomplicated malaria in children using the covariate age.

RESULTS

The magnitude of the inconsistency appears to be decreasing with increasing age for each comparison. For one comparison, direct and indirect evidence differ for age 1 (P = .05), and this brings results for age 1 for all comparisons into question.

CONCLUSION

When fitting models including interactions, the consistency of direct and indirect evidence must be assessed across the range of covariates included in the trials. Clinical inferences are only valid for covariate values for which results are consistent.

Common pitfalls and mistakes in the set-up, analysis and interpretation of results in network meta-analysis: what clinicians should look for in a published article

OBJECTIVE

Several tools have been developed to evaluate the extent to which the findings from a network meta-analysis would be valid; however, applying these tools is a time-consuming task and often requires specific expertise. Clinicians have little time for critical appraisal, and they need to understand the key elements that help them select network meta-analyses that deserve further attention, optimising time and resources. This paper is aimed at providing a practical framework to assess the methodological robustness and reliability of results from network meta-analysis.

METHODS

As a working example, we selected a network meta-analysis about drug treatments for generalised anxiety disorder, which was published in 2011 in the British Medical Journal. The same network meta-analysis was previously used to illustrate the potential of this methodology in a methodological paper published in JAMA.

RESULTS

We reanalysed the 27 studies included in this network following the methods reported in the original article and compared our findings with the published results. We showed how different methodological approaches and the presentation of results can affect conclusions from network meta-analysis. We divided our results into three sections, according to the specific issues that should always be addressed in network meta-analysis: (1) understanding the evidence base, (2) checking the statistical analysis and (3) checking the reporting of findings.

CONCLUSIONS

The validity of the results from network meta-analysis depends on the plausibility of the transitivity assumption. The risk of bias introduced by limitations of individual studies must be considered first and judgement should be used to infer about the plausibility of transitivity. Inconsistency exists when treatment effects from direct and indirect evidence are in disagreement. Unlike transitivity, inconsistency can be always evaluated statistically, and it should be specifically investigated and reported in the published paper. Network meta-analysis allows researchers to list treatments in preferential order; however, in this paper we demonstrated that rankings could be misleading if based on the probability of being the best. Clinicians should always be interested in the effect sizes rather than the naive rankings.

Diagnostics for generalized linear hierarchical models in network meta-analysis

Network meta-analysis (NMA) combines direct and indirect evidence comparing more than 2 treatments. Inconsistency arises when these 2 information sources differ. Previous work focuses on inconsistency detection, but little has been done on how to proceed after identifying inconsistency. The key issue is whether inconsistency changes an NMA's substantive conclusions. In this paper, we examine such discrepancies from a diagnostic point of view. Our methods seek to detect influential and outlying observations in NMA at a trial-by-arm level. These observations may have a large effect on the parameter estimates in NMA, or they may deviate markedly from other observations. We develop formal diagnostics for a Bayesian hierarchical model to check the effect of deleting any observation. Diagnostics are specified for generalized linear hierarchical NMA models and investigated for both published and simulated datasets. Results from our example dataset using either contrast- or arm-based models and from the simulated datasets indicate that the sources of inconsistency in NMA tend not to be influential, though results from the example dataset suggest that they are likely to be outliers. This mimics a familiar result from linear model theory, in which outliers with low leverage are not influential. Future extensions include incorporating baseline covariates and individual-level patient data.

Keeping Children Safe: a multicentre programme of research to increase the evidence base for preventing unintentional injuries in the home in the under-fives

Background

Unintentional injuries among 0- to 4-year-olds are a major public health problem incurring substantial NHS, individual and societal costs. However, evidence on the effectiveness and cost-effectiveness of preventative interventions is lacking.

Aim

To increase the evidence base for thermal injury, falls and poisoning prevention for the under-fives.

Methods

Six work streams comprising five multicentre case–control studies assessing risk and protective factors, a study measuring quality of life and injury costs, national surveys of children’s centres, interviews with children’s centre staff and parents, a systematic review of barriers to, and facilitators of, prevention and systematic overviews, meta-analyses and decision analyses of home safety interventions. Evidence from these studies informed the design of an injury prevention briefing (IPB) for children’s centres for preventing fire-related injuries and implementation support (training and facilitation). This was evaluated by a three-arm cluster randomised controlled trial comparing IPB and support (IPB+), IPB only (no support) and usual care. The primary outcome was parent-reported possession of a fire escape plan. Evidence from all work streams subsequently informed the design of an IPB for preventing thermal injuries, falls and poisoning.

Results

Modifiable risk factors for falls, poisoning and scalds were found. Most injured children and their families incurred small to moderate health-care and non-health-care costs, with a few incurring more substantial costs. Meta-analyses and decision analyses found that home safety interventions increased the use of smoke alarms and stair gates, promoted safe hot tap water temperatures, fire escape planning and storage of medicines and household products, and reduced baby walker use. Generally, more intensive interventions were the most effective, but these were not always the most cost-effective interventions. Children’s centre and parental barriers to, and facilitators of, injury prevention were identified. Children’s centres were interested in preventing injuries, and believed that they could prevent them, but few had an evidence-based strategic approach and they needed support to develop this. The IPB was implemented by children’s centres in both intervention arms, with greater implementation in the IPB+ arm. Compared with usual care, more IPB+ arm families received advice on key safety messages, and more families in each intervention arm attended fire safety sessions. The intervention did not increase the prevalence of fire escape plans [adjusted odds ratio (AOR) IPB only vs. usual care 0.93, 95% confidence interval (CI) 0.58 to 1.49; AOR IPB+ vs. usual care 1.41, 95% CI 0.91 to 2.20] but did increase the proportion of families reporting more fire escape behaviours (AOR IPB only vs. usual care 2.56, 95% CI 1.38 to 4.76; AOR IPB+ vs. usual care 1.78, 95% CI 1.01 to 3.15). IPB-only families were less likely to report match play by children (AOR 0.27, 95% CI 0.08 to 0.94) and reported more bedtime fire safety routines (AOR for a 1-unit increase in the number of routines 1.59, 95% CI 1.09 to 2.31) than usual-care families. The IPB-only intervention was less costly and marginally more effective than usual care. The IPB+ intervention was more costly and marginally more effective than usual care.

Limitations

Our case–control studies demonstrate associations between modifiable risk factors and injuries but not causality. Some injury cost estimates are imprecise because of small numbers. Systematic reviews and meta-analyses were limited by the quality of the included studies, the small numbers of studies reporting outcomes and significant heterogeneity, partly explained by differences in interventions. Network meta-analysis (NMA) categorised interventions more finely, but some variation remained. Decision analyses are likely to underestimate cost-effectiveness for a number of reasons. IPB implementation varied between children’s centres. Greater implementation may have resulted in changes in more fire safety behaviours.

Conclusions

Our studies provide new evidence about the effectiveness of, as well as economic evaluation of, home safety interventions. Evidence-based resources for preventing thermal injuries, falls and scalds were developed. Providing such resources to children’s centres increases their injury prevention activity and some parental safety behaviours.

Future work

Further randomised controlled trials, meta-analyses and NMAs are needed to evaluate the effectiveness and cost-effectiveness of home safety interventions. Further work is required to measure NHS, family and societal costs and utility decrements for childhood home injuries and to evaluate complex multicomponent interventions such as home safety schemes using a single analytical model.

Trial registration

Current Controlled Trials ISRCTN65067450 and ClinicalTrials.gov NCT01452191.

Funding

The National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 5, No. 14. See the NIHR Journals Library website for further project information.