Indirect comparisons: a review of reporting and methodological quality

A comparison of two models for detecting inconsistency in network meta-analysis

The application of network meta-analysis is becoming increasingly widespread, and for a successful implementation, it requires that the direct comparison result and the indirect comparison result should be consistent. Because of this, a proper detection of inconsistency is often a key issue in network meta-analysis as whether the results can be reliably used as a clinical guidance. Among the existing methods for detecting inconsistency, two commonly used models are the design-by-treatment interaction model and the side-splitting models. While the original side-splitting model was initially estimated using a Bayesian approach, in this context, we employ the frequentist approach. In this paper, we review these two types of models comprehensively as well as explore their relationship by treating the data structure of network meta-analysis as missing data and parameterizing the potential complete data for each model. Through both analytical and numerical studies, we verify that the side-splitting models are specific instances of the design-by-treatment interaction model, incorporating additional assumptions or under certain data structure. Moreover, the design-by-treatment interaction model exhibits robust performance across different data structures on inconsistency detection compared to the side-splitting models. Finally, as a practical guidance for inconsistency detection, we recommend utilizing the design-by-treatment interaction model when there is a lack of information about the potential location of inconsistency. By contrast, the side-splitting models can serve as a supplementary method especially when the number of studies in each design is small, enabling a comprehensive assessment of inconsistency from both global and local perspectives.

A comparison between psilocybin and esketamine in treatment-resistant depression using number needed to treat (NNT): A systematic review

BACKGROUND

Inadequate outcomes with monoamine-based treatments in depressive disorders are common and provide the impetus for mechanistically-novel treatments. Esketamine is a proven treatment recently approved for adults with Treatment-Resistant Depression (TRD) while psilocybin is an investigational treatment. Translation of the clinical meaningfulness for these foregoing agents in adults with TRD is required. Herein we evaluate the Number Needed to Treat (NNT) and Harm (NNH) of esketamine and psilocybin in adults with TRD.

METHODS

We conducted a systematic review of randomized controlled trials, comparing the clinical efficacy of oral psilocybin to the co-commencement of intranasal esketamine with an oral antidepressant in adults with TRD.

RESULTS

25 mg psilocybin had a significant reduction in depressive symptoms at 21-days post-dose, the NNT was 5 [95 % CI = 3.1, 18.5]. Psilocybin-induced nausea had a significant NNH = 5. Fixed-dosed esketamine at 56 mg and 84 mg had a significant effect at 28-days post-dose, (NNT of 7 [95 % CI56mg = 3.5, 46.7], [95 % CI84mg = 3.6, 142.2]). Esketamine-induced headache, nausea, dizziness, and dissociation had NNHs <10.

LIMITATIONS

The preliminary results may only reflect a small portion of the patient population. These results require replication and longer term studies investigating maintenance therapy.

CONCLUSION

Relatively few pharmacologic agents are proven safe and effective in adults with TRD. NNT estimates for investigational psilocybin and esketamine in TRD indicate clinical meaningfulness. The NNH profile for both aforementioned agents is clinically acceptable. Our results underscore the clinical relevance of these treatment options in adults with TRD.

Low awareness of the transitivity assumption in complex networks of interventions: a systematic survey from 721 network meta-analyses

BACKGROUND

The transitivity assumption is the cornerstone of network meta-analysis (NMA). Violating transitivity compromises the credibility of the indirect estimates and, by extent, the estimated treatment effects of the comparisons in the network. The present study offers comprehensive empirical evidence on the completeness of reporting and evaluating transitivity in systematic reviews with multiple interventions.

METHODS

We screened the datasets of two previous empirical studies, resulting in 361 systematic reviews with NMA published between January 2011 and April 2015. We updated our evidence base with an additional 360 systematic reviews with NMA published between 2016 and 2021, employing a pragmatic approach. We devised assessment criteria for reporting and evaluating transitivity using relevant methodological literature and compared their reporting frequency before and after the PRISMA-NMA statement.

RESULTS

Systematic reviews published after PRISMA-NMA were more likely to provide a protocol (odds ratio (OR): 3.94, 95% CI: 2.79-5.64), pre-plan the transitivity evaluation (OR: 3.01, 95% CI: 1.54-6.23), and report the evaluation and results (OR: 2.10, 95% CI: 1.55-2.86) than those before PRISMA-NMA. However, systematic reviews after PRISMA-NMA were less likely to define transitivity (OR: 0.57, 95% CI: 0.42-0.79) and discuss the implications of transitivity (OR: 0.48, 95% CI: 0.27-0.85) than those published before PRISMA-NMA. Most systematic reviews evaluated transitivity statistically than conceptually (40% versus 12% before PRISMA-NMA, and 54% versus 11% after PRISMA-NMA), with consistency evaluation being the most preferred (34% before versus 47% after PRISMA-NMA). One in five reviews inferred the plausibility of the transitivity (22% before versus 18% after PRISMA-NMA), followed by 11% of reviews that found it difficult to judge transitivity due to insufficient data. In justifying their conclusions, reviews considered mostly the comparability of the trials (24% before versus 30% after PRISMA-NMA), followed by the consistency evaluation (23% before versus 16% after PRISMA-NMA).

CONCLUSIONS

Overall, there has been a slight improvement in reporting and evaluating transitivity since releasing PRISMA-NMA, particularly in items related to the systematic review report. Nevertheless, there has been limited attention to pre-planning the transitivity evaluation and low awareness of the conceptual evaluation methods that align with the nature of the assumption.

Twenty years of network meta-analysis: Continuing controversies and recent developments

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.

Methodological review of NMA bias concepts provides groundwork for the development of a list of concepts for potential inclusion in a new risk of bias tool for network meta-analysis (RoB NMA Tool)

INTRODUCTION

Network meta-analyses (NMAs) have gained popularity and grown in number due to their ability to provide estimates of the comparative effectiveness of multiple treatments for the same condition. The aim of this study is to conduct a methodological review to compile a preliminary list of concepts related to bias in NMAs.

METHODS AND ANALYSIS

We included papers that present items related to bias, reporting or methodological quality, papers assessing the quality of NMAs, or method papers. We searched MEDLINE, the Cochrane Library and unpublished literature (up to July 2020). We extracted items related to bias in NMAs. An item was excluded if it related to general systematic review quality or bias and was included in currently available tools such as ROBIS or AMSTAR 2. We reworded items, typically structured as questions, into concepts (i.e. general notions).

RESULTS

One hundred eighty-one articles were assessed in full text and 58 were included. Of these articles, 12 were tools, checklists or journal standards; 13 were guidance documents for NMAs; 27 were studies related to bias or NMA methods; and 6 were papers assessing the quality of NMAs. These studies yielded 99 items of which the majority related to general systematic review quality and biases and were therefore excluded. The 22 items we included were reworded into concepts specific to bias in NMAs.

CONCLUSIONS

A list of 22 concepts was included. This list is not intended to be used to assess biases in NMAs, but to inform the development of items to be included in our tool.

Umbrella review and network meta-analysis of diagnostic imaging test accuracy studies in Differentiating between brain tumor progression versus pseudoprogression and radionecrosis

PURPOSE

In this study we gathered and analyzed the available evidence regarding 17 different imaging modalities and performed network meta-analysis to find the most effective modality for the differentiation between brain tumor recurrence and post-treatment radiation effects.

METHODS

We conducted a comprehensive systematic search on PubMed and Embase. The quality of eligible studies was assessed using the Assessment of Multiple Systematic Reviews-2 (AMSTAR-2) instrument. For each meta-analysis, we recalculated the effect size, sensitivity, specificity, positive and negative likelihood ratios, and diagnostic odds ratio from the individual study data provided in the original meta-analysis using a random-effects model. Imaging technique comparisons were then assessed using NMA. Ranking was assessed using the multidimensional scaling approach and by visually assessing surface under the cumulative ranking curves.

RESULTS

We identified 32 eligible studies. High confidence in the results was found in only one of them, with a substantial heterogeneity and small study effect in 21% and 9% of included meta-analysis respectively. Comparisons between MRS Cho/NAA, Cho/Cr, DWI, and DSC were most studied. Our analysis showed MRS (Cho/NAA) and 18F-DOPA PET displayed the highest sensitivity and negative likelihood ratios. 18-FET PET was ranked highest among the 17 studied techniques with statistical significance. APT MRI was the only non-nuclear imaging modality to rank higher than DSC, with statistical insignificance, however.

CONCLUSION

The evidence regarding which imaging modality is best for the differentiation between radiation necrosis and post-treatment radiation effects is still inconclusive. Using NMA, our analysis ranked FET PET to be the best for such a task based on the available evidence. APT MRI showed promising results as a non-nuclear alternative.

OssaNMA: An R package for using information from network meta-analyses to optimize the power and sample allocation of a new two-arm trial

Randomized clinical trials (RCTs) are designed for measuring the effectiveness of the treatments and testing a hypothesis regarding the relative effect between two or more treatments. Trial designers are often interested in maximizing power when the total sample size is fixed or minimizing the required total sample size to reach a pre-specified power. One approach to maximizing power proposed by previous researchers is to leverage prior evidence using meta-analysis (NMA) to inform the sample size determination of a new trial. For example, researchers may be interested in designing a two-arm trial comparing treatments A and B which are already in the existing trial network but do not have any direct comparison. The researchers' intention is to incorporate the result into an existing network for meta-analysis. Here we develop formulas to address these options and use simulations to validate our formula and evaluate the performance of different analysis methods in terms of power. We also implement our proposed method into the R package OssaNMA and publish an R Shiny app for the convenience of the application. The goal of the package is to enable researchers to readily adopt the proposed approach which can improve the power of an RCT and is therefore resource-saving. In the R Shiny app, We also provide the option to include the cost of each treatment which would enable researchers to compare the total treatment cost associated with each design and analysis approach. Further, we explore the effect of allocation to treatment group on study power when the a priori plan is to incorporate the new trial result into an existing network for meta-analysis.

Treatment-duration-wise harm profile of insulin-sodium-glucose co-transporter inhibitor co-treatment in type 1 diabetes mellitus patients

Background

The treatment duration of insulin-sodium-glucose co-transporter inhibitors (SGLTis) co-treatment of type 1 diabetes mellitus (T1DM) patients in randomized controlled trials (RCTs) varies by 1-52 weeks. Henceforth, treatment duration-wise, we compared the following insulin-treatment adjuncts- mega- versus low-dose SGLTis, SGLTis versus placebo, and different SGLTi dosages.

Method

Double-blinded RCTs reporting the above were searched (using terms like insulin-dependent, "juvenile-onset diabetes," and "sodium glucose cotransport*") in the PubMed, Embase, and Scopus databases and appraised using a Cochrane tool. The risks across different SGLTi-dosages were compared using network meta-analysis. Random-effect pairwise meta-analysis was performed for the remaining harm juxtapositions. Meta-analyses were performed for the following treatment durations- < 4 weeks, 4 to < 24 weeks, and ≥ 24 weeks. For meta-analysis and certainty of evidence assessment, we used the Stata statistical software and the GRADE method, respectively.

Results

A total of 15 (low risks of bias) studies sourcing data from about 7,330 T1DM patients were reviewed. Meta-analysis findings of ≥ 24 weeks long trials were- a. SGLTi-insulin co-treatment increased the genital infection (GI) (RR: 3.51; 95% CI: 2.59, 4.77), diabetic ketoacidosis (DKA) and (RR: 3.25; 95% CI:1.29, 8.16), and serious side effects (RR: 1.43; 95% CI: 1.05, 1.94) risk. b. SGLT2i-insulin increased the GI risk (RR: 3.77; 95% CI: 2.31, 6.16; high-quality evidence). c. Sotagliflozin-insulin increased the GI (RR: 3.36; 95% CI: 2.28, 4.96) and DKA (RR: 6.69; 95% CI: 2.75, 16.32) risk (both high-quality evidence). Compared to low-dose, megadose SGLTi treatment for 4 to < 24 weeks increased the GI risk. The remaining analyses were not statistically significantly different.

Conclusion

On moderate to long-term treatment (24-52 weeks) of T1DM patients, insulin-SGLT2i co-treatment was associated with GI risk, and insulin-sotagliflozin co-treatment was associated with DKA and GI risk.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40200-023-01192-7.

Evaluation of the Reporting Standard Guidelines of Network Meta-Analyses in Physical Therapy: A Systematic Review

The concept of network meta-analyses (NMA) has been introduced to the field of physical therapy. However, the reporting standard guidelines of these studies have not been evaluated. In this systematic review, we included all published NMA physical therapy studies that compared the clinical efficacy of three or more interventions to evaluate whether NMAs in physical therapy exhibit adequate reporting recommendations. PubMed, EMBASE, Web of Science, and the Cochrane Library were searched up to 30 June 2022. Among the 252 identified articles, 19 NMAs including 805 randomized controlled trials were included. We applied both preferred reporting items for systematic reviews and meta-analysis (PRISMA) and PRISMA-NMA checklists, which are 27- and 32-item reporting standard guidelines assessment tools, respectively. Protocol registrations (68.4%), risk of bias across studies (63.2%), additional analysis (57.9%), and funding (31.6%) were problematic items considering the PRISMA guidelines. Four studies reported all five new NMA-reporting items, and 15 (78.9%) did not address items S1-5 from the PRISMA-NMA guidelines. The median score (interquartile range) of the reporting standard guidelines was 27.0 (25.8-28.0). The identified shortcomings of published NMAs should be addressed while training researchers, and they should be encouraged to apply PRISMA-NMA, as a recognized tool for assessing NMA reporting guidelines is required.

Using information from network meta-analyses to optimize the power and sample allocation of a subsequent trial with a new treatment

BACKGROUND

A critical step in trial design is determining the sample size and sample allocation to ensure the proposed study has sufficient power to test the hypothesis of interest: superiority, equivalence, or non-inferiority. When data are available from prior trials and leveraged with the new trial to answer the scientific questions, the value of society's investment in prior research is increased. When prior information is available, the trial design including the sample size and allocation should be adapted accordingly, yet the current approach to trial design does not utilize such information. Ensuring we maximize the value of prior research is essential as there are always constraints on resources, either physical or financial, and designing a trial with adequate power can be a challenge.

METHODS

We propose an approach to increasing the power of a new trial by incorporating evidence from a network meta-analysis into the new trial design and analysis. We illustrate the methodology through an example network meta-analysis, where the goal is to identify the optimal allocation ratio for the new three-arm trial, which involves the reference treatment, the new treatment, and the negative control. The primary goal of the new trial is to show that the new treatment is non-inferior to the reference treatment. It may also be of interest to know if the new treatment is superior to the negative control. We propose an optimal treatment allocation strategy which is derived from minimizing the standard error of the log odds ratio estimate of the comparison of interest. We conducted a simulation study to assess the proposed methods to design a new trial while borrowing information from the existing network meta-analysis and compare it to even allocation methods.

RESULTS

Using mathematical derivation and simulations, we document that our proposed approach can borrow information from a network meta-analysis to modify the treatment allocation ratio and increase the power of the new trial given a fixed total sample size or to reduce the total sample size needed to reach a desired power.

CONCLUSIONS

When prior evidence about the hypotheses of interest is available, the traditional equal allocation strategy is not the most powerful approach anymore. Our proposed methodology can improve the power of trial design, reduce the cost of trials, and maximize the utility of prior investments in research.

Prophylactic cyclo-oxygenase inhibitor drugs for the prevention of morbidity and mortality in preterm infants: a network meta-analysis

BACKGROUND

Patent ductus arteriosus (PDA) is associated with significant morbidity and mortality in preterm infants. Cyclooxygenase inhibitors (COX-I) may prevent PDA-related complications. Controversy exists on which COX-I drug is the most effective and has the best safety profile in preterm infants.

OBJECTIVES

To compare the effectiveness and safety of prophylactic COX-I drugs and 'no COXI prophylaxis' in preterm infants using a Bayesian network meta-analysis (NMA).

SEARCH METHODS

Searches of Cochrane CENTRAL via Wiley, OVID MEDLINE and Embase via Elsevier were conducted on 9 December 2021. We conducted independent searches of clinical trial registries and conference abstracts; and scanned the reference lists of included trials and related systematic reviews.

SELECTION CRITERIA

We included randomised controlled trials (RCTs) that enrolled preterm or low birth weight infants within the first 72 hours of birth without a prior clinical or echocardiographic diagnosis of PDA and compared prophylactic administration of indomethacin or ibuprofen or acetaminophen versus each other, placebo or no treatment.

DATA COLLECTION AND ANALYSIS

We used the standard methods of Cochrane Neonatal. We used the GRADE NMA approach to assess the certainty of evidence derived from the NMA for the following outcomes: severe intraventricular haemorrhage (IVH), mortality, surgical or interventional PDA closure, necrotizing enterocolitis (NEC), gastrointestinal perforation, chronic lung disease (CLD) and cerebral palsy (CP).

MAIN RESULTS

We included 28 RCTs (3999 preterm infants). Nineteen RCTs (n = 2877) compared prophylactic indomethacin versus placebo/no treatment, 7 RCTs (n = 914) compared prophylactic ibuprofen versus placebo/no treatment and 2 RCTs (n = 208) compared prophylactic acetaminophen versus placebo/no treatment. Nine RCTs were judged to have high risk of bias in one or more domains.We identified two ongoing trials on prophylactic acetaminophen. Bayesian random-effects NMA demonstrated that prophylactic indomethacin probably led to a small reduction in severe IVH (network RR 0.66, 95% Credible Intervals [CrI] 0.49 to 0.87; absolute risk difference [ARD] 43 fewer [95% CrI, 65 fewer to 16 fewer] per 1000; median rank 2, 95% CrI 1-3; moderate-certainty), a moderate reduction in mortality (network RR 0.85, 95% CrI 0.64 to 1.1; ARD 24 fewer [95% CrI, 58 fewer to 16 more] per 1000; median rank 2, 95% CrI 1-4; moderate-certainty) and surgical PDA closure (network RR 0.40, 95% CrI 0.14 to 0.66; ARD 52 fewer [95% CrI, 75 fewer to 30 fewer] per 1000; median rank 2, 95% CrI 1-2; moderate-certainty) compared to placebo. Prophylactic indomethacin resulted in trivial difference in NEC (network RR 0.76, 95% CrI 0.35 to 1.2; ARD 16 fewer [95% CrI, 42 fewer to 13 more] per 1000; median rank 2, 95% CrI 1-3; high-certainty), gastrointestinal perforation (network RR 0.92, 95% CrI 0.11 to 3.9; ARD 4 fewer [95% CrI, 42 fewer to 137 more] per 1000; median rank 1, 95% CrI 1-3; moderate-certainty) or CP (network RR 0.97, 95% CrI 0.44 to 2.1; ARD 3 fewer [95% CrI, 62 fewer to 121 more] per 1000; median rank 2, 95% CrI 1-3; low-certainty) and may result in a small increase in CLD (network RR 1.10, 95% CrI 0.93 to 1.3; ARD 36 more [95% CrI, 25 fewer to 108 more] per 1000; median rank 3, 95% CrI 1-3; low-certainty). Prophylactic ibuprofen probably led to a small reduction in severe IVH (network RR 0.69, 95% CrI 0.41 to 1.14; ARD 39 fewer [95% CrI, 75 fewer to 18 more] per 1000; median rank 2, 95% CrI 1-4; moderate-certainty) and moderate reduction in surgical PDA closure (network RR 0.24, 95% CrI 0.06 to 0.64; ARD 66 fewer [95% CrI, from 82 fewer to 31 fewer] per 1000; median rank 1, 95% CrI 1-2; moderate-certainty) compared to placebo. Prophylactic ibuprofen may result in moderate reduction in mortality (network RR 0.83, 95% CrI 0.57 to 1.2; ARD 27 fewer [95% CrI, from 69 fewer to 32 more] per 1000; median rank 2, 95% CrI 1-4; low-certainty) and leads to trivial difference in NEC (network RR 0.73, 95% CrI 0.31 to 1.4; ARD 18 fewer [95% CrI, from 45 fewer to 26 more] per 1000; median rank 1, 95% CrI 1-3; high-certainty), or CLD (network RR 1.00, 95% CrI 0.83 to 1.3; ARD 0 fewer [95% CrI, from 61 fewer to 108 more] per 1000; median rank 2, 95% CrI 1-3; low-certainty). The evidence is very uncertain on effect of ibuprofen on gastrointestinal perforation (network RR 2.6, 95% CrI 0.42 to 20.0; ARD 76 more [95% CrI, from 27 fewer to 897 more] per 1000; median rank 3, 95% CrI 1-3; very low-certainty). The evidence is very uncertain on the effect of prophylactic acetaminophen on severe IVH (network RR 1.17, 95% CrI 0.04 to 55.2; ARD 22 more [95% CrI, from 122 fewer to 1000 more] per 1000; median rank 4, 95% CrI 1-4; very low-certainty), mortality (network RR 0.49, 95% CrI 0.16 to 1.4; ARD 82 fewer [95% CrI, from 135 fewer to 64 more] per 1000; median rank 1, 95% CrI 1-4; very low-certainty), or CP (network RR 0.36, 95% CrI 0.01 to 6.3; ARD 70 fewer [95% CrI, from 109 fewer to 583 more] per 1000; median rank 1, 95% CrI 1-3; very low-certainty). In summary, based on ranking statistics, both indomethacin and ibuprofen were equally effective (median ranks 2 respectively) in reducing severe IVH and mortality. Ibuprofen (median rank 1) was more effective than indomethacin in reducing surgical PDA ligation (median rank 2). However, no statistically-significant differences were observed between the COX-I drugs for any of the relevant outcomes.

AUTHORS' CONCLUSIONS

Prophylactic indomethacin probably results in a small reduction in severe IVH and moderate reduction in mortality and surgical PDA closure (moderate-certainty), may result in a small increase in CLD (low-certainty) and results in trivial differences in NEC (high-certainty), gastrointestinal perforation (moderate-certainty) and cerebral palsy (low-certainty). Prophylactic ibuprofen probably results in a small reduction in severe IVH and moderate reduction in surgical PDA closure (moderate-certainty), may result in a moderate reduction in mortality (low-certainty) and trivial differences in CLD (low-certainty) and NEC (high-certainty). The evidence is very uncertain about the effect of acetaminophen on any of the clinically-relevant outcomes.

Effectiveness of crizotinib versus entrectinib in ROS1-positive non-small-cell lung cancer using clinical and real-world data

Aims: To compare clinical trial results for crizotinib and entrectinib in ROS1-positive non-small-cell lung cancer and compare clinical trial data and real-world outcomes for crizotinib. Patients & methods: We analyzed four phase I-II studies using a simulated treatment comparison (STC). A STC of clinical trial versus real-world evidence compared crizotinib clinical data to real-world outcomes. Results: Adjusted STC found nonsignificant trends favoring crizotinib over entrectinib: objective response rate, risk ratio = 1.04 (95% CI: 0.85-1.28); median duration of response, mean difference = 16.11 months (95% CI: -1.57- 33.69); median progression-free survival, mean difference = 3.99 months (95% CI: -6.27-14.25); 12-month overall survival, risk ratio = 1.01 (95% CI: 0.90-1.12). Nonsignificant differences were observed between the trial end point values and the real-world evidence for crizotinib. Conclusions: Crizotinib and entrectinib have comparable efficacy in ROS1-positive non-small-cell lung cancer.

Prevalence of evidence of inconsistency and its association with network structural characteristics in 201 published networks of interventions

BACKGROUND

Network meta-analysis (NMA) has attracted growing interest in evidence-based medicine. Consistency between different sources of evidence is fundamental to the reliability of the NMA results. The purpose of the present study was to estimate the prevalence of evidence of inconsistency and describe its association with different NMA characteristics.

METHODS

We updated our collection of NMAs with articles published up to July 2018. We included networks with randomised clinical trials, at least four treatment nodes, at least one closed loop, a dichotomous primary outcome, and available arm-level data. We assessed consistency using the design-by-treatment interaction (DBT) model and testing all the inconsistency parameters globally through the Wald-type chi-squared test statistic. We estimated the prevalence of evidence of inconsistency and its association with different network characteristics (e.g., number of studies, interventions, intervention comparisons, loops). We evaluated the influence of the network characteristics on the DBT p-value via a multivariable regression analysis and the estimated Pearson correlation coefficients. We also evaluated heterogeneity in NMA (consistency) and DBT (inconsistency) random-effects models.

RESULTS

We included 201 published NMAs. The p-value of the design-by-treatment interaction (DBT) model was lower than 0.05 in 14% of the networks and lower than 0.10 in 20% of the networks. Networks including many studies and comparing few interventions were more likely to have small DBT p-values (less than 0.10), which is probably because they yielded more precise estimates and power to detect differences between designs was higher. In the presence of inconsistency (DBT p-value lower than 0.10), the consistency model displayed higher heterogeneity than the DBT model.

CONCLUSIONS

Our findings show that inconsistency was more frequent than what would be expected by chance, suggesting that researchers should devote more resources to exploring how to mitigate inconsistency. The results of this study highlight the need to develop strategies to detect inconsistency (because of the relatively high prevalence of evidence of inconsistency in published networks), and particularly in cases where the existing tests have low power.

Network Meta-Analysis Techniques for Synthesizing Prevention Science Evidence

Network meta-analysis is a popular statistical technique for synthesizing evidence from studies comparing multiple interventions. Benefits of network meta-analysis, over more traditional pairwise meta-analysis approaches, include evaluating efficacy/safety of interventions within a single framework, increased precision, comparing pairs of interventions that have never been directly compared in a trial, and providing a hierarchy of interventions in terms of their effectiveness. Network meta-analysis is relatively underutilized in prevention science. This paper therefore presents a primer of network meta-analysis for prevention scientists who wish to apply this method or to critically appraise evidence from publications using the method. We introduce the key concepts and assumptions of network meta-analysis, namely, transitivity and consistency, and demonstrate their applicability to the field of prevention science. We then illustrate the method using a network meta-analysis examining the comparative effectiveness of brief alcohol interventions for preventing hazardous drinking among college students. We provide data and code for all examples. Finally, we discuss considerations that are particularly relevant in network meta-analyses in the field of prevention, such as including non-randomized evidence.

Network meta-analysis-based comparison of first-line steroid-sparing adjuvants in the treatment of pemphigus vulgaris and pemphigus foliaceus

BACKGROUND

Steroid-sparing adjuvants may enhance oral glucocorticoid benefits in pemphigus treatment. Selecting the optimal therapeutic option among various first-line steroid-sparing adjuvants is often a clinical challenge due to the lack of head-to-head clinical trials.

OBJECTIVE

To determine the best first-line steroid-sparing adjuvants for pemphigus treatment.

METHODS

Randomized controlled trials comparing different steroid-sparing adjuvants in patients with pemphigus were identified through a systematic literature search and subjected to a network meta-analysis. The primary outcomes were the proportion of remission and the mean cumulative glucocorticoid dose.

RESULTS

Ten trials involving 592 patients were analyzed. Among the 7 steroid-sparing adjuvants evaluated, rituximab was the most effective for achieving remission and was more effective than steroid alone (odds ratio, 14.35; 95% confidence interval [CI], 4.71-43.68). Rituximab, azathioprine, and cyclophosphamide pulse therapy enabled the reduction of the cumulative glucocorticoid doses compared to the use of steroid alone: mean differences, -11,830.5 mg (95% CI, -14,089.48 to -9571.52), -3032.48 mg (-4700.74 to -1364.22), and -2469.54 mg (-4128.42 to -810.66), respectively.

LIMITATIONS

The results were driven primarily by a small number of studies, and the effect estimates are imprecise because of indirect comparisons.

CONCLUSION

Network meta-analysis showed that rituximab appears to be an efficacious, well tolerated steroid-sparing adjuvant for pemphigus.

Pregabalin and gabapentin in neuropathic pain management after spinal cord injury: a systematic review and meta-analysis

Neuropathic pain after spinal cord injury (SCI) has a significant negative impact on the patients’ quality of life. The objective of this systematic review is to examine the safety and efficacy of pregabalin (PGB) and gabapentin (GBP) in the treatment of neuropathic pain due to SCI. PubMed, the Cochrane Library, Embase, Scopus, and the Web of Science were searched up to December 2018. The reference lists of key and review studies were reviewed for additional citations. The quality of the studies was evaluated using the Cochrane Collaboration’s tools for assessing the risk of bias. A meta-analysis was performed for primary and secondary outcomes. Eight studies were eligible for inclusion. Meta-analysis of PGB vs. placebo showed that PGB was effective for neuropathic pain (standardized mean difference [SMD] = −0.40; 95% confidence interval [CI]: −0.78, −0.01), anxiety (MD = −0.68; 95% CI: −0.77, −0.59), depression (mean difference [MD] = −0.99; 95% CI: −1.08, −0.89), and sleep interference (MD = −1.08; 95% CI: −1.13, −1.02). Also, GBP was more effective than a placebo for reducing pain. No significant difference was observed between the efficacy of the two drugs (MD = −0.37; 95% CI: −1.67, 0.93). There was no significant difference between the two drugs for discontinuation due to adverse events (risk ratio = 3.00; 95% CI: 0.81, 11.15). PGB and GBP were effective vs. placebos in decreasing neuropathic pain after SCI. Also, there was no significant difference between the two drugs for decreasing pain and adverse events.

A criterion-based approach to systematic and transparent comparative effectiveness: a case study in psoriatic arthritis

Aim: Indirect treatment comparisons are used when no direct comparison is available. Comparison networks should satisfy the transitivity assumption, that is, equal likelihood of treatment assignment for a given patient based on comparability of studies. Materials & methods: Seven criteria were evaluated across 18 randomized controlled trials in psoriatic arthritis: inclusion/exclusion criteria, clinical trial design and follow-up, patient-level baseline characteristics, disease severity, prior therapies, concomitant and extended-trial treatment and placebo response differences. Results: Across studies, placebo was a common comparator, and key efficacy end points were reported. Collectively, several potential sources of insufficient transitivity were identified, most often related to trial design and population differences. Conclusion: Potential challenges in satisfying transitivity occur frequently and should be evaluated thoroughly.

BUGSnet: an R package to facilitate the conduct and reporting of Bayesian network Meta-analyses

Background

Several reviews have noted shortcomings regarding the quality and reporting of network meta-analyses (NMAs). We suspect that this issue may be partially attributable to limitations in current NMA software which do not readily produce all of the output needed to satisfy current guidelines.

Results

To better facilitate the conduct and reporting of NMAs, we have created an R package called “BUGSnet” (Bayesian inference Using Gibbs Sampling to conduct a Network meta-analysis). This R package relies upon Just Another Gibbs Sampler (JAGS) to conduct Bayesian NMA using a generalized linear model. BUGSnet contains a suite of functions that can be used to describe the evidence network, estimate a model and assess the model fit and convergence, assess the presence of heterogeneity and inconsistency, and output the results in a variety of formats including league tables and surface under the cumulative rank curve (SUCRA) plots. We provide a demonstration of the functions contained within BUGSnet by recreating a Bayesian NMA found in the second technical support document composed by the National Institute for Health and Care Excellence Decision Support Unit (NICE-DSU). We have also mapped these functions to checklist items within current reporting and best practice guidelines.

Conclusion

BUGSnet is a new R package that can be used to conduct a Bayesian NMA and produce all of the necessary output needed to satisfy current scientific and regulatory standards. We hope that this software will help to improve the conduct and reporting of NMAs.

Delivery room interventions to prevent bronchopulmonary dysplasia in preterm infants: a protocol for a systematic review and network meta-analysis

INTRODUCTION

As gestational age decreases, incidence of bronchopulmonary dysplasia (BPD) and chronic lung disease increases. There are many interventions used in the delivery room to prevent acute lung injury and consequently BPD in these patients. The availability of different treatment options often poses a practical challenge to the practicing neonatologist when it comes to making an evidence-based choice as the multitude of pairwise systematic reviews including Cochrane reviews that are currently available only provide a narrow perspective through head-to-head comparisons.

METHODS AND ANALYSIS

We will conduct a systematic review of all randomised controlled trials evaluating delivery room interventions within the first golden hour after birth for prevention of BPD. The primary outcome includes BPD. Secondary outcomes include death at 36 weeks of postmenstrual age or before discharge; severe intraventricular haemorrhage (grade 3 or 4 based on the Papile criteria); any air leak syndromes (including pneumothorax or pulmonary interstitial emphysema); retinopathy of prematurity (any stage) and neurodevelopmental impairment at 18-24 months. We will search from their inception to August 2018, the following databases: Medline, EMBASE and Cochrane Central Register of Controlled Trials as well as grey literature resources. Two reviewers will independently screen titles and abstracts, review full texts, extract information and assess the risk of bias and the confidence in the estimate (with Grading of Recommendations Assessment, Development and Evaluation approach). This review will use Bayesian network meta-analysis approach which allows the comparison of the multiple delivery room interventions for prevention of BPD. We will perform a Bayesian network meta-analysis to combine the pooled direct and indirect treatment effect estimates for each outcome, effectiveness and safety of delivery room interventions for prevention of BPD.

ETHICS AND DISSEMINATION

The proposed protocol is a network meta-analysis, which has been registered on PROSPERO International prospective register of systematic reviews (CRD42018078648). The results will provide an evidence-based guide to choosing the right sequence of early postnatal interventions that will be associated with the least likelihood of inducing lung injury and BPD in preterm infants. Furthermore, we will identify knowledge gaps and will encourage further research for other therapeutic options. Therefore, its results will be disseminated through peer-reviewed publications and conference presentations. Due to the nature of the design, no ethics approval is necessary.

Determining the Sample Size for Future Trials of Hearing Instruments for Unilaterally Deaf Adults: An Application of Network Meta-analysis

OBJECTIVE

Previous trials have compared the efficacy of hearing instruments to no intervention in adults with single-sided deafness (SSD) or the relative efficacy of different instruments. Network meta-analysis (NMA) was used to refine estimates of effect sizes to determine required sample sizes for further trials.

DATA SOURCES

PubMed, EMBASE, MEDLINE, Cochrane, CINAHL, and DARE databases were searched with no restrictions on language, with studies to February 2015 included.

STUDY SELECTION

Studies were included that 1) assessed hearing instruments in adults with SSD; 2) compared instruments with other instruments, placebo, or no intervention; 3) measured speech perception in quiet/noise and listening ability; 4) were prospective controlled or observational studies.

DATA EXTRACTION

The following data were extracted: sample size in each group, type of intervention and comparator, type of outcomes, mean outcome scores and their 95% confidence intervals.

DATA SYNTHESIS

Random-effects meta-analysis was conducted to determine pooled effects for each outcome based on direct evidence alone. NMA used graph-theoretical method to determine pooled effects based on indirect evidence. Sample size calculations were conducted for each outcome for each class of evidence.

CONCLUSIONS

The incorporation of indirect evidence had substantial impacts on some effect sizes but negligible impacts on other effects. The most notable impacts were on self-reported listening ability and measures of speech perception in noise. Changes in effect size estimates and required sample sizes resulting from the incorporation of indirect evidence highlight areas of uncertainty where trials may be feasible to conduct.

The uptake of Bayesian methods in biomedical meta-analyses: A scoping review (2005-2016)

AIM

Bayesian statistical methods can allow for more complete and accurate incorporation of evidence in meta-analyses. However, these methods remain under-utilized.

METHODS

A scoping review was conducted to examine the proportion of biomedical meta-analyses that used Bayesian methods in the period 2005-2016. The review also examined the reproducibility of the work, the cited sources, the reasons for it, its success or failure, the type of model and prior distributions, and whether a mixture of Bayesian and frequentist methods were employed.

RESULTS

We found that 1% of meta-analyses are Bayesian and that the reporting and conduct of these were often poor. Data were published in 41% of analyses, and programs to run the analysis in 18%. Network meta-analysis was the most common reason and became increasingly popular in recent years. In the majority of papers, models and distributions were either not reported or explained in such brief and ambiguous terms as to be uninformative.

CONCLUSIONS

More use needs to be made of Bayesian meta-analysis, and reporting needs to be improved. Greater awareness of these methods and access to training in them is essential.

Frontline treatment for transplant-eligible multiple myeloma: A 6474 patients network meta-analysis

Autologous transplantation continues to be the cornerstone of younger and fit multiple myeloma patients. It is known that frontline induction therapy before transplantation can influence post-transplant results. Therefore, best frontline treatment for transplant-eligible patients should be based on best available evidence to guide therapy. Furthermore, until now due to data scarcity, it was not possible to thoroughly compare lenalidomide to other regimens in this setting. We performed a systematic review and network (mixed treatment comparison) meta-analysis of 21 clinical trial publications, enrolling 6474 patients and comparing 11 different treatment frontline setting regimens regarding survival, response, and safety outcomes. OS analysis showed superiority of CRD (cyclophosphamide-lenalidomide-dexamethasone) over TD-based (thalidomide-dexamethasone, HR = 0.76,0.62-0.90), VAD-based (HR = 0.71,0.52-0.90), and Z-Dex (idarubicin-dexamethasone, HR = 0.37,0.17-0.76) regimens. Concerning PFS, VTD (bortezomib-thalidomide-dexametasone) showed superior results when compared with TD-based (HR = 0.66,0.51-0.84), VAD-based (HR = 0.61,0.46-0.82), Z-Dex (HR = 0.42,0.22-0.78), and high dose dexamethasone (Dex, HR = 0.62,0.41-0.90) regimens. Bortezomib/thalidomide regimens were not superior to lenalidomide, considering these outcomes. Also, concerning complete and overall response, VTD ranked first among other regimens, showing clear superiority over thalidomide-only containing protocols. Safety outcome evaluated infectious, cardiac, gastrointestinal, neurological, thrombotic, and hematological grade 3 to 4 adverse events. Risk of thrombotic events was higher with TAD (thalidomide-doxorubicin-dexamethasone), neurological with PAD (bortezomib-doxorubicin-dexamethasone), infectious with Dex, hematological with Z-Dex, gastrointestinal with VTD, and cardiac with PAD regimens. Our study endorses current recommendations on combined immunomodulatory drugs and proteasome inhibitors frontline regimens (in triplets) in transplant-eligible multiple myeloma patients, but also formally demonstrates the favorable performance of lenalidomide in overall and progression-free survival, when compared with bortezomib/thalidomide protocols.

Vaginal progesterone is as effective as cervical cerclage to prevent preterm birth in women with a singleton gestation, previous spontaneous preterm birth, and a short cervix: updated indirect comparison meta-analysis

BACKGROUND

An indirect comparison meta-analysis published in 2013 reported that both vaginal progesterone and cerclage are equally efficacious for preventing preterm birth and adverse perinatal outcomes in women with a singleton gestation, previous spontaneous preterm birth, and a sonographic short cervix. The efficacy of vaginal progesterone has been challenged after publication of the OPPTIMUM study. However, this has been resolved by an individual patient-data meta-analysis (Am J Obstet Gynecol. 2018;218:161-180).

OBJECTIVE

To compare the efficacy of vaginal progesterone and cerclage in preventing preterm birth and adverse perinatal outcomes in women with a singleton gestation, previous spontaneous preterm birth, and a midtrimester sonographic short cervix.

DATA SOURCES

MEDLINE, EMBASE, LILACS, and CINAHL (from their inception to March 2018); Cochrane databases, bibliographies, and conference proceedings.

STUDY ELIGIBILITY CRITERIA

Randomized controlled trials comparing vaginal progesterone to placebo/no treatment or cerclage to no cerclage in women with a singleton gestation, previous spontaneous preterm birth, and a sonographic cervical length <25 mm.

STUDY APPRAISAL AND SYNTHESIS METHODS

Updated systematic review and adjusted indirect comparison meta-analysis of vaginal progesterone vs cerclage using placebo/no cerclage as the common comparator. The primary outcomes were preterm birth <35 weeks of gestation and perinatal mortality. Pooled relative risks (RRs) with 95% confidence intervals were calculated.

RESULTS

Five trials comparing vaginal progesterone vs placebo (265 women) and 5 comparing cerclage vs no cerclage (504 women) were included. Vaginal progesterone, compared to placebo, significantly reduced the risk of preterm birth <35 and <32 weeks of gestation, composite perinatal morbidity/mortality, neonatal sepsis, composite neonatal morbidity, and admission to the neonatal intensive care unit (RRs from 0.29 to 0.68). Cerclage, compared to no cerclage, significantly decreased the risk of preterm birth <37, <35, <32, and <28 weeks of gestation, composite perinatal morbidity/mortality, and birthweight <1500 g (RRs from 0.64 to 0.70). Adjusted indirect comparison meta-analyses did not show statistically significant differences between vaginal progesterone and cerclage in the reduction of preterm birth or adverse perinatal outcomes.

CONCLUSION

Vaginal progesterone and cerclage are equally effective for preventing preterm birth and improving perinatal outcomes in women with a singleton gestation, previous spontaneous preterm birth, and a midtrimester sonographic short cervix. The choice of treatment will depend on adverse events and cost-effectiveness of interventions and patient/physician's preferences.

History and publication trends in the diffusion and early uptake of indirect comparison meta-analytic methods to study drugs: animated coauthorship networks over time

OBJECTIVE

To characterise the early diffusion of indirect comparison meta-analytic methods to study drugs.

DESIGN

Systematic literature synthesis.

DATA SOURCES

Cochrane Database of Systematic Reviews, EMBASE, MEDLINE, Scopus and Web of Science.

STUDY SELECTION

English language papers that used indirect comparison meta-analytic methods to study the efficacy or safety of three or more interventions, where at least one was a drug.

DATA EXTRACTION

The number of publications and authors was plotted by year and type: methodological contribution, review or empirical application. Author and methodological details were summarised for empirical applications, and animated coauthorship networks were created to visualise contributors by country and affiliation type (academia, industry, government or other) over time.

RESULTS

We identified 477 papers (74 methodological contributions, 42 reviews and 361 empirical applications) by 1689 distinct authors from 1997 to 2013. Prior to 2002, only three applications were published, with contributions from the USA (n=2) and Canada (n=1). The number of applications gradually increased annually with rapid uptake between 2011 and 2013 (n=254, 71%). Early diffusion occurred primarily in Europe with the first application credited to the UK in 2003. Application spread to other European countries in 2005, and may have been supported by regulatory requirements for drug approval. By the end of 2013, contributions included 49% credited to Europe (22% UK, 27% other), 37% credited to North America (11% Canada, 26% USA) and 14% from other regions.

CONCLUSION

Indirect comparison meta-analytic methods are an important innovation for health research. Although Canada and the USA were the first to apply these methods, Europe led their diffusion. The increase in uptake of these methods may have been facilitated by acceptance by regulatory agencies, which are calling for more comparative drug effect data to assist in drug accessibility and reimbursement decisions.

The clinical effectiveness and cost-effectiveness of abatacept, adalimumab, etanercept and tocilizumab for treating juvenile idiopathic arthritis: a systematic review and economic evaluation

BACKGROUND

Juvenile idiopathic arthritis (JIA) is characterised by joint pain, swelling and a limitation of movement caused by inflammation. Subsequent joint damage can lead to disability and growth restriction. Treatment commonly includes disease-modifying antirheumatic drugs (DMARDs), such as methotrexate. Clinical practice now favours newer drugs termed biologic DMARDs where indicated.

OBJECTIVE

To assess the clinical effectiveness and cost-effectiveness of four biologic DMARDs [etanercept (Enbrel(®), Pfizer), abatacept (Orencia(®), Bristol-Myers Squibb), adalimumab (Humira(®), AbbVie) and tocilizumab (RoActemra(®), Roche) - with or without methotrexate where indicated] for the treatment of JIA (systemic or oligoarticular JIA are excluded).

DATA SOURCES

Electronic bibliographic databases including MEDLINE, EMBASE, The Cochrane Library and the Database of Abstracts of Reviews of Effects were searched for published studies from inception to May 2015 for English-language articles. Bibliographies of related papers, systematic reviews and company submissions were screened and experts were contacted to identify additional evidence.

REVIEW METHODS

Systematic reviews of clinical effectiveness, health-related quality of life and cost-effectiveness were undertaken in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. A cost-utility decision-analytic model was developed to compare the estimated cost-effectiveness of biologic DMARDs versus methotrexate. The base-case time horizon was 30 years and the model took a NHS perspective, with costs and benefits discounted at 3.5%.

RESULTS

Four placebo-controlled randomised controlled trials (RCTs) met the inclusion criteria for the clinical effectiveness review (one RCT evaluating each biologic DMARD). Only one RCT included UK participants. Participants had to achieve an American College of Rheumatology Pediatric (ACR Pedi)-30 response to open-label lead-in treatment in order to be randomised. An exploratory adjusted indirect comparison suggests that the four biologic DMARDs are similar, with fewer disease flares and greater proportions of ACR Pedi-50 and -70 responses among participants randomised to continued biologic DMARDs. However, confidence intervals were wide, the number of trials was low and there was clinical heterogeneity between trials. Open-label extensions of the trials showed that, generally, ACR responses remained constant or even increased after the double-blind phase. The proportions of adverse events and serious adverse events were generally similar between the treatment and placebo groups. Four economic evaluations of biologic DMARDs for patients with JIA were identified but all had limitations. Two quality-of-life studies were included, one of which informed the cost-utility model. The incremental cost-effectiveness ratios (ICERs) for adalimumab, etanercept and tocilizumab versus methotrexate were £38,127, £32,526 and £38,656 per quality-adjusted life year (QALY), respectively. The ICER for abatacept versus methotrexate as a second-line biologic was £39,536 per QALY.

LIMITATIONS

The model does not incorporate the natural history of JIA in terms of long-term disease progression, as the current evidence is limited. There are no head-to-head trials of biologic DMARDs, and clinical evidence for specific JIA subtypes is limited.

CONCLUSIONS

Biologic DMARDs are superior to placebo (with methotrexate where permitted) in children with (predominantly) polyarticular course JIA who have had an insufficient response to previous treatment. Randomised comparisons of biologic DMARDs with long-term efficacy and safety follow-up are needed to establish comparative effectiveness. RCTs for JIA subtypes for which evidence is lacking are also required.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42015016459.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Graphs of study contributions and covariate distributions for network meta-regression

Background

Meta‐regression results must be interpreted taking into account the range of covariate values of the contributing studies. Results based on interpolation or extrapolation may be unreliable. In network meta‐regression (NMR) models, which include covariates in network meta‐analyses, results are estimated using direct and indirect evidence; therefore, it may be unclear which studies and covariate values contribute to which result. We propose graphs to help understand which trials and covariate values contribute to each NMR result and to highlight extrapolation or interpolation.

Methods

We introduce methods to calculate the contribution that each trial and covariate value makes to each result and compare them with existing methods. We show how to construct graphs including a network covariate distribution diagram, covariate‐contribution plot, heat plot, contribution‐NMR plot, and heat‐NMR plot. We demonstrate the methods using a dataset with treatments for malaria using the covariate average age and a dataset of topical fluoride interventions for preventing dental caries using the covariate randomisation year.

Results

For the malaria dataset, no contributing trials had an average age between 7–25 years and therefore results were interpolated within this range. For the fluoride dataset, there are no contributing trials randomised between 1954–1959 for most comparisons therefore, within this range, results would be extrapolated.

Conclusions

Even in a fully connected network, an NMR result may be estimated from trials with a narrower covariate range than the range of the whole dataset. Calculating contributions and graphically displaying them aids interpretation of NMR result by highlighting extrapolated or interpolated results.

Recommendations for assessing the risk of bias in systematic reviews of health-care interventions

OBJECTIVES

Risk-of-bias assessment is a central component of systematic reviews, but little conclusive empirical evidence exists on the validity of such assessments. In the context of such uncertainty, we present pragmatic recommendations that promote transparency and reproducibility in processes, address methodological advances in the risk-of-bias assessment, and can be applied consistently across review topics.

STUDY DESIGN AND SETTING

Epidemiological study design principles; available empirical evidence, risk-of-bias tools, and guidance; and workgroup consensus.

RESULTS

We developed recommendations for assessing the risk of bias of studies of health-care interventions specific to framing the focus and scope of risk-of-bias assessment; selecting the risk-of-bias categories; choosing assessment instruments; and conducting, analyzing, and presenting results of risk-of-bias assessments. Key recommendations include transparency and reproducibility of judgments, separating risk of bias from other constructs such as applicability and precision, and evaluating the risk of bias per outcome. We recommend against certain past practices, such as focusing on reporting quality, relying solely on study design or numerical quality scores, and automatically downgrading for industry sponsorship.

CONCLUSION

Risk-of-bias assessment remains a challenging but essential step in systematic reviews. We presented standards to promote transparency of judgments.

Comparison between long-acting injectable aripiprazole versus paliperidone palmitate in the treatment of schizophrenia: systematic review and indirect treatment comparison

We investigated the relative efficacy and tolerability of aripiprazole once monthly (AOM) versus paliperidone palmitate (PP) for treating schizophrenia. Extensive databases searches on short-term, placebo-controlled, randomized studies of AOM and PP were performed. Indirect treatment comparisons were performed between the two long-acting injectable antipsychotics (LAIAs). The primary efficacy endpoint was the mean change in the Positive and Negative Syndrome Scale total score from baseline between each LAIA and placebo. The effect sizes were mean differences and odds ratio (ORs) with 95% confidence intervals (CIs) for the primary efficacy endpoint and safety/tolerability between two LAIAs, respectively. Mean difference in the primary efficacy endpoint was significantly different, favouring AOM over PP (OR: -6.4; 95% CI: -11.402 to -1.358); sensitivity analyses and noninferiority test (AOM vs. PP) confirmed the primary results. The overall early dropout rate was not significantly different between AOM and PP (OR: 1.223; 95% CI: 0.737-2.03). However, there was a significant difference in the early dropout rate in terms of lack of efficacy favouring AOM over PP (OR: 0.394; 95% CI: 0.185-0.841). Within the context of the inherent limitations of the current analysis, our results may suggest that there may be relative advantages for AOM over PP in the short-term treatment of schizophrenia.

Incidence of Severe Nephrotoxicity With Cisplatin Based on Renal Function Eligibility Criteria: Indirect Comparison Meta-analysis

BACKGROUND

The objective of this meta-analysis was to indirectly compare incidence of nephrotoxicity in trials using cisplatin (CIS) for treatment of solid tumors when renal function was assessed using serum creatinine (SCr) or creatinine clearance (CrCl) for eligibility criteria.

METHODS

Randomized trials comparing CIS-containing with non-CIS-containing chemotherapy regimens were identified in PubMed. Included studies were performed from 1990 to 2010, used SCr or CrCl as an eligibility criterion, and reported incidence of grade ≥3 nephrotoxicity for both treatment arms using World Health Organization (WHO) or National Cancer Institute (NCI) toxicity criteria. The relative risk (RR) of grade ≥3 nephrotoxicity associated with CIS versus non-CIS regimens was examined. Subgroup analyses, adjusted indirect comparison, and metaregression were used to compare SCr and CrCl.

RESULTS

The literature search identified 2359 studies, 42 studies met all the inclusion criteria (N=9521 patients). SCr was used as an eligibility criterion in 20 studies (N=4704), CrCl was used in 9 studies (N=1650), and either was used in 13 studies (N=3167). The overall RR for developing nephrotoxicity with CIS versus non-CIS treatment was 1.75 (P=0.005). Subgroup analyses showed an increased risk when SCr was used (RR=2.60, P=0.005) but not when CrCl was used (RR=1.50, P=0.19). Both the adjusted indirect comparison and metaregression showed a nonsignificantly reduced risk of nephrotoxicity when CrCl was used.

CONCLUSIONS

CIS-based therapy was associated with a significant increase in severe nephrotoxicity. The risk of severe nephrotoxicity appears to be lower when CrCl was used to determine whether people should be treated with CIS.

Improving Endpoint Detection to Support Automated Systematic Reviews

Authors of biomedical articles use comparison sentences to communicate the findings of a study, and to compare the results of the current study with earlier studies. The Claim Framework defines a comparison claim as a sentence that includes at least two entities that are being compared, and an endpoint that captures the way in which the entities are compared. Although automated methods have been developed to identify comparison sentences from the text, identifying the role that a specific noun plays (i.e. entity or endpoint) is much more difficult. Automated methods have been successful at identifying the second entity, but classification models were unable to clearly differentiate between the first entity and the endpoint. We show empirically that establishing if head noun is an amount or measure provides a statistically significant improvement that increases the endpoint precision from 0.42 to 0.56 on longer and from 0.51 to 0.58 on shorter sentences and recall from 0.64 to 0.71 on longer and from 0.69 to 0.74 on shorter sentences. The differences were not statistically significant for the second compared entity.

The Development of a Checklist to Enhance Methodological Quality in Intervention Programs

The methodological quality of primary studies is an important issue when performing meta-analyses or systematic reviews. Nevertheless, there are no clear criteria for how methodological quality should be analyzed. Controversies emerge when considering the various theoretical and empirical definitions, especially in relation to three interrelated problems: the lack of representativeness, utility, and feasibility. In this article, we (a) systematize and summarize the available literature about methodological quality in primary studies; (b) propose a specific, parsimonious, 12-items checklist to empirically define the methodological quality of primary studies based on a content validity study; and (c) present an inter-coder reliability study for the resulting 12-items. This paper provides a precise and rigorous description of the development of this checklist, highlighting the clearly specified criteria for the inclusion of items and a substantial inter-coder agreement in the different items. Rather than simply proposing another checklist, however, it then argues that the list constitutes an assessment tool with respect to the representativeness, utility, and feasibility of the most frequent methodological quality items in the literature, one that provides practitioners and researchers with clear criteria for choosing items that may be adequate to their needs. We propose individual methodological features as indicators of quality, arguing that these need to be taken into account when designing, implementing, or evaluating an intervention program. This enhances methodological quality of intervention programs and fosters the cumulative knowledge based on meta-analyses of these interventions. Future development of the checklist is discussed.

Epidemiology Characteristics, Methodological Assessment and Reporting of Statistical Analysis of Network Meta-Analyses in the Field of Cancer

Because of the methodological complexity of network meta-analyses (NMAs), NMAs may be more vulnerable to methodological risks than conventional pair-wise meta-analysis. Our study aims to investigate epidemiology characteristics, conduction of literature search, methodological quality and reporting of statistical analysis process in the field of cancer based on PRISMA extension statement and modified AMSTAR checklist. We identified and included 102 NMAs in the field of cancer. 61 NMAs were conducted using a Bayesian framework. Of them, more than half of NMAs did not report assessment of convergence (60.66%). Inconsistency was assessed in 27.87% of NMAs. Assessment of heterogeneity in traditional meta-analyses was more common (42.62%) than in NMAs (6.56%). Most of NMAs did not report assessment of similarity (86.89%) and did not used GRADE tool to assess quality of evidence (95.08%). 43 NMAs were adjusted indirect comparisons, the methods used were described in 53.49% NMAs. Only 4.65% NMAs described the details of handling of multi group trials and 6.98% described the methods of similarity assessment. The median total AMSTAR-score was 8.00 (IQR: 6.00-8.25). Methodological quality and reporting of statistical analysis did not substantially differ by selected general characteristics. Overall, the quality of NMAs in the field of cancer was generally acceptable.

The Global Research Collaboration of Network Meta-Analysis: A Social Network Analysis

Background and Objective

Research collaborations in biomedical research have evolved over time. No studies have addressed research collaboration in network meta-analysis (NMA). In this study, we used social network analysis methods to characterize global collaboration patterns of published NMAs over the past decades.

Methods

PubMed, EMBASE, Web of Science and the Cochrane Library were searched (at 9^th July, 2015) to include systematic reviews incorporating NMA. Two reviewers independently selected studies and cross-checked the standardized data. Data was analyzed using Ucinet 6.0 and SPSS 17.0. NetDraw software was used to draw social networks.

Results

771 NMAs published in 336 journals from 3459 authors and 1258 institutions in 49 countries through the period 1997–2015 were included. More than three-quarters (n = 625; 81.06%) of the NMAs were published in the last 5-years. The BMJ (4.93%), Current Medical Research and Opinion (4.67%) and PLOS One (4.02%) were the journals that published the greatest number of NMAs. The UK and the USA (followed by Canada, China, the Netherlands, Italy and Germany) headed the absolute global productivity ranking in number of NMAs. The top 20 authors and institutions with the highest publication rates were identified. Overall, 43 clusters of authors (four major groups: one with 37 members, one with 12 members, one with 11 members and one with 10 members) and 21 clusters of institutions (two major groups: one with 62 members and one with 20 members) were identified. The most prolific authors were affiliated with academic institutions and private consulting firms. 181 consulting firms and pharmaceutical industries (14.39% of institutions) were involved in 199 NMAs (25.81% of total publications). Although there were increases in international and inter-institution collaborations, the research collaboration by authors, institutions and countries were still weak and most collaboration groups were small sizes.

Conclusion

Scientific production on NMA is increasing worldwide with research leadership of Western countries (most notably, the UK, the USA and Canada). More authors, institutions and nations are becoming involved in research collaborations, but frequently with limited international collaborations.

Efficacy and Safety of Ferric Carboxymaltose and Other Formulations in Iron-Deficient Patients: A Systematic Review and Network Meta-analysis of Randomised Controlled Trials

Background

Iron deficiency is very common in a number of medical conditions. Ferric carboxymaltose is a new stable iron preparation that can be administered in single infusions over short periods of time. The aim of this study was to conduct a systematic review of randomised controlled trials (RCTs) regarding the efficacy and safety of the novel complex compared with other iron formulations. In addition, the feasibility of a network meta-analysis for indirect comparisons was investigated.

Methods

A systematic literature review was performed for published RCTs on the use of ferric carboxymaltose in iron deficiency between July and October 2014. Indirect comparisons were also addressed using terms referring to competing iron formulations. We further supported the qualitative results of the systematic review by a network meta-analysis that allows pooling the evidence around different intervention outcomes in the absence of trials involving a direct comparison.

Results

The initial search yielded 1027 citations, which was decreased to 21 studies eligible for inclusion in the review. Studies were heterogeneous in the number of patients randomised, iron deficiency-related conditions addressed, trial inclusion criteria, time horizon, treatment dosage and outcomes assessed. Six studies with the same time horizon (i.e. 6 weeks) were included in the network meta-analysis. Considering the differences between final and initial outcome values for each iron formulation, the mean difference of these differences (delta) was estimated for each couple of treatments involving ferric carboxymaltose. Significant improvements in serum ferritin (µg/l) were obtained with ferric carboxymaltose compared to oral iron (delta 172.8; 95 % CI 66.7–234.4) and in haemoglobin (g/dl) with respect to ferric gluconate (delta 0.6; 95 % CI 0.2–0.9), oral iron (delta 0.8; 95 % CI 0.6–0.9) and placebo (delta 2.1; 95 % CI 1.2–3.0).

Conclusions

All currently available intravenous iron preparations appear to be safe and effective, but ferric carboxymaltose seems to provide a better and quicker correction of haemoglobin and serum ferritin levels in iron-deficient patients.

Antibiotics for eradicating meningococcal carriages: Network meta-analysis and investigation of evidence inconsistency

AIM

To compare different antibiotics for eradicating the carriage of Neisseria meningitidis (N. meningitidis), and to investigate heterogeneity and evidence inconsistency.

METHODS

From a search of PubMed and published systematic reviews, we identified 23 trials evaluating 15 antibiotics that could be connected in a trial network. The outcome of interest is the eradication of N. meningitidis. We used WinBUGS to conduct random-effects, mixed treatment comparisons. Heterogeneity and evidence inconsistency was investigated by meta-regression modelling and examining characteristics of trial participants and interventions evaluated.

RESULTS

Rifampin, ciprofloxacin, minocycline, ceftriaxone, and azythromycin were statistically significantly (P < 0.05) more effective than placebo. The probability of being the best was 67.0% for a combination of rifampin and minocycline, 25.0% for ceftriaxone, 1.7% for azythromycin, and below 1% for the remaining regimens. Significant inconsistency between the direct and indirect estimates was observed for the comparison of rifampin and ciprofloxacin (P < 0.01), which may be caused by different types of carriers and different doses of ciprofloxacin.

CONCLUSION

A range of prophylactic antibiotic regimens are effective for eradicating meningococcal carriages, and treatment choice will depend on the individual priorities of the patients and physicians. In clinical situations where complete eradication is considered to be of the utmost importance, a combination of rifampin and minocycline seems to offer the highest likelihood of success. Ceftriaxone as a single intramuscular injection is also likely to be more effective as compared with the other two antibiotics (ciprofloxacin or rifampin) recommended by the current guidelines.

Effectiveness and safety of treatments used for the management of patent ductus arteriosus (PDA) in preterm infants: a protocol for a systematic review and network meta-analysis

INTRODUCTION

Management of patent ductus arteriosus (PDA) in preterm infants is one of the most controversial topics in neonatal medicine. The availability of different pharmacotherapeutic options often poses a practical challenge to the practising neonatologist as to which one to choose as a therapeutic option. Our objectives are to determine the relative merits of the available pharmacotherapeutic options for the management of PDA.

METHODS AND ANALYSIS

We will conduct a systematic review of all randomised controlled trials evaluating the use of intravenous or oral: indomethacin, ibuprofen and acetaminophen for the treatment of PDA in preterm infants. The primary outcome is failure of closure of the PDA. Secondary outcomes are neonatal mortality, need for surgical closure, duration of ventilator support, chronic lung disease, intraventricular haemorrhage, periventricular leukomalacia, necrotising enterocolitis, gastrointestinal bleeding, time to full enteral feeds and oliguria. We will search Medline, Embase and Cochrane Central Register of Controlled Trials (CENTRAL) as well as grey literature resources. Two reviewers will independently screen titles and abstracts, review full texts, extract information, and assess the risk of bias (ROB) and the confidence in the estimate (with Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach). Subgroup analysis according to gestational age, birth weight, different doses of interventions, time of administration of the first dose of the intervention, and echocardiographic definition of haemodynamically significant PDA and ROB are planned. We will perform a Bayesian network meta-analysis to combine the pooled direct and indirect treatment effect estimates for each outcome, if adequate data are available.

ETHICS AND DISSEMINATION

The results will help to reduce the uncertainty about the safety and effectiveness of the interventions, will identify knowledge gaps or will encourage further research for other therapeutic options. Therefore, its results will be disseminated through peer-reviewed publications and conference presentations. On the basis of the nature of its design, no ethics approval is necessary for this study.

TRIAL REGISTRATION NUMBER

CRD42015015797.

Evidence regarding lingual fixed orthodontic appliances' therapeutic and adverse effects is insufficient

Data sourcesMedline, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Cochrane Central Register of Controlled Trials, Virtual Health Library and Web of Science were systematically searched up to July 2015 without limitations. Scopus, Google Scholar, ClinicalTrials.gov, the ISRCTN registry as well as reference lists of the trials included and relevant reviews were manually searched.Study selectionRandomised (RCTs) and prospective non-randomised clinical trials (non-RCTs) on human patients that compared therapeutic and adverse effects of lingual and labial appliances were considered. One reviewer initially screened titles and subsequently two reviewers independently screened the selected abstracts and full texts.Data extraction and synthesisThe data were extracted independently by the reviewers. Missing or unclear information, ongoing trials and raw data from split-mouth trials were requested from the authors of the trials. The quality of the included trials and potential bias across studies were assessed using Cochrane's risk of bias tool and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. For parallel trials, mean difference (MD) and the relative risk (RR) were used for continuous (objective speech performance, subjective speech performance, intercanine width, intermolar width and sagittal anchorage loss) and binary outcomes (eating difficulty), respectively. The standardised mean difference (SMD) was chosen to pool, after conversion, the outcome (oral discomfort) that assessed both binary and continuous. Random-effects meta-analyses were conducted, followed by subgroup and sensitivity analyses.ResultsThirteen papers pertaining to 11 clinical trials (three parallel RCTs, one split-mouth RCT and seven parallel prospective non-RCTs) were included with a total of 407 (34% male/66% female) patients. All trials had at least one bias domain at high risk of bias. Compared with labial appliances, lingual appliances were associated with increased overall oral discomfort, increased speech impediment (measured using auditory analysis), worse speech performance assessed by laypersons, increased eating difficulty and decreased intermolar width. On the other hand, lingual appliances were associated with increased intercanine width and significantly decreased anchorage loss of the maxillary first molar during space closure. However, the quality of all analyses included was judged as very low because of the high risk of bias of the included trials, inconsistency and imprecision.ConclusionsBased on existing trials there is insufficient evidence to make robust recommendations for lingual fixed orthodontic appliances regarding their therapeutic or adverse effects, as the quality of evidence was low.

Clinical effectiveness of cell therapies in patients with chronic liver disease and acute-on-chronic liver failure: a systematic review protocol

BACKGROUND

Chronic liver disease (CLD) is a major health burden worldwide. Liver cirrhosis, a form of CLD is the fifth most common cause of death in the UK. Acute-on-chronic liver failure (ACLF) is the result of an acute insult superimposed on patients with liver cirrhosis as a result of precipitating events such as infection or bleeding. ACLF has a high associated mortality as a result of multi-organ failure. The only effective treatment for CLD is liver transplantation, but the treatment is limited by shortage of donor organs. As a result, alternative treatments such as cell therapies have been studied in patients with liver diseases. This study will systematically review the evidence on clinical effectiveness of cell therapies in patients.

METHODS

All types of study design that investigate the effectiveness of cell therapies (haematopoietic, mesenchymal and unsorted cell types) of autologous or allogeneic origin and/or the use of granulocyte colony-stimulating factor in patients with CLD including ACLF will be included (except case reports). Both autologous and allogenic cell types will be included. The primary outcomes of interest are survival, model for end-stage liver disease score, quality of life and adverse events. Secondary outcomes include liver function tests, Child-Pugh score and events of liver decompensation. A literature search will be conducted in the following databases: MEDLINE, MEDLINE in Process, EMBASE and Cochrane Library (CENTRAL, CDSR, DARE, HTA databases). Trial registers will be searched for ongoing trials, as will conference proceedings. Reference lists of relevant articles and systematic reviews will be screened. Randomised controlled trial (RCT) evidence is likely to be scant; therefore, controlled trials and concurrently controlled observational studies will be primarily analysed and uncontrolled observational studies will be analysed where primary outcomes are not reported in the control studies or where uncontrolled studies have longer follow-up. Initial screening of studies will be carried by one reviewer with a proportion checked by another reviewer. Full-text selection will be performed by two reviewers independently against the pre-defined selection criteria. The data collection and the risk of bias assessment will be completed by one reviewer and counter checked by another reviewer for all selected studies. Where appropriate, data will be meta-analysed for each study design, therapy and outcome. Data specifically on ACLF will be treated as a subgroup.

DISCUSSION

This systematic review will identify the available evidence on the effectiveness of cell therapies in patients with CLD and in ACLF subgroup. The findings will aid decision-making by clinicians and health service leaders.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42016016104.

Parameterization of a disease progression simulation model for sequentially treated metastatic human epidermal growth factor receptor 2 positive breast cancer patients

Background The objective of this study is twofold: 1) to propose a simulation model for HER2+ metastatic breast cancer (mBC) which could further be used to assess the overall cost-effectiveness of the treatment sequences that would maximize survival of patients, and 2) to estimate transitional probabilities between treatment lines required to parameterize the simulation model, in the absence of individual patient data (IPD). Methods Individual patient data (IPD) were reconstructed for treatment lines composing four treatment sequences. Parametric models were tested to select the model that best fits the IPD. The transitional probability equations, used for disease progression modeling, were obtained by substituting the parameters of the general equation for transitional probabilities by the parameters estimated from fitted distributions. Results The log-logistic model best fitted the reconstructed data for progression-free and overall survival curves for each line of treatment. The shapes and scales of the log-logistic models were used to develop the transitional probability equations for the HER2+ mBC simulation model.

KEY LIMITATIONS

The estimation of the transitional probabilities depends heavily on the accuracy of the IPD reconstruction. Nonetheless, analytical and graphical tests can be performed to check the face validity of the reconstructed data. Additionally, sensitivity analyses can be conducted to test the impact of uncertainty surrounding the estimated parameters defining equations for transitional probabilities. Conclusion The results of this study can be used as input in model-based economic evaluations of sequential therapy for HER2+ mBC.