Use of network meta-analysis in systematic reviews: a survey of authors

Methodological and reporting quality assessment of network meta-analyses in anesthesiology: a systematic review and meta-epidemiological study

PURPOSE

The scientific rigour of the conduct and reporting of anesthesiology network meta-analyses (NMAs) is unknown. This systematic review and meta-epidemiological study assessed the methodological and reporting quality of NMAs in anesthesiology.

METHODS

We searched four databases, including MEDLINE, PubMed, Embase, and the Cochrane Systematic Reviews Database, for anesthesiology NMAs published from inception to October 2020. We assessed the compliance of NMAs against A Measurement Tool to Assess Systematic Reviews (AMSTAR-2), Preferred Reporting Items for Systematic Reviews and Meta-Analyses Statement for Network Meta-Analyses (PRISMA-NMA), and PRISMA checklists. We measured the compliance across various items in AMSTAR-2 and PRISMA checklists and provided recommendations to improve quality.

RESULTS

Using the AMSTAR-2 rating method, 84% (52/62) of NMAs were rated "critically low." Quantitatively, the median [interquartile range] AMSTAR-2 score was 55 [44-69]%, while the PRISMA score was 70 [61-81]%. Methodological and reporting scores showed a strong correlation (R = 0.78). Anesthesiology NMAs had a higher AMSTAR-2 score and PRISMA score if they were published in higher impact factor journals (P = 0.006 and P = 0.01, respectively) or followed PRISMA-NMA reporting guidelines (P = 0.001 and P = 0.002, respectively). Network meta-analyses from China had lower scores (P < 0.001 and P < 0.001, respectively). Neither score improved over time (P = 0.69 and P = 0.67, respectively).

CONCLUSION

The current study highlights numerous methodological and reporting deficiencies in anesthesiology NMAs. Although the AMSTAR tool has been used to assess the methodological quality of NMAs, dedicated tools for conducting and assessing the methodological quality of NMAs are urgently required.

STUDY REGISTRATION

PROSPERO (CRD42021227997); first submitted 23 January 2021.

Retrieval barriers in individual participant data reviews with network meta-analysis

OBJECTIVES

Individual participant data (IPD) from randomised controlled trials (RCTs) can be used in network meta-analysis (NMA) to underpin patient care and are the best analyses to support the development of guidelines about the use of healthcare interventions for a specific condition. However, barriers to IPD retrieval pose a major threat. The aim of this study was to present barriers we encountered during retrieval of IPD from RCTs in two published systematic reviews with IPD-NMA.

METHODS

We evaluated retrieval of IPD from RCTs for IPD-NMA in Alzheimer's dementia and type 1 diabetes. We requested IPD from authors, industry sponsors and data repositories, and recorded IPD retrieval, reasons for IPD unavailability, and retrieval challenges.

RESULTS

In total, we identified 108 RCTs: 78 industry sponsored, 11 publicly sponsored and 19 with no funding information. After failing to obtain IPD from any trial authors, we requested it from industry sponsors. Seven of the 17 industry sponsors shared IPD for 12 950 participants (59%) through proprietary-specific data sharing platforms from 26 RCTs (33%). We found that lack of RCT identifiers (eg, National Clinical Trial number) and unclear data ownership were major challenges in IPD retrieval. Incomplete information in retrieved datasets was another important problem that led to exclusion of RCTs from the NMA. There were also practical challenges in obtaining IPD from or analysing it within platforms, and additional costs were incurred in accessing IPD this way.

CONCLUSIONS

We found no clear evidence of retrieval bias (where IPD availability was linked to trial findings) in either IPD-NMA, but because retrieval bias could impact NMA findings, subsequent decision-making and guideline development, this should be considered when assessing risk of bias in IPD syntheses.

Critical evaluation of reporting quality of network meta-analyses assessing the effectiveness of acupuncture

BACKGROUND

and purpose: Network meta-analyses (NMAs) comparing the effectiveness of multiple acupuncture have been published but the key concepts underlying NMAs have not been properly reported. This critical evaluation aims to assess the completeness of reporting for NMAs of acupuncture to enhance the validity of findings.

MATERIALS AND METHODS

Five databases were searched. The characteristics and reporting quality based on the Preferred Reporting Items for Systematic Reviews and Meta-analyses Extension Statement for reporting of NMA (PRISMA-NMA) were evaluated. NMAs were categorized into two groups according to the publication date, reporting guideline, and intervention. The group differences and correlation coefficients were calculated.

RESULTS

Forty-two NMAs of acupuncture were evaluated. The reporting quality for new items in the key concepts of NMAs was low (median 70.0 % (interquartile range 60.0-80.0)). While the issue of consistency was well reported, the assumption of transitivity and network geometry showed poor reporting. Seventeen studies that followed the PRISMA-NMA guideline showed a higher reporting rate for essential concepts of NMA. The recency of publication did not guarantee clear reporting.

CONCLUSION

The reporting quality of NMAs of acupuncture was low. The researchers should follow the guidelines on the reporting of NMAs.

Assessing the Quality and Coherence of Network Meta-Analyses of Biologics in Plaque Psoriasis: What Does All This Evidence Synthesis Tell Us?

INTRODUCTION

A range of treatments are available for moderate-to-severe psoriasis; however, there remains a paucity of direct comparisons of these in head-to-head trials. Network meta-analyses (NMA) allow comparisons of these to support clinical decision making. This systematic literature review assesses the methodological quality of NMAs available for moderate-to-severe psoriasis and compares their methods and results. Their validity and applicability for current practice is also assessed.

METHODS

A systematic review of published NMAs of at least two biologics for moderate-to-severe psoriasis was undertaken. Embase, MEDLINE, MEDLINE In-Process, and the Cochrane Library were last searched on 19 February 2020. The quality of NMAs was assessed using the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) criteria. NMA methodology, funding, and results were compared and differences in results explored.

RESULTS

Twenty-five analyses evaluating up to 19 different treatments at 8-24 weeks, and two analyses at 1 year, were included. Psoriasis Area Severity Index (PASI) response was assessed in 23, facilitating comparisons between NMAs. All NMAs met at least half of the ISPOR criteria. The major limitations were explaining the rationale for methodology, exploring effect modifiers, and consistency between direct and indirect estimates. The analyses differed in model type (Bayesian or frequentist), analysis of PASI response (binomial or multinomial), and analysis of different treatment doses (separate or pooled). PASI results were broadly similar, except for the Cochrane Collaboration NMA which provided lower estimates of treatment efficacy versus placebo. This analysis differed methodologically from others, including pooling data for different doses.

CONCLUSIONS

Based on PASI 90 at induction, the majority of recent NMAs came to similar conclusions: interleukin (IL) 17 inhibitors (brodalumab, ixekizumab, secukinumab), IL-23 inhibitors (guselkumab and risankizumab) and infliximab were most efficacious, supporting the validity of NMAs in this clinical area. Decisions should be made using high-quality, up-to-date NMAs with assumptions relevant to clinical practice.

Topical Fluoride to Prevent Root Caries: Systematic Review with Network Meta-analysis

The aim of this systematic review and network meta-analysis was to summarize the direct and indirect clinical evidence on the effectiveness of professionally applied and self-applied topical fluorides in preventing dental root caries. Controlled clinical trials with any follow-up duration were included. MEDLINE, PubMed, Embase, Scopus, and Cochrane Library were searched. Two reviewers independently carried out the selection of studies, data extraction, risk-of-bias assessments, and assessment of the certainty in the evidence using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. Fixed effects model and frequentist approach were used in the network meta-analyses. Nine clinical trials involving 4,030 participants were included. Five professionally applied and 7 self-applied topical fluoride agents or combinations were included in the meta-analyses. Compared to control group, 38% silver diamine fluoride solution, 5% sodium fluoride varnish, and 1.2% acidulated phosphate fluoride reduced root caries increment after 2 y (ranging from 0.59 to 0.85 mean decayed or filled root [DF-root]). Fluoride mouth rinse and fluoride toothpaste, used alone or in combination, reduced root caries increment after 1 y (ranging from 0.29 to 1.90 mean DF-root). Among the professionally applied topical fluorides reviewed, an annually applied 38% silver diamine fluoride (SDF) solution combined with oral health education is most likely to be the most effective in preventing dental root caries. Among the reviewed self-applied topical fluoride methods, daily use of a 0.2% sodium fluoride (NaF) mouth rinse is most likely to be the most effective, followed by 1100 ppm to 1500 ppm fluoride toothpaste plus 0.05% NaF mouth rinse, and 1100 ppm to 1500 ppm fluoride toothpaste.

Preoxygenation before intubation in adult patients with acute hypoxemic respiratory failure: a network meta-analysis of randomized trials

Background

Patients with acute hypoxemic respiratory failure are at risk for life-threatening complications during endotracheal intubation. Preoxygenation might help reduce the risk of hypoxemia and intubation-related complications. This network meta-analysis summarizes the efficacy and safety of preoxygenation methods in adult patients with acute hypoxemic respiratory failure.

Methods

We searched PubMed, EMBASE, and the Cochrane Library Central Register of Controlled Trials through April 2019 for randomized controlled trials (RCT) that studied the use of conventional oxygen therapy (COT), high-flow nasal cannula (HFNC), noninvasive ventilation (NIV), and HFNC and NIV as preoxygenation before intubation in patients with acute hypoxemic respiratory failure. Citations’ screening, study selection, data extraction, and risk of bias assessment were independently performed by two authors. The primary outcome was the lowest SpO₂ during the intubation procedure.

Results

We included 7 RCTs (959 patients). Patients preoxygenated with NIV had significantly less desaturation than patients treated with COT (mean difference, MD 5.53, 95% CI 2.71, 8.34) and HFNC (MD 3.58, 95% CI 0.59, 6.57). Both NIV (odds ratio, OR 0.43, 95% CI 0.21, 0.87) and HFNC (OR 0.49, 95% CI 0.28, 0.88) resulted in a lower risk of intubation-related complications than COT. There were no significant mortality differences among the use of NIV, HFNC, COT, and HFNC and NIV during preoxygenation.

Conclusions

In adult patients with acute hypoxemic respiratory failure, NIV is a safe and probably the most effective preoxygenation method.

The Effect of Disease Modifying Therapies on Disability Progression in Multiple Sclerosis: A Systematic Overview of Meta-Analyses

Background: Disease modifying therapy (DMT) efficacy trials make an essential contribution to the development of evidence-based clinical treatments and practices for people with multiple sclerosis (MS). Meta-analysis is a critical part of this process and provides a powerful tool to assess the effects of DMT on MS progression. However, although there have been several meta-analyses on the effect of DMT on MS disease progression, they often do not reach the same conclusions.

Objective: Our aim was to better understand and contextualize the results of meta-analyses evaluating DMT, identify differences in methodology that might explain their differing conclusions, and highlight areas for future research that will improve our ability to develop clinical recommendations.

Methods: We conducted an overview of systematic reviews with meta-analyses assessing the efficacy of DMT on disability progression in people with MS in PubMed (Medline) and the Cochrane Database of Systematic Reviews.

Results: We included 22 meta-analyses in this overview: eight general (on >3 DMT), 11 specific (on ≤3 DMT), 2 that evaluated subsets, and 1 that evaluated long-term effects. We found that there is good evidence that DMT improve short-term (≤2–3 years) disability progression outcomes relative to placebo in people with relapsing-remitting MS. However, results varied substantially between meta-analyses, and there is little evidence of their efficacy in other populations or over longer periods. The relative effects of individual DMT also remain unclear. The variance in results between meta-analyses may be related to the substantial differences in inclusion criteria, which was reflected in the limited overlap in included studies, as well as the year of meta-analysis publication. Of the 123 total unique studies included in the general meta-analyses, 77 (62.6%) were included in only one meta-analysis. This incongruence was also evident in the included DMT. Six of the 16 (37.5%) DMT evaluated in the general meta-analyses were only included in one meta-analysis.

Conclusions: Translating DMT efficacy studies into evidence-based clinical practice requires greater methodological consistency in meta-analyses, more data on the relative effects of DMT through head-to-head clinical trials, and better reporting of adverse events.

Interventions for treating nausea and vomiting in pregnancy: a network meta-analysis and trial sequential analysis of randomized clinical trials

INTRODUCTION

Several drugs were explored for their utility in treating nausea and vomiting in pregnancy (NVP). The present study is a network meta-analysis of such drugs.

METHODS

Electronic databases were searched for randomized clinical trials that have compared active interventions (with placebo or other active interventions) for treating NVP. Nausea scores were the primary outcome and changes in nausea scores, emetic episodes, adverse events, and vomiting control were the key secondary outcomes. Weighted mean difference was the effect estimate for continuous variable and odds ratio for the numerical variable. Random-effects model was used and the strength of the evidence was graded.

RESULTS

Fifty studies were included in the systematic review and 42 in the meta-analysis. Acupuncture, chamomile, dimenhydrinate, doxylamine/vitamin B6, ginger, quince, metoclopramide, and vitamin B6 were associated with reduced nausea scores compared to placebo. Of these interventions, ginger and vitamin B6 were also associated with better vomiting control and less incidence of adverse events. Adequate evidence supporting the use exists only for ginger and the quality of evidence for this comparison is moderate. Strength of evidence for all other comparisons is very low.

CONCLUSION

Present evidence is conclusive on the therapeutic benefits of ginger in treating NVP. Although favorable results were obtained for several other interventions, the strength of evidence is very low. The results of this network meta-analysis should be interpreted with extreme caution as it might change with the advent of data from future head-to-head clinical trials.

Mapping the characteristics of network meta-analyses on drug therapy: A systematic review

BACKGROUND

Network meta-analysis (NMA) is a new tool developed to overcome some limitations of pairwise meta-analyses. NMAs provide evidence on more than two comparators simultaneously. This study aimed to map the characteristics of the published NMAs on drug therapy comparisons.

METHODS

A systematic review of NMAs comparing pharmacological interventions was performed. Searches in Medline (PubMed) and Scopus along with manual searches were conducted. The main characteristics of NMAs were systematically collected: publication metadata, criteria for drug inclusion, statistical methods used, and elements reported. A methodological quality score with 25 key elements was created and applied to the included NMAs. To identify potential trends, the median of the publication year distribution was used as a cut-off.

RESULTS

The study identified 365 NMAs published from 2003 to 2016 in more than 30 countries. Randomised controlled trials were the primary source of data, with only 5% including observational studies, and 230 NMAs used a placebo as a comparator. Less than 15% of NMAs were registered in PROSPERO or a similar system. One third of studies followed PRISMA and less than 9% Cochrane recommendations. Around 30% presented full-search strategies of the systematic review, and 146 NMAs stated the selection criteria for drug inclusion. Over 75% of NMAs presented network plots, but only half described their geometry. Statistical parameters (model fit, inconsistency, convergence) were properly reported by one third of NMAs. Although 216 studies exhibited supplemental material, no data set of primary studies was available. The methodological quality score (mean 13·9; SD 3·8) presented a slightly positive trend over the years.

CONCLUSION

The map of the published NMAs emphasises the potential of this tool to gather evidence in healthcare, but it also identified some weaknesses, especially in the report, which limits its transparency and reproducibility.

Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review

BACKGROUND

The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations.

METHODS

We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated).

RESULTS

We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions.

CONCLUSIONS

Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.