Methodological review to develop a list of bias items used to assess reviews incorporating network meta-analysis: protocol and rationale

Methodological review of NMA bias concepts provides groundwork for the development of a list of concepts for potential inclusion in a new risk of bias tool for network meta-analysis (RoB NMA Tool)

INTRODUCTION

Network meta-analyses (NMAs) have gained popularity and grown in number due to their ability to provide estimates of the comparative effectiveness of multiple treatments for the same condition. The aim of this study is to conduct a methodological review to compile a preliminary list of concepts related to bias in NMAs.

METHODS AND ANALYSIS

We included papers that present items related to bias, reporting or methodological quality, papers assessing the quality of NMAs, or method papers. We searched MEDLINE, the Cochrane Library and unpublished literature (up to July 2020). We extracted items related to bias in NMAs. An item was excluded if it related to general systematic review quality or bias and was included in currently available tools such as ROBIS or AMSTAR 2. We reworded items, typically structured as questions, into concepts (i.e. general notions).

RESULTS

One hundred eighty-one articles were assessed in full text and 58 were included. Of these articles, 12 were tools, checklists or journal standards; 13 were guidance documents for NMAs; 27 were studies related to bias or NMA methods; and 6 were papers assessing the quality of NMAs. These studies yielded 99 items of which the majority related to general systematic review quality and biases and were therefore excluded. The 22 items we included were reworded into concepts specific to bias in NMAs.

CONCLUSIONS

A list of 22 concepts was included. This list is not intended to be used to assess biases in NMAs, but to inform the development of items to be included in our tool.

Clinical validation of grouping conservative treatments in neck pain for use in a network meta-analysis: a Delphi consensus study

BACKGROUND

A network meta-analysis aims to help clinicians make clinical decisions on the most effective treatment for a certain condition. Neck pain is multifactorial, with various classification systems and treatment options. Classifying patients and grouping interventions in clinically relevant treatment nodes for a NMA is essential, but this process is poorly defined.

OBJECTIVE

Our aim is to obtain consensus among experts on neck pain classifications and the grouping of interventions into nodes for a future network meta-analysis.

DESIGN

A Delphi consensus study involving neck pain experts worldwide.

METHODS

We invited authors of neck pain clinical practice guidelines published from 2014 onwards. The Delphi baseline questionnaire was developed based on the findings of a scoping review, including four items on classifications and 19 nodes. Participants were asked to record their level of agreement on a seven-point Likert scale or using Yes/No/Not sure answer options for the various statements. We used descriptive analysis to summarise the responses on each statement with content analysis of the free-text comments.

RESULTS

In total, 18/80 experts (22.5%) agreed to participate in one or more Delphi rounds. We needed three rounds to reach consensus for two classification of neck pain: one based on aetiology and one on duration. In addition, we also reached consensus on the grouping of interventions, including a definition of each node, with the number of nodes reduced to 17.

CONCLUSION

With this consensus we clinically validated two neck pain classifications and grouped conservative treatments into 17 well-defined and clinically relevant nodes.

Methodological and reporting quality assessment of network meta-analyses in anesthesiology: a systematic review and meta-epidemiological study

PURPOSE

The scientific rigour of the conduct and reporting of anesthesiology network meta-analyses (NMAs) is unknown. This systematic review and meta-epidemiological study assessed the methodological and reporting quality of NMAs in anesthesiology.

METHODS

We searched four databases, including MEDLINE, PubMed, Embase, and the Cochrane Systematic Reviews Database, for anesthesiology NMAs published from inception to October 2020. We assessed the compliance of NMAs against A Measurement Tool to Assess Systematic Reviews (AMSTAR-2), Preferred Reporting Items for Systematic Reviews and Meta-Analyses Statement for Network Meta-Analyses (PRISMA-NMA), and PRISMA checklists. We measured the compliance across various items in AMSTAR-2 and PRISMA checklists and provided recommendations to improve quality.

RESULTS

Using the AMSTAR-2 rating method, 84% (52/62) of NMAs were rated "critically low." Quantitatively, the median [interquartile range] AMSTAR-2 score was 55 [44-69]%, while the PRISMA score was 70 [61-81]%. Methodological and reporting scores showed a strong correlation (R = 0.78). Anesthesiology NMAs had a higher AMSTAR-2 score and PRISMA score if they were published in higher impact factor journals (P = 0.006 and P = 0.01, respectively) or followed PRISMA-NMA reporting guidelines (P = 0.001 and P = 0.002, respectively). Network meta-analyses from China had lower scores (P < 0.001 and P < 0.001, respectively). Neither score improved over time (P = 0.69 and P = 0.67, respectively).

CONCLUSION

The current study highlights numerous methodological and reporting deficiencies in anesthesiology NMAs. Although the AMSTAR tool has been used to assess the methodological quality of NMAs, dedicated tools for conducting and assessing the methodological quality of NMAs are urgently required.

STUDY REGISTRATION

PROSPERO (CRD42021227997); first submitted 23 January 2021.

Knowledge user survey and Delphi process to inform development of a new risk of bias tool to assess systematic reviews with network meta-analysis (RoB NMA tool)

BACKGROUND

Network meta-analysis (NMA) is increasingly used in guideline development and other aspects of evidence-based decision-making. We aimed to develop a risk of bias (RoB) tool to assess NMAs (RoB NMA tool). An international steering committee recommended that the RoB NMA tool to be used in combination with the Risk of Bias in Systematic reviews (ROBIS) tool (i.e. because it was designed to assess biases only) or other similar quality appraisal tools (eg, A MeaSurement Tool to Assess systematic Reviews 2 [AMSTAR 2]) to assess quality of systematic reviews. The RoB NMA tool will assess NMA biases and limitations regarding how the analysis was planned, data were analysed and results were presented, including the way in which the evidence was assembled and interpreted.

OBJECTIVES

Conduct (a) a Delphi process to determine expert opinion on an item's inclusion and (b) a knowledge user survey to widen its impact.

DESIGN

Cross-sectional survey and Delphi process.

METHODS

Delphi panellists were asked to rate whether items should be included. All agreed-upon item were included in a second round of the survey (defined as 70% agreement). We surveyed knowledge users' views and preferences about the importance, utility and willingness to use the RoB NMA tool to evaluate evidence in practice and in policymaking. We included 12 closed and 10 open-ended questions, and we followed a knowledge translation plan to disseminate the survey through social media and professional networks.

RESULTS

22 items were entered into a Delphi survey of which 28 respondents completed round 1, and 22 completed round 2. Seven items did not reach consensus in round 2. A total of 298 knowledge users participated in the survey (14% respondent rate). 75% indicated that their organisation produced NMAs, and 78% showed high interest in the tool, especially if they had received adequate training (84%). Most knowledge users and Delphi panellists preferred a tool to assess both bias in individual NMA results and authors' conclusions. Response bias in our sample is a major limitation as knowledge users working in high-income countries were more represented. One of the limitations of the Delphi process is that it depends on the purposive selection of experts and their availability, thus limiting the variability in perspectives and scientific disciplines.

CONCLUSIONS

This Delphi process and knowledge user survey informs the development of the RoB NMA tool.

Phosphodiesterase type 5 inhibitors and premature ejaculation: an overview of systematic reviews/meta-analyses using the AMSTAR 2, ROBIS, and GRADE tools

Introduction

The place of phosphodiesterase type 5 inhibitors (PDE5-Is) in treating premature ejaculation (PE) remains a therapeutic challenge.

Objectives

(1) To summarize the evidence of the efficacy and safety of PDE5-Is from published systematic reviews/meta-analyses (SRs/MAs). (2) To evaluate the reporting, methodological quality, and evidence quality of SRs/MAs concerning PE.

Methods

Nine databases were searched to retrieve SRs/MAs on using PDE5-Is for PE from inception to July 2022. Methodological quality and risk of bias were assessed with the AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews) and ROBIS (Risk of Bias in Systematic Reviews). GRADE criteria (Grading of Recommendations, Assessment, Development, and Evaluations) were used to assess the evidence quality.

Results

The literature search revealed 15 relevant SRs/MAs covering 28 primary studies (9 pairwise MAs and 6 network MAs) rated as low or very low quality and high risk of bias except 1 review that was rated as moderate quality and low risk of bias. Among the 27 outcome measures related to efficacy and safety, the quality of evidence according to GRADE criteria was low in 4 and critically low in 23. Oral PDE5-Is have demonstrated a possible benefit over placebo in lifelong PE and mixed PE. The results of pairwise and network MAs advocated that the combined use of PDE5-Is and SSRIs is of possible benefit as compared with either SSRIs or PDE5-Is alone. The total adverse effects were more frequent with PDE5-Is than placebo.

Conclusion

PDE5-Is are of a possible benefit than placebo in lifelong PE and mixed PE. The results favor coadministration of PDE5-Is plus SSRIs over SSRIs alone or PDE5-I monotherapy. These conclusions should be interpreted cautiously due to the low methodological quality and low quality of evidence of most available reviews. Additional higher-quality randomized controlled trials, SRs, and MAs are warranted to provide a better estimate of any effect size.