History and publication trends in the diffusion and early uptake of indirect comparison meta-analytic methods to study drugs: animated coauthorship networks over time

Methodological review of NMA bias concepts provides groundwork for the development of a list of concepts for potential inclusion in a new risk of bias tool for network meta-analysis (RoB NMA Tool)

INTRODUCTION

Network meta-analyses (NMAs) have gained popularity and grown in number due to their ability to provide estimates of the comparative effectiveness of multiple treatments for the same condition. The aim of this study is to conduct a methodological review to compile a preliminary list of concepts related to bias in NMAs.

METHODS AND ANALYSIS

We included papers that present items related to bias, reporting or methodological quality, papers assessing the quality of NMAs, or method papers. We searched MEDLINE, the Cochrane Library and unpublished literature (up to July 2020). We extracted items related to bias in NMAs. An item was excluded if it related to general systematic review quality or bias and was included in currently available tools such as ROBIS or AMSTAR 2. We reworded items, typically structured as questions, into concepts (i.e. general notions).

RESULTS

One hundred eighty-one articles were assessed in full text and 58 were included. Of these articles, 12 were tools, checklists or journal standards; 13 were guidance documents for NMAs; 27 were studies related to bias or NMA methods; and 6 were papers assessing the quality of NMAs. These studies yielded 99 items of which the majority related to general systematic review quality and biases and were therefore excluded. The 22 items we included were reworded into concepts specific to bias in NMAs.

CONCLUSIONS

A list of 22 concepts was included. This list is not intended to be used to assess biases in NMAs, but to inform the development of items to be included in our tool.

Treatment classifications and interventions for neck pain: a scoping review

OBJECTIVES

Our aim is to provide an overview of how neck pain is classified in the literature, define and group conservative interventions into 'nodes', and develop draft networks of interventions in preparation for a network meta-analysis (NMA).

STUDY DESIGN AND SETTINGS

We performed a scoping review. For feasibility reasons, we searched for randomized clinical trials (RCTs) via neck pain clinical practice guidelines (CPGs) published from 2014. We used standardized data extraction forms to extract data about classification of neck pain and interventions evaluated in the included RCTs. We calculated frequencies of neck pain classifications and grouped interventions into nodes based on the definitions used in Cochrane reviews. Draft network graphs comparing interventions were constructed using the online Shiny R application CINEMA.

RESULTS

We included 242 RCTs from seven CPGs, evaluating 28,581 patients. We found three different classification systems of which The Neck Pain Task Force classification was used most often. We defined and grouped all interventions into 19 discrete potential nodes.

CONCLUSION

We found a wide variation in neck pain classifications and conservative interventions. Grouping the interventions was challenging and needs further evaluation before conducting a final NMA.

Knowledge user survey and Delphi process to inform development of a new risk of bias tool to assess systematic reviews with network meta-analysis (RoB NMA tool)

BACKGROUND

Network meta-analysis (NMA) is increasingly used in guideline development and other aspects of evidence-based decision-making. We aimed to develop a risk of bias (RoB) tool to assess NMAs (RoB NMA tool). An international steering committee recommended that the RoB NMA tool to be used in combination with the Risk of Bias in Systematic reviews (ROBIS) tool (i.e. because it was designed to assess biases only) or other similar quality appraisal tools (eg, A MeaSurement Tool to Assess systematic Reviews 2 [AMSTAR 2]) to assess quality of systematic reviews. The RoB NMA tool will assess NMA biases and limitations regarding how the analysis was planned, data were analysed and results were presented, including the way in which the evidence was assembled and interpreted.

OBJECTIVES

Conduct (a) a Delphi process to determine expert opinion on an item's inclusion and (b) a knowledge user survey to widen its impact.

DESIGN

Cross-sectional survey and Delphi process.

METHODS

Delphi panellists were asked to rate whether items should be included. All agreed-upon item were included in a second round of the survey (defined as 70% agreement). We surveyed knowledge users' views and preferences about the importance, utility and willingness to use the RoB NMA tool to evaluate evidence in practice and in policymaking. We included 12 closed and 10 open-ended questions, and we followed a knowledge translation plan to disseminate the survey through social media and professional networks.

RESULTS

22 items were entered into a Delphi survey of which 28 respondents completed round 1, and 22 completed round 2. Seven items did not reach consensus in round 2. A total of 298 knowledge users participated in the survey (14% respondent rate). 75% indicated that their organisation produced NMAs, and 78% showed high interest in the tool, especially if they had received adequate training (84%). Most knowledge users and Delphi panellists preferred a tool to assess both bias in individual NMA results and authors' conclusions. Response bias in our sample is a major limitation as knowledge users working in high-income countries were more represented. One of the limitations of the Delphi process is that it depends on the purposive selection of experts and their availability, thus limiting the variability in perspectives and scientific disciplines.

CONCLUSIONS

This Delphi process and knowledge user survey informs the development of the RoB NMA tool.

Use of Instrumental Variable Analyses for Evaluating Comparative Effectiveness in Empirical Applications of Oncology: A Systematic Review

PURPOSE

This systematic review aims to characterize the use and trends of instrumental variables (IVs) in oncology research, assess the quality and completeness of IV reporting, and evaluate the agreement and interpretation of IV results in comparison with other techniques used for determining comparative effectiveness in observational research.

METHODS

We performed a systematic search of observational empirical oncology papers evaluating the comparative effectiveness of cancer treatments using IV methods. EMBASE and MEDLINE (through June 2021) were used for a keyword search; Scopus and Web of Science were used for a citation search. Publication details and characteristics of IV analysis and reporting were extracted from each study to examine the uptake and quality of IV applications.

RESULTS

Sixty-five empirical papers were identified from February 2001 through June 2021. Geographic variation (50.8%) was the most common type of IV used, and the majority of IV applications constructed binary instruments (53.8%). Concurrent analyses using another non-IV method to adjust for confounding were conducted in 56 (86.2%) studies, 17 (30.4%) of which produced results divergent from IV approaches. We observed a modest uptake of IV methods between 2011 and 2021 together with its dissemination, which remained fairly limited to the United States (76.9%). The quality and completeness of IV reporting varied greatly. The underlying assumptions required for a valid IV analysis were only accounted for in full by 20 (30.8%) studies.

CONCLUSION

There are limited use and variable quality of IV analyses in oncology. Future research should look to establish standards to better facilitate the quality, transparency, and completeness of IV reporting in this setting.

Methodological review to develop a list of bias items used to assess reviews incorporating network meta-analysis: protocol and rationale

INTRODUCTION

Systematic reviews with network meta-analysis (NMA; ie, multiple treatment comparisons, indirect comparisons) have gained popularity and grown in number due to their ability to provide comparative effectiveness of multiple treatments for the same condition. The methodological review aims to develop a list of items relating to biases in reviews with NMA. Such a list will inform a new tool to assess the risk of bias in NMAs, and potentially other reporting or quality checklists for NMAs which are being updated.

METHODS AND ANALYSIS

We will include articles that present items related to bias, reporting or methodological quality, articles assessing the methodological quality of reviews with NMA, or papers presenting methods for NMAs. We will search Ovid MEDLINE, the Cochrane library and difficult to locate/unpublished literature. Once all items have been extracted, we will combine conceptually similar items, classifying them as referring to bias or to other aspects of quality (eg, reporting). When relevant, reporting items will be reworded into items related to bias in NMA review conclusions, and then reworded as signalling questions.

ETHICS AND DISSEMINATION

No ethics approval was required. We plan to publish the full study open access in a peer-reviewed journal, and disseminate the findings via social media (Twitter, Facebook and author affiliated websites). Patients, healthcare providers and policy-makers need the highest quality evidence to make decisions about which treatments should be used in healthcare practice. Being able to critically appraise the findings of systematic reviews that include NMA is central to informed decision-making in patient care.