Comparison of methodological quality rating of systematic reviews on neuropathic pain using AMSTAR and R-AMSTAR

Summary of the best evidence for the prevention and management of myasthenic crisis after thymectomy

Background

Myasthenic crisis (MC) may occur after thymectomy in patients with myasthenia gravis (MG), but effective preventive interventions can reduce the occurrence of this complication. Previous research on MC focused on risk factors, emergency treatment, etc., which was relatively scattered and did not form a comprehensive management framework. This study sought to retrieve and summarize the relevant evidence on the prevention and management of postoperative MC to provide a theoretical reference for clinical medical staff.

Methods

According to the evidence pyramid model, relevant articles were retrieved from UpToDate, British Medical Journal (BMJ) Best Practice, World Health Organization (WHO), Scottish Intercollegiate Guidelines Network (SIGN), Guidelines International Network (GIN), Australian Joanna Briggs Institute (JBI) Healthcare Database, Medlive, PubMed, Cochrane Library, Embase, Web of Science, China National Knowledge Infrastructure (CNKI), and Wanfang. The types of evidence included clinical guidelines, expert consensus articles, clinical decisions, systematic reviews, and randomized controlled trials (RCTs). The quality evaluations were conducted using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) evaluation tool for guidelines, the Australian JBI Evidence-Based Healthcare Center evaluation tool for expert consensus articles, the Critical Appraisal for Summaries of Evidence (CASE) evaluation tool for clinical decisions, the Assessment of Multiple Systematic Reviews (AMSTAR) evaluation tool for systematic reviews, and the Cochrane risk-of-bias tool for RCTs.

Results

A total of 12 articles were included in this study, including three clinical guidelines, three expert consensus articles, three clinical decisions, two systematic reviews, and one RCT. From these articles, we summarized 39 pieces of evidence on the prevention and management of postoperative MC.

Conclusions

This study summarized the best evidence on the prevention and management of postoperative MC and provided to clinical staffs evidence-based clinical approaches to help reduce the incidence of this complication.

Characteristics and Methodological Quality of the Top 50 Most Influential Articles on Stroke Rehabilitation: A Bibliometric Analysis

ABSTRACT

This study aimed to conduct a comprehensive review of the top 50 most influential articles on stroke rehabilitation to investigate characteristics, such as the number of citations, year of publication, study design, and research topic, as well as to assess the evidence level and methodological quality. Moreover, we performed a supplementary assessment of the top 10 articles published within the past 5 yrs in the same domain, aiming to discern potential shifts in trends and methodological quality. Web of Science was used to search for articles on stroke rehabilitation. The data extracted from the articles included title, journal impact factor, year of publication, total number of citations, article topic, study design, and others. The level of evidence and methodological quality were assessed by two reviewers. Noninvasive brain stimulation and robotic rehabilitation were frequently discussed in the top 50 articles. We found that there was no difference in methodology quality between the top 50 articles in all years and the top ten articles in the past 5 yrs. Furthermore, the number of citations and citation density were not associated with the methodological quality. The findings suggest that the number of citations alone may not be a reliable indicator of research quality.

Most systematic reviews reporting adherence to AMSTAR 2 had critically low methodological quality: a cross-sectional meta-research study

OBJECTIVES

To analyze the methodological quality and characteristics of systematic reviews (SRs) that reported they were conducted in line with the AMSTAR 2 (A MeaSurement Tool to Assess Systematic Reviews).

STUDY DESIGN AND SETTING

This was a cross-sectional meta-research study. We searched MEDLINE and Embase. We included full reports of SRs reporting the study was conducted, prepared, or designed in line with the AMSTAR 2. Eligible SRs were those published from January 1, 2018, until May 3, 2022. We assessed the methodological quality of the included SRs using AMSTAR 2.

RESULTS

We included a total of 45 records. There were 43 SRs and 2 SR protocols. Among them, most were SRs of interventions that included primary studies on humans. More than half had a meta-analysis. According to our overall AMSTAR 2 assessments of included SRs, 35 SRs were of critically low confidence, 7 SRs were of low confidence, and one SR was of high confidence. There were no SRs of moderate confidence.

CONCLUSION

Even when authors indicate in their manuscripts that the SR was conducted/prepared/designed in line with the AMSTAR 2, it does not necessarily imply it is of high or even moderate confidence according to AMSTAR 2. A self-assessment with AMSTAR 2 could be required for submission and carefully checked by the editors/peer reviewers.

Interventions for treating pain and disability in adults with complex regional pain syndrome- an overview of systematic reviews

BACKGROUND

Complex regional pain syndrome (CRPS) is a chronic pain condition that usually occurs in a limb following trauma or surgery. It is characterised by persisting pain that is disproportionate in magnitude or duration to the typical course of pain after similar injury. There is currently no consensus regarding the optimal management of CRPS, although a broad range of interventions have been described and are commonly used. This is the first update of the original Cochrane review published in Issue 4, 2013.

OBJECTIVES

To summarise the evidence from Cochrane and non-Cochrane systematic reviews of the efficacy, effectiveness, and safety of any intervention used to reduce pain, disability, or both, in adults with CRPS.

METHODS

We identified Cochrane reviews and non-Cochrane reviews through a systematic search of Ovid MEDLINE, Ovid Embase, Cochrane Database of Systematic Reviews, CINAHL, PEDro, LILACS and Epistemonikos from inception to October 2022, with no language restrictions. We included systematic reviews of randomised controlled trials that included adults (≥18 years) diagnosed with CRPS, using any diagnostic criteria.  Two overview authors independently assessed eligibility, extracted data, and assessed the quality of the reviews and certainty of the evidence using the AMSTAR 2 and GRADE tools respectively. We extracted data for the primary outcomes pain, disability and adverse events, and the secondary outcomes quality of life, emotional well-being, and participants' ratings of satisfaction or improvement with treatment.  MAIN RESULTS: We included six Cochrane and 13 non-Cochrane systematic reviews in the previous version of this overview and five Cochrane and 12 non-Cochrane reviews in the current version. Using the AMSTAR 2 tool, we judged Cochrane reviews to have higher methodological quality than non-Cochrane reviews. The studies in the included reviews were typically small and mostly at high risk of bias or of low methodological quality. We found no high-certainty evidence for any comparison.  There was low-certainty evidence that bisphosphonates may reduce pain intensity post-intervention (standardised mean difference (SMD) -2.6, 95% confidence interval (CI) -1.8 to -3.4, P = 0.001; I2 = 81%; 4 trials, n = 181) and moderate-certainty evidence that they are probably associated with increased adverse events of any nature (risk ratio (RR) 2.10, 95% CI 1.27 to 3.47; number needed to treat for an additional harmful outcome (NNTH) 4.6, 95% CI 2.4 to 168.0; 4 trials, n = 181).  There was moderate-certainty evidence that lidocaine local anaesthetic sympathetic blockade probably does not reduce pain intensity compared with placebo, and low-certainty evidence that it may not reduce pain intensity compared with ultrasound of the stellate ganglion. No effect size was reported for either comparison. There was low-certainty evidence that topical dimethyl sulfoxide may not reduce pain intensity compared with oral N-acetylcysteine, but no effect size was reported. There was low-certainty evidence that continuous bupivacaine brachial plexus block may reduce pain intensity compared with continuous bupivacaine stellate ganglion block, but no effect size was reported. For a wide range of other commonly used interventions, the certainty in the evidence was very low and provides insufficient evidence to either support or refute their use. Comparisons with low- and very low-certainty evidence should be treated with substantial caution. We did not identify any RCT evidence for routinely used pharmacological interventions for CRPS such as tricyclic antidepressants or opioids.

AUTHORS' CONCLUSIONS

Despite a considerable increase in included evidence compared with the previous version of this overview, we identified no high-certainty evidence for the effectiveness of any therapy for CRPS. Until larger, high-quality trials are undertaken, formulating an evidence-based approach to managing CRPS will remain difficult. Current non-Cochrane systematic reviews of interventions for CRPS are of low methodological quality and should not be relied upon to provide an accurate and comprehensive summary of the evidence.

Gwet's AC1 is not a substitute for Cohen's kappa - A comparison of basic properties

Gwet's AC1 has been proposed as an alternative to Cohen's kappa in evaluating the agreement between two binary ratings. This approach is becoming increasingly popular, and researchers have been criticized for still using Cohen's kappa. However, a rigorous discussion of properties of Gwet's AC1 is still missing. In this paper several basic properties of Gwet's AC1 are investigated and compared with those of Cohen's kappa, in particular the dependence on the prevalence of positive ratings for a given agreement rate and the behaviour in case of no association or maximal disagreement. Both approaches compare the observed agreement rate with a comparative number. Cohen's kappa uses an expected agreement rate as comparator, whereas Gwet's AC1 uses an expected disagreement rate. Consequently, for a fixed agreement rate, Gwet's AC1 increases with increasing difference of the prevalence of positive ratings from 0.5. In contrast, Cohen's kappa decreases. Gwet's AC1 can take positive and negative values in the case of no association between the two raters, whereas Cohen's kappa is 0. Due to these fundamental differences, Gwet's AC1 should not be seen as a substitute for Cohen's kappa. In particular, the verbal classification of kappa values by Landis & Koch should not be applied to Gwet's AC1.

Pharmacological treatments for low back pain in adults: an overview of Cochrane Reviews

BACKGROUND

Pharmacological interventions are the most used treatment for low back pain (LBP). Use of evidence from systematic reviews of the effects of pharmacological interventions for LBP published in the Cochrane Library, is limited by lack of a comprehensive overview.

OBJECTIVES

To summarise the evidence from Cochrane Reviews of the efficacy, effectiveness, and safety of systemic pharmacological interventions for adults with non-specific LBP.

METHODS

The Cochrane Database of Systematic Reviews was searched from inception to 3 June 2021, to identify reviews of randomised controlled trials (RCTs) that investigated systemic pharmacological interventions for adults with non-specific LBP. Two authors independently assessed eligibility, extracted data, and assessed the quality of the reviews and certainty of the evidence using the AMSTAR 2 and GRADE tools. The review focused on placebo comparisons and the main outcomes were pain intensity, function, and safety.

MAIN RESULTS

Seven Cochrane Reviews that included 103 studies (22,238 participants) were included. There is high confidence in the findings of five reviews, moderate confidence in one, and low confidence in the findings of another. The reviews reported data on six medicines or medicine classes: paracetamol, non-steroidal anti-inflammatory drugs (NSAIDs), muscle relaxants, benzodiazepines, opioids, and antidepressants. Three reviews included participants with acute or sub-acute LBP and five reviews included participants with chronic LBP. Acute LBP Paracetamol There was high-certainty evidence for no evidence of difference between paracetamol and placebo for reducing pain intensity (MD 0.49 on a 0 to 100 scale (higher scores indicate worse pain), 95% CI -1.99 to 2.97), reducing disability (MD 0.05 on a 0 to 24 scale (higher scores indicate worse disability), 95% CI -0.50 to 0.60), and increasing the risk of adverse events (RR 1.07, 95% CI 0.86 to 1.33). NSAIDs There was moderate-certainty evidence for a small between-group difference favouring NSAIDs compared to placebo at reducing pain intensity (MD -7.29 on a 0 to 100 scale (higher scores indicate worse pain), 95% CI -10.98 to -3.61), high-certainty evidence for a small between-group difference for reducing disability (MD -2.02 on a 0-24 scale (higher scores indicate worse disability), 95% CI -2.89 to -1.15), and very low-certainty evidence for no evidence of an increased risk of adverse events (RR 0.86, 95% CI 0. 63 to 1.18). Muscle relaxants and benzodiazepines There was moderate-certainty evidence for a small between-group difference favouring muscle relaxants compared to placebo for a higher chance of pain relief (RR 0.58, 95% CI 0.45 to 0.76), and higher chance of improving physical function (RR 0.55, 95% CI 0.40 to 0.77), and increased risk of adverse events (RR 1.50, 95% CI 1. 14 to 1.98). Opioids None of the included Cochrane Reviews aimed to identify evidence for acute LBP. Antidepressants No evidence was identified by the included reviews for acute LBP. Chronic LBP Paracetamol No evidence was identified by the included reviews for chronic LBP. NSAIDs There was low-certainty evidence for a small between-group difference favouring NSAIDs compared to placebo for reducing pain intensity (MD -6.97 on a 0 to 100 scale (higher scores indicate worse pain), 95% CI -10.74 to -3.19), reducing disability (MD -0.85 on a 0-24 scale (higher scores indicate worse disability), 95% CI -1.30 to -0.40), and no evidence of an increased risk of adverse events (RR 1.04, 95% CI -0.92 to 1.17), all at intermediate-term follow-up (> 3 months and ≤ 12 months postintervention). Muscle relaxants and benzodiazepines There was low-certainty evidence for a small between-group difference favouring benzodiazepines compared to placebo for a higher chance of pain relief (RR 0.71, 95% CI 0.54 to 0.93), and low-certainty evidence for no evidence of difference between muscle relaxants and placebo in the risk of adverse events (RR 1.02, 95% CI 0.67 to 1.57). Opioids There was high-certainty evidence for a small between-group difference favouring tapentadol compared to placebo at reducing pain intensity (MD -8.00 on a 0 to 100 scale (higher scores indicate worse pain), 95% CI -1.22 to -0.38), moderate-certainty evidence for a small between-group difference favouring strong opioids for reducing pain intensity (SMD -0.43, 95% CI -0.52 to -0.33), low-certainty evidence for a medium between-group difference favouring tramadol for reducing pain intensity (SMD -0.55, 95% CI -0.66 to -0.44) and very low-certainty evidence for a small between-group difference favouring buprenorphine for reducing pain intensity (SMD -0.41, 95% CI -0.57 to -0.26). There was moderate-certainty evidence for a small between-group difference favouring strong opioids compared to placebo for reducing disability (SMD -0.26, 95% CI -0.37 to -0.15), moderate-certainty evidence for a small between-group difference favouring tramadol for reducing disability (SMD -0.18, 95% CI -0.29 to -0.07), and low-certainty evidence for a small between-group difference favouring buprenorphine for reducing disability (SMD -0.14, 95% CI -0.53 to -0.25). There was low-certainty evidence for a small between-group difference for an increased risk of adverse events for opioids (all types) compared to placebo; nausea (RD 0.10, 95% CI 0.07 to 0.14), headaches (RD 0.03, 95% CI 0.01 to 0.05), constipation (RD 0.07, 95% CI 0.04 to 0.11), and dizziness (RD 0.08, 95% CI 0.05 to 0.11). Antidepressants There was low-certainty evidence for no evidence of difference for antidepressants (all types) compared to placebo for reducing pain intensity (SMD -0.04, 95% CI -0.25 to 0.17) and reducing disability (SMD -0.06, 95% CI -0.40 to 0.29).

AUTHORS' CONCLUSIONS

We found no high- or moderate-certainty evidence that any investigated pharmacological intervention provided a large or medium effect on pain intensity for acute or chronic LBP compared to placebo. For acute LBP, we found moderate-certainty evidence that NSAIDs and muscle relaxants may provide a small effect on pain, and high-certainty evidence for no evidence of difference between paracetamol and placebo. For safety, we found very low- and high-certainty evidence for no evidence of difference with NSAIDs and paracetamol compared to placebo for the risk of adverse events, and moderate-certainty evidence that muscle relaxants may increase the risk of adverse events. For chronic LBP, we found low-certainty evidence that NSAIDs and very low- to high-certainty evidence that opioids may provide a small effect on pain. For safety, we found low-certainty evidence for no evidence of difference between NSAIDs and placebo for the risk of adverse events, and low-certainty evidence that opioids may increase the risk of adverse events.

Definition, harms, and prevention of redundant systematic reviews

BACKGROUND

Along with other types of research, it has been stated that the extent of redundancy in systematic reviews has reached epidemic proportions. However, it was also emphasized that not all duplication is bad, that replication in research is essential, and that it can help discover unfortunate behaviors of scientists. Thus, the question is how to define a redundant systematic review, the harmful consequences of such reviews, and what we could do to prevent the unnecessary amount of this redundancy.

MAIN BODY

There is no consensus definition of a redundant systematic review. Also, it needs to be defined what amount of overlap between systematic reviews is acceptable and not considered a redundancy. One needs to be aware that it is possible that the authors did not intend to create a redundant systematic review. A new review on an existing topic, which is not an update, is likely justified only when it can be shown that the previous review was inadequate, for example, due to suboptimal methodology. Redundant meta-analyses could have scientific, ethical, and economic questions for researchers and publishers, and thus, they should be avoided, if possible. Potential solutions for preventing redundant reviews include the following: (1) mandatory prospective registration of systematic reviews; (2) editors and peer reviewers rejecting duplicate/redundant and inadequate reviews; (3) modifying the reporting checklists for systematic reviews; (4) developing methods for evidence-based research (EBR) monitoring; (5) defining systematic reviews; (6) defining the conclusiveness of systematic reviews; (7) exploring interventions for the adoption of methodological advances; (8) killing off zombie reviews (i.e., abandoned registered reviews); (9) better prevention of duplicate reviews at the point of registration; (10) developing living systematic reviews; and (11) education of researchers.

CONCLUSIONS

Disproportionate redundancy of the same or very similar systematic reviews can lead to scientific, ethical, economic, and societal harms. While it is not realistic to expect that the creation of redundant systematic reviews can be completely prevented, some preventive measures could be tested and implemented to try to reduce the problem. Further methodological research and development in this field will be welcome.

Thirty-year survey of bibliometrics used in the research literature of pain: Analysis, evolution, and pitfalls

During the last decades, the emergence of Bibliometrics and the progress in Pain research have led to a proliferation of bibliometric studies on the medical and scientific literature of pain (B/P). This study charts the evolution of the B/P literature published during the last 30 years. Using various searching techniques, 189 B/P studies published from 1993 to August 2022 were collected for analysis-half were published since 2018. Most of the selected B/P publications use classic bibliometric analysis of Pain in toto, while some focus on specific types of Pain with Headache/Migraine, Low Back Pain, Chronic Pain, and Cancer Pain dominating. Each study is characterized by the origin (geographical, economical, institutional, …) and the medical/scientific context over a specified time span to provide a detailed landscape of the Pain research literature. Some B/P studies have been developed to pinpoint difficulties in appropriately identifying the Pain literature or to highlight some general publishing pitfalls. Having observed that most of the recent B/P studies have integrated newly emergent software visualization tools (SVTs), we found an increase of anomalies and suggest that readers exercise caution when interpreting results in the B/P literature details.

Screening for in vitro systematic reviews: a comparison of screening methods and training of a machine learning classifier

OBJECTIVE

Existing strategies to identify relevant studies for systematic review may not perform equally well across research domains. We compare four approaches based on either human or automated screening of either title and abstract or full text, and report the training of a machine learning algorithm to identify in vitro studies from bibliographic records.

METHODS

We used a systematic review of oxygen-glucose deprivation (OGD) in PC-12 cells to compare approaches. For human screening, two reviewers independently screened studies based on title and abstract or full text, with disagreements reconciled by a third. For automated screening, we applied text mining to either title and abstract or full text. We trained a machine learning algorithm with decisions from 2000 randomly selected PubMed Central records enriched with a dataset of known in vitro studies.

RESULTS

Full-text approaches performed best, with human (sensitivity: 0.990, specificity: 1.000 and precision: 0.994) outperforming text mining (sensitivity: 0.972, specificity: 0.980 and precision: 0.764). For title and abstract, text mining (sensitivity: 0.890, specificity: 0.995 and precision: 0.922) outperformed human screening (sensitivity: 0.862, specificity: 0.998 and precision: 0.975). At our target sensitivity of 95% the algorithm performed with specificity of 0.850 and precision of 0.700.

CONCLUSION

In this in vitro systematic review, human screening based on title and abstract erroneously excluded 14% of relevant studies, perhaps because title and abstract provide an incomplete description of methods used. Our algorithm might be used as a first selection phase in in vitro systematic reviews to limit the extent of full text screening required.

Trends in Level of Evidence of Systematic Reviews in Sports Medicine, 2010-2020 : A Systematic Review and Meta-analysis

Background:

Popularization of systematic reviews has been met with controversy because of concerns that the primary literature for certain topics may not be suited for systematic review and meta-analysis.

Purpose:

To assess the rate of publication of systematic reviews based on their level of evidence (LOE) in influential orthopaedic sports medicine journals and commonly studied topics in sports medicine.

Study Design:

Systematic review.

Methods:

An electronic search was performed using the PubMed database of studies published from January 2010 to December 2020. The advanced search function was used to identify systematic reviews from the Journal of Shoulder and Elbow Surgery (JSES), American Journal of Sports Medicine (AJSM), Arthroscopy, British Journal of Sports Medicine (BJSM), Journal of Bone and Joint Surgery–American Volume (JBJS), and Sports Medicine (SM Auckland), as well as reviews of the most common areas of sports medicine research, including rotator cuff repair (RCR), shoulder instability (SI), anterior cruciate ligament reconstruction (ACLR), and meniscal repair. The LOE was assigned to each included study according to the Oxford Centre for Evidence-Based Medicine. Studies were grouped as LOE 1-2, LOE 3-5, and nonclinical systematic reviews. A negative binomial regression was used to determine the changes in publication rate over time.

Results:

A total of 2162 systematic reviews were included in this study. From 2010 to 2020, the rate of publication of LOE 3-5 systematic reviews increased significantly among most of the surveyed journals (AJSM, P < .0001; Arthroscopy, P = .01; BJSM, P < .0001; JSES, P < .0001; SM Auckland, P < .0001), with the exception of JBJS (P = .57). The rate of publication of LOE 1-2 systematic reviews increased in AJSM (P < .0001), Arthroscopy (P = .02), BJSM (P < .0001), and SM Auckland (P < .0001); however, no significant changes were seen in JBJS (P = .08) or JSES (P = .15). The publication rate of LOE 3-5 systematic reviews increased for all sports medicine topics surveyed (meniscal repair, P < .0001; RCR, P < .0001; SI, P < .0001; ACLR, P < .0001). However, the publication rate of LOE 1-2 studies only increased for RCR (P = .0003) and ACLR (P < .0001).

Conclusion:

The rate of publication of LOE 3-5 systematic reviews exponentially increased in orthopaedic sports medicine journals over the past decade, outpacing the publication rate of LOE 1-2 systematic reviews.

Adopting AMSTAR 2 critical appraisal tool for systematic reviews: speed of the tool uptake and barriers for its adoption

Background

In 2007, AMSTAR (A MeaSurement Tool to Assess systematic Reviews), a critical appraisal tool for systematic reviews (SRs), was published, and it has since become one of the most widely used instruments for SR appraisal. In September 2017, AMSTAR 2 was published as an updated version of the tool. This mixed-methods study aimed to analyze the extent of the AMSTAR 2 uptake and explore potential barriers to its uptake.

Methods

We analyzed the frequency of AMSTAR or AMSTAR 2 use in articles published in 2018, 2019 and 2020. We surveyed authors who have used AMSTAR but not AMSTAR 2 in the analyzed time frame to identify their reasons and barriers. The inclusion criterion for those authors was that the month of manuscript submission was after September 2017, i.e. after AMSTAR 2 was published.

Results

We included 871 studies. The majority (N = 451; 52%) used AMSTAR 2, while 44% (N = 382) used AMSTAR, 4% (N = 31) used R-AMSTAR and others used a combination of tools. In 2018, 81% of the analyzed studies used AMSTAR, while 16% used AMSTAR 2. In 2019, 52% used AMSTAR, while 44% used AMSTAR 2. Among articles published in 2020, 28% used AMSTAR, while AMSTAR 2 was used by 69%.

An author survey indicated that the authors did not use AMSTAR 2 mostly because they were not aware of it, their protocol was already established, or data collection completed at the time when the new tool was published. Barriers towards AMSTAR 2 use were lack of quantitative assessment, insufficient awareness, length, difficulties with a specific item.

Conclusion

In articles published in 2018-2020, that were submitted to a journal after AMSTAR 2 tool was published, almost half of the authors (44%) still used AMSTAR, the old version of the tool. However, the use of AMSTAR has been declining in each subsequent year. Our survey indicated that editors and peer-reviewers did not ask the authors to use the new version of the tool. Few barriers towards using AMSTAR 2 were identified, and thus it is anticipated that the use of the old version of AMSTAR will continue to decline.

Traduction franco-canadienne de l’Assessment of Systematic Reviews Revised (AMSTAR 2) : validation transculturelle et fidélité interjuges

Objective: Produce a French-Canadian translation of AMSTAR 2, affirm its content validity, and examine interrater reliability. Methods: Based on Vallerand's methodological approach, we conducted forward and parallel inverse-translations. Subsequently, an expert panel evaluated the translations to create a preliminary experimental French-Canadian version. A second expert panel examined this version and proposed additional modifications. Twenty future health professionals then rated the second experimental version for ambiguity on a scale (from 1 to 7). The principal co-investigators then reviewed the problematic elements and proposed a pre-official version. To ascertain content validity, a final back-translation was conducted resulting in the official version. Four judges evaluated 13 systematic reviews using the official French-Canadian version of AMSTAR 2. The Kappa coefficient was used to evaluate interrater reliability. Results: This rigorous adaptation enabled the development of a Franco-Canadian version of AMSTAR 2. Its application demonstrated low ambiguity (mean 1.15; SD 0.26) as well as good overall interrater reliability (total κ > 0.64) across all items. Conclusion: The French-Canadian version of AMSTAR 2 can now support francophone clinicians, educators, and managers in Canada as they undertake evidence-based practice.

Self-Efficacy of Older People Using Technology to Self-Manage COPD, Hypertension, Heart Failure, or Dementia at Home: An Overview of Systematic Reviews

BACKGROUND AND OBJECTIVES

Although telehealth research among the general population is voluminous, the quality of studies is low and results are mixed. Little is known specifically concerning older people and their self-efficacy to engage with and benefit from such technologies. This article reviews the evidence for which self-care telehealth technology supports the self-efficacy of older people with long-term conditions (LTCs) living at home.

RESEARCH DESIGN AND METHODS

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses Statement (PRISMA) guidelines, this overview of systematic reviews focused on four LTCs and the concept of "self-efficacy." Quality was appraised using R-AMSTAR and study evaluation was guided by the PRISMS taxonomy for reporting of self-management support. Heterogeneous data evidencing technology-enhanced self-efficacy were narratively synthesized.

RESULTS

Five included articles contained 74 primary studies involving 9,004 participants with chronic obstructive pulmonary disease, hypertension, heart failure, or dementia. Evidence for self-care telehealth technology supporting the self-efficacy of older people with LTCs living at home was limited. Self-efficacy was rarely an outcome, also attrition and dropout rates and mediators of support or education. The pathway from telehealth to self-efficacy depended on telehealth modes and techniques promoting healthy lifestyles. Increased self-care and self-monitoring empowered self-efficacy, patient activation, or mastery.

DISCUSSION AND IMPLICATIONS

Future research needs to focus on the process by which the intervention works and the effects of mediating variables and mechanisms through which self-management is achieved. Self-efficacy, patient activation, and motivation are critical components to telehealth's adoption by the patient and hence to the success of self-care in self-management of LTCs. Their invisibility as outcomes is a limitation.

Performance of conceptual framework elements for the retrieval of qualitative health literature: a case study

Objective

A growing volume of studies address methods for performing systematic reviews of qualitative studies. One such methodological aspect is the conceptual framework used to structure the review question and plan the search strategy for locating relevant studies. The purpose of this case study was to evaluate the retrieval potential of each element of conceptual frameworks in qualitative systematic reviews in the health sciences.

Methods

The presence of elements from conceptual frameworks in publication titles, abstracts, and controlled vocabulary in CINAHL and PubMed was analyzed using a set of qualitative reviews and their included studies as a gold standard. Using a sample of 101 publications, we determined whether particular publications could be retrieved if a specific element from the conceptual framework was used in the search strategy.

Results

We found that the relative recall of conceptual framework elements varied considerably, with higher recall for patient/population (99%) and research type (97%) and lower recall for intervention/phenomenon of interest (74%), outcome (79%), and context (61%).

Conclusion

The use of patient/population and research type elements had high relative recall for qualitative studies. However, other elements should be used with great care due to lower relative recall.

Efficacy of reminders for increasing volunteer engagement in translating Cochrane plain language summaries: a pilot randomised controlled trial

OBJECTIVES

The aim of this study was to pilot test the effectiveness of reminders versus no intervention for increasing the number of translated Cochrane plain language summaries (PLSs) among volunteer translators.

STUDY DESIGN

Parallel-group randomised controlled trial.

SETTING

Cochrane Croatia translation project.

PARTICIPANTS

Adults who volunteered to translate Cochrane PLSs within the Cochrane Croatia translation project.

INTERVENTION

The participants were randomly allocated to intervention (receiving up to four bi-weekly email reminders to translate PLSs) or control group (no intervention).

PRIMARY OUTCOME

The number of translated PLSs within the 6-month trial period.

RESULTS

We included 80 participants. The median number of translated PLSs after 6 months was 9 in the intervention group (95% CI 2.0 to 15.0) and 4 in the control group (95% CI 2.9 to 7.0), but this was not significantly different (p=0.181, Mann-Whitney U test). There was no difference between the groups in the number of translations after 3 months, the average time-to-translation after 3 or 6 months, or the satisfaction at the end of the study period. The number of reminders received and the number of translated summaries were negatively correlated (r=-0.50; 95% CI -0.70 to -0.22).

CONCLUSIONS

Our pilot trial showed that reminders do not seem to be significantly effective in increasing the number of PLS translations. Future studies could explore whether different frequency, timing and content of reminders have an influence on an increase in the engagement among volunteer translators of evidence synthesis.

TRIAL REGISTRATION

NCT03534791.

Appraising the quality of meta-analysis for breast cancer treatment in the adjuvant setting: A systematic review

INTRODUCTION

Breast cancer is the tumor with highest incidence in women worldwide and adjuvant treatment is extremely important to achieve disease control. Given the relevance of systematic reviews, their rigor should be warranted to avoid biased conclusions. Our objective was to investigate the methodological quality of meta-analysis of early breast cancer adjuvant treatment.

MATERIAL AND METHODS

Comprehensive searches were performed using electronic databases from 1/1/2007 to 11/12/2018. All studies identified as a systematic review with meta-analysis investigating the efficacy of breast cancer adjuvant treatments were included. Two reviewers independently assessed titles and abstracts, then full-texts for eligibility. Quality was assessed using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) version 2 tool.

RESULTS

Of 950 citations retrieved, 66 studies (7.0%) were deemed eligible. Methodological quality was highly variable, median AMSTAR score 8.5 (IQR 7-9.5) and range 0-16. There was a weak positive correlation between journal impact factor and AMSTAR score (r = 0.17) and citation rate and AMSTAR score (r = 0.16). Cochrane Systematic Reviews were of higher quality than reviews from other journals. Overall confidence was critically low for 61 (92.4%) studies, and the least well-reported domains were the statement of conflict of interest and funding source for the included studies (4.6%), the report of a pre-defined study protocol (15.2%), and the description of details of excluded studies (6.1%).

CONCLUSIONS

Our findings reinforce concerns about the design, conduction and interpretation of meta-analysis in current literature. Methodological quality should be carefully considered and journal editors, decision makers and readers in general, must follow a critical approach to this studies.

Reasons and factors associated with inconclusiveness of systematic reviews about interventions for neuropathic pain

Aim: Systematic reviews (SRs) are frequently inconclusive. The aim of this study was to analyze factors associated with conclusiveness of SRs about efficacy and safety of interventions for neuropathic pain (NeuP). Materials & methods: The study protocol was registered in the PROSPERO database (No. CRD42015025831). Five electronic databases (Medical Literature Analysis and Retrieval System Online, Cochrane Database of Systematic Reviews, Cumulative Index for Nursing and Allied Health Literature, Database of Abstracts of Reviews of Effects and Psychological Information Database) were searched until July 2018 for SRs about NeuP management. Conclusion statements for efficacy and safety, and characteristics of SRs were analyzed. Conclusiveness was defined as explicit statement by the SR authors that one intervention is better/similar to the other in terms of efficacy and safety. Methodological quality of SRs was assessed with the AMSTAR (A MeaSurement Tool to Assess systematic Reviews) tool. Results: Of 160 SRs, 37 (23%) were conclusive for efficacy and/or safety. In the SRs, conclusions about safety were missing in half of the analyzed abstracts, and a third of the full texts. Conclusive SRs included significantly more trials and participants, searched more databases, had more authors, conducted meta-analysis, analyzed quality of evidence, and had lower methodological quality than inconclusive SRs. The most common reasons for the lack of conclusiveness indicated by the SR authors were the small number of participants and trials, and the high heterogeneity of included studies. Conclusion: Most SRs about NeuP treatment were inconclusive. Sources of inconclusiveness of NeuP reviews need to be further studied, and SR authors need to provide conclusions about both safety and efficacy of interventions.

Systematic reviews evaluating the effectiveness of motor control exercises in patients with non-specific low back pain do not consider its principles - A review

BACKGROUND

Motor control exercise (MCEs), spinal stabilization or core stability exercises were developed with the aim of restoring the coordination, control, and capacity of the trunk muscles and systematic reviews (SR) evaluating their effectiveness has shown conflicting results. This we hypothesized was due to the non-consideration of principles of neuroplasticity. The objective of this review was to review the operating definitions used in these reviews for these exercises and evaluate if these reviews have considered and satisfied the principles of these exercises in persons with NSLBP, both acute and chronic.

METHODS

The available evidence to address the research question was sought in the reviews published in English between January 2006 and April 2019 using the population, intervention, comparison, and outcome format. Data were extracted against the following factors: satisfy the principles of specific inclusion criteria; interventions; experience-dependent plasticity; and measure any one of the concepts of motor control. The quality of the evidence obtained was graded using the National Institute for Health and Care Excellence protocol and the quality of SRs evaluated using the R-AMSTAR.

RESULTS

Eleven reviews on core stability/spinal stabilization exercises and four reviews on MCE were considered in this review. The results showed that most of the studies considered by the reviews did not adhere to the principles of these exercises.

CONCLUSION

There is wide heterogeneity in the understanding, administration, and progression of exercises. The exercises were implemented without considering the potential for neuroplasticity of the nervous system and the principles of motor learning.

Clinical performance of zirconia implants: A meta-review

STATEMENT OF PROBLEM

The clinical effectiveness of zirconia implants as an alternative to titanium implants is still controversial.

PURPOSE

The purpose of this analysis was to identify and evaluate systematic reviews reporting on the clinical outcomes of zirconia implants for oral rehabilitation.

MATERIAL AND METHODS

An electronic search was undertaken on MEDLINE, Embase, and the Cochrane Oral Health Reviews databases up to December 24, 2018, without language restriction. Eligible reviews were screened and assessed. The eligibility criteria were systematic reviews or meta-analyses, implant survival rate, implant success, marginal bone loss, peri-implant soft tissue status, and biologic and functional complications of zirconia implants. Two review authors independently evaluated the quality assessment of the secondary studies by applying the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) tool.

RESULTS

Nine reviews fulfilled the inclusion criteria and were evaluated. Seven reviews were classified as moderate and 2 as high quality. The overall AMSTAR's quality of these reports was moderate. In the primary studies contained in these reviews, zirconia implant clinical outcomes were found to be similar or inferior to those for titanium implants. The few primary clinical studies contained in these reviews were not homogeneous among each other, presented poor methodology, and only offered promising short-term outcomes due to the lack of long-term follow-ups.

CONCLUSIONS

Based on this meta-review, in spite of short-term promising results of zirconia implants, evidence with long term is lacking.

Methodological steps used by authors of systematic reviews and meta-analyses of clinical trials: a cross-sectional study

Background

The quality of systematic reviews and meta-analyses (SR/MAs) depends on the extent of the methods used. We investigated the methodological steps used by authors of SR/MAs of clinical trials via an author survey.

Methods

We conducted an email-based cross-sectional study by contacting corresponding authors of SR/MAs that were published in 2015 and 2016 and retrieved through the PubMed database. The 27-item questionnaire was developed to study the methodological steps used by authors when conducting a SR/MA and the demographic characteristics of the respondent. Besides the demographic characteristics, methodological questions regarding the source, extraction and synthesis of data were included.

Results

From 10,292 emails sent, 384 authors responded and were included in the final analysis. Manual searches were carried out by 69.2% of authors, while 87.3% do updated searches, 49.2% search grey literature, 74.9% use the Cochrane tool for risk of bias assessment, 69.8% assign more than two reviewers for data extraction, 20.5% use digital software to extract data from graphs, 57.9% use raw data in the meta-analysis, and 43.8% meta-analyze both adjusted and non-adjusted data. There was a positive correlation of years of experience in conducting of SR/MAs with both searching grey literature (P = 0.0003) and use of adjusted and non-adjusted data (P = 0.006).

Conclusions

Many authors still do not carry out many of the vital methodological steps to be taken when performing any SR/MA. The experience of the authors in SR/MAs is highly correlated with use of the recommended tips for SR/MA conduct. The optimal methodological approach for researchers conducting a SR/MA should be standardized.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0780-2) contains supplementary material, which is available to authorized users.

Capacity building for knowledge translation: A survey about the characteristics and motivation of volunteer translators of Cochrane plain language summaries

BACKGROUND

Cochrane systematic reviews have plain language summaries (PLSs) that are translated into multiple languages, mostly by volunteers. We aimed to survey volunteer translators to find out their characteristics, motivation, and suggestions to further improve this translation project.

METHODS

We surveyed 176 registered volunteer translators from the Cochrane Croatia PLS translation project. A 28-item survey, created for the purpose of this study, was administered via SurveyMonkey in November-December 2017. Translators received an invitation to the survey and two follow-up reminders via e-mail. Primary and secondary outcomes were the characteristics and motivation of volunteers.

RESULTS

We received 106 responses (60% response rate) to the survey. The translators were on average (standard deviation) 32 ± 10 years old, they were mostly women (74%), and most of them were medical doctors (29%) or pharmacists (13%). The majority found out about the translation project from acquaintances and colleagues. Most of them indicated that they have still had high motivation for the project, but for the majority, the number of translations decreased over time. When asked what could motivate them to translate more Cochrane PLSs, or to start translating if they did not translate anything yet, the most common answers were: feedback about translation quality, reminders, a workshop for translators and Facebook group for translators.

CONCLUSION

Cochrane volunteer PLS translators are a motivated and highly educated group, but their translation output is decreasing with time. Reminders, feedback and education are interventions that should be tested to increase volunteer engagement in such initiatives.

Characteristics, trend, and methodological quality of systematic reviews and meta-analyses in nuclear medicine: A bibliometric analysis of studies published between 2005 and 2016

To evaluate the characteristics, trend, and quality of systematic reviews and meta-analyses in nuclear medicine.We performed a PubMed search to identify systematic reviews and meta-analyses published between 2005 and 2016 in the field of nuclear medicine. The following data were extracted: journal name, impact factor, type of study, topics with cancer type, imaging modalities, authors (number, country, affiliation, presence of nuclear medicine specialists and statisticians, discordance between the first and corresponding authors), funding, methodological quality, methods used for quality assessment, and statistical methods.We included 185 nuclear medicine articles. Meta-analyses (n = 164; 88.6%) were published about 7 times more frequently than systematic reviews. Oncology was the most commonly studied topic (n = 125, 67.6%). The first authors were most frequently located in China (n = 73; 39.5%). PET was the most commonly used modality (n = 150; 81.1%). Both the number of authors and the ratio of discordance between the first and corresponding authors tended to progressively increase over time.The mean AMSTAR score increased over time (5.77 in 2005-2008, 6.71 in 2009-2012, and 7.44 in 2013-2016). The proportion of articles with quality assessment increased significantly (20/26 in 2005-2008, 54/65 in 2009-2012, and 79/94 in 2013-2016). The most commonly used assessment tool was quality assessment of diagnostic accuracy studies (n = 85; 54.9%).The number and quality of systematic reviews and meta-analyses in nuclear medicine have significantly increased over the review period; however, the quality of these articles varies. Efforts to overcome specific weaknesses of the methodologies can provide opportunities for quality improvement.