Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals

A practical guide to living evidence: reducing the knowledge-to-practice gap

Living evidence involves continuous evidence surveillance to incorporate new relevant evidence into systematic reviews and clinical practice guideline recommendations as soon as it becomes available. Thus, living evidence may improve the timeliness of recommendation updates and reduce the knowledge-to-practice gap. When considering a living evidence model, several processes and practical aspects need to be explored. Some of these include identifying the need for a living evidence model, funding, governance structure, time, team skills and capabilities, frequency of updates, approval and endorsement, and publication and dissemination.

Cochrane systematic reviews in orthodontics: trends across updates

BACKGROUND

Systematic reviews (SR) are regularly updated to reflect new evidence. However, updates are time-consuming and costly, and therefore should ideally be informed by new high-quality research. The purpose of this study is to assess trends in the quantity, quality, and recency of evidence intervening updates of orthodontic SR.

METHODS

SR relevant to orthodontics with at least two versions were identified from the Cochrane Database. The number, risk of bias, and year of publication of included trials were recorded for each update. Multivariate regression was conducted to assess factors affecting the risk of bias in trials, and the proportions within SR.

RESULTS

Forty-five SR inclusive of updates were included. The median number of trials was three per review and this increased across subsequent versions. Seven reviews (15.6%) included no evidence, and 40.74% of updates included no new evidence. Most of the primary research was considered high risk of bias (57.3%), although this was reduced marginally across updates. The proportion of studies considered low risk did not improve significantly between updates. There was no impact of publication year of clinical trials on the risk of bias (P = 0.349). However, average age of trials included in a systematic review significantly affected the proportion of low risk-of-bias studies (P = 0.039).

CONCLUSIONS

SR are frequently updated without including new evidence. New evidence that is included is commonly deemed to be at high risk of bias. Targeted strategies to improve the efficient use of resources and improve research quality should be considered.

Where is the exit? The ethical importance of exit plans in clinical trials with neural implants

BACKGROUND

As clinical trials involving implantable neural devices (INDs) increase in frequency and attract greater public attention, it is paramount to ensure they are conducted in alignment with fundamental ethical guidelines. Particular focus must be placed on the often underexplored aspect of trial termination for INDs.

OBJECTIVE

To systematically review the ethical challenges encountered in clinical trials for INDs at the juncture of trial termination.

METHODS

We conducted a rapid review using PubMed with two specific search queries, including all publications addressing ethical issues in the context of IND clinical trials. Priority was given to publications focusing on the end of treatment or the discontinuation of clinical studies or trials.

RESULTS

We identified three primary groups of ethical challenges: patient-centric challenges, challenges faced by the research and physician team, and manufacturer-related issues. Further analysis highlights the importance of initiating early, transparent discussions regarding trial cessation protocols, ensuring that all stakeholders-patients, healthcare providers, researchers, and manufacturers-are equitably considered. Additionally, we found a discrepancy between current discontinuation strategies and international ethical guidelines. To address this, we emphasize the ethical obligation to establish comprehensive exit strategies that align with the principles in the Declaration of Helsinki and the CIOMS/WHO guidelines.

CONCLUSION

Our findings highlight the need for increased attention to the ethical and practical aspects of exit strategies and encourage further empirical research to address gaps in current practices. This would ensure that the discontinuation of IND trials is handled with ethical rigor, prioritizing the interests and well-being of all stakeholders involved.

Evaluation of traditional Chinese exercise for knee osteoarthritis (KOA): an overview of systematic reviews

BACKGROUND

Knee osteoarthritis (KOA) has become a public health issue. Several systematic reviews (SRs) and meta-analyses (MAs) indicate that traditional Chinese exercise (TCE) may be an effective treatment for reducing pain and stiffness and improving physical function in people with knee osteoarthritis (KOA).

OBJECTIVES

To evaluate the literature quality and evidence for the systematic reviews of TCE for KOA and provide evidence to support the clinical application of TCE for KOA.

METHODS

Eight databases were searched from their inception to January 3, 2023, to retrieve relevant literature, including China National Knowledge Infrastructure (CNKI), Wanfang, Chinese Scientific Journal Database (VIP), China Biology Medical literature database (CBM), PubMed, Embase, Web of Science and Cochrane Library, without restrictions on publication date or language. AMSTAR-2 and PRISMA 2020 assessed the methodological and reporting quality of included SRs/MAs. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system was utilized to evaluate the quality of evidence.

RESULTS

A total of 18 SRs/MAs were included. The methodological quality was "very low" based on AMSTAR-2. The overall reporting quality was deficient based on PRISMA 2020. The quality of Chinese and English literature differed, with English literature being superior in methodological and reporting quality. Among 93 pieces of evidence obtained, 46 (49.46%) were of very low quality, 34 (36.56%) were of low quality, 13 (13.98%) were of moderate quality, and none were of high quality. TCE was supported by 76 pieces of evidence (81.72%).

CONCLUSION

TCE appears beneficial and safe for managing KOA. However, due to the relatively low methodological and evidentiary quality of included SRs/MAs, clinicians should interpret these findings cautiously.

Are Newer Drugs Better? An Analysis of Neonatal Pharmacological Treatments across Generations

INTRODUCTION

We evaluated the relative effects of newer versus older medications for neonatal conditions and trends in margin of superiority across generations.

MATERIALS AND METHODS

We assessed network meta-analyses (NMAs) on neonatal pharmacological interventions identified from MEDLINE, Cochrane, and PROSPERO. Interventions were chronologically arranged based on the earliest study and compared for their effects against placebo or no treatment and their immediate predecessor. We assessed the time trend in effect sizes using the Mann-Kendall test.

RESULTS

From 8,048 retrieved records, 10 neonatal NMAs covering 352 trials and 102,653 participants were included. Compared to placebo, 56/61 (91.8%) interventions showed superiority with 23 (37.7%) statistically significant. Compared to previous generation, 47/72 (65.3%) showed superiority with 3 (4.2%) statistically significant. No significant trends in effect sizes were observed across generations for most conditions (p = 0.09-1).

CONCLUSIONS

We found no evidence that newer generation medications in neonatal care are consistently more effective than older generation medications.

Protocol for the development of a 'trustworthy' living systematic review and meta analyses of manual therapy interventions to treat neuromusculoskeletal impairments

BACKGROUND

Preprocessed research resources are believed to be highly 'trustworthy' when translating research to clinical practice. However, the overall 'trustworthiness' is unknown if this evidence contains randomized clinical trials (RCTs) where prospective has not been/cannot be verified, has low confidence in estimated effects, and if they are not up to date.

OBJECTIVES

This protocol will be used to create a baseline benchmark for a series of trustworthy living systematic reviews (SRs) regarding manual therapy interventions.

METHODS

Data will originate from RCTs related to manual therapy neuromusculoskeletal interventions, indexed in 6 search engines in English from 1 January 2010, to the present. Two blinded reviewers will identify the RCTs and extract data using Covidence. The data will be synthesized based on consensus and analyzed using the Cochrane collaboration's Review Manager.

EXPECTED OUTCOMES

It is expected that there will be a shortage of RCTs with at least a moderate confidence in estimated effects that will allow for strong practice recommendations.

DISCUSSION

Identifying evidence that can be translated into strong practice recommendations is essential to identify beneficial and harmful interventions, decrease practice variability, and identify neuromusculoskeletal manual therapy interventions that require further disciplined methodological focus.

Methodological quality and reporting quality of COVID-19 living systematic review: a cross-sectional study

OBJECTIVES

The main objective of this study is to evaluate the methodological quality and reporting quality of living systematic reviews (LSRs) on Coronavirus disease 2019 (COVID-19), while the secondary objective is to investigate potential factors that may influence the overall quality of COVID-19 LSRs.

METHODS

Six representative databases, including Medline, Excerpta Medica Database (Embase), Cochrane Library, China national knowledge infrastructure (CNKI), Wanfang Database, and China Science, Technology Journal Database (VIP) were systematically searched for COVID-19 LSRs. Two authors independently screened articles, extracted data, and then assessed the methodological and reporting quality of COVID-19 LSRs using the "A Measurement Tool to Assess systematic Reviews-2" (AMSTAR-2) tool and "Preferred Reporting Items for Systematic reviews and Meta-Analyses" (PRISMA) 2020 statement, respectively. Univariate linear regression and multivariate linear regression were used to explore eight potential factors that might affect the methodological quality and reporting quality of COVID-19 LSRs.

RESULTS

A total of 64 COVID-19 LSRs were included. The AMSTAR-2 evaluation results revealed that the number of "yes" responses for each COVID-19 LSR was 13 ± 2.68 (mean ± standard deviation). Among them, 21.9% COVID-19 LSRs were rated as "high", 4.7% as "moderate", 23.4% as "low", and 50% as "critically low". The evaluation results of the PRISMA 2020 statement showed that the sections with poor adherence were methods, results and other information. The number of "yes" responses for each COVID-19 LSR was 21 ± 4.18 (mean ± standard deviation). The number of included studies and registration are associated with better methodological quality; the number of included studies and funding are associated with better reporting quality.

CONCLUSIONS

Improvement is needed in the methodological and reporting quality of COVID-19 LSRs. Researchers conducting COVID-19 LSRs should take note of the quality-related factors identified in this study to generate evidence-based evidence of higher quality.

TFOS Lifestyle - Evidence quality report: Advancing the evaluation and synthesis of research evidence

Evidence-based practice is a dominant paradigm in healthcare that emphasizes the importance of ensuring the translation of the best available, relevant research evidence into practice. An Evidence Quality Subcommittee was established to provide specialized methodological support and expertise to promote rigorous and evidence-based approaches for the Tear Film and Ocular Surface Society (TFOS) Lifestyle Epidemic reports. The present report describes the purpose, scope, and activity of the Evidence Quality Subcommittee in the undertaking of high-quality narrative-style literature reviews, and leading prospectively registered, reliable systematic reviews of high priority research questions, using standardized methods for each topic area report. Identification of predominantly low or very low certainty evidence across the eight systematic reviews highlights a need for further research to define the efficacy and/or safety of specific lifestyle interventions on the ocular surface, and to clarify relationships between certain lifestyle factors and ocular surface disease. To support the citation of reliable systematic review evidence in the narrative review sections of each report, the Evidence Quality Subcommittee curated topic-specific systematic review databases and relevant systematic reviews underwent standardized reliability assessment. Inconsistent methodological rigor was noted in the published systematic review literature, emphasizing the importance of internal validity assessment. Based on the experience of implementing the Evidence Quality Subcommittee, this report makes suggestions for incorporation of such initiatives in future international taskforces and working groups. Content areas broadly relevant to the activity of the Evidence Quality Subcommittee, including the critical appraisal of research, clinical evidence hierarchies (levels of evidence), and risk of bias assessment, are also outlined.

Outcomes of Remote Patient Monitoring in Peritoneal Dialysis: A Meta-Analysis and Review of Practical Implications for COVID-19 Epidemics

The present study seeks to determine clinical outcomes associated with remote patient monitoring of peritoneal dialysis (RPM-PD), with potential implications during COVID-19 outbreaks. We performed a systematic review in the PubMed, Embase, and Cochrane databases. We combined all study-specific estimates using the inverse-variant weighted averages of logarithmic relative risk (RR) in the random-effects models. Confidence interval (CI) including the value of 1 was used as evidence to produce a statistically significant estimate. Twenty-two studies were included in our meta-analysis. Quantitative analysis demonstrated that RPM-PD patients had lower rates of technique failure (log RR = -0.32; 95% CI, -0.59 to -0.04), lower hospitalization rates (standardized mean difference = -0.84; 95% CI, -1.24 to -0.45), and lower mortality rates (log RR = -0.26; 95% CI, -0.44 to -0.08) compared with traditional PD monitoring. RPM-PD has better outcomes in multiple spheres of outcomes when compared with conventional monitoring and likely increases system resilience during disruptions of healthcare operations.

Recommendations for Improving the Quality of Prehospital Evidence-Based Guidelines

Evidence-based guidelines that provide recommendations for clinical care or operations are increasingly being published to inform the EMS community. The quality of evidence evaluation and methodological rigor undertaken to develop and publish these recommendations vary. This can negatively affect dissemination, education, and implementation efforts. Guideline developers and end users could be better informed by efforts across medical specialties to improve the quality of guidelines, including the use of specific criteria that have been identified within the highest quality guidelines. In this special contribution, we aim to describe the current state of published guidelines available to the EMS community informed by two recent systematic reviews of existing prehospital evidenced based guidelines (EBGs). We further aim to provide a description of key elements of EBGs, methods that can be used to assess their quality, and concrete recommendations for guideline developers to improve the quality of evidence evaluation, guideline development, and reporting. Finally, we outline six key recommendations for improving prehospital EBGs, informed by systematic reviews of prehospital guidelines performed by the Prehospital Guidelines Consortium.

Screening for in vitro systematic reviews: a comparison of screening methods and training of a machine learning classifier

OBJECTIVE

Existing strategies to identify relevant studies for systematic review may not perform equally well across research domains. We compare four approaches based on either human or automated screening of either title and abstract or full text, and report the training of a machine learning algorithm to identify in vitro studies from bibliographic records.

METHODS

We used a systematic review of oxygen-glucose deprivation (OGD) in PC-12 cells to compare approaches. For human screening, two reviewers independently screened studies based on title and abstract or full text, with disagreements reconciled by a third. For automated screening, we applied text mining to either title and abstract or full text. We trained a machine learning algorithm with decisions from 2000 randomly selected PubMed Central records enriched with a dataset of known in vitro studies.

RESULTS

Full-text approaches performed best, with human (sensitivity: 0.990, specificity: 1.000 and precision: 0.994) outperforming text mining (sensitivity: 0.972, specificity: 0.980 and precision: 0.764). For title and abstract, text mining (sensitivity: 0.890, specificity: 0.995 and precision: 0.922) outperformed human screening (sensitivity: 0.862, specificity: 0.998 and precision: 0.975). At our target sensitivity of 95% the algorithm performed with specificity of 0.850 and precision of 0.700.

CONCLUSION

In this in vitro systematic review, human screening based on title and abstract erroneously excluded 14% of relevant studies, perhaps because title and abstract provide an incomplete description of methods used. Our algorithm might be used as a first selection phase in in vitro systematic reviews to limit the extent of full text screening required.

Characteristics of Living Systematic Review for COVID-19

Purpose

The systematic review aims to analyze and summarize the characteristics of living systematic review (LSR) for coronavirus disease 2019 (COVID-19).

Methods

Six databases including Medline, Excerpta Medica (Embase), Cochrane Library, China National Knowledge Infrastructure (CNKI), Wanfang Database and China Science, and Technology Journal Database (VIP), were searched as the source of basic information and methodology of LSR. Descriptive analytical methods were used to analyze the included COVID-19 LSRs, and the study characteristics of COVID-19 LSRs were further assessed.

Results

Sixty-four COVID-19 LSRs were included. Eighty-nine point one percent of LSRs were published on Science Citation Index (SCI) journals, and 64.1% publication with an impact factor (IF) >5 and 17.2% with an IF >15 among SCI journals. The first unit of the published LSRs for COVID-19 came from 19 countries, with the largest contribution from the UK (17.2%, 11/64). Forty point six percent of LSRs for COVID-19 were related to therapeutics topic which was considered the most concerned perspective for LSRs for COVID-19. Seventy-six point six percent of LSRs focused on the general population, with less attention to children, pregnant women and the elderly. However, the LSR for COVID-19 was reported incomplete on “living” process, including 40.6% of studies without search frequency, 79.7% of studies without screening frequency, 20.3% of studies without update frequency, and 65.6% of studies without the timing or criteria of transitioning LSR out of living mode.

Conclusion

Although researchers in many countries have applied LSRs to COVID-19, most of the LSRs for COVID-19 were incomplete in reporting on the “living” process and less focused on special populations. This could reduce the confidence of health-care providers and policy makers in the results of COVID-19 LSR, thereby hindering the translation of evidence on COVID-19 LSR into clinical practice. It was necessary to explicitly enact preferred reporting items for systematic reviews and meta-analyses (PRISMA) to improve the reporting quality of LSR and support ongoing efforts of therapeutics research for special patients with COVID-19.

Accuracy and Efficiency of Machine Learning-Assisted Risk-of-Bias Assessments in "Real-World" Systematic Reviews : A Noninferiority Randomized Controlled Trial

BACKGROUND

Automation is a proposed solution for the increasing difficulty of maintaining up-to-date, high-quality health evidence. Evidence assessing the effectiveness of semiautomated data synthesis, such as risk-of-bias (RoB) assessments, is lacking.

OBJECTIVE

To determine whether RobotReviewer-assisted RoB assessments are noninferior in accuracy and efficiency to assessments conducted with human effort only.

DESIGN

Two-group, parallel, noninferiority, randomized trial. (Monash Research Office Project 11256).

SETTING

Health-focused systematic reviews using Covidence.

PARTICIPANTS

Systematic reviewers, who had not previously used RobotReviewer, completing Cochrane RoB assessments between February 2018 and May 2020.

INTERVENTION

In the intervention group, reviewers received an RoB form prepopulated by RobotReviewer; in the comparison group, reviewers received a blank form. Studies were assigned in a 1:1 ratio via simple randomization to receive RobotReviewer assistance for either Reviewer 1 or Reviewer 2. Participants were blinded to study allocation before starting work on each RoB form.

MEASUREMENTS

Co-primary outcomes were the accuracy of individual reviewer RoB assessments and the person-time required to complete individual assessments. Domain-level RoB accuracy was a secondary outcome.

RESULTS

Of the 15 recruited review teams, 7 completed the trial (145 included studies). Integration of RobotReviewer resulted in noninferior overall RoB assessment accuracy (risk difference, -0.014 [95% CI, -0.093 to 0.065]; intervention group: 88.8% accurate assessments; control group: 90.2% accurate assessments). Data were inconclusive for the person-time outcome (RobotReviewer saved 1.40 minutes [CI, -5.20 to 2.41 minutes]).

LIMITATION

Variability in user behavior and a limited number of assessable reviews led to an imprecise estimate of the time outcome.

CONCLUSION

In health-related systematic reviews, RoB assessments conducted with RobotReviewer assistance are noninferior in accuracy to those conducted without RobotReviewer assistance.

PRIMARY FUNDING SOURCE

University College London and Monash University.

Cochrane systematic reviews on traditional Chinese medicine: What matters-the quantity or quality of evidence?

BACKGROUND

Systematic reviews on traditional Chinese medicine (TCM) are constantly increasing. However, if these reviews are to be of practical value, the evidence needs to be relevant, valid, and adequately reported. Cochrane Systematic Reviews (CSRs) are considered as high-quality systematic reviews that can inform health care decision making. Our aim was to provide an overview of the scope, findings, quality and impact of CSRs on the benefits and harms associated with TCM interventions for the treatment and prevention of disease to provide new information for clinical practice and future research.

METHODS

The Cochrane Database of Systematic Reviews was searched up to May 2021, and descriptive characteristics were extracted. The correspondence between the questions asked in the CSRs and the available evidence, conclusions and certainty of findings (according to GRADE assessment), methodological quality (AMSTAR 2), and impact (Altmetric Attention Score [AAS], total citations by guideline, and total citations in Web of Science [WoS]) of CSRs were extracted. Tabular and graphical summaries of these descriptive characteristics were constructed.

RESULTS

Of 104 CSRs on TCM identified, 70 diseases belonged to 16 disease systems and contained 1642 primary studies with 157,943 participants. Interventions included Chinese herbal medicine (n = 70), acupuncture (n = 28), TCM exercises (n = 4), and moxibustion (n = 2). Among 1642 primary studies, 662 studies included an intervention group treated with at least one TCM therapy and 980 studies included a combination of therapies. Promising outcomes from the 104 CSRs were divided into endpoint outcomes (34 diseases), doctor- or patient-reported outcomes (27 diseases), and surrogate outcomes (37 diseases). Despite the presence of promising outcomes, only 5/104 CSRs drew overall positive conclusions, 42 CSRs concluded the evidence was insufficient, and 54 failed to draw firm conclusions. GRADE assessments were reported in 41.3% of the CSRs, and the ratings were mostly low or very low. Comparing the questions asked and results obtained, there was frequently a lack of information about specific outcomes. Only 16 CSRs obtained results for all outcomes listed in the methods section. According to AMSTAR 2, 51 CSRs (49.0%) were of low quality. The total number of citations in the WoS was 2135 (mean ± SD: 20.8 ± 21.2), and 38.5% of the CSRs had been cited in guidelines 95 times.

CONCLUSION

Although TCM is commonly used, evidence of its effectiveness remains largely inconclusive. Rigorous high-quality trials are needed to support the performance of high-quality reviews and to increase the evidence base. It is critical to emphasize quality over quantity in future TCM research.

Usefulness of Cochrane Reviews in Clinical Guideline Development-A Survey of 585 Recommendations

The Danish Health Authority develops clinical practice guidelines to support clinical decision-making based on the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system and prioritizes using Cochrane reviews. The objective of this study was to explore the usefulness of Cochrane reviews as a source of evidence in the development of clinical recommendations. Evidence-based recommendations in guidelines published by the Danish Health Authority between 2014 and 2021 were reviewed. For each recommendation, it was noted if and how Cochrane reviews were utilized. In total, 374 evidence-based recommendations and 211 expert consensus recommendations were published between 2014 and 2021. Of the 374 evidence-based recommendations, 106 included evidence from Cochrane reviews. In 28 recommendations, all critical and important outcomes included evidence from Cochrane reviews. In 36 recommendations, a minimum of all critical outcomes included evidence from Cochrane reviews, but not all important outcomes. In 33 recommendations, some but not all critical outcomes included evidence from Cochrane reviews. Finally, in nine recommendations, some of the important outcomes included evidence from Cochrane reviews. In almost one-third of the evidence-based recommendations, Cochrane reviews were used to inform clinical recommendations. This evaluation should inform future evaluations of Cochrane review uptake in clinical practice guidelines concerning outcomes important for clinical decision-making.

Nursing interventions to cover patients' basic needs in the intensive care context - A systematic review

AIM

To examine the content, theoretical frameworks and effectiveness of nursing interventions utilizing patient-reported outcome measures (PROMs) in the intensive care unit (ICU).

DESIGN

A systematic review and narrative synthesis following the guidelines of the preferred reporting items for systematic reviews and meta-analysis checklist.

METHODS

We searched the MEDLINE, CINAHL, PsycINFO, SweMed and Cochrane controlled trials register (CENTRAL) databases for studies evaluating interventions primarily delivered by nurses in the ICU. Two independent reviewers performed study selection, data extraction and risk of bias.

RESULTS

Twenty-two studies were included, whereas only seven studies used a theoretical framework. The interventions were heterogeneous in content, duration and choice of PROMs. Outcomes were related to covering patients' and families' basic needs, described by Henderson as essential functions of nursing. Several studies reported positive intervention effects, and nurses' communication and psychosocial care were considered essential components of nursing interventions in the ICU.

The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrations

BACKGROUND

Clinical trial registries can be used as sources of clinical evidence for systematic review synthesis and updating. Our aim was to evaluate methods for identifying clinical trial registrations that should be screened for inclusion in updates of published systematic reviews.

METHODS

A set of 4644 clinical trial registrations (ClinicalTrials.gov) included in 1089 systematic reviews (PubMed) were used to evaluate two methods (document similarity and hierarchical clustering) and representations (L2-normalised TF-IDF, Latent Dirichlet Allocation, and Doc2Vec) for ranking 163,501 completed clinical trials by relevance. Clinical trial registrations were ranked for each systematic review using seeding clinical trials, simulating how new relevant clinical trials could be automatically identified for an update. Performance was measured by the number of clinical trials that need to be screened to identify all relevant clinical trials.

RESULTS

Using the document similarity method with TF-IDF feature representation and Euclidean distance metric, all relevant clinical trials for half of the systematic reviews were identified after screening 99 trials (IQR 19 to 491). The best-performing hierarchical clustering was using Ward agglomerative clustering (with TF-IDF representation and Euclidean distance) and needed to screen 501 clinical trials (IQR 43 to 4363) to achieve the same result.

CONCLUSION

An evaluation using a large set of mined links between published systematic reviews and clinical trial registrations showed that document similarity outperformed hierarchical clustering for identifying relevant clinical trials to include in systematic review updates.

Reporting and methodological quality of systematic literature reviews evaluating the associations between e-cigarette use and cigarette smoking behaviors: a systematic quality review

Introduction

Several published systematic reviews have examined the potential associations between e-cigarette use and cigarette smoking, but their methodological and/or reporting quality have not yet been assessed. This systematic quality review followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and AMSTAR (A MeaSurement Tool to Assess systematic Reviews) 2 to evaluate the quality of systematic reviews investigating potential associations between e-cigarette use and cigarette smoking.

Materials and methods

PubMed/MEDLINE, Embase, and PsycINFO were searched from 01 January 2007 to 24 June 2020. Methodological quality was assessed using AMSTAR 2, and reporting quality was assessed using PRISMA guidelines.

Results

Of 331 potentially relevant systematic reviews, 20 met predefined inclusion criteria. Most reviews (n = 15; 75%) reported on e-cigarette use and cigarette smoking cessation, while three reported on e-cigarette use and cigarette smoking initiation (15%); and two reported on cigarette smoking initiation and cessation (10%). According to AMSTAR 2 guidelines, 18 of the 20 reviews (90%) were “critically low” in overall confidence of the results, while two were ranked “low.” Additionally, reporting quality varied across the reviews, with only 60% reporting at least half of the PRISMA items.

Discussion

Methodological limitations were identified across reviews examining potential associations between e-cigarette use and cigarette smoking behaviors, indicating that findings from these reviews should be interpreted with caution.

Conclusions

Future systematic reviews in this field should strive to adhere to AMSTAR 2 and PRISMA guidelines, to provide high quality syntheses of the available data with transparent and complete reporting.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12954-021-00570-9.

Effectiveness of acupuncture therapy for postherpetic neuralgia: an umbrella review protocol

INTRODUCTION

Several systematic reviews and meta-analysis indicate that acupuncture and related therapies may be a valuable adjunctive technique to pharmacological interventions for pain management of postherpetic neuralgia (PHN). However, the robustness of the results of these studies has not been evaluated. The aim of this proposed umbrella review is to provide more reliable evidence of the effectiveness of acupuncture therapy for PHN based on medical references for healthcare decision makers.

METHODS AND ANALYSIS

PubMed, EMBASE, The Cochrane Library, Web of Science, Chinese BioMedical Literature Database, VIP Database for Chinese Technical Periodicals, China National Knowledge Infrastructure and Wan fang Database will be used to retrieve reviews. The time of publication will be limited from inception to March 2021. Two reviewers will screen all retrieved articles independently to identify their eligibility and extract the data. The quality will be assessed independently by two trained reviewers using Assessment of Multiple Systematic Reviews-2 for methodological quality, Risk of Bias in Systematic Review for level of bias, Preferred Reporting Items for Systematic Reviews and Meta-Analysis for reporting quality and Grading of Recommendations Assessment, Development and Evaluation for the quality of evidence. Any disagreements will be settled by discussion or the involvement of a third reviewer.

ETHICS AND DISSEMINATION

The protocol of this review does not require ethical approval because the research will be based on publicly available data. The findings will be disseminated through publication in peer-reviewed international journals or presentation in academic conference.

PROSPERO REGISTRATION NUMBER

CRD42020173341.

REPORTING CHECKLIST

PRISMA-P, 2015.

Efficacy of reminders for increasing volunteer engagement in translating Cochrane plain language summaries: a pilot randomised controlled trial

OBJECTIVES

The aim of this study was to pilot test the effectiveness of reminders versus no intervention for increasing the number of translated Cochrane plain language summaries (PLSs) among volunteer translators.

STUDY DESIGN

Parallel-group randomised controlled trial.

SETTING

Cochrane Croatia translation project.

PARTICIPANTS

Adults who volunteered to translate Cochrane PLSs within the Cochrane Croatia translation project.

INTERVENTION

The participants were randomly allocated to intervention (receiving up to four bi-weekly email reminders to translate PLSs) or control group (no intervention).

PRIMARY OUTCOME

The number of translated PLSs within the 6-month trial period.

RESULTS

We included 80 participants. The median number of translated PLSs after 6 months was 9 in the intervention group (95% CI 2.0 to 15.0) and 4 in the control group (95% CI 2.9 to 7.0), but this was not significantly different (p=0.181, Mann-Whitney U test). There was no difference between the groups in the number of translations after 3 months, the average time-to-translation after 3 or 6 months, or the satisfaction at the end of the study period. The number of reminders received and the number of translated summaries were negatively correlated (r=-0.50; 95% CI -0.70 to -0.22).

CONCLUSIONS

Our pilot trial showed that reminders do not seem to be significantly effective in increasing the number of PLS translations. Future studies could explore whether different frequency, timing and content of reminders have an influence on an increase in the engagement among volunteer translators of evidence synthesis.

TRIAL REGISTRATION

NCT03534791.

Systematic Reviews and Meta-Analysis in Spine Surgery-How Good Are They in Methodological Quality? A Systematic Review

STUDY DESIGN

Systematic review.

OBJECTIVES

To assess the methodological quality of systematic reviews and meta-analyses in spine surgery over the past 2 decades.

MATERIALS AND METHODS

We conducted independent and in duplicate systematic review of the published systematic reviews and meta-analyses between 2000 and 2019 from PubMed Central and Cochrane Database pertaining to spine surgery involving surgical intervention. We searched bibliographies to identify additional relevant studies. Methodological quality was evaluated with AMSTAR score and graded with AMSTAR 2 criteria.

RESULTS

A total of 96 reviews met the eligibility criteria, with mean AMSTAR score of 7.51 (SD = 1.98). Based on AMSTAR 2 criteria, 13.5% (n = 13) and 18.7% (n = 18) of the studies had high and moderate level of confidence of results, respectively, without any critical flaws. A total of 29.1% (n = 28) of the studies had at least 1 critical flaw and 38.5% (n = 37) of the studies had more than 1 critical flaw, so that their results have low and critically low confidence, respectively. Failure to analyze the conflict of interest of authors of primary studies included in review and lack of list of excluded studies with justification were the most common critical flaw. Regression analysis demonstrated that studies with funding and studies published in recent years were significantly associated with higher methodological quality.

CONCLUSION

Despite improvement in methodological quality of systematic reviews and meta-analyses in spine surgery in current decade, a substantial proportion continue to show critical flaws. With increasing number of review articles in spine surgery, stringent measures must be taken to adhere to methodological quality by following PRISMA and AMSTAR guidelines to attain higher standards of evidence in published literature.

Evidence-Based Decision-Making 2: Systematic Reviews and Meta-Analysis

The number of studies published in the biomedical literature has dramatically increased over the last few decades. This massive proliferation of literature makes clinical medicine increasingly complex, and information from multiple studies is often needed to inform a particular clinical decision. However, available studies often vary in their design, methodological quality, and population studied, and may define the research question of interest quite differently. This can make it challenging to synthesize the conclusions of multiple studies. In addition, since even highly cited trials may be challenged over time, clinical decision-making requires ongoing reconciliation of studies which provide different answers to the same question. Because it is often impractical for readers to track down and review all the primary studies, systematic reviews and meta-analyses are an important source of evidence on the diagnosis, prognosis and treatment of any given disease. This chapter summarizes methods for conducting and reading systematic reviews and meta-analyses, as well as describes potential advantages and disadvantages of these publications.

Reading, Conducting, and Developing Systematic Review and Individual Patient Data Meta-Analyses in Psychiatry for Treatment Issues

Introduction: Individual participant data meta-analyses (IPD-MAs) include the raw data from relevant randomised clinical trials (RCTs) and involve secondary analyses of the data. Performed since the late 1990s, ~50 such meta-analyses have been carried out in psychiatry, mostly in the field of treatment. IPD-MAs are particularly relevant for three objectives: (1) evaluation of the average effect of an intervention by combining effects from all included trials, (2) evaluation of the heterogeneity of an intervention effect and sub-group analyses to approach personalised psychiatry, (3) mediation analysis or surrogacy evaluation to replace a clinical (final) endpoint for the evaluation of new treatments with intermediate or surrogate endpoints. The objective is to describe the interest and the steps of an IPD-MA method applied to the field of psychiatric therapeutic research. Method: The method is described in three steps. First, the identification of the relevant trials with an explicit description of the inclusion/exclusion criteria for the RCT to be incorporated in the IPD-MA and a definition of the intervention, the population, the context and the relevant points (outcomes or moderators). Second, the data management with the standardisation of collected variables and the evaluation and the assessment of the risk-of-bias for each included trial and of the global risk. Third, the statistical analyses and their interpretations, depending on the objective of the meta-analysis. All steps are illustrated with examples in psychiatry for treatment issues, excluding study protocols. Conclusion: The meta-analysis of individual patient data is challenging. Only strong collaborations between all stakeholders can make such a process efficient. An "ecosystem" that includes all stakeholders (questions of interest prioritised by the community, funders, trialists, journal editors, institutions, …) is required. International medical societies can play a central role in favouring the emergence of such communities.

Can statistical adjustment guided by causal inference improve the accuracy of effect estimation? A simulation and empirical research based on meta-analyses of case-control studies

BACKGROUND

Statistical adjustment is often considered to control confounding bias in observational studies, especially case-control studies. However, different adjustment strategies may affect the estimation of odds ratios (ORs), and in turn affect the results of their pooled analyses. Our study is aimed to investigate how to deal with the statistical adjustment in case-control studies to improve the validity of meta-analyses.

METHODS

Three types of adjustment strategies were evaluated including insufficient adjustment (not all preset confounders were adjusted), full adjustment (all confounders were adjusted under the guidance of causal inference), and improper adjustment (covariates other than confounders were adjusted). We carried out a series of Monte Carlo simulation experiments based on predesigned scenarios, and assessed the accuracy of effect estimations from meta-analyses of case-control studies by combining ORs calculated according to different adjustment strategies. Then we used the data from an empirical review to illustrate the replicability of the simulation results.

RESULTS

For all scenarios with different strength of causal relations, combining ORs that were comprehensively adjusted for confounders would get the most precise effect estimation. By contrast, combining ORs that were not sufficiently adjusted for confounders or improperly adjusted for mediators or colliders would easily introduce bias in causal interpretation, especially when the true effect of exposure on outcome was weak or none. The findings of the simulation experiments were further verified by the empirical research.

CONCLUSIONS

Statistical adjustment guided by causal inference are recommended for effect estimation. Therefore, when conducting meta-analyses of case-control studies, the causal relationship formulated by exposure, outcome, and covariates should be firstly understood through a directed acyclic graph, and then reasonable original ORs could be extracted and combined by suitable methods.

Acupuncture for the Treatment of Alzheimer's Disease: An Overview of Systematic Reviews

Background: Acupuncture may be an effective complementary treatment for Alzheimer's disease (AD). The aim of this study was to summarize the evidence provided by systematic reviews (SRs)/meta-analyses (MAs) on the effect of acupuncture on AD. Methods: Eight electronic databases were searched from their inception until October 19, 2020. The methodological quality, reporting quality, and risk of bias of the included SRs were assessed by the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), the Risk of Bias in Systematic Reviews (ROBIS) tool, and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). Moreover, the evidence quality of the outcome measures was assessed by the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE). Results: Eleven SRs/MAs met all inclusion criteria. According to the results of the AMSTAR-2, all included reviews were rated critically as being of low quality. With PRISMA, the reporting checklist was relatively complete, but some reporting weaknesses remained in the topics of the protocol and registration, search strategy, risk of bias, additional analyses, and funding. Based on the ROBIS tool, only two SRs/MAs had a low risk of bias. With the GRADE system, no high-quality evidence was found, and only seven outcomes provided moderate-quality evidence. Among the downgraded factors, the risk of bias within the original trials was ranked first, followed by inconsistency, imprecision, and publication bias. Conclusions: Acupuncture is a promising complementary treatment for AD. However, due to the low quality of the SRs/MAs supporting these results, high-quality studies with rigorous study designs and larger samples are needed before widespread recommendations can be made.

Quality of evidence of the efficacy of therapeutic interventions on patient-important outcomes in Cochrane's systematic reviews' abstracts: A survey

OBJECTIVE

The objective of this study was to evaluate the proportion of therapeutics that have proven their efficacy on patient-important outcomes with a high quality of evidence among Cochrane systematic reviews (SRs).

METHODS

We surveyed a random sample of 400 SRs' abstracts published between September 2012 and December 2015, which compared therapeutic interventions with at least a placebo or no intervention control. The primary endpoint was the proportion of SRs with a statistically significant efficacy on a patient-important outcome and with a high quality of evidence.

RESULTS

Among the 400 abstracts surveyed, 32 (8%) found efficacy on a patient-important outcome with a high quality of evidence. Half of the 400 SRs (50.2%) evaluated a pharmacological intervention and 12% of these found efficacy of the intervention on a patient-important outcome with a reported high quality of evidence.

CONCLUSION

Based on an analysis of 400 abstracts of SRs from the Cochrane Collaboration, we found that there is a low number of therapeutic interventions which have proven their efficacy on patient-important outcomes with a high quality of evidence.

Cochrane reviews of educational and self-management interventions to guide nursing practice: A review

BACKGROUND

The burden of chronic disease on healthcare services worldwide continues to grow, and the increased development of educational interventions which help patients to better manage their own condition is evident internationally.

OBJECTIVES

This paper reports on findings of an updated review of Cochrane systematic reviews of interventions designed to improve patients' knowledge and skills to manage chronic disease, with particular reference to nursing contribution and practice.

METHODS

A broad search strategy was used to search the Cochrane Database of Systematic Reviews to identify reviews of patient education, self-management, and self-care studies. Two reviewers independently assessed eligibility for inclusion and extracted data from the reviews.

FINDINGS

From a total of 882 reviews, 63 met the inclusion criteria, and 900 studies were identified. Most (68%, n = 43) of the 63 reviews were judged by Cochrane reviewers to provide inadequate evidence of the effectiveness of the interventions reviewed. Information on the profession of the person delivering the intervention was often not available, although 78% (n = 49) of reviews mentioned that nurses were involved in a proportion of studies delivering interventions either independently or as part of a multi-professional team.

CONCLUSION

Educational programmes have definite benefits for patients suffering from asthma, chronic obstructive pulmonary disorder and stroke, and are promising in areas such as diabetes, epilepsy, cancer care, and mental health. However, it still is not clear what the active ingredients of many successful interventions are. Further research is needed to establish the impact of technology on programme delivery, and to develop programmes tailored for patients with multiple health problems.

Half of Cochrane reviews were published more than 2 years after the protocol

OBJECTIVES

The objective of the study is to examine the time from publication of the protocol for a Cochrane review to publication of the Cochrane review for the entire Cochrane Database of Systematic Reviews (CDSR).

STUDY DESIGN AND SETTINGS

Cochrane reviews from the CDSR published between 1995 and 2019 were assessed. Characteristics of the reviews were extracted, and time from publication of protocol to publication of review was calculated. These times were grouped for relevant characteristics and visualized through charts and tables to illustrate trends.

RESULTS

Of the total 8,201 reviews in the CDSR, 6,764 were included. The median publication time was 2 years (range 0 days to 21.7 years). Reviews that were published more than 5 years after the protocol made up 11% of all included reviews, whereas 19% of reviews were published within a year. The median publication time for the individual Cochrane Review Groups ranged from 15 to 39 months.

CONCLUSION

Half of Cochrane reviews were published later than Cochrane's aim of 2 years. Furthermore, the Cochrane Review Groups' median times from publication of protocol to publication of review varied widely.

Is psychoeducation for parents and teachers of children and adolescents with ADHD efficacious? A systematic literature review

Objective

To identify evidence from comparative studies on the effects of psychoeducation programs on clinical outcomes in children and adolescents with ADHD.

Method

Articles published between January 1980 and July 2010 were searched through electronic databases and hand search. A qualitative systematic review of comparative studies of psychoeducation in ADHD was performed. Psychoeducation was considered if studies use a specific therapeutic program focusing on the didactically communication of information and provide patients and families with coping skills.

Results

Seven studies were identified (four randomized-controlled trials, three uncontrolled pre-post treatment designs). Studies differed on whether psychoeducation approaches were applied to parents of ADHD children (three studies), to ADHD children/adolescents and their families (three studies) or to their teachers (one study). Positive outcomes measured as improvement on a number of different variables, including patient's behavior, parent and child satisfaction, child's knowledge of ADHD, children's opinion of the use of medication and adherence to medical recommendations were found.

Conclusions

Although available evidence is limited and some findings may be difficult to be interpreted, the positive role of psychoeducation and other educational interventions in children and adolescents with ADHD in regard to several outcome measures is supported by most of the literature referenced in this review.

Assessing the methodological and reporting quality of clinical systematic reviews and meta-analyses in paediatric urology: can practices on contemporary highest levels of evidence be built?

INTRODUCTION

Systematic reviews and meta-analyses provide a comprehensive summary of research studies and are used to assess clinical evidence, form policy and construct guidelines. This is pertinent to childhood surgery with issues of consent and condition prevalence. The aims of this study were to evaluate the methodological and reporting quality of these reviews and to identify how these reviews might guide clinical practice amongst those conditions most commonly encountered and managed by practicing paediatric urologists.

METHODS

A systematic search of the English literature was performed to identify systematic reviews and meta-analyses focusing on clinical paediatric urology (1/1/1992-1/12/2018) to include common paediatric urological conditions managed by paediatric urology residents/fellows. To these reviews, Assessing the Methodological Quality of Systematic Reviews (AMSTAR)-2 and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) scores were applied. Univariate linear regression and descriptive statistical methods were performed.

RESULTS

From an initial literature review of 1723 articles, 227 were included in the analysis. Inter-reviewer agreement was high amongst 3 independent reviewers (κ = 0.92). Eighty-four percent of systematic reviews and meta-analyses were published since 2009 following publication of the PRISMA guidelines. The overall impact factor was 3.38 (0.83-17.58), with adherence to AMSTAR-2 criteria 48.46% and PRISMA criteria 70.1%. From a methodological perspective, 15% of reviews were of moderate quality, 65% were of low quality and 20% reviews were of critically low quality, with none found to have good quality reporting.

CONCLUSIONS

Despite the continued increase of systematic reviews and meta-analyses in paediatric urology from which many guidelines are based, a significant number of reviews contain poor methodology and, to a lesser extent, poor reporting quality. Journals should consider having specific 'a priori' criteria based on checklists before publication of manuscripts to ensure the highest possible reporting quality.

The effectiveness of acupuncture for Parkinson's disease: An overview of systematic reviews

OBJECTIVES

Acupuncture is an alternative therapy for Parkinson's disease (PD), but its efficacy and safety are controversial. This overview aimed to summarize the existing evidence from systematic reviews (SRs) and meta-analyses (MAs) in order to assess the effectiveness of acupuncture as a treatment for PD.

METHODS

Seven electronic databases were searched from their inception until July 2019. The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) and Assessment of Multiple Systematic Reviews 2 (AMSTAR2) checklists were used to assess evidence quality and methodological quality, respectively. The outcomes of study were calculated using mean differences (MDs) and risk ratios (RRs) with 95 % confidence intervals (CIs). A meta-analysis was performed using RevMan 5.3 software.

RESULTS

A total of 12 SRs/MAs were included. All 12 SRs/MAs had more than one critical weakness in AMSTAR 2 and were considered of critically low methodological quality. The quality of evidence was unsatisfactory according to the GRADE checklist. Meta-analyses showed that acupuncture combined with drug for the treatment of PD can significantly improve the total effectiveness rate compared with drug alone (RR = 1.25, 95 % CI 1.16-1.34, P < 0.001). It was also found that acupuncture combined with drug significantly improved the UPDRS I-IV total summed scores (WMD=-6.18, 95 % CI -10.32 to -2.04, P < 0.001) and Webster scores (WMD=-4.20, 95 % CI -7.59 to -0.81, P < 0.001).

CONCLUSION

Acupuncture might improve the UPDRS score, Webster score, and total effective rate in treatment of PD. It might be a safe and useful adjunctive treatment for patients with PD. However, we should interpret the findings of these reviews with caution, considering the overall limited methodological and reporting quality.

Scoping Reviews, Systematic Reviews, and Meta-Analysis: Applications in Veterinary Medicine

Evidence-based decision making is a hallmark of effective veterinary clinical practice. Scoping reviews, systematic reviews, and meta-analyses all are methods intended to provide transparent and replicable ways of summarizing a body of research to address an important clinical or public health issue. As these methods increasingly are being used by researchers and read by practitioners, it is important to understand the distinction between these techniques and to understand what research questions they can, and cannot, address. This review provides an overview of scoping reviews, systematic reviews, and meta-analysis, including a discussion of the method and uses. A sample dataset and coding to conduct a simple meta-analysis in the statistical program R also are provided. Scoping reviews are a descriptive approach, designed to chart the literature around a particular topic. The approach involves an extensive literature search, following by a structured mapping, or charting, of the literature. The results of scoping reviews can help to inform future research by identifying gaps in the existing literature and also can be used to identify areas where there may be a sufficient depth of literature to warrant a systematic review. Systematic reviews are intended to address a specific question by identifying and summarizing all of the available research that has addressed the review question. Questions types that can be addressed by a systematic review include prevalence/incidence questions, and questions related to etiology, intervention efficacy, and diagnostic test accuracy. The systematic review process follows structured steps with multiple reviewers working in parallel to reduce the potential for bias. An extensive literature search is undertaken and, for each relevant study identified by the search, a formal extraction of data, including the effect size, and assessment of the risk of bias is performed. The results from multiple studies can be combined using meta-analysis. Meta-analysis provides a summary effect size, and allows heterogeneity of effect among studies to be quantified and explored. These evidence synthesis approaches can provide scientific input to evidence-based clinical decision-making for veterinarians and regulatory bodies, and also can be useful for identifying gaps in the literature to enhance the efficiency of future research in a topic area.

A summary of the evidence and evaluation of the effectiveness of nonpharmacological interventions for mild cognitive impairment

Objective

To summarize and evaluate the evidence of guidelines and systematic reviews (SRs) of nonpharmacological interventions for mild cognitive impairment (MCI) to support the development of future guidelines and clinical decisions for MCI patients.

Methods

Scottish Intercollegiate Guideline Network (SIGN), National Institute for Health and Clinical Excellence (NICE), American Academy of Neurology (AAN), Registered Nurses Association of Ontario (RNAO), Web of Science, PubMed, Cochrane Library, CNAHL, VIP, China National Knowledge Infrastructure (CNKI), and Wanfang Database were searched for relevant publications, including guidelines and SRs, from January 2014 to March 2019. Two authors independently screened articles, extracted data, and assessed the publications for adherence to the inclusion criteria. Appraisal of Guidelines for Research and Evaluation (AGREE II) was used to assess the quality of the guidelines, and Assessment of Multiple Systematic Reviews (AMSTAR 2) was used to assess the quality of SRs. In addition, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach was used to evaluate the quality of outcomes.

Results

Thirty-two articles were retrieved, including 1 guideline and 31 SRs. Fourteen SRs of physical exercise for MCI, six articles describing cognitive interventions, four articles describing acupuncture, and seven articles assessing dietary interventions (including four articles employing a Mediterranean diet, one article using vitamin B supplementation, and two articles assessing the effects of tea, coffee, and caffeine) were included. The quality of the articles was very low for 4 (13%), low for 10 (32%), and moderate for 17 (55%).

Conclusions

Based on the evidence available to date, nonpharmacological interventions may improve the current cognitive function of persons with MCI. In particular, physical exercise, cognitive interventions, and acupuncture exerted promising effects. However, due to the limited number and quality of the included publications, additional high-quality reviews are needed to further confirm.

Feasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation

BACKGROUND

Living systematic reviews (LSRs) offer an approach to keeping high-quality evidence synthesis continually up to date, so the most recent, relevant and reliable evidence can be used to inform policy and practice, resulting in improved quality of care and patient health outcomes. However, they require modifications to authoring and editorial processes and pose technical and publishing challenges. Several teams within Cochrane and the international Living Evidence Network have been piloting living systematic reviews.

METHODS

We conducted a mixed-methods evaluation with participants involved in six LSRs (three Cochrane and three non-Cochrane). Up to three semi-structured interviews were conducted with 27 participants involved with one or more of the pilot LSRs. Interviews explored participants' experiences contributing to the LSR, barriers and facilitators to their conduct and opportunities for future development. Pilot team members also completed monthly surveys capturing time for key tasks and the number of citations screened for each review.

RESULTS

Across the pilot LSRs, search frequency was monthly to three-monthly, with some using tools such as machine learning and Cochrane Crowd to screen searches. Varied approaches were used to communicate updates to readers. The number of citations screened varied widely between the reviews, from three to 300 citations per month. The amount of time spent per month by the author team on each review also varied from 5 min to 32 h. Participants were enthusiastic to be involved in the LSR pilot. They highlighted the importance of a motivated and well-organised team; the value of technology enablers to improve workflow efficiencies; the need to establish reliable and efficient processes to sustain living reviews; and the potential for saving time and effort in the long run. Participants highlighted challenges with the current publication processes, managing ongoing workload and the lack of resources to support LSRs in the long term.

CONCLUSIONS

Findings to date support feasibility and acceptability of LSR production. There are challenges that need to be addressed for living systematic reviews to be sustainable and have maximum value. The findings from this study will be used in discussions with the Cochrane community, key decision makers and people more broadly concerned with LSRs to identify and develop priorities for scale-up.

Preventive Treatments for Psychosis: Umbrella Review (Just the Evidence)

Background: Indicated primary prevention in young people at Clinical High Risk for Psychosis (CHR-P) is a promising avenue for improving outcomes of one of the most severe mental disorders but their effectiveness has recently been questioned. Methods: Umbrella review. A multi-step independent literature search of Web of Science until January 11, 2019, identified interventional meta-analyses in CHR-P individuals. The individual randomised controlled trials that were analysed by the meta-analyses were extracted. A review of ongoing trials and a simulation of living meta-analysis complemented the analysis. Results: Seven meta-analyses investigating preventive treatments in CHR-P individuals were included. None of them produced pooled effect sizes across psychological, pharmacological, or other types of interventions. The outcomes analysed encompassed risk of psychosis onset, the acceptability of treatments, the severity of attenuated positive/negative psychotic symptoms, depression, symptom-related distress, social functioning, general functioning, and quality of life. These meta-analyses were based on 20 randomised controlled trials: the vast majority defined the prevention of psychosis onset as their primary outcome of interest and only powered to large effect sizes. There was no evidence to favour any preventive intervention over any other (or control condition) for improving any of these clinical outcomes. Caution is required when making clinical recommendations for the prevention of psychosis in individuals at risk. Discussion: Prevention of psychosis from a CHR-P state has been, and should remain, the primary outcome of interventional research, refined and complemented by other clinically meaningful outcomes. Stagnation of knowledge should promote innovative and collaborative research efforts, in line with the progressive and incremental nature of medical knowledge. Advancements will most likely be associated with the development of new experimental therapeutics that are ongoing along with the ability to deconstruct the high heterogeneity within CHR-P populations. This would require the estimation of treatment-specific effect sizes through living individual participant data meta-analyses, controlling risk enrichment during recruitment, statistical power, and embedding precision medicine within youth mental health services that can accommodate sequential prognosis and advanced trial designs. Conclusions: The evidence-based challenges and proposed solutions addressed by this umbrella review can inform the next generation of research into preventive treatments for psychosis.

Reporting quality of Cochrane systematic reviews with Chinese herbal medicines

BACKGROUND

Chinese herbal medicines (CHMs) are the major interventions of traditional Chinese medicine (TCM), which are typically administered as either single herbs or formulas. The Cochrane systematic reviews (SRs) of CHMs are essential references for evaluating the efficacy and safety of CHMs interventions; they are expected to be accurate and reliable. This study aimed to assess the reporting quality of these SRs, particularly whether necessary information related to CHM was adequately reported.

METHODS

The Cochrane Database was systematically searched for all SRs of CHM that were published up to 31 December 2017. The primary analysis was to assess their reporting quality based on 27-item of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) and 9-item of CHM-related information designed according to TCM theory. Descriptive statistics were additionally used to analyze their baseline characteristics.

RESULTS

A total of 109 Cochrane SRs of CHM were identified from 1999 to 2017. For 27-item of PRISMA, 26 had the reporting compliances higher than 50%, of which 11 were fully reporting (100%). However, for CHM-related information, 65 (59.6%) SRs did not report the specific name of the CHM in the title, 42 (38.5%) lacked TCM-related rationales in the introduction, 62 (56.9%) did not include CHM-related characteristics in the additional analyses, and 77 (70.6%) did not analyze CHM results in terms of TCM-related theories in the discussion. Of 97 SRs that included clinical trials, 38 (39.2%) did not provide the details of composition and dosage of CHMs, 85 (87.6%) did not report the CHM sources, 13 (13.4%) did not provide the dosage form, 95 (97.9%) lacked CHM quality control information, and 57 (58.8%) did not describe details of the controls. For 62 (72.9%) of 85 SRs that included meta-analysis, it was impossible to assess whether meta-analysis had been properly conducted due to inadequate reporting of CHM interventions.

CONCLUSION

Although the Cochrane SRs of CHM showed reporting compliance with PRISMA checklist, their reporting quality needs improvement, especially about full reporting of CHM interventions and of TCM-related rationales. Reporting guideline of "PRISMA extension for CHM interventions" should be developed thus to improve their quality.

Barriers to the registration and conduct of Cochrane systematic reviews of traditional East Asian medicine therapies

INTRODUCTION

Traditional East Asian medicine (TEAM) is widely used in Asia and increasingly in the West. Systematic reviews (SRs) are the best summaries of the potential benefits or harms of interventions, and Cochrane is a leading international SR organization. Cochrane perspectives on the barriers to the initiation and completion of Cochrane SRs of TEAM therapies were solicited.

METHODS

Cochrane Review Groups (CRGs) were identified from the online listing of CRGs at cochrane.org and a link to an online survey was e-mailed to the primary contact for each CRG.

RESULTS

Forty-eight responses were received on behalf of 49/53 (92%) CRGs. Most CRGs had experience producing TEAM reviews, primarily in acupuncture or herbal medicine. The main barriers to taking on a new TEAM review were difficulty in understanding and assessing the intervention, and the low priority of TEAM topics. Problems with the quality and accessibility of randomized trials in TEAM were cited as a major concern. CRGs suggested that the quality and accessibility of randomized trials should be improved, that the methodological and language expertise of authors should be enhanced, and that further peer review expertise should be made available to CRGs.

CONCLUSIONS

TEAM topics are covered in Cochrane reviews but are often considered low-priority. This survey highlights Cochrane concerns about the quality of the underlying evidence base and the training of the author teams as barriers to successful SR completion. Specific approaches are proposed to increase the number of TEAM reviews and address the limitations of TEAM research processes within Cochrane.

Measuring the impact of screening automation on meta-analyses of diagnostic test accuracy

BACKGROUND

The large and increasing number of new studies published each year is making literature identification in systematic reviews ever more time-consuming and costly. Technological assistance has been suggested as an alternative to the conventional, manual study identification to mitigate the cost, but previous literature has mainly evaluated methods in terms of recall (search sensitivity) and workload reduction. There is a need to also evaluate whether screening prioritization methods leads to the same results and conclusions as exhaustive manual screening. In this study, we examined the impact of one screening prioritization method based on active learning on sensitivity and specificity estimates in systematic reviews of diagnostic test accuracy.

METHODS

We simulated the screening process in 48 Cochrane reviews of diagnostic test accuracy and re-run 400 meta-analyses based on a least 3 studies. We compared screening prioritization (with technological assistance) and screening in randomized order (standard practice without technology assistance). We examined if the screening could have been stopped before identifying all relevant studies while still producing reliable summary estimates. For all meta-analyses, we also examined the relationship between the number of relevant studies and the reliability of the final estimates.

RESULTS

The main meta-analysis in each systematic review could have been performed after screening an average of 30% of the candidate articles (range 0.07 to 100%). No systematic review would have required screening more than 2308 studies, whereas manual screening would have required screening up to 43,363 studies. Despite an average 70% recall, the estimation error would have been 1.3% on average, compared to an average 2% estimation error expected when replicating summary estimate calculations.

CONCLUSION

Screening prioritization coupled with stopping criteria in diagnostic test accuracy reviews can reliably detect when the screening process has identified a sufficient number of studies to perform the main meta-analysis with an accuracy within pre-specified tolerance limits. However, many of the systematic reviews did not identify a sufficient number of studies that the meta-analyses were accurate within a 2% limit even with exhaustive manual screening, i.e., using current practice.

Less is more in solid-dominant lung cancer? Sublobar resection versus lobectomy for solid-dominant stage IA non-small-cell lung cancer: A meta-analysis study

Although lobectomy is well established as the standard surgical procedure for stage IA non-small-cell lung cancer (NSCLC), sublobar resection is increasingly preferred, particularly in intentional segmentectomy for radiologically less-invasive small NSCLC. However, the indication for sublobar resection of radiologically pure solid or solid-dominant NSCLC remains controversial, owing to its invasive pathological characteristics. Therefore, the present meta-analysis was conducted to compare the efficacy of sublobar resection with lobectomy for treating solid-dominant stage IA NSCLC. An electronic search was conducted using four online databases from their dates of inception to April 2017. The hazard ratio (HR) was used as a summary statistic for censored outcomes and the odds ratio (OR) was used as the summary statistic for dichotomous variables. A total of nine studies met the selection criteria, including a total of 2,265 patients (1,728 patients underwent lobectomy, 425 segmentectomy and 112 wedge resection). From the available data, patients treated with a sublobar resection had a higher risk of local recurrence compared with patients treated with lobectomy [OR=1.89; 95% confidence interval (CI), 1.02–3.50; P=0.04]. However, no obvious difference in local recurrence was found in a subgroup analysis of segmentectomy compared with lobectomy (OR=1.19; 95% CI, 0.68–2.10; P=0.61). Sublobar resection was not associated with a significantly negative impact on distant recurrence (OR=1.09; 95% CI, 0.55–2.16; P=0.796). Patients in the sublobar resection group had no significant differences in recurrence-free survival (RFS; HR=1.43; 95% CI, 0.76–2.69; P=0.27) and overall survival (OS; HR=0.96; 95% CI, 0.75–1.23; P=0.77) compared with those in the lobectomy group. In the subgroup analysis of anatomic segmentectomy compared with lobectomy, there was no significant difference in RFS, with mild inter-study heterogeneity. The current meta-analysis suggested that segmentectomy had a comparable oncologic efficacy to lobectomy for solid-dominant stage IA NSCLC. Therefore, segmentectomy may be a feasible alternative in selected cases of solid-dominant stage IA NSCLC. However, these findings should be confirmed by prospective randomized controlled trials in the future.

Characteristics of stakeholder involvement in systematic and rapid reviews: a methodological review in the area of health services research

OBJECTIVE

Engaging stakeholders in reviews is considered to generate more relevant evidence and to facilitate dissemination and use. As little is known about stakeholder involvement, we assessed the characteristics of their engagement in systematic and rapid reviews and the methodological quality of included studies. Stakeholders were people with a particular interest in the research topic.

DESIGN

Methodological review.

SEARCH STRATEGY

Four databases (Medline, Embase, Cochrane database of systematic reviews, databases of the University of York, Center for Reviews and Dissemination (CRD)) were searched based on an a priori protocol. Four types of reviews (Cochrane and non-Cochrane systematic reviews, rapid and CRD rapid reviews) were retrieved between January 2011 and October 2015, pooled by potential review type and duplicates excluded. Articles were randomly ordered and screened for inclusion and exclusion criteria until 30 reviews per group were reached. Their methodological quality was assessed using AMSTAR and stakeholder characteristics were collected.

RESULTS

In total, 57 822 deduplicated citations were detected with potential non-Cochrane systematic reviews being the biggest group (56 986 records). We found stakeholder involvement in 13% (4/30) of Cochrane, 20% (6/30) of non-Cochrane, 43% (13/30) of rapid and 93% (28/30) of CRD reviews. Overall, 33% (17/51) of the responding contact authors mentioned positive effects of stakeholder involvement. A conflict of interest statement remained unmentioned in 40% (12/30) of non-Cochrane and in 27% (8/30) of rapid reviews, but not in Cochrane or CRD reviews. At most, half of non-Cochrane and rapid reviews mentioned an a priori study protocol in contrast to all Cochrane reviews.

CONCLUSION

Stakeholder engagement was not general practice, except for CRD reviews, although it was more common in rapid reviews. Reporting factors, such as including an a priori study protocol and a conflict of interest statement should be considered in conjunction with involving stakeholders.

Silver diamine fluoride for managing carious lesions: an umbrella review

BACKGROUND

This umbrella review comprehensively appraised evidence for silver diamine fluoride (SDF) to arrest and prevent root and coronal caries by summarizing systematic reviews. Adverse events were explored.

METHODS

Following Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines, PubMed, Embase, Cochrane Library, PROSPERO register and Joanna Briggs Institute Database of Systematic Reviews were searched for systematic reviews investigating SDF for caries prevention or arrest (1970-2018) without language restrictions. Systematic reviews were selected, data extracted, and risk of bias assessed using ROBIS by two independent reviewers, in duplicate. Corrected covered area was calculated to quantify studies' overlap across reviews.

RESULTS

Eleven systematic reviews were included; four focussing on SDF for root caries in adults and seven on coronal caries in children. These cited 30 studies (4 root caries; 26 coronal caries) appearing 63 times. Five systematic reviews were of "low", one "unclear" and five "high" risk of bias. Overlap of studies was very high (50% root caries; 17% coronal caries). High overlap and heterogeneity, mainly comparators and outcome measures, precluded meta-analysis. Results were grouped by aim and outcomes to present an overview of direction and magnitude of effect. SDF had a positive effect on prevention and arrest of coronal and root caries, consistently outperforming comparators (fluoride varnish, Atraumatic Restorative Treatment, placebo). For root caries prevention, the prevented fraction (PF) was 25-71% higher for SDF compared to placebo (two systematic reviews with three studies) and PF = 100-725% for root caries arrest (one systematic review with two studies). For coronal caries prevention, PF = 70-78% (two systematic reviews with two studies) and PF = 55-96% for coronal caries arrest (one systematic review with two studies) with arrest rates of 65-91% (four systematic reviews with six studies). Eight systematic reviews reported adverse events, seven of which reported arrested lesions black staining.

CONCLUSION

Systematic reviews consistently supported SDF's effectiveness for arresting coronal caries in the primary dentition and arresting and preventing root caries in older adults for all comparators. There is insufficient evidence to draw conclusions on SDF for prevention in primary teeth and prevention and arrest in permanent teeth in children. No serious adverse events were reported.

Capacity building for knowledge translation: A survey about the characteristics and motivation of volunteer translators of Cochrane plain language summaries

BACKGROUND

Cochrane systematic reviews have plain language summaries (PLSs) that are translated into multiple languages, mostly by volunteers. We aimed to survey volunteer translators to find out their characteristics, motivation, and suggestions to further improve this translation project.

METHODS

We surveyed 176 registered volunteer translators from the Cochrane Croatia PLS translation project. A 28-item survey, created for the purpose of this study, was administered via SurveyMonkey in November-December 2017. Translators received an invitation to the survey and two follow-up reminders via e-mail. Primary and secondary outcomes were the characteristics and motivation of volunteers.

RESULTS

We received 106 responses (60% response rate) to the survey. The translators were on average (standard deviation) 32 ± 10 years old, they were mostly women (74%), and most of them were medical doctors (29%) or pharmacists (13%). The majority found out about the translation project from acquaintances and colleagues. Most of them indicated that they have still had high motivation for the project, but for the majority, the number of translations decreased over time. When asked what could motivate them to translate more Cochrane PLSs, or to start translating if they did not translate anything yet, the most common answers were: feedback about translation quality, reminders, a workshop for translators and Facebook group for translators.

CONCLUSION

Cochrane volunteer PLS translators are a motivated and highly educated group, but their translation output is decreasing with time. Reminders, feedback and education are interventions that should be tested to increase volunteer engagement in such initiatives.

What public health strategies work to reduce the tobacco demand among young people? An umbrella review of systematic reviews and meta-analyses

OBJECTIVE

To perform an umbrella review of systematic reviews and meta-analyses of health policy and health promotion strategies to reduce the tobacco demand in adolescents, youth and young adults.

METHODS

Reviewers independently performed an electronic database search, reviewed titles and abstracts, assessed articles' eligibility for inclusion and quality, and extracted relevant data. Only systematic reviews and meta-analyses reporting data on tobacco policies and interventions focusing on individuals aged <25 years were included. The Framework Convention on Tobacco Control was used to guide data synthesis.

RESULTS

13 articles were included. Studies were of mixed quality with five studies ranked as critically low and seven as high quality. Overall, mixed results were found on the effectiveness for tobacco policies and interventions. Strategies such as increasing taxes on tobacco products were most promising.

CONCLUSION

Though data on a variety of measures to reduce smoking is available, conclusions concerning the effectiveness are inconclusive. Tobacco policies and interventions have the potential to reduce smoking, but conclusions are hampered due to both lack of high-quality trials and numerous biases in primary studies. Further high-quality research is required to examine the effectiveness of interventions and policies to reduce the tobacco demand in adolescents, youth and young adults.

Protocol registration or development may benefit the design, conduct and reporting of dose-response meta-analysis: empirical evidence from a literature survey

Background

To investigate the prevalence of protocol registration (or development) among published dose-response meta-analyses (DRMAs), and whether DRMAs with a protocol are better than those not.

Methods

Three databases were searched for eligible DRMAs. The modified AMSTAR (14 items) and PRISMA checklists (26 items) were used to assess the methodological and reporting quality, with each item assigned 1 point if it met the requirement or 0 if not. We matched (1,2) DRMAs with registered or published protocol to those not, by region and publication years. The summarized quality score and compliance rate of each item were compared between the two groups. Multivariable regression was employed to see if protocol registration or development was associated with total quality score.

Results

We included 529 DRMAs, with 45 (8.51%) completed protocol registration or development. We observed a higher methodological score for DRMAs with protocol than the matched controls (9.47 versus 8.58, P <  0.01); this embodied in 4 out of 14 items of AMSTAR [e.g., Duplicate data extraction (rate difference, RD = 0.17, 95% CI: 0.04, 0.30; P = 0.01). A higher reporting score (cubic transformed) for DRMAs with protocol than the matched controls was also observed (11,875.00 versus 10,229.53, P <  0.01); which embodied in 6 out of 26 items of PRISMA [e.g. Describe methods for publication bias (RD = 0.08, 95% CI: 0.01, 0.14; P = 0.02)]. Regression analysis suggested positive association between protocol registration or development and total reporting score (P = 0.012) while not for methodological score (P = 0.87).

Conclusions

Only a small proportion of DRMAs completed protocol registration or development, and those with protocol were better reported than those not. Protocol registration or development is highly desirable.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0715-y) contains supplementary material, which is available to authorized users.