Abbreviated literature searches were viable alternatives to comprehensive searches: a meta-epidemiological study

An exploration of available methods and tools to improve the efficiency of systematic review production: a scoping review

BACKGROUND

Systematic reviews (SRs) are time-consuming and labor-intensive to perform. With the growing number of scientific publications, the SR development process becomes even more laborious. This is problematic because timely SR evidence is essential for decision-making in evidence-based healthcare and policymaking. Numerous methods and tools that accelerate SR development have recently emerged. To date, no scoping review has been conducted to provide a comprehensive summary of methods and ready-to-use tools to improve efficiency in SR production.

OBJECTIVE

To present an overview of primary studies that evaluated the use of ready-to-use applications of tools or review methods to improve efficiency in the review process.

METHODS

We conducted a scoping review. An information specialist performed a systematic literature search in four databases, supplemented with citation-based and grey literature searching. We included studies reporting the performance of methods and ready-to-use tools for improving efficiency when producing or updating a SR in the health field. We performed dual, independent title and abstract screening, full-text selection, and data extraction. The results were analyzed descriptively and presented narratively.

RESULTS

We included 103 studies: 51 studies reported on methods, 54 studies on tools, and 2 studies reported on both methods and tools to make SR production more efficient. A total of 72 studies evaluated the validity (n = 69) or usability (n = 3) of one method (n = 33) or tool (n = 39), and 31 studies performed comparative analyses of different methods (n = 15) or tools (n = 16). 20 studies conducted prospective evaluations in real-time workflows. Most studies evaluated methods or tools that aimed at screening titles and abstracts (n = 42) and literature searching (n = 24), while for other steps of the SR process, only a few studies were found. Regarding the outcomes included, most studies reported on validity outcomes (n = 84), while outcomes such as impact on results (n = 23), time-saving (n = 24), usability (n = 13), and cost-saving (n = 3) were less often evaluated.

CONCLUSION

For title and abstract screening and literature searching, various evaluated methods and tools are available that aim at improving the efficiency of SR production. However, only few studies have addressed the influence of these methods and tools in real-world workflows. Few studies exist that evaluate methods or tools supporting the remaining tasks. Additionally, while validity outcomes are frequently reported, there is a lack of evaluation regarding other outcomes.

Rapid review: A review of methods and recommendations based on current evidence

Rapid review (RR) could accelerate the traditional systematic review (SR) process by simplifying or omitting steps using various shortcuts. With the increasing popularity of RR, numerous shortcuts had emerged, but there was no consensus on how to choose the most appropriate ones. This study conducted a literature search in PubMed from inception to December 21, 2023, using terms such as "rapid review" "rapid assessment" "rapid systematic review" and "rapid evaluation". We also scanned the reference lists and performed citation tracking of included impact studies to obtain more included studies. We conducted a narrative synthesis of all RR approaches, shortcuts and studies assessing their effectiveness at each stage of RRs. Based on the current evidence, we provided recommendations on utilizing certain shortcuts in RRs. Ultimately, we identified 185 studies focusing on summarizing RR approaches and shortcuts, or evaluating their impact. There was relatively sufficient evidence to support the use of the following shortcuts in RRs: limiting studies to those published in English-language; conducting abbreviated database searches (e.g., only searching PubMed/MEDLINE, Embase, and CENTRAL); omitting retrieval of grey literature; restricting the search timeframe to the recent 20 years for medical intervention and the recent 15 years for reviewing diagnostic test accuracy; conducting a single screening by an experienced screener. To some extent, the above shortcuts were also applicable to SRs. This study provided a reference for future RR researchers in selecting shortcuts, and it also presented a potential research topic for methodologists.

What are the best methods for rapid reviews of the research evidence? A systematic review of reviews and primary studies

Rapid review methodology aims to facilitate faster conduct of systematic reviews to meet the needs of the decision-maker, while also maintaining quality and credibility. This systematic review aimed to determine the impact of different methodological shortcuts for undertaking rapid reviews on the risk of bias (RoB) of the results of the review. Review stages for which reviews and primary studies were sought included the preparation of a protocol, question formulation, inclusion criteria, searching, selection, data extraction, RoB assessment, synthesis, and reporting. We searched 11 electronic databases in April 2022, and conducted some supplementary searching. Reviewers worked in pairs to screen, select, extract data, and assess the RoB of included reviews and studies. We included 15 systematic reviews, 7 scoping reviews, and 65 primary studies. We found that several commonly used shortcuts in rapid reviews are likely to increase the RoB in the results. These include restrictions based on publication date, use of a single electronic database as a source of studies, and use of a single reviewer for screening titles and abstracts, selecting studies based on the full-text, and for extracting data. Authors of rapid reviews should be transparent in reporting their use of these shortcuts and acknowledge the possibility of them causing bias in the results. This review also highlights shortcuts that can save time without increasing the risk of bias. Further research is needed for both systematic and rapid reviews on faster methods for accurate data extraction and RoB assessment, and on development of more precise search strategies.

Rapid reviews methods series: Guidance on literature search

This paper is part of a series of methodological guidance from the Cochrane Rapid Reviews Methods Group. Rapid reviews (RR) use modified systematic review methods to accelerate the review process while maintaining systematic, transparent and reproducible methods. In this paper, we address considerations for RR searches. We cover the main areas relevant to the search process: preparation and planning, information sources and search methods, search strategy development, quality assurance, reporting, and record management. Two options exist for abbreviating the search process: (1) reducing time spent on conducting searches and (2) reducing the size of the search result. Because screening search results is usually more resource-intensive than conducting the search, we suggest investing time upfront in planning and optimising the search to save time by reducing the literature screening workload. To achieve this goal, RR teams should work with an information specialist. They should select a small number of relevant information sources (eg, databases) and use search methods that are highly likely to identify relevant literature for their topic. Database search strategies should aim to optimise both precision and sensitivity, and quality assurance measures (peer review and validation of search strategies) should be applied to minimise errors.

In-depth evaluation of machine learning methods for semi-automating article screening in a systematic review of mechanistic literature

We aimed to evaluate the performance of supervised machine learning algorithms in predicting articles relevant for full-text review in a systematic review. Overall, 16,430 manually screened titles/abstracts, including 861 references identified relevant for full-text review were used for the analysis. Of these, 40% (n = 6573) were sub-divided for training (70%) and testing (30%) the algorithms. The remaining 60% (n = 9857) were used as a validation set. We evaluated down- and up-sampling methods and compared unigram, bigram, and singular value decomposition (SVD) approaches. For each approach, Naïve Bayes, Support Vector Machines (SVM), regularized logistic regressions, neural networks, random forest, Logit boost, and XGBoost were implemented using simple term frequency or Tf-Idf feature representations. Performance was evaluated using sensitivity, specificity, precision and area under the Curve. We combined predictions of the best-performing algorithms (Youden Index ≥0.3 with sensitivity/specificity≥70/60%). In a down-sample unigram approach, Naïve Bayes, SVM/quanteda text models with Tf-Idf, and linear SVM e1071 package with Tf-Idf achieved >90% sensitivity at specificity >65%. Combining the predictions of the 10 best-performing algorithms improved the performance to reach 95% sensitivity and 64% specificity in the validation set. Crude screening burden was reduced by 61% (5979) (adjusted: 80.3%) with 5% (27) false negativity rate. All the other approaches yielded relatively poorer performances. The down-sampling unigram approach achieved good performance in our data. Combining the predictions of algorithms improved sensitivity while screening burden was reduced by almost two-third. Implementing machine learning approaches in title/abstract screening should be investigated further toward refining these tools and automating their implementation.

Methodological Quality of Physical Therapy-Related Trials Published in Open Access and Subscription Journal: A Cross-sectional Meta-Epidemiological Study

OBJECTIVE

We aimed to compare the methodological quality of physical therapy-related trials published in open access with that of trials published in subscription-based journals, adjusting for subdiscipline, intervention type, endorsement of the Consolidated Standards of Reporting Trials, impact factor, and publication language.

DESIGN

In this meta-epidemiological study, we searched the Physiotherapy Evidence Database on May 8, 2021, to include any physical therapy-related trials published from January 1, 2020. We extracted variables such as Consolidated Standards of Reporting Trials endorsement, the Physiotherapy Evidence Database score, and publication type. We compared the Physiotherapy Evidence Database score between the publication types using a multivariable generalized estimating equation by adjusting for covariates.

RESULTS

A total of 2743 trials were included, with a mean total Physiotherapy Evidence Database score (standard deviation) of 5.8 (±1.5). Trials from open access journals had a lower total Physiotherapy Evidence Database score than those from subscription-based journals (5.5 ± 1.5 vs. 5.9 ± 1.5, mean difference = -0.4; 95% confidence interval = 0.3-0.5). Generalized estimating equation revealed that open access publication was significantly associated with the total Physiotherapy Evidence Database score (mean difference = -0.42; P < 0.001).

CONCLUSIONS

In the recent physical therapy-related trials, open access publications demonstrated lower methodological quality than subscription-based publications, although with a small difference.

CAT HPPR: a critical appraisal tool to assess the quality of systematic, rapid, and scoping reviews investigating interventions in health promotion and prevention

BACKGROUND

For over three decades researchers have developed critical appraisal tools (CATs) for assessing the scientific quality of research overviews. Most established CATs for reviews in evidence-based medicine and evidence-based public health (EBPH) focus on systematic reviews (SRs) with studies on experimental interventions or exposure included. EBPH- and implementation-oriented organisations and decision-makers, however, often seek access to rapid reviews (RRs) or scoping reviews (ScRs) for rapid evidence synthesis and research field exploration. Until now, no CAT is available to assess the quality of SRs, RRs, and ScRs following a unified approach. We set out to develop such a CAT.

METHODS

The development process of the Critical Appraisal Tool for Health Promotion and Prevention Reviews (CAT HPPR) included six phases: (i) the definition of important review formats and complementary approaches, (ii) the identification of relevant CATs, (iii) prioritisation, selection and adaptation of quality criteria using a consensus approach, (iv) development of the rating system and bilingual guidance documents, (v) engaging with experts in the field for piloting/optimising the CAT, and (vi) approval of the final CAT. We used a pragmatic search approach to identify reporting guidelines/standards (n = 3; e.g. PRISMA, MECIR) as well as guidance documents (n = 17; e.g. for reviews with mixed-methods approach) to develop working definitions for SRs, RRs, ScRs, and other review types (esp. those defined by statistical methods or included data sources).

RESULTS

We successfully identified 14 relevant CATs, predominantly for SRs (e.g. AMSTAR 2), and extracted 46 items. Following consensual discussions 15 individual criteria were included in our CAT and tailored to the review types of interest. The CAT was piloted with 14 different reviews which were eligible to be included in a new German database looking at interventions in health promotion and prevention in different implementation settings.

CONCLUSIONS

The newly developed CAT HPPR follows a unique uniformed approach to assess a set of heterogeneous reviews (e.g. reviews from problem identification to policy evaluations) to assist end-users needs. Feedback of external experts showed general feasibility and satisfaction with the tool. Future studies should further formally test the validity of CAT HPPR using larger sets of reviews.

[Evidence syntheses in public health: An overview]

BACKGROUND

Since the beginning of the COVID-19 pandemic, there has been a high demand for rapid evidence syntheses to answer urgent public health questions. This article provides an overview of different types of reviews for public health questions and a synthesis of existing recommendations for the preparation of reviews. The aim is to support the planning of one's own review and the critical evaluation of published reviews.

METHODS

The basis of this summary is an extensive search for guidelines and recommendations for different review types. Furthermore, internal journal clubs were held to determine knowledge needs and to critically discuss the various review types. Relating to the dissemination of results, fact sheets were developed for the individual review types including the most important information, prerequisites and work steps, as well as a decision tree for identifying the appropriate review type for the respective question.

RESULTS

Of the review types identified, Systematic, Rapid, Scoping, Umbrella, and Narrative Reviews were considered in more detail because they are particularly relevant to public health issues. Together with scoping and umbrella reviews, systematic reviews have the highest resource requirements due to the demands for extensive, systematic evidence synthesis and reproducibility. Rapid methods can accelerate the review process, for example by a very narrowly formulated question, a limited literature search, or the execution of certain steps by one instead of two persons.

DISCUSSION

Systematic Reviews may be considered as the gold standard, but they were developed primarily for clinical questions relating to interventions. This article, however, focusses on review types that consider the diversity of questions as well as the predominant use of quantitative methods in the field of public health. The fact sheets developed and the decision tree should enable low-threshold access to reviews while linking the perspectives of research and resource planning. They complement existing guidelines and recommendations.

CONCLUSION

To answer the diverse spectrum of public health questions, various types of reviews with various requirements and approaches are available. Given this diversity, a systematic introduction can be helpful for researchers planning or assessing a review.

Efficient searching for NICE public health guidelines: Would using fewer sources still find the evidence?

Systematic searches are integral to identifying the evidence that is used in National Institute for Health and Care Excellence (NICE) public health guidelines (PHGs). This study analyses the sources, including bibliographic databases and other techniques, required for PHGs. The aims were to analyse the sources used to identify the publications included in NICE PHGs; and to assess whether fewer sources could have been searched to retrieve these publications. Data showing how the included publications had been identified was collated using search summary tables. Three scenarios were created to test various combinations of sources to determine whether fewer sources could have been used. The sample included 29 evidence reviews, compiled using 13 searches, to support 10 PHG topics. Across the PHGs, 23 databases and six other techniques retrieved included publications. A mean reduction in total results of 6.5% could have been made if the minimum set of sources plus Cochrane Library, Embase, and MEDLINE were searched. On average, Cochrane Library, Embase, and MEDLINE contributed 76.8% of the included publications, with other databases adding 11% and other techniques 12.2%. None of the searches had a minimum set that was comprised entirely of databases. There was not a core set of sources for PHGs. A range of databases and techniques, covering a multi-disciplinary evidence base, was required to identify all included publications. It would be possible to reduce the number of sources searched and make some gains in productivity. It is important to create a tailored set of sources to do an efficient search.

A Tailored Approach: A model for literature searching in complex systematic reviews

Our previous work identified that nine leading guidance documents for seven different types of systematic review advocated the same process of literature searching. We defined and illustrated this process and we named it ‘the Conventional Approach’. The Conventional Approach appears to meet the needs of researchers undertaking literature searches for systematic reviews of clinical interventions. In this article, we report a new and alternate process model of literature searching called ‘A Tailored Approach’. A Tailored Approach is indicated as a search process for complex reviews which do not focus on the evaluation of clinical interventions. The aims of this article are to (1) explain the rationale for, and the theories behind, the design of A Tailored Approach; (2) report the current conceptual illustration of A Tailored Approach and to describe a user’s interaction with the process model; and (3) situate the elements novel to A Tailored Approach (when compared with the Conventional Approach) in the relevant literature. A Tailored Approach suggests investing time at the start of a review, to develop the information needs from the research objectives, and to tailor the search approach to studies or data. Tailored Approaches should be led by the information specialist (librarian) but developed by the research team. The aim is not necessarily to focus on comprehensive retrieval. Further research is indicated to evaluate the use of supplementary search methods, methods of team-working to define search approaches, and to evaluate the use of conceptual models of information retrieval for testing and evaluation.

Include or not to include conference abstracts in systematic reviews? Lessons learned from a large Cochrane network meta-analysis including 585 trials

BACKGROUND

Systematic reviews attempt to gather all available evidence. Controversy exists regarding effort and benefit of including study results presented at conferences only. We recently published a Cochrane network meta-analysis (NMA) including 585 randomized controlled trials comparing drugs for prevention of postoperative nausea and vomiting (PONV). Studies published as conference abstracts only were excluded. This study aimed to include all eligible studies published as abstracts only, assessing their added value regarding reporting quality and effect on the review's interpretation.

METHODS

Conference abstracts were searched in the review's excluded studies and conference proceedings of anaesthesiologic societies. We assessed their reporting quality regarding review's eligibility criteria, Cochrane 'risk of bias' assessment tool 1.0, and adherence to CONSORT (Consolidated Standards of Reporting Trials) for abstracts. Abstracts were included in sensitivity NMA, and impact on the NMA structure was investigated.

RESULTS

We identified 90 abstracts. A total of 14% (13/90) were eligible. A total of 86% (77/90) are awaiting classification due to insufficient reporting of review's eligibility criteria. In abstracts awaiting classification, sufficient information was missing on standardization of anaesthesia in 71% (55/77), age of participants in 56% (43/77), and outcome details in 46% (36/77). A total of 73% (66/90) of abstracts lacked sufficient information on 15/25 data extraction items. Reported study characteristics of abstracts were comparable to included studies of the review. A total of 62% (56/90) of abstract trials were assessed as overall high risk of bias due to poor reporting. Median adherence to CONSORT for abstracts was 24% (IQR, 18 to 29%). Six of the 13 eligible abstracts reported relevant outcome data in sufficient detail for NMA on seven outcomes of the Cochrane review. Inclusion of abstracts did not substantially change the network structure, network effect estimates, ranking of treatments, or the conclusion. Certainty of evidence for headache on palonosetron use was upgraded from very low to low.

CONCLUSIONS

Most conference abstracts on PONV were insufficiently reported regarding review's narrow inclusion criteria and could not be included in NMA. The resource-intensive search and evaluation of abstracts did not substantially extent the full-text evidence base of the review, given the few adequately reported abstracts. Conferences should oblige authors to adhere to CONSORT for abstracts.

Paper 2: Performing rapid reviews

BACKGROUND

Health policy-makers must often make decisions in compressed time frames and with limited resources. Hence, rapid reviews have become a pragmatic alternative to comprehensive systematic reviews. However, it is important that rapid review methods remain rigorous to support good policy development and decisions. There is currently little evidence about which streamlined steps in a rapid review are less likely to introduce unacceptable levels of uncertainty while still producing a product that remains useful to policy-makers.

METHODS

This paper summarizes current research describing commonly used methods and practices that are used to conduct rapid reviews and presents key considerations and options to guide methodological choices for a rapid review.

RESULTS

The most important step for a rapid review is for an experienced research team to have early and ongoing engagement with the people who have requested the review. A clear research protocol, derived from a needs assessment conducted with the requester, serves to focus the review, defines the scope of the rapid review, and guides all subsequent steps. Common recommendations for rapid review methods include tailoring the literature search in terms of databases, dates, and languages. Researchers can consider using a staged search to locate high-quality systematic reviews and then subsequently published primary studies. The approaches used for study screening and selection, data extraction, and risk-of-bias assessment should be tailored to the topic, researcher experience, and available resources. Many rapid reviews use a single reviewer for study selection, risk-of-bias assessment, or data abstraction, sometimes with partial or full verification by a second reviewer. Rapid reviews usually use a descriptive synthesis method rather than quantitative meta-analysis. Use of brief report templates and standardized production methods helps to speed final report publication.

CONCLUSIONS

Researchers conducting rapid reviews need to make transparent methodological choices, informed by stakeholder input, to ensure that rapid reviews meet their intended purpose. Transparency is critical because it is unclear how or how much streamlined methods can bias the conclusions of reviews. There are not yet internationally accepted standards for conducting or reporting rapid reviews. Thus, this article proposes interim guidance for researchers who are increasingly employing these methods.

A search of just four key databases would identify most randomized controlled trials of acupuncture: a meta-epidemiological study

INTRODUCTION

Little research has been conducted to assess which specific databases should be searched when performing a systematic review (SR) on acupuncture. The current study aimed to identify key databases and the optimal database combination to retrieve randomized controlled trials (RCTs) on acupuncture for inclusion in SRs.

METHODS

A systematic search for SRs in the field of acupuncture was conducted in order to identify target databases and RCTs were extracted from the SRs that had searched all target databases. The proportions of SRs that had achieved 100%, 95% or 90% recall of RCTs and the total recall of RCTs in various combinations of target databases were calculated. Sensitivity analysis was performed on those SRs that included 10 or more RCTs.

RESULTS

CNKI, WanFang, VIP, PubMed, CENTRAL and Embase were regarded as target databases. A total of 4349 acupuncture RCTs were extracted from 286 SRs. Searching all six target databases retrieved 99.3% of all RCTs while 99.1% were recalled by searching the combination of CNKI, WanFang, PubMed and CENTRAL. There were no significant differences on total recall of RCTs (p=0.549) or in the proportion of SRs with 90% recall of RCTs (97.2% vs. 97.6%; p=0.794) between searching the above four and the full six target databases. Most results were similar in the sensitivity analysis.

CONCLUSION

The combined retrieval power of CNKI, WanFang, PubMed and CENTRAL was considered an efficient choice to retrieve acupuncture RCTs included in SRs. This article is protected by copyright. All rights reserved.

Searching two or more databases decreased the risk of missing relevant studies: a metaresearch study

OBJECTIVE

Assessing changes in coverage, recall, review conclusions and references not found when searching fewer databases.

STUDY DESIGN AND SETTING

In 60 randomly selected Cochrane reviews, we checked included study publications' coverage (indexation) and recall (findability) using different search approaches with MEDLINE, Embase, and CENTRAL and related them to authors' conclusions and certainty. We assessed characteristics of unfound references.

RESULTS

1989/2080 included references, were indexed in ≥1 database (coverage=96%). In reviews where using one of our search approaches would not change conclusions and certainty (n=44-54), median coverage and recall were highest (range 87.9-100.0% and 78.2-93.3%, respectively). Here, searching ≥2 databases reached >95% coverage and ≥87.9% recall. In reviews with unchanged conclusions but less certainty (n=2-8): 63.3-79.3% coverage and 45.0-75.0% recall. In reviews with opposite conclusions (n=1-3): 63.3-96.6% and 52.1-78.7%. In reviews where a conclusion was no longer possible (n=3-7): 60.6%-86.0% and 20.0-53.8%. The 265 references that were indexed but unfound were more often abstractless (30% vs 11%) and older (28% vs. 17% published before 1991) than found references.

CONCLUSION

Searching ≥2 databases improves coverage and recall and decreases the risk of missing eligible studies. If researchers suspect that relevant articles are difficult to find, supplementary search methods should be used.

The role of hyperinsulinaemia in screening for prediabetes in the adolescent population: A systematic literature review

BACKGROUND/AIMS

Present screening methods for Type 2 diabetes (T2DM) fall short of detecting prediabetes. This paper summarises the literature on the utility of insulin measurements (hyperinsulinemia) in detecting prediabetes in adolescents.

METHODS

A systematic literature review was conducted using EMBASE and Medline. Relevant data on hyperinsulinemia in the adolescent population is narrated.

RESULTS

The database search identified 174 potential articles; 106 underwent a full-paper review, and 36 were included.

CONCLUSION

Elevated fasting insulin is a marker of impaired insulin resistance and pending beta-cell dysfunction in at-risk adolescents and can be an early indicator of prediabetes.

An overview of methodological approaches in systematic reviews

AIM

The aim of this overview is to identify and collate evidence from existing published systematic review (SR) articles evaluating various methodological approaches used at each stage of an SR.

METHODS

The search was conducted in five electronic databases from inception to November 2020 and updated in February 2022: MEDLINE, Embase, Web of Science Core Collection, Cochrane Database of Systematic Reviews, and APA PsycINFO. Title and abstract screening were performed in two stages by one reviewer, supported by a second reviewer. Full-text screening, data extraction, and quality appraisal were performed by two reviewers independently. The quality of the included SRs was assessed using the AMSTAR 2 checklist.

RESULTS

The search retrieved 41,556 unique citations, of which 9 SRs were deemed eligible for inclusion in final synthesis. Included SRs evaluated 24 unique methodological approaches used for defining the review scope and eligibility, literature search, screening, data extraction, and quality appraisal in the SR process. Limited evidence supports the following (a) searching multiple resources (electronic databases, handsearching, and reference lists) to identify relevant literature; (b) excluding non-English, gray, and unpublished literature, and (c) use of text-mining approaches during title and abstract screening.

CONCLUSION

The overview identified limited SR-level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process, as well as some methodological modifications currently used in expedited SRs. Overall, findings of this overview highlight the dearth of published SRs focused on SR methodologies and this warrants future work in this area.

Opportunities to improve reporting of rapid response in health technology assessment

Introduction

Mini health technology assessment (HTA) reports have been used to support policy makers and health systems by providing a timely summary of scientific evidence. The objective of this meta-epidemiologic study was to evaluate the quality of reporting of mini-HTA reports published in Brazil.

Methods

An electronic search for all mini-HTA reports published between 2014 and March 2019 was conducted in the SISREBRATS and CONITEC databases. The study selection and data extraction were performed by two independent assessors. The following data were extracted: bibliographic data; research question; characteristics of the population, health technologies and outcomes assessed; eligibility criteria; information about searches and study selection; risk of bias assessment; quality of evidence assessment; synthesis of results; and recommendation about the technology evaluated. A descriptive analysis was used to summarize the information retrieved from all the included mini-HTA reports.

Results

We included 103 mini-HTA reports, the great majority of which (92.3 percent) focused on the coverage of the technologies in the healthcare system, with more than 60 percent being about drugs. Only five mini-HTA reports (4.8 percent) gave reasons for the choice of outcomes, and fifteen (14.5 percent) discriminated between primary and secondary outcomes. All mini-HTAs reported the databases searched and 99 percent of them reported using Medline. Sixty percent of the mini-HTA reported assessing the risk of bias, and 52 percent reported assessing the quality of evidence.

Conclusion

The quality of reporting of the mini-HTA reports performed in Brazil is insufficient and needs to be improved to guarantee transparency and replicability.

Literature searching methods or guidance and their application to public health topics: A narrative review

Background

Information specialists conducting searches for systematic reviews need to consider key questions around which and how many sources to search. This is particularly important for public health topics where evidence may be found in diverse sources.

Objectives

The objective of this review is to give an overview of recent studies on information retrieval guidance and methods that could be applied to public health evidence and used to guide future searches.

Methods

A literature search was performed in core databases and supplemented by browsing health information journals and citation searching. Results were sifted and reviewed.

Results

Seventy‐two papers were found and grouped into themes covering sources and search techniques. Public health topics were poorly covered in this literature.

Discussion

Many researchers follow the recommendations to search multiple databases. The review topic influences decisions about sources. Additional sources covering grey literature eliminate bias but are time‐consuming and difficult to search systematically. Public health searching is complex, often requiring searches in multidisciplinary sources and using additional methods.

Conclusions

Search planning is advisable to enable decisions about which and how many sources to search. This could improve with more work on modelling search scenarios, particularly in public health topics, to examine where publications were found and guide future research.

School-based interventions to prevent anxiety, depression and conduct disorder in children and young people: a systematic review and network meta-analysis

Background

Schools in the UK increasingly have to respond to anxiety, depression and conduct disorder as key causes of morbidity in children and young people.

Objective

The objective was to assess the comparative effectiveness of educational setting-based interventions for the prevention of anxiety, depression and conduct disorder in children and young people.

Design

This study comprised a systematic review, a network meta-analysis and an economic evaluation.

Data sources

The databases MEDLINE, EMBASE™ (Elsevier, Amsterdam, the Netherlands), PsycInfo® (American Psychological Association, Washington, DC, USA) and Cochrane Central Register of Controlled Trials (CENTRAL) were searched to 4 April 2018, and the NHS Economic Evaluation Database (NHS EED) was searched on 22 May 2019 for economic evaluations. No language or date filters were applied.

Main outcomes

The main outcomes were post-intervention self-reported anxiety, depression or conduct disorder symptoms.

Review methods

Randomised/quasi-randomised trials of universal or targeted interventions for the prevention of anxiety, depression or conduct disorder in children and young people aged 4–18 years were included. Screening was conducted independently by two reviewers. Data extraction was conducted by one reviewer and checked by a second. Intervention- and component-level network meta-analyses were conducted in OpenBUGS. A review of the economic literature and a cost–consequence analysis were conducted.

Results

A total of 142 studies were included in the review, and 109 contributed to the network meta-analysis. Of the 109 studies, 57 were rated as having an unclear risk of bias for random sequence generation and allocation concealment. Heterogeneity was moderate. In universal secondary school settings, mindfulness/relaxation interventions [standardised mean difference (SMD) –0.65, 95% credible interval (CrI) –1.14 to –0.19] and cognitive–behavioural interventions (SMD –0.15, 95% CrI –0.34 to 0.04) may be effective for anxiety. Cognitive–behavioural interventions incorporating a psychoeducation component may be effective (SMD –0.30, 95% CrI –0.59 to –0.01) at preventing anxiety immediately post intervention. There was evidence that exercise was effective in preventing anxiety in targeted secondary school settings (SMD –0.47, 95% CrI –0.86 to –0.09). There was weak evidence that cognitive–behavioural interventions may prevent anxiety in universal (SMD –0.07, 95% CrI –0.23 to 0.05) and targeted (SMD –0.38, 95% CrI –0.84 to 0.07) primary school settings. There was weak evidence that cognitive–behavioural (SMD –0.04, 95% CrI –0.16 to 0.07) and cognitive–behavioural + interpersonal therapy (SMD –0.18, 95% CrI –0.46 to 0.08) may be effective in preventing depression in universal secondary school settings. Third-wave (SMD –0.35, 95% CrI –0.70 to 0.00) and cognitive–behavioural interventions (SMD –0.11, 95% CrI –0.28 to 0.05) incorporating a psychoeducation component may be effective at preventing depression immediately post intervention. There was no evidence of intervention effectiveness in targeted secondary, targeted primary or universal primary school settings post intervention. The results for university settings were unreliable because of inconsistency in the network meta-analysis. A narrative summary was reported for five conduct disorder prevention studies, all in primary school settings. None reported the primary outcome at the primary post-intervention time point. The economic evidence review reported heterogeneous findings from six studies. Taking the perspective of a single school budget and based on cognitive–behavioural therapy intervention costs in universal secondary school settings, the cost–consequence analysis estimated an intervention cost of £43 per student.

Limitations

The emphasis on disorder-specific prevention excluded broader mental health interventions and restricted the number of eligible conduct disorder prevention studies. Restricting the study to interventions delivered in the educational setting may have limited the number of eligible university-level interventions.

Conclusions

There was weak evidence of the effectiveness of school-based, disorder-specific prevention interventions, although effects were modest and the evidence not robust. Cognitive–behavioural therapy-based interventions may be more effective if they include a psychoeducation component.

Future work

Future trials for prevention of anxiety and depression should evaluate cognitive–behavioural interventions with and without a psychoeducation component, and include mindfulness/relaxation or exercise comparators, with sufficient follow-up. Cost implications must be adequately measured.

Study registration

This study is registered as PROSPERO CRD42016048184.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 9, No. 8. See the NIHR Journals Library website for further project information.

A Scoping Review and Content Analysis of Common Depressive Symptoms of Young People

School nurses are the most accessible health care providers for many young people including adolescents and young adults. Early identification of depression results in improved outcomes, but little information is available comprehensively describing depressive symptoms specific to this population. The aim of this study was to develop a taxonomy of depressive symptoms that were manifested and described by young people based on a scoping review and content analysis. Twenty-five journal articles that included narrative descriptions of depressive symptoms in young people were included. A total of 60 depressive symptoms were identified and categorized into five dimensions: behavioral (n = 8), cognitive (n = 14), emotional (n = 15), interpersonal (n = 13), and somatic (n = 10). This comprehensive depression symptom taxonomy can help school nurses to identify young people who may experience depression and will support future research to better screen for depression.

The decisions and processes involved in a systematic search strategy: a hierarchical framework

OBJECTIVE

The decisions and processes that may compose a systematic search strategy have not been formally identified and categorized. This study aimed to (1) identify all decisions that could be made and processes that could be used in a systematic search strategy and (2) create a hierarchical framework of those decisions and processes.

METHODS

The literature was searched for documents or guides on conducting a literature search for a systematic review or other evidence synthesis. The decisions or processes for locating studies were extracted from eligible documents and categorized into a structured hierarchical framework. Feedback from experts was sought to revise the framework. The framework was revised iteratively and tested using recently published literature on systematic searching.

RESULTS

Guidance documents were identified from expert organizations and a search of the literature and Internet. Data were extracted from 74 eligible documents to form the initial framework. The framework was revised based on feedback from 9 search experts and further review and testing by the authors. The hierarchical framework consists of 119 decisions or processes sorted into 17 categories and arranged under 5 topics. These topics are "Skill of the searcher," "Selecting information to identify," "Searching the literature electronically," "Other ways to identify studies," and "Updating the systematic review."

CONCLUSIONS

The work identifies and classifies the decisions and processes used in systematic searching. Future work can now focus on assessing and prioritizing research on the best methods for successfully identifying all eligible studies for a systematic review.

Overlaps of multiple database retrieval and citation tracking in dementia care research: a methodological study

OBJECTIVE

We aimed to determine overlaps and optimal combination of multiple database retrieval and citation tracking for evidence synthesis, based on a previously conducted scoping review on facilitators and barriers to implementing nurse-led interventions in dementia care.

METHODS

In our 2019 scoping review, we performed a comprehensive literature search in eight databases (CENTRAL, CINAHL, Embase, Emcare, MEDLINE, Ovid Nursing Database, PsycINFO, and Web of Science Core Collection) and used citation tracking. We retrospectively analyzed the coverage and overlap of 10,527 retrieved studies published between 2015 and 2019. To analyze database overlap, we used cross tables and multiple correspondence analysis (MCA).

RESULTS

Of the retrieved studies, 6,944 were duplicates and 3,583 were unique references. Using our search strategies, considerable overlaps can be found in some databases, such as between MEDLINE and Web of Science Core Collection or between CINAHL, Emcare, and PsycINFO. Searching MEDLINE, CINAHL, and Web of Science Core Collection and using citation tracking were necessary to retrieve all included studies of our scoping review.

CONCLUSIONS

Our results can contribute to enhancing future search practice related to database selection in dementia care research. However, due to limited generalizability, researchers and librarians should carefully choose databases based on the research question. More research on optimal database retrieval in dementia care research is required for the development of methodological standards.

Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews

OBJECTIVES

To develop methods guidance to support the conduct of rapid reviews (RRs) produced within Cochrane and beyond, in response to requests for timely evidence syntheses for decision-making purposes including urgent health issues of high priority.

STUDY DESIGN AND SETTING

Interim recommendations were informed by a scoping review of the underlying evidence, primary methods studies conducted, and a survey sent to 119 representatives from 20 Cochrane entities, who were asked to rate and rank RR methods across stages of review conduct. Discussions among those with expertise in RR methods further informed the list of recommendations with accompanying rationales provided.

RESULTS

Based on survey results from 63 respondents (53% response rate), 26 RR methods recommendations are presented for which there was a high or moderate level of agreement or scored highest in the absence of such agreement. Where possible, how recommendations align with Cochrane methods guidance for systematic reviews is highlighted.

CONCLUSION

The Cochrane Rapid Reviews Methods Group offers new, interim guidance to support the conduct of RRs. Because best practice is limited by the lack of currently available evidence for some RR methods shortcuts taken, this guidance will need to be updated as additional abbreviated methods are evaluated.

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and an expert survey

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in health-related systematic literature searching. Our study will be guided by the following research questions: What are the benefits of citation tracking for health-related systematic literature searching? Which perspectives and experiences do experts in the field of literature retrieval methods have with regard to citation tracking in health-related systematic literature searching? Methods: Our study will have two parts: a scoping review and an expert survey. The scoping review aims at identifying methodological studies on benefits or problems of citation tracking in health-related systematic literature searching with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE, The Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science Core Collection, two information science databases, and free web searching. Two reviewers will independently assess full texts of selected abstracts. We will conduct direct backward and forward citation tracking on included articles. The results of the scoping review will inform our expert survey through which we aim to learn about experts΄ perspectives and experiences. We will narratively synthesize the results and derive recommendations for performing health-related systematic reviews.

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and a Delphi study

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in systematic literature searching for health-related topics. Our study will be guided by the following research questions: What is the benefit of citation tracking for systematic literature searching for health-related topics? Which methods, citation indexes, and other tools are used for citation tracking? What terminology is used for citation tracking methods? Methods: Our study will have two parts: a scoping review and a Delphi study. The scoping review aims at identifying methodological studies on the benefit and use of citation tracking in systematic literature searching for health-related topics with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE (Ovid), CINAHL (EBSCOhost), Web of Science Core Collection, two information science databases, web searching, and contact experts in the field. Two reviewers will independently perform study selection. We will conduct direct backward and forward citation tracking on included articles. Data from included studies will be extracted using a prespecified extraction sheet and presented in both tabular and narrative form. The results of the scoping review will inform the subsequent Delphi study through which we aim to derive consensus recommendations for the future practice and research of citation tracking.

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and a Delphi study

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in systematic literature searching for health-related topics. Our study will be guided by the following research questions: What is the benefit of citation tracking for systematic literature searching for health-related topics? Which methods, citation indexes, and other tools are used for citation tracking? What terminology is used for citation tracking methods? Methods: Our study will have two parts: a scoping review and a Delphi study. The scoping review aims at identifying methodological studies on the benefit and use of citation tracking in systematic literature searching for health-related topics with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE (Ovid), CINAHL (EBSCOhost), Web of Science Core Collection, two information science databases, web searching, and contact experts in the field. Two reviewers will independently perform study selection. We will conduct direct backward and forward citation tracking on included articles. Data from included studies will be extracted using a prespecified extraction sheet and presented in both tabular and narrative form. The results of the scoping review will inform the subsequent Delphi study through which we aim to derive consensus recommendations for the future practice and research of citation tracking.

Rapid reviews: A critical perspective

BACKGROUND

The high scientific uncertainty of many far-reaching and serious political decisions during the "coronavirus crisis" underpins the enormous importance of having evidence syntheses that are quickly available and at the same time reliable. As these requirements can only be insufficiently fulfilled by systematic reviews due to the high amount of time required, abbreviated evidence syntheses in the form of rapid reviews are becoming increasingly popular.

PURPOSE

This commentary aims to enhance methodological and methodical discussions and research about abbreviated evidence syntheses.

METHODS

A selective literature search and evaluation focussing on research dealing with rapid reviews.

RESULTS

In rapid reviews, a wide variety of methods can be used to speed up the process of literature search and evaluation, while at the same time maintaining the principles of methodological quality and transparent reporting. But do rapid reviews currently keep what they promise? We discuss the increasing trend towards rapid reviews, giving the currently available evidence on the topic some critical reflection. Following this discussion, we will finally derive demands that go beyond the topic of rapid reviews alone.

Abbreviated and comprehensive literature searches led to identical or very similar effect estimates: a meta-epidemiological study

OBJECTIVES

The objective of this study was to assess the agreement of treatment effect estimates from meta-analyses based on abbreviated or comprehensive literature searches.

STUDY DESIGN AND SETTING

This was a meta-epidemiological study. We abbreviated 47 comprehensive Cochrane review searches and searched MEDLINE/Embase/CENTRAL alone, in combination, with/without checking references (658 new searches). We compared one meta-analysis from each review with recalculated ones based on abbreviated searches.

RESULTS

The 47 original meta-analyses included 444 trials (median 6 per review [interquartile range (IQR) 3-11]) with 360045 participants (median 1,371 per review [IQR 685-8,041]). Depending on the search approach, abbreviated searches led to identical effect estimates in 34-79% of meta-analyses, to different effect estimates with the same direction and level of statistical significance in 15-51%, and to opposite effects (or effects could not be estimated anymore) in 6-13%. The deviation of effect sizes was zero in 50% of the meta-analyses and in 75% not larger than 1.07-fold. Effect estimates of abbreviated searches were not consistently smaller or larger (median ratio of odds ratio 1 [IQR 1-1.01]) but more imprecise (1.02-1.06-fold larger standard errors).

CONCLUSION

Abbreviated literature searches often led to identical or very similar effect estimates as comprehensive searches with slightly increased confidence intervals. Relevant deviations may occur.

Asthma control factors in the Gulf Cooperation Council (GCC) countries and the effectiveness of ICS/LABA fixed dose combinations: a dual rapid literature review

Background

Asthma control is influenced by multiple factors. These factors must be considered when appraising asthma interventions and their effectiveness in the Gulf Cooperation Council (GCC) countries (Bahrain, Kuwait, Oman, Qatar, Saudi Arabia and United Arab Emirates [UAE]). Based on published studies, the most prevalent asthma treatment in these countries are fixed dose combinations (FDC) of inhaled corticosteroid and long-acting beta-agonist (ICS/LABA). This study is a rapid review of the literature on: (a) factors associated with asthma control in the GCC countries and (b) generalisability of ICS/LABA FDC effectiveness studies.

Methods

To review local factors associated with asthma control and, generalisability of published ICS/LABA FDC studies, two rapid reviews were conducted. Review 1 targeted literature pertaining to asthma control factors in GCC countries. Eligible studies were appraised, and clustering methodology used to summarise factors. Review 2 assessed ICS/LABA FDC studies in conditions close to actual clinical practice (i.e. effectiveness studies). Eligibility was determined by reviewing study characteristics. Evaluation of studies focused on randomised controlled trials (RCTs). In both reviews, initial (January 2018) and updated (November 2019) searches were conducted in EMBASE and PubMed databases. Eligible studies were appraised using the Critical Appraisal Skills Program (CASP) checklists.

Results

We identified 51 publications reporting factors associated with asthma control. These publications reported studies conducted in Saudi Arabia (35), Qatar (5), Kuwait (5), UAE (3), Oman (1) and multiple countries (2). The most common factors associated with asthma control were: asthma-related education (13 articles), demographics (11articles), comorbidities (11 articles) and environmental exposures (11 articles). Review 2 identified 61 articles reporting ICS/LABA FDC effectiveness studies from countries outside of the GCC. Of these, six RCTs were critically appraised. The adequacy of RCTs in informing clinical practice varied when appraised against previously published criteria.

Conclusions

Asthma-related education was the most recurring factor associated with asthma control in the GCC countries. Moreover, the generalisability of ICS/LABA FDC studies to this region is variable. Hence, asthma patients in the region, particularly those on ICS/LABA FDC, will continue to require physician review and oversight. While our findings provide evidence for local treatment guidelines, further research is required in GCC countries to establish the causal pathways through which asthma-related education influence asthma control for patients on ICS/LABA FDC therapy.

Combining abbreviated literature searches with single-reviewer screening: three case studies of rapid reviews

BACKGROUND

Decision-makers increasingly request rapid answers to clinical or public health questions. To save time, personnel, and financial resources, rapid reviews streamline the methodological steps of the systematic review process. We aimed to explore the validity of a rapid review approach that combines a substantially abbreviated literature search with a single-reviewer screening of abstracts and full texts using three case studies.

METHODS

We used a convenience sample of three ongoing Cochrane reviews as reference standards. Two reviews addressed oncological topics and one addressed a public health topic. For each of the three topics, three reviewers screened the literature independently. Our primary outcome was the change in conclusions between the rapid reviews and the respective Cochrane reviews. In case the rapid approach missed studies, we recalculated the meta-analyses for the main outcomes and asked Cochrane review authors if the new body of evidence would change their original conclusion compared with the reference standards. Additionally, we assessed the sensitivity of the rapid review approach compared with the results of the original Cochrane reviews.

RESULTS

For the two oncological topics (case studies 1 and 2), the three rapid reviews each yielded the same conclusions as the Cochrane reviews. However, the authors would have had less certainty about their conclusion in case study 2. For case study 3, the public health topic, only one of the three rapid reviews led to the same conclusion as the Cochrane review. The other two rapid reviews provided insufficient information for the authors to draw conclusions. Using the rapid review approach, the sensitivity was 100% (3 of 3) for case study 1. For case study 2, the three rapid reviews identified 40% (4 of 10), 50% (5 of 10), and 60% (6 of 10) of the included studies, respectively; for case study 3, the respective numbers were 38% (8 of 21), 43% (9 of 21), and 48% (10 of 21).

CONCLUSIONS

Within the limitations of these case studies, a rapid review approach that combines abbreviated literature searches with single-reviewer screening may be feasible for focused clinical questions. For complex public health topics, sensitivity seems to be insufficient.

Single-reviewer abstract screening missed 13 percent of relevant studies: a crowd-based, randomized controlled trial

OBJECTIVES

To determine the accuracy of single-reviewer screening in correctly classifying abstracts as relevant or irrelevant for literature reviews.

STUDY DESIGN AND SETTING

We conducted a crowd-based, parallel-group randomized controlled trial. Using the Cochrane Crowd platform, we randomly assigned eligible participants to 100 abstracts each of a pharmacological or a public health topic. After completing a training exercise, participants screened abstracts online based on predefined inclusion and exclusion criteria. We calculated sensitivities and specificities of single- and dual-reviewer screening using two published systematic reviews as reference standards.

RESULTS

Two hundred and eighty participants made 24,942 screening decisions on 2,000 randomly selected abstracts from the reference standard reviews. On average, each abstract was screened 12 times. Overall, single-reviewer abstract screening missed 13% of relevant studies (sensitivity: 86.6%; 95% confidence interval [CI], 80.6%-91.2%). By comparison, dual-reviewer abstract screening missed 3% of relevant studies (sensitivity: 97.5%; 95% CI, 95.1%-98.8%). The corresponding specificities were 79.2% (95% CI, 77.4%-80.9%) and 68.7% (95% CI, 66.4%-71.0%), respectively.

CONCLUSIONS

Single-reviewer abstract screening does not appear to fulfill the high methodological standards that decisionmakers expect from systematic reviews. It may be a viable option for rapid reviews, which deliberately lower methodological standards to provide decision makers with accelerated evidence synthesis products.

Excluding non-English publications from evidence-syntheses did not change conclusions: a meta-epidemiological study

OBJECTIVES

We aimed to assess whether limiting the inclusion criteria solely to English-language publications affected the overall conclusions of evidence syntheses.

STUDY DESIGN AND SETTING

Our analyses used a dataset of a previous methods study that included 59 randomly selected Cochrane intervention reviews with no language restrictions. First, we ascertained the publication language of all 2,026 included publications. Next, we excluded studies based on the following criteria: (1) publication solely in non-English language, or (2) main publication (in case of multiple publications of the same study) in non-English language. We then re-calculated meta-analyses for outcomes that were presented in the main summary of findings tables of the Cochrane reports. If the direction of the effect estimate or the statistical significance changed, authors of the respective Cochrane reviews were consulted to assess whether the new evidence base would have changed their conclusions. The primary outcome of our analyses examined the proportion of conclusions that would change with the exclusion of non-English publications. We set the threshold for the approach as noninferior if the upper limit of the 95% confidence interval of the proportion of changed conclusions did not cross a margin of 10%.

RESULTS

Across all 59 Cochrane reviews, 29 (49%) included 80 non-English publications. For 16 (27%) of these Cochrane reviews, the exclusion of non-English publications resulted in the exclusion of at least one study. In the remaining 13 Cochrane reviews, the non-English publications were not the only or main publication of the study or they did not contribute to the main summary of the findings table, so their exclusion did not result in an exclusion of the study. Overall, the exclusion of non-English publications led to the exclusion of 31 studies contributing to 40 outcomes. For 38 of the 40 outcomes, the exclusion of non-English studies did not markedly alter the size or direction of effect estimates or statistical significance. In two outcomes, the statistical significance changed, but authors would have still drawn the same conclusion, albeit with less certainty. Thus, the proportion of changed conclusions in our sample was 0.0% (95% CI 0.0-0.6), which indicated the noninferiority of the approach. However, the majority of excluded studies were small.

CONCLUSION

Exclusion of non-English publications from systematic reviews on clinical interventions had a minimal effect on overall conclusions and could be a viable methodological shortcut, especially for rapid reviews.

Measuring the impact of screening automation on meta-analyses of diagnostic test accuracy

BACKGROUND

The large and increasing number of new studies published each year is making literature identification in systematic reviews ever more time-consuming and costly. Technological assistance has been suggested as an alternative to the conventional, manual study identification to mitigate the cost, but previous literature has mainly evaluated methods in terms of recall (search sensitivity) and workload reduction. There is a need to also evaluate whether screening prioritization methods leads to the same results and conclusions as exhaustive manual screening. In this study, we examined the impact of one screening prioritization method based on active learning on sensitivity and specificity estimates in systematic reviews of diagnostic test accuracy.

METHODS

We simulated the screening process in 48 Cochrane reviews of diagnostic test accuracy and re-run 400 meta-analyses based on a least 3 studies. We compared screening prioritization (with technological assistance) and screening in randomized order (standard practice without technology assistance). We examined if the screening could have been stopped before identifying all relevant studies while still producing reliable summary estimates. For all meta-analyses, we also examined the relationship between the number of relevant studies and the reliability of the final estimates.

RESULTS

The main meta-analysis in each systematic review could have been performed after screening an average of 30% of the candidate articles (range 0.07 to 100%). No systematic review would have required screening more than 2308 studies, whereas manual screening would have required screening up to 43,363 studies. Despite an average 70% recall, the estimation error would have been 1.3% on average, compared to an average 2% estimation error expected when replicating summary estimate calculations.

CONCLUSION

Screening prioritization coupled with stopping criteria in diagnostic test accuracy reviews can reliably detect when the screening process has identified a sufficient number of studies to perform the main meta-analysis with an accuracy within pre-specified tolerance limits. However, many of the systematic reviews did not identify a sufficient number of studies that the meta-analyses were accurate within a 2% limit even with exhaustive manual screening, i.e., using current practice.

[Increasing the efficiency of guideline production: a narrative review]

BACKGROUND

The development of high-quality clinical practice guidelines is laborious and time-consuming. New methods have become available to streamline this process. However, the awareness of these methods should be improved.

METHODS

Selective literature search in PubMed/MEDLINE und Embase.

RESULTS

Simple tools such as surveys or voting systems can facilitate the organization, planning and communication. Adequate methods should be used to prioritize all potential questions that should be addressed in the guideline. Published or ongoing international guidelines and systematic reviews can be used meaningfully for the planned guideline. In the case of guideline updates, it should be determined whether all parts of the guideline require an update of the evidence. The need for an update should be investigated. The concept of living guidelines has the biggest potential to provide gains in efficiency. Living guidelines are continuously updated based on new evidence instead of being regularly updated at a predefined time.

CONCLUSIONS

New methods allowing for more efficient guideline production have been developed and, in part, already been introduced. Before starting with the production of a guideline (or its update), the potential advantages and disadvantages/risks of the corresponding methods should be balanced.

Challenges of rapid reviews for diagnostic test accuracy questions: a protocol for an international survey and expert consultation

BACKGROUND

Assessment of diagnostic tests, broadly defined as any element that aids in the collection of additional information for further clarification of a patient's health status, has increasingly become a critical issue in health policy and decision-making. Diagnostic evidence, including the accuracy of a medical test for a target condition, is commonly appraised using standard systematic review methodology. Owing to the considerable time and resources required to conduct these, rapid reviews have emerged as a pragmatic alternative by tailoring methods according to the decision maker's circumstances. However, it is not known if streamlining methodological aspects has an impact on the validity of evidence synthesis. Furthermore, due to the particular nature and complexity of the appraisal of diagnostic accuracy, there is need for detailed guidance on how to conduct rapid reviews of diagnostic tests. In this study, we aim to identify the methods currently used by rapid review developers to synthesize evidence on diagnostic test accuracy, as well as to analyze potential shortcomings and challenges related to these methods.

METHODS

We will carry out a two-fold approach: (1) an international survey of professionals working in organizations that develop rapid reviews of diagnostic tests, in terms of the methods and resources used by these agencies when conducting rapid reviews, and (2) semi-structured interviews with senior-level individuals to further explore and validate the findings from the survey and to identify challenges in conducting rapid reviews. We will use STATA 15.0 for quantitative analyses and framework analysis for qualitative analyses. We will ensure protection of data during all stages.

DISCUSSION

The main result of this research will be a map of methods and resources currently used for conducting rapid reviews of diagnostic test accuracy, as well as methodological shortcomings and potential solutions in diagnostic knowledge synthesis that require further research.

Rapid reviews may produce different results to systematic reviews: a meta-epidemiological study

Objective

To simulate possible changes in systematic review results if rapid review methods were used.

Study Design and Setting

We recalculated meta-analyses for binary primary outcomes in Cochrane systematic reviews, simulating rapid review methods. We simulated searching only PubMed, excluding older articles (5, 7, 10, 15, and 20 years before the search date), excluding smaller trials (<50, <100, and <200 participants), and using the largest trial only. We examined percentage changes in pooled odds ratios (ORs) (classed as no important change [<5%], small [<20%], moderate [<30%], or large [≥30%]), statistical significance, and biases observed using rapid methods.

Results

Two thousand five hundred and twelve systematic reviews (16,088 studies) were included. Rapid methods resulted in the loss of all data in 3.7–44.7% of meta-analyses. Searching only PubMed had the smallest risk of changed ORs (19% [477/2,512] were small changes or greater; 10% [260/2,512] were moderate or greater). Changes in ORs varied substantially with each rapid review method; 8.4–21.3% were small, 1.9–8.8% were moderate, and 4.7–34.1% were large. Changes in statistical significance occurred in 6.5–38.6% of meta-analyses. Changes from significant to nonsignificant were most common (2.1–13.7% meta-analyses). We found no evidence of bias with any rapid review method.

Conclusion

Searching PubMed only might be considered where a ∼10% risk of the primary outcome OR changing by >20% could be tolerated. This could be the case in scoping reviews, resource limitation, or where syntheses are needed urgently. Other situations, such as clinical guidelines and regulatory decisions, favor more comprehensive systematic review methods.

Dexmedetomidine in prevention and treatment of postoperative and intensive care unit delirium: a systematic review and meta-analysis

BACKGROUND

To determine the preventive and therapeutic effect of dexmedetomidine on intensive care unit (ICU) delirium.

METHODS

The literature search using PubMed and the Cochrane Central Register of Controlled Trials was performed (August 1, 2018) to detect all randomized controlled trials (RCTs) of adult ICU patients receiving dexmedetomidine. Articles were included if they assessed the influence of dexmedetomidine compared to a sedative agent on incidence of ICU delirium or treatment of this syndrome. Accordingly, relevant articles were allocated to the following two groups: (1) articles that assessed the delirium incidence (incidence comparison) or articles that assessed the treatment of delirium (treatment comparison). Incidence of delirium and delirium resolution were the primary outcomes. We combined treatment effects comparing dexmedetomidine versus (1) placebo, (2) standard sedatives, and (3) opioids in random-effects meta-analyses. Risk of bias for each included RCT was assessed following Cochrane standards.

RESULTS

The literature search resulted in 28 articles (25 articles/4975 patients for the incidence comparison and three articles/166 patients for the treatment comparison). In the incidence comparison, heterogeneity was present in different subgroups. Administration of dexmedetomidine was associated with significantly lower overall incidence of delirium when compared to placebo (RR 0.52; 95% CI 0.39-0.70; I² = 37%), standard sedatives (RR 0.63; 95% CI 0.46-0.86; I² = 69%), as well as to opioids (RR 0.61; 95% CI 0.44-0.83; I² = 0%). Use of dexmedetomidine significantly increased the risks of bradycardia and hypotension. Limited data were available on circulatory insufficiency and mortality. In the treatment comparison, the comparison drugs in the three RCTs were placebo, midazolam, and haloperidol. The resolution of delirium was measured differently in each study. Two out of the three studies indicated clear favorable effects for dexmedetomidine (i.e., compared to placebo and midazolam). The study comparing dexmedetomidine with haloperidol was a pilot study (n = 20) with high variability in the results.

CONCLUSION

Findings suggest that dexmedetomidine reduces incidence and duration of ICU delirium. Furthermore, our systematic searches show that there is limited evidence if a delirium shall be treated with dexmedetomidine.

Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies

BACKGROUND

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving readers clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence. Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before. The purpose of this review is to determine if a shared model of the literature searching process can be detected across systematic review guidance documents and, if so, how this process is reported in the guidance and supported by published studies.

METHOD

A literature review. Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks. Published studies were identified through 'pearl growing', citation chasing, a search of PubMed using the systematic review methods filter, and the authors' topic knowledge. The relevant sections within each guidance document were then read and re-read, with the aim of determining key methodological stages. Methodological stages were identified and defined. This data was reviewed to identify agreements and areas of unique guidance between guidance documents. Consensus across multiple guidance documents was used to inform selection of 'key stages' in the process of literature searching.

RESULTS

Eight key stages were determined relating specifically to literature searching in systematic reviews. They were: who should literature search, aims and purpose of literature searching, preparation, the search strategy, searching databases, supplementary searching, managing references and reporting the search process.

CONCLUSIONS

Eight key stages to the process of literature searching in systematic reviews were identified. These key stages are consistently reported in the nine guidance documents, suggesting consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews. Further research to determine the suitability of using the same process of literature searching for all types of systematic review is indicated.