Identifying evidence for public health guidance: a comparison of citation searching with Web of Science and Google Scholar

Beyond Google Scholar, Scopus, and Web of Science: An evaluation of the backward and forward citation coverage of 59 databases' citation indices

Citation indices providing information on backward citation (BWC) and forward citation (FWC) links are essential for literature discovery, bibliographic analysis, and knowledge synthesis, especially when language barriers impede document identification. However, the suitability of citation indices varies. While some have been analyzed, the majority, whether new or established, lack comprehensive evaluation. Therefore, this study evaluates the citation coverage of the citation indices of 59 databases, encompassing the widely used Google Scholar, Scopus, and Web of Science alongside many others never previously analyzed, such as the emerging Lens, Scite, Dimensions, and OpenAlex or the subject-specific PubMed and JSTOR. Through a comprehensive analysis using 259 journal articles from across disciplines, this research aims to guide scholars in selecting indices with broader document coverage and more accurate and comprehensive backward and forward citation links. Key findings highlight Google Scholar, ResearchGate, Semantic Scholar, and Lens as leading options for FWC searching, with Lens providing superior download capabilities. For BWC searching, the Web of Science Core Collection can be recommended over Scopus for accuracy. BWC information from publisher databases such as IEEE Xplore or ScienceDirect was generally found to be the most accurate, yet only available for a limited number of articles. The findings will help scholars conducting systematic reviews, meta-analyses, and bibliometric analyses to select the most suitable databases for citation searching.

Expert searchers identified time, team, technology and tension as challenges when carrying out supplementary searches for systematic reviews: A thematic network analysis

BACKGROUND

Systematic reviews require detailed planning of complex processes which can present logistical challenges. Understanding these logistical challenges can help with planning and execution of tasks OBJECTIVES: To describe the perspectives of expert searchers on the main logistical challenges when carrying out supplementary searches for systematic reviews, in particular, forward citation searching and web searching.

METHODS

Qualitative interviews were undertaken with 15 experts on searching for studies for systematic reviews (e.g. information specialists) working in health and social care research settings. Interviews were undertaken by video-call between September 2020 and June 2021. Data analysis used thematic network analysis.

RESULTS

We identified three logistical challenges of using forward citation searching and web searching which were organised under the global theme of 'tension': time, team and technology. Several subthemes were identified which supported the organising themes, including allocating time, justifying time and keeping to time; reviewer expectations and contact with review teams; and access to resources and reference management.

CONCLUSION

Forward citation searching and web searching are logistically challenging search methods for a systematic review. An understanding of these challenges should encourage expert searchers and review teams to maintain open channels of communication, which should also facilitate improved working relationships.

Efficient searching for NICE public health guidelines: Would using fewer sources still find the evidence?

Systematic searches are integral to identifying the evidence that is used in National Institute for Health and Care Excellence (NICE) public health guidelines (PHGs). This study analyses the sources, including bibliographic databases and other techniques, required for PHGs. The aims were to analyse the sources used to identify the publications included in NICE PHGs; and to assess whether fewer sources could have been searched to retrieve these publications. Data showing how the included publications had been identified was collated using search summary tables. Three scenarios were created to test various combinations of sources to determine whether fewer sources could have been used. The sample included 29 evidence reviews, compiled using 13 searches, to support 10 PHG topics. Across the PHGs, 23 databases and six other techniques retrieved included publications. A mean reduction in total results of 6.5% could have been made if the minimum set of sources plus Cochrane Library, Embase, and MEDLINE were searched. On average, Cochrane Library, Embase, and MEDLINE contributed 76.8% of the included publications, with other databases adding 11% and other techniques 12.2%. None of the searches had a minimum set that was comprised entirely of databases. There was not a core set of sources for PHGs. A range of databases and techniques, covering a multi-disciplinary evidence base, was required to identify all included publications. It would be possible to reduce the number of sources searched and make some gains in productivity. It is important to create a tailored set of sources to do an efficient search.

A Tailored Approach: A model for literature searching in complex systematic reviews

Our previous work identified that nine leading guidance documents for seven different types of systematic review advocated the same process of literature searching. We defined and illustrated this process and we named it ‘the Conventional Approach’. The Conventional Approach appears to meet the needs of researchers undertaking literature searches for systematic reviews of clinical interventions. In this article, we report a new and alternate process model of literature searching called ‘A Tailored Approach’. A Tailored Approach is indicated as a search process for complex reviews which do not focus on the evaluation of clinical interventions. The aims of this article are to (1) explain the rationale for, and the theories behind, the design of A Tailored Approach; (2) report the current conceptual illustration of A Tailored Approach and to describe a user’s interaction with the process model; and (3) situate the elements novel to A Tailored Approach (when compared with the Conventional Approach) in the relevant literature. A Tailored Approach suggests investing time at the start of a review, to develop the information needs from the research objectives, and to tailor the search approach to studies or data. Tailored Approaches should be led by the information specialist (librarian) but developed by the research team. The aim is not necessarily to focus on comprehensive retrieval. Further research is indicated to evaluate the use of supplementary search methods, methods of team-working to define search approaches, and to evaluate the use of conceptual models of information retrieval for testing and evaluation.

citationchaser: a tool for transparent and efficient forward and backward citation chasing in systematic searching

Systematic searching aims to find all possibly relevant research from multiple sources, the basis for an unbiased and comprehensive evidence base. Along with bibliographic databases, systematic reviewers use a variety of additional methods to minimise procedural bias. Citation chasing exploits connections between research articles to identify relevant records for a review by making use of explicit mentions of one article within another. Citation chasing is a popular supplementary search method because it helps to build on the work of primary research and review authors. It does so by identifying potentially relevant studies that might otherwise not be retrieved by other search methods; for example, because they did not use the review authors' search terms in the specified combinations in their titles, abstracts, or keywords. Here, we briefly provide an overview of citation chasing as a method for systematic reviews. Furthermore, given the challenges and high resource requirements associated with citation chasing, the limited application of citation chasing in otherwise rigorous systematic reviews, and the potential benefit of identifying terminologically disconnected but semantically linked research studies, we have developed and describe a free and open source tool that allows for rapid forward and backward citation chasing. We introduce citationchaser, an R package and Shiny app for conducting forward and backward citation chasing from a starting set of articles. We describe the sources of data, the backend code functionality, and the user interface provided in the Shiny app. This article is protected by copyright. All rights reserved.

Thirty Years of Implementing the Photovoice Method: Insights From a Review of Reviews

As the use of the photovoice method has proliferated over the past three decades, several literature reviews have been conducted about implementing the method in the context of health promotion research and practice. Challenges emerged from the literature frequently enough to be reported in several reviews. As such, the purpose of this "review of reviews" was to identify and describe the various challenges to implementing photovoice as reported in published literature reviews. Taken together, the sum of the reviews' critiques about photovoice may be of particular use for developing solutions regarding challenges that limit the utility and outcomes of the method. The literature reviews included in this study were identified using PubMed, CINAHL, Google Scholar, and backward/forward chaining. The final 15 review articles included in the study were analyzed for the challenges of implementing photovoice described in each review. Four major themes emerged from the reviews: (1) inconsistent adherence to the photovoice method, (2) inconsistent evaluation of photovoice outcomes and impacts, (3) implementation challenges with specific populations, and (4) inconsistent reporting and adherence to ethical procedures. For those conducting photovoice projects in the future, the implications of the findings include maximizing participant engagement in the project from start to finish, evaluating photovoice projects for effectiveness in producing community change, and reporting ethical procedures.

Literature searching methods or guidance and their application to public health topics: A narrative review

Background

Information specialists conducting searches for systematic reviews need to consider key questions around which and how many sources to search. This is particularly important for public health topics where evidence may be found in diverse sources.

Objectives

The objective of this review is to give an overview of recent studies on information retrieval guidance and methods that could be applied to public health evidence and used to guide future searches.

Methods

A literature search was performed in core databases and supplemented by browsing health information journals and citation searching. Results were sifted and reviewed.

Results

Seventy‐two papers were found and grouped into themes covering sources and search techniques. Public health topics were poorly covered in this literature.

Discussion

Many researchers follow the recommendations to search multiple databases. The review topic influences decisions about sources. Additional sources covering grey literature eliminate bias but are time‐consuming and difficult to search systematically. Public health searching is complex, often requiring searches in multidisciplinary sources and using additional methods.

Conclusions

Search planning is advisable to enable decisions about which and how many sources to search. This could improve with more work on modelling search scenarios, particularly in public health topics, to examine where publications were found and guide future research.

Short-term physical health effects of sleep disruptions attributed to the acute hospital environment: a systematic review

The sleep disruption experienced by patients admitted to hospital may have a negative effect on health but the nature and magnitude of the effect has not been conclusively outlined. The review was designed to examine the impact of sleep disruption associated with being a hospital inpatient, on short-term physical health outcomes in adult patients. Searches comprised journal databases, gray literature sources, and backward and forward citation searching. Two reviewers independently screened the records. Original studies of adult hospitalized patients' sleep were included if physical outcomes were also measured. Interventional studies were excluded. The methodological quality was assessed independently by 2 reviewers using CASP checklists. Sleep assessment measures and results, physical outcomes and contextual data were extracted. Results were synthesized according to frequently reported outcomes: delirium, pain intensity, physical strength, and respiratory function. A meta-analysis was not performed; studies were heterogeneous and reporting was limited. Of 9919 retrieved records, 26 published studies were included (published: 2001-2020). Risk of bias was moderately high. Confounding factors were poorly reported. Total sleep time was either normal or reduced. Sleep was disrupted: arousal indices were high (mean: 0 5-21/h); slow wave sleep proportions were limited. Subjective sleep quality was poor. The association between sleep reduction or disruption and short-term health outcomes was negative, mixed or equivocal and included increased delirium, higher pain intensity, poorer strength, and adverse respiratory function. The impact of sleep disruption on outcomes for hospitalized patients is not well defined.

What you see depends on where you sit: The effect of geographical location on web-searching for systematic reviews: A case study

There is limited guidance on how to web-search in systematic reviews and concern relates to the reproducibility of searches using search engines such as Google. The aim of this paper is to address one potential source of variation in Google searches: does the geographical location of a researcher affect Google search returns? Using a virtual private network, we ran the same web-search for the medical technology Dasatinib in 12 different countries. Two researchers independently extracted the search returns by country organised by page rank. We compared: C1. any difference in the items returned by Google searches between countries and C2. any difference in the page rank of items returned between countries. Searches were undertaken on Monday September 28th 2020. From 12 countries, 43 items were identified. For C1: 19 items were common to all 12 countries. Twenty-four items were missed by searches in some countries. This means that there were differences in search returns between countries. For C2: a randomised trial reported by Raddich et al was the first search return for all countries. All other items, common to all countries, varied in their page-rank. We find that geographic location would appear to influence Google search returns based on the findings of this case study. The findings suggest that recording the location of the researcher undertaking web-searching may now be an important factor to report alongside detail on steps taken to minimise personalisation of web-searches covered by recent guidance. This finding also has implications for stopping-rules.

A technical review of three clinical trials register resources indicates where improvements to the search interfaces are needed

Clinical trials registers form an important part of the search for studies in systematic reviews of intervention effectiveness but the search interfaces and functionality of registers can be challenging to search systematically and resource intensive to search well. We report a technical review of the search interfaces of three leading trials register resources: ClinicalTrials.gov, the EU Clinical Trials Register and the WHO International Clinical Trials Registers Platform. The technical review used a validated checklist to identify areas where the search interfaces of these trials register resources performed well, where performance was adequate, where performance was poor, and to identify differences between search interfaces. The review found low overall scores for each of the interfaces (ClinicalTrials.gov 55/165, the EU Clinical Trials Register 25/165, the WHO International Clinical Trials Registers Platform 32/165). This finding suggests a need for joined-up dialogue between the producers of the registers and researchers who search them via these interfaces. We also set out a series of four proposed changes which might improve the search interfaces. Trials registers are an invaluable resource in systematic reviews of intervention effectiveness. With the continued growth in systematic reviews, and initiatives such as 'AllTrials', there is an anticipated need for these resources. We conclude that small changes to the search interfaces, and improved dialogue with providers, might improve the future search functionality of these valuable resources.

Searching for Programme theories for a realist evaluation: a case study comparing an academic database search and a simple Google search

BACKGROUND

Realist methodologies are increasingly being used to evaluate complex interventions in health and social care. Programme theory (ideas and assumptions of how a particular intervention works) development is the first step in a realist evaluation or a realist synthesis, with literature reviews providing important evidence to support this. Deciding how to search for programme theories is challenging and there is limited guidance available. Using an example of identifying programme theories for a realist evaluation of Pressure Ulcer Risk Assessment Instruments in clinical practice, the authors explore and compare several different approaches to literature searching and highlight important methodological considerations for those embarking on a programme theory review.

METHODS

We compared the performance of an academic database search with a simple Google search and developed an optimised search strategy for the identification primary references (i.e. documents providing the clearest examples of programme theories) associated with the use of Pressure Ulcer Risk Assessment Instruments (PU-RAIs). We identified the number of primary references and the total number of references retrieved per source. We then calculated the number needed to read (NNR) expressed as the total number of titles and abstracts screened to identify one relevant reference from each source.

RESULTS

The academic database search (comprising CINAHL, The Cochrane Library, EMBASE, HMIC, Medline) identified 2 /10 primary references with a NNR of 1395.The Google search identified 7/10 primary references with a NNR of 10.1. The combined NNR was 286.3. The optimised search combining Google and CINAHL identified 10/10 primary references with a NNR of 40.2.

CONCLUSION

The striking difference between the efficiency of the review's academic database and Google searches in finding relevant references prompted an in-depth comparison of the two types of search. The findings indicate the importance of including grey literature sources such as Google in this particular programme theory search, while acknowledging the need for transparency of methods. Further research is needed to facilitate improved guidance for programme theory searches to enhance practice in the realist field and to save researcher time and therefore resource.

Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources

Rigorous evidence identification is essential for systematic reviews and meta-analyses (evidence syntheses) because the sample selection of relevant studies determines a review's outcome, validity, and explanatory power. Yet, the search systems allowing access to this evidence provide varying levels of precision, recall, and reproducibility and also demand different levels of effort. To date, it remains unclear which search systems are most appropriate for evidence synthesis and why. Advice on which search engines and bibliographic databases to choose for systematic searches is limited and lacking systematic, empirical performance assessments. This study investigates and compares the systematic search qualities of 28 widely used academic search systems, including Google Scholar, PubMed, and Web of Science. A novel, query-based method tests how well users are able to interact and retrieve records with each system. The study is the first to show the extent to which search systems can effectively and efficiently perform (Boolean) searches with regards to precision, recall, and reproducibility. We found substantial differences in the performance of search systems, meaning that their usability in systematic searches varies. Indeed, only half of the search systems analyzed and only a few Open Access databases can be recommended for evidence syntheses without adding substantial caveats. Particularly, our findings demonstrate why Google Scholar is inappropriate as principal search system. We call for database owners to recognize the requirements of evidence synthesis and for academic journals to reassess quality requirements for systematic reviews. Our findings aim to support researchers in conducting better searches for better evidence synthesis.

Resources for forwards citation searching for implementation studies in dementia care: A case study comparing Web of Science and Scopus

BACKGROUND

Forwards citation searching is a valuable method for finding relevant studies in reviews where concepts are not clearly defined. Scopus and Web of Science can both be used for forwards citation searching but there is little evidence comparing the resources for this purpose.

METHOD

104 source records relevant to a scoping review of implementation and dissemination strategies in dementia care were checked for inclusion on Web of Science and Scopus. The number of citing references was recorded. Where citing references appeared unique to one of the resources, they were checked for inclusion on the other resource to assess the performance of citation links.

RESULTS

1397 citing references were returned by Scopus and 1010 were returned by Web of Science. For the unique citing references returned by Web of Science (n = 52), 36 were subsequently found to be on Scopus but had failed to be picked up as citing a source record. Of the unique citing references returned by Scopus (n = 355), 83 were found to be on Web of Science but had failed to be picked up as citing a source record. 26 additional relevant records were identified for the review by forwards citation searching. All were found on Scopus; six would have been missed by searching Web of Science alone.

CONCLUSION

Citation searching using Scopus alone would have found all additional relevant studies for the review. Both Scopus and Web of Science failed to return all citing references from the source records, even where they were present on the database, indicating poor links between citations.

Burn injuries caused by e-cigarette explosions: A systematic review of published cases

INTRODUCTION

E-cigarettes have the potential to cause burns from batteries that explode. Although e-cigarette explosion burns have been reported by the media (e.g. local online news, blogs), there is a need for a comprehensive review of published medical case reports regarding these injuries.

METHODS

CINAHL and PubMed were systematically searched using common terms regarding e-cigarettes (electronic cigarette, e-cigarette, vape, vaping, electronic nicotine delivery systems) in every combination with the term ‘explosion’. Peer-reviewed articles were included if they: were written in English, described case reports of burn injuries caused by e-cigarette explosions, and were published in any year. Cases were categorized by demographics, location of the e-cigarette explosion, burned body areas, types of burns, total body surface area of burns, the need for skin grafting, and the length of hospital stay.

RESULTS

Thirty-one articles were included in the review and described 164 cases. Most patients (90%) were male and between 20 to 29 years old. In the majority of cases (65%), e-cigarettes exploded in pockets, compared to exploding in the face or hand. Common burned areas included the thigh, hand, genitals, and face. Burn severity was typically second-degree burns (35%) or a combination of second-degree and third-degree burns (20%). In all, 48 patients required skin grafting, with 19 reporting a median hospital stay of 5 days.

CONCLUSIONS

This review has several implications, including the need for regulation of batteries, education regarding battery safety, and leveraging images of the severity of e-cigarette explosion burns to discourage the use of e-cigarettes.

Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies

BACKGROUND

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving readers clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence. Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before. The purpose of this review is to determine if a shared model of the literature searching process can be detected across systematic review guidance documents and, if so, how this process is reported in the guidance and supported by published studies.

METHOD

A literature review. Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks. Published studies were identified through 'pearl growing', citation chasing, a search of PubMed using the systematic review methods filter, and the authors' topic knowledge. The relevant sections within each guidance document were then read and re-read, with the aim of determining key methodological stages. Methodological stages were identified and defined. This data was reviewed to identify agreements and areas of unique guidance between guidance documents. Consensus across multiple guidance documents was used to inform selection of 'key stages' in the process of literature searching.

RESULTS

Eight key stages were determined relating specifically to literature searching in systematic reviews. They were: who should literature search, aims and purpose of literature searching, preparation, the search strategy, searching databases, supplementary searching, managing references and reporting the search process.

CONCLUSIONS

Eight key stages to the process of literature searching in systematic reviews were identified. These key stages are consistently reported in the nine guidance documents, suggesting consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews. Further research to determine the suitability of using the same process of literature searching for all types of systematic review is indicated.

Disclosure of domestic violence and sexual assault within the context of abortion: meta-ethnographic synthesis of qualitative studies protocol

BACKGROUND

One third of women will have an abortion in their lifetime (Kerr, QUT Law Rev 14:15, 2014; Aston and Bewley, Obstetrician & Gynaecologist 11:163-8, 2009). These women are more likely to have experienced domestic violence or sexual assault than women who continue with their pregnancies. Frontline health personnel involved in the care of women seeking abortions are uniquely positioned to support patients who choose to disclose their violence. Yet, the disclosure of domestic violence or sexual assault within the context of abortion is not well understood. To enhance service provision, it is important to understand the disclosure experience, that is, how frontline health personnel manage such disclosures and how victims/survivors perceive this experience. This review aims to provide a systematic synthesis of qualitative literature to increase understanding of the phenomena and identify research gaps.

METHODS

A meta-ethnography of qualitative evidence following PRISMA-P recommendations for reporting systematic reviews will be performed to better understand the experiences of domestic violence and sexual assault disclosure from the perspective of frontline health personnel providing support and women seeking an abortion. A three-stage search strategy including database searching, citation searching and Traditional Pearl Growing will be applied starting with the terms "domestic violence", "sexual assault", "disclosure" and "abortion", their common synonyms and MeSH terms. The database search will include CINAHL, MEDLINE, Embase and PsycINFO. Published studies from 1970, written in English and from all countries will be included. Two reviewers will screen titles and abstracts and if suitable will then perform a full-text review. To attribute weight to each study, two reviewers will perform the critical appraisal using a modified version of the "Guidelines for Extracting Data and Quality Assessing Primary Studies in Educational Research". Data extraction and coding will occur using EPPI-Reviewer 4 and will be carried out by two reviewers.

DISCUSSION

The reviewers will illuminate what transpires at the interface when women seeking an abortion in the context of domestic violence and sexual assault meet frontline health personnel. Increased knowledge in this area will improve the frontline health personnel's practices and responsiveness to women who seek out healthcare in the context of violence.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42016051136.

Optimal literature search for systematic reviews in surgery

BACKGROUND

The aim of the present study was to determine empirically which electronic databases contribute best to a literature search in surgical systematic reviews.

METHODS

For ten published systematic reviews, the systematic literature searches were repeated in the databases MEDLINE, Web of Science, CENTRAL, and EMBASE. On the basis of these reviews, a gold standard set of eligible articles was created. Recall (%), precision (%), unique contribution (%), and numbers needed to read (NNR) were calculated for each database, as well as for searches of citing references and of the reference lists of related systematic reviews (hand search).

RESULTS

CENTRAL yielded the highest recall (88.4%) and precision (8.3%) for randomized controlled trials (RCT), MEDLINE for non-randomized studies (NRS; recall 92.6%, precision 5.2%). The most effective combination of two databases plus hand searching for RCT was MEDLINE/CENTRAL (98.6% recall, NNR 97). Adding EMBASE marginally increased the recall to 99.3%, but with an NNR of 152. For NRS, the most effective combination was MEDLINE/Web of Science (99.5% recall, NNR 60).

CONCLUSIONS

For surgical systematic reviews, the optimal literature search for RCT employs MEDLINE and CENTRAL. For surgical systematic reviews of NRS, Web of Science instead of CENTRAL should be searched. EMBASE does not contribute substantially to reviews with a surgical intervention.

A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: a methodological review

BACKGROUND

The purpose and contribution of supplementary search methods in systematic reviews is increasingly acknowledged. Numerous studies have demonstrated their potential in identifying studies or study data that would have been missed by bibliographic database searching alone. What is less certain is how supplementary search methods actually work, how they are applied, and the consequent advantages, disadvantages and resource implications of each search method. The aim of this study is to compare current practice in using supplementary search methods with methodological guidance.

METHODS

Four methodological handbooks in informing systematic review practice in the UK were read and audited to establish current methodological guidance. Studies evaluating the use of supplementary search methods were identified by searching five bibliographic databases. Studies were included if they (1) reported practical application of a supplementary search method (descriptive) or (2) examined the utility of a supplementary search method (analytical) or (3) identified/explored factors that impact on the utility of a supplementary method, when applied in practice.

RESULTS

Thirty-five studies were included in this review in addition to the four methodological handbooks. Studies were published between 1989 and 2016, and dates of publication of the handbooks ranged from 1994 to 2014. Five supplementary search methods were reviewed: contacting study authors, citation chasing, handsearching, searching trial registers and web searching.

CONCLUSIONS

There is reasonable consistency between recommended best practice (handbooks) and current practice (methodological studies) as it relates to the application of supplementary search methods. The methodological studies provide useful information on the effectiveness of the supplementary search methods, often seeking to evaluate aspects of the method to improve effectiveness or efficiency. In this way, the studies advance the understanding of the supplementary search methods. Further research is required, however, so that a rational choice can be made about which supplementary search strategies should be used, and when.

Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry

OBJECTIVES

To summarise logistical aspects of recently completed systematic reviews that were registered in the International Prospective Register of Systematic Reviews (PROSPERO) registry to quantify the time and resources required to complete such projects.

DESIGN

Meta-analysis.

DATA SOURCES AND STUDY SELECTION

All of the 195 registered and completed reviews (status from the PROSPERO registry) with associated publications at the time of our search (1 July 2014).

DATA EXTRACTION

All authors extracted data using registry entries and publication information related to the data sources used, the number of initially retrieved citations, the final number of included studies, the time between registration date to publication date and number of authors involved for completion of each publication. Information related to funding and geographical location was also recorded when reported.

RESULTS

The mean estimated time to complete the project and publish the review was 67.3 weeks (IQR=42). The number of studies found in the literature searches ranged from 27 to 92 020; the mean yield rate of included studies was 2.94% (IQR=2.5); and the mean number of authors per review was 5, SD=3. Funded reviews took significantly longer to complete and publish (mean=42 vs 26 weeks) and involved more authors and team members (mean=6.8 vs 4.8 people) than those that did not report funding (both p<0.001).

CONCLUSIONS

Systematic reviews presently take much time and require large amounts of human resources. In the light of the ever-increasing volume of published studies, application of existing computing and informatics technology should be applied to decrease this time and resource burden. We discuss recently published guidelines that provide a framework to make finding and accessing relevant literature less burdensome.