Adoption of peer review of literature search strategies in knowledge synthesis from 2009 to 2018: An overview

Evaluation of the interim Cochrane rapid review methods guidance-A mixed-methods study on the understanding of and adherence to the guidance

The Cochrane Rapid Review Methods Group (RRMG) first released interim guidance in March 2020 to support authors in conducting rapid reviews (RRs). The objective of this mixed-methods study was to assess the adherence and investigate authors' understanding of the RRMG guidance. We identified all documents citing the Interim Cochrane RRMG guidance up to February 17, 2022 and performed an exploratory adherence analysis. We interviewed 20 RR authors to assess the recommendations' comprehensibility and reasons for any deviations. Further, we surveyed nine authors of COVID-19-related Cochrane reviews for their reasons for not conducting a RR. We analyzed 128 RRs (111 non-Cochrane, 17 Cochrane) that cited the RRMG guidance. Several recommendations were not followed by a large proportion of RR authors such as stepwise approach to study design inclusion or peer review of search strategies, whereas others were exceeded, for example, dual independent screening of abstracts/full texts. The most reported reasons for deviating from the guidance were time constraints, unclarities in the recommended approach, or inapplicability to the specific RR. Overall, the guidance was viewed as user-friendly; however, without pre-existing knowledge of systematic review (SR) conduct, the application was perceived as difficult. The main reasons for conducting a full SR over a RR were late availability of the guidance, preset mandate to conduct a SR, uncertainty regarding methodological distinctions between SR and RR, and inapplicability to the evidence base. Clarifications are warranted throughout the Interim Cochrane RRMG guidance to ensure that users with various experience levels can understand and apply its recommendations accordingly.

Why do researchers co-author evidence syntheses with librarians? A mixed-methods study

Librarians and information specialists are experts in designing comprehensive literature searches, such as those needed for Evidence Syntheses (ES). The contributions of these professionals to ES research teams have several documented benefits, especially when they collaborate on the project. However, librarian co-authorship is relatively rare. This study explores researcher motivations for working with librarians at the co-author level through a mixed methods design. Interviews with researchers identified 20 potential motivations that were then tested through an online questionnaire sent to authors of recently published ES. Consistent with previous findings, most respondents did not have a librarian co-author on their ES, though 16% acknowledged one in their manuscript and 10% consulted one but did not document the contribution. Search expertise was the most common motivation both to and not to co-author with librarians. Those that had or were interested in co-authoring stated that they wanted the librarians' search expertise, while those who had not or were not interested stated that they already had the necessary search expertise. Researchers who were motivated by methodological expertise and availability were more likely to have co-authored their ES with a librarian. No motivations were negatively associated with librarian co-authorship. These findings provide an overview of the motivations that influence researchers to bring a librarian into an ES investigatory team. More research is needed to substantiate the validity of these motivations.

The National Institute for Health and Care Excellence information specialist development pathway: Developing the skills, knowledge and confidence to quality assure search strategies

Quality assurance (QA) is an important process in ensuring that systematic reviews and other evidence syntheses are supported by a high-quality search. This paper describes how the National Institute for Health and Care Excellence (NICE) in the UK established a development pathway to ensure its information specialists had the skills, knowledge and confidence to undertake search QA. The key component of the pathway is that it blends technical knowledge with interpersonal skills. The pathway develops technical skills in the early steps before using peer support activities to build confidence while undertaking a range of searches. QA is effective when the search lead communicates the contextual information that has influenced search development. QA is treated as a collaboration to get the right search for the review. The key requirements for search QA, alongside technical knowledge, are communication, collaboration and negotiation skills.

Avoiding searching for outcomes called for additional search strategies: A study of Cochrane review searches

OBJECTIVE

A search strategy for a systematic review that use the PICO-model as framework, should include the population, the intervention(s), and the type(s) of study design. According to existing guidelines outcome should generally be excluded from the search strategy unless the search is multistranded. However, a recent study found that approximately 10% (51) of recent Cochrane reviews on interventions included outcomes in their literature search strategies. This study aims to analyze the alternatives to including outcome in a search strategy, by analyzing these recent Cochrane reviews.

STUDY DESIGN

This study analyses the 51 Cochrane reviews that included outcomes in their literature search strategies and analyzes the results of alternative search strategies that follow current recommendations.

RESULTS

Despite a small study sample of 51 reviews, the results show that many of the reviews excluded some of the recommended elements due to very broadly defined elements (e.g., all interventions or all people). Furthermore, excluding outcome from the search strategy is followed by an enormous increase in the number of retrieved records making it unmanageable to screen, if using a single stranded search strategy.

CONCLUSION

Recommendations for search strategies in difficult cases are called for.

Modular literature review: a novel systematic search and review method to support priority setting in health policy and practice

BACKGROUND

There is an unmet need for review methods to support priority-setting, policy-making and strategic planning when a wide variety of interventions from differing disciplines may have the potential to impact a health outcome of interest. This article describes a Modular Literature Review, a novel systematic search and review method that employs systematic search strategies together with a hierarchy-based appraisal and synthesis of the resulting evidence.

METHODS

We designed the Modular Review to examine the effects of 43 interventions on a health problem of global significance. Using the PICOS (Population, Intervention, Comparison, Outcome, Study design) framework, we developed a single four-module search template in which population, comparison and outcome modules were the same for each search and the intervention module was different for each of the 43 interventions. A series of literature searches were performed in five databases, followed by screening, extraction and analysis of data. "ES documents", source documents for effect size (ES) estimates, were systematically identified based on a hierarchy of evidence. The evidence was categorised according to the likely effect on the outcome and presented in a standardised format with quantitative effect estimates, meta-analyses and narrative reporting. We compared the Modular Review to other review methods in health research for its strengths and limitations.

RESULTS

The Modular Review method was used to review the impact of 46 antenatal interventions on four specified birth outcomes within 12 months. A total of 61,279 records were found; 35,244 were screened by title-abstract. Six thousand two hundred seventy-two full articles were reviewed against the inclusion criteria resulting in 365 eligible articles.

CONCLUSIONS

The Modular Review preserves principles that have traditionally been important to systematic reviews but can address multiple research questions simultaneously. The result is an accessible, reliable answer to the question of "what works?". Thus, it is a well-suited literature review method to support prioritisation, decisions and planning to implement an agenda for health improvement.

Improving peer review of systematic reviews by involving librarians and information specialists: protocol for a randomized controlled trial

Background

Problems continue to exist with the reporting quality and risk of bias in search methods and strategies in systematic reviews and related review types. Peer reviewers who are not familiar with what is required to transparently and fully report a search may not be prepared to review the search components of systematic reviews, nor may they know what is likely to introduce bias into a search. Librarians and information specialists, who have expertise in searching, may offer specialized knowledge that would help improve systematic review search reporting and lessen risk of bias, but they are underutilized as methodological peer reviewers.

Methods

This study will evaluate the effect of adding librarians and information specialists as methodological peer reviewers on the quality of search reporting and risk of bias in systematic review searches. The study will be a pragmatic randomized controlled trial using 150 systematic review manuscripts submitted to BMJ and BMJ Open as the unit of randomization. Manuscripts that report on completed systematic reviews and related review types and have been sent for peer review are eligible. For each manuscript randomized to the intervention, a librarian/information specialist will be invited as an additional peer reviewer using standard practices for each journal. First revision manuscripts will be assessed in duplicate for reporting quality and risk of bias, using adherence to 4 items from PRISMA-S and assessors’ judgements on 4 signaling questions from ROBIS Domain 2, respectively. Identifying information from the manuscripts will be removed prior to assessment.

Discussion

The primary outcomes for this study are quality of reporting as indicated by differences in the proportion of adequately reported searches in first revision manuscripts between intervention and control groups and risk of bias as indicated by differences in the proportions of first revision manuscripts with high, low, and unclear bias. If the intervention demonstrates an effect on search reporting or bias, this may indicate a need for journal editors to work with librarians and information specialists as methodological peer reviewers.

Trial registration

Open Science Framework. Registered on June 17, 2021, at 10.17605/OSF.IO/W4CK2.

10,000 steps a day? Activity trackers, health information literacy and the potential role of Information Science & Health Libraries

Acknowledging the new ways in which activity tracking technologies (ATTs) such as Apple Watch, Fitbit and Garmin are changing the ways in which we understand human-information interactions, Aylin Imeri calls for information science and health libraries to extend their health information literacy remit to support ATT user information needs.